Medicare Health Outcomes Survey Supporting Statement
C. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
1) Overview of the Medicare HOS structure
We will continue to administer and operate the Medicare HOS using the same historical protocol as from 1998 through 2006. The Medicare HOS is a longitudinal survey that assesses the physical and mental health status of Medicare beneficiaries enrolled in managed care in the United States. Beginning in 1998 and continuing annually, a new baseline cohort is created from a randomly selected sample of 1,000 Medicare managed care enrollees from each applicable Medicare contract market area. In plans with fewer than 1,000 Medicare beneficiaries, the sample includes the entire enrolled Medicare population that meets the inclusion criteria. Medicare beneficiaries who are continuously enrolled in the same health plan for at least six months are eligible for sampling. Baseline respondents are surveyed again two years later (the follow-up measurement).
The data collection protocol includes a combination of mail and telephone surveys. An advance postcard, followed by up to two survey instruments, is mailed to each Medicare beneficiary drawn into the study. These mail surveys are available in English, Spanish and Chinese. Non-responses are followed up with up to six telephone calls. English and Spanish CATI procedures are followed to collect information during the telephone follow-up phase.
Although the Medicare HOS is switching from a 36 to 12 item core, the 12-item survey has been applied and validated relative to the 36-item survey (as described in Section B 15).
2) Description of sample selection and universe
A. Potential respondent universe.
The target universe is current Medicare beneficiaries enrolled in managed care, living in the 50 states, the Virgin Islands, and Puerto Rico. Aged and disabled Medicare beneficiaries continuously enrolled in the same MA plan for six months are eligible. Institutionalized members are eligible. Members with end state renal disease (ESRD) are excluded.
B. Sample Sizes.
Target sample sizes.
The baseline sample size for each MA Plan is 1,000. The survey protocol is designed to achieve a 70 percent response rate for the baseline survey. The targeted response rate for the follow-up survey is 80 percent. The initial sample size and targeted baseline and follow-up response rates are designed to achieve an analytic sample size with adequate statistical power to detect significant variation between health plans on physical and mental health outcomes.
Baseline Survey sampling.
Due to variations in health plan population size, three sampling approaches are used.
a) In MCOs with Medicare populations greater than or equal to 3,000 members, 1,000 members are randomly selected for the Baseline Survey. To reduce burden on survey respondents, members who were sampled for and returned a completed survey the previous year are excluded from sampling.
b) In MCOs with populations of 1,001 to 2,999 members, 1000 members are randomly selected for the Baseline Survey. To ensure a sample size of 1,000, members who were sampled for and responded with a completed survey the previous year are not excluded from sampling.
c) In MCOs with populations less than or equal to 1,000 members, all eligible members who meet the eligible population criteria are included in the sample.
Follow-Up Survey sampling
Eligible members include all Baseline (two years prior) respondents for whom a valid physical component summary score and mental component summary score were calculated and who remain enrolled in the same MA plan.
C. Response rates.
A total survey response rate is calculated for each sample that is the total number of complete surveys divided by all eligible members of the sample. Eligible members of the sample include the entire random sample minus members assigned a survey disposition code of ineligible. In other words, invalid beneficiaries are removed from the denominator in the response rate calculations.
The total survey response rate is calculated as follows:
Number of Complete Surveys
Entire random sample – [Ineligible for any of the following reasons: Deceased + Not enrolled in MCO + End-stage renal disease + Language barrier + Bad address and nonworking and unlisted phone number or person unknown at the dialed phone number]
Attachment 4 illustrates survey response rates from 1998 through 2005 (the latest completed year of data collection). Over the course of time, the refinement of the Medicare HOS measure has resulted in changes to the definition of a “completed survey”. In general, for Cohorts 1-3 Baseline, a completed survey is defined as a survey that has at least 80% of the questions answered. However, beginning with Cohort 1 Follow Up and Cohort 4 Baseline, a completed survey is defined as a survey with calculatable physical and mental health summary scores.
Given that the definition of a completed survey has evolved over time, this table should not be utilized for response rate comparisons across the cohorts. (For the Medicare HOS-modified, a completed survey is defined as a survey that, at a minimum, had responses to all six activities of daily living measures.)
For the baseline cohorts, the reporting units represent the individual plans sampled for the survey. However for the follow up cohorts, the reporting units have been adjusted to accommodate selected plan consolidations and service area reductions at the time of performance measurement reporting (which typically occurs in the year subsequent to the collection of the follow up data).
3) Data collection procedures
This section describes the procedures used for the national survey. It includes a summary of the questionnaire content, and a general discussion of the case-mix and risk-adjustment methodology. Finally, there is a discussion of rules for allowing proxy response.
A. Years 2007 through 2009 data collection procedures.
(i) Mail phase.
The mail component of the survey uses standardized questionnaires, cover letters and postcards. Because of the challenge in corresponding with a diverse population, it is necessary to support material in multiple languages. The three most common languages encountered with the Medicare population are English, Spanish and Chinese. Both the standard and modified instruments (Attachments 5 and 7, respectively) plus associated supporting correspondence (Attachments 6 and 8) are available in each of the three identified languages.
To ensure comparability, the survey vendor may not change the wording of the survey questions, the response categories or the order of the questions. The survey vendor may modify the format and layout of the questionnaire, adhering to specified formatting parameters.
The Pre-notification Postcard and Letter for First Questionnaire contain English and Spanish text. The Letter for First Questionnaire is double-sided. One side of the letter contains English text and the other side contains Spanish text. The Spanish text invites Spanish-speaking members to request a Spanish version of the questionnaire by calling the survey vendor’s toll-free customer support number.
Completed questionnaires can be manually key-entered into the computer, or optically scanned. To ensure quality for key-entered data, two separate data entry specialists must independently key answers for each questionnaire. A comparison of the separate entries identifies data entry errors that need adjudication by a supervisor.
(ii) Telephone phase.
The telephone component uses a standardized CATI script and specific design specification. The survey vendor is responsible for programming the scripts and specifications into its existing CATI software. To ensure the comparability of survey results, the survey vendor cannot change the wording of survey questions, the response categories or order of the questions.
Following the mail portion of the process, survey vendors identify members who did not respond to the mail survey and members who returned a blank or incomplete mail questionnaire (a questionnaire with less than 80 percent of questions complete). These members are eligible for telephone interviews.
The survey vendor attempts to contact non-respondents by telephone so that at least six telephone calls are attempted at different times of day, on different days of the week and in different weeks.
The survey vendor establishes training programs for all personnel involved in the telephone phase of the process. It establishes quality control procedures and monitors staff performance to ensure the integrity of the telephone interviewing process. The survey vendor monitors 10 percent of CATI interviews to evaluate the quality of interviewing and provides feedback and additional training as necessary.
B. Interview content.
There are two survey instruments used by the Medicare HOS. While both are similar, there are some distinct differences. These two instruments are: Medicare HOS 2.0 and Medicare HOS 2.0-Modified (HOS 2.0-M).
While the Medicare HOS 2.0 is the primary instrument used for data collection, considerations were made for how the Medicare HOS 2.0 is administered to vulnerable Medicare beneficiaries at greatest risk for poor health outcomes. These beneficiaries are enrolled in specialized Medicare plans—PACE programs, Minnesota Senior Health Options, Minnesota Disability Health Options, Wisconsin Partnership Program, and Massachusetts MassHealth Senior Care Options. The main goal of the Medicare HOS 2.0-M is to assess the frailty of the population in these plans in order to adjust Medicare payments.
PACE is a capitated benefit authorized by the Balanced Budget Act of 1997 (BBA) that features a comprehensive service delivery system and integrated Medicare and Medicaid financing. The PACE model was developed to address the needs of long-term care clients, providers, and payers. For most participants, the comprehensive service package permits them to continue living at home while receiving services rather than be institutionalized. Capitated financing allows providers to deliver all services participants need, rather than be limited to those reimbursable under the Medicare and Medicaid fee-for-service systems.
The Medicare HOS 2.0-M survey is a modified version of the Medicare Health Outcomes Survey. The instrument assesses the physical and mental health functioning of the Program members to generate information for payment adjustment. It includes 12 physical and mental health status questions, one question about memory loss interfering with daily activities, and one question about urinary incontinence. If the participant received assistance completing the questionnaire, the respondent was asked why a proxy was needed, how the proxy assisted the participant and the staff position of the proxy.
The Medicare HOS 2.0 is comprised of three major components: 1) the VR-12, as the core component; 2) questions to gather information for case-mix and risk-adjustment; and 3) questions added by CMS to gather information required by the 1997 Balanced Budget Act.
(i) The VR-12.
The VR-12 consists of items falling into 8 concepts of health. Items are scored and summarized into a physical component summary (PCS) and a mental component summary (MCS).
The taxonomy underlying the construction of the VR-12 scales (concepts) and summary measures has three levels: 1) fourteen items; 2) eight scales that aggregate one or two items each; 3) two summary measures that aggregate the eight scales.
Scales covered in the VR-12 are:
Physical Functioning (PF) consists of two questions that ask respondents to indicate the extent to which their health limits their physical activities.
Role—Physical (RP) consists of two questions that assess whether respondents’ physical health limits them in the kind of work or other usual activities they perform both in terms of time and performance.
Bodily Pain (BP) consists of one question that determines the extent to which pain interferes with the respondent’s normal activities.
General Health (GH) consists of one question that asks respondents to rate their current health status overall.
Vitality (VT) consists of one question that asks respondents to rate their well being by indicating how frequently they experience energy.
Social Functioning (SF) consists of one question that asks respondents to indicate limitations in social functioning due specifically to health.
Role—Emotional (RE) consists of two questions that assess whether emotional problems have caused respondents to accomplish less in their work or other usual activities in terms of time and performance.
Mental Health (MH) consists of two questions that ask respondents how frequently they experience feelings representing the four major mental health dimensions: anxiety; depression; loss of behavioral / emotional control; and psychological well being.
In addition, a two-item measure of Change in Health asks respondents to rate their general physical health and emotional problems now compared to their health one year ago.
(ii) Questions to gather information for case-mix and risk-adjustment.
Case-mix and risk-adjustment are essential for meaningful and valid plan-to-plan comparison of health outcomes. By including variables to control for differences in demographics and socioeconomic characteristics, chronic medical conditions, and Medicare HOS study design variables, regression techniques can be used to adjust the scales and summary measures
Demographics include questions on beneficiary age, gender, race, education, marital status, and annual household income.
Thirteen chronic medical conditions are included in the questionnaire. These conditions are: hypertension; angina pectoris or coronary artery disease; congestive heart failure; myocardial infarction or heart attack; other heart conditions, such as heart valve defects or arrhythmias; stroke; emphysema, asthma, or Chronic Obstructive Pulmonary Disease (COPD); inflammatory bowel disease, including Crohn’s disease and ulcerative colitis; arthritis of the hip or knee; arthritis of the hand or wrist; sciatica; diabetes, hyperglycemia, or glycouria; and any cancer (other than skin cancer).
Study design variables include who completed the survey, the mode of survey administration, CMS region, and the survey vendor.
(iii) Questions to gather comparative information on managed care plans.
Six Activities of Daily Living (ADLs) are included to determine self-reported difficulty with performance of daily tasks. ADLs include bathing, dressing, eating, getting in or out of chairs, walking, and using the toilet.
Three depression screening questions include levels of depression that encompass: feeling sad / blue for two weeks in the past year; depression for much of the past year; or depression for two years or more in life.
Three Healthy Days questions, which encompass the number of days in the past thirty days that physical health was not good, mental health was not good, and activities were limited due to poor physical or mental health.
HEDIS® Effectiveness of Care includes four measures on the management of urinary incontinence, physical activity in older adults, fall risk management in older adults, and osteoporosis testing in older women.
C. Adjustment methodologies.
(i) Case-mix adjustment.
The analytic sample for the Medicare HOS is limited to those seniors (age 65 and over) with calculatable physical component summary and mental component summary scores. Regression techniques are used to case mix adjust these measures for each beneficiary.
As mentioned in C.3.B.(ii), models used to adjust the scales and summary measures included variables to control for differences in demographic and socioeconomic characteristics, chronic medical conditions, and study design variables. A series of three different models is used for each measure since all beneficiaries do not have completed data for all of the covariates. Only one model, the most comprehensive model possible, was used for each beneficiary and an adjusted score was calculated for each. Since the Chinese language version of the survey does not contain the marital status and income questions, only model three is used for adjustment of the scores for these respondents.
Model One: If the beneficiary has completed data for all of the covariates, then the adjusted scores are calculated using Model One which contains all variables.
Model Two: If the beneficiary has completed data for all covariates but annual household income, Model Two is used.
Model Three: If the beneficiary does not have enough completed data for Model One or Two, then Model Three is used. The variables included in Model Three, which are available for all beneficiaries sampled, are age, gender, race, mode of survey administration, CMS region, and survey vendor.
See (Attachment 9) for a detailed breakdown of covariates used in the Case Mix Adjustment of PCS and MCS measures for the Cohort 8 Baseline Data User’s Guide. This methodology will remain in practice, although minor refinements could be explored to maintain the accuracy of the measures.
(ii) Risk Adjustment.
In addition to the VR-12 Health Survey, the Medicare HOS questionnaire collects information for purposes of a standardized plan-to-plan risk adjustment. Additional items include variables for morbid conditions, activities of daily living, and socio-demographic characteristics. Risk adjustment accounts for patient-associated factors before comparing outcomes across different health plans or populations. Therefore, risk adjustment enables a fair comparison between health plans.
For each respondent, a change in functional health score will be estimated by subtracting the first (baseline) questionnaire score from the second (follow up) score. Taking into account an expected decline in health and risk-adjustment factors, change in physical and mental health status can be: better, the same, or worse than expected.
Because outcome is defined as a change score, each respondent serves as his or her own ‘control’. In addition plan-to-plan risk adjustments are based on morbid conditions at baseline, income, household size, social support, education, race and gender. Results are aggregated across respondents for each health care plan.
Proxy rules.
While sampled members are encouraged to respond directly to the mail or telephone survey, not all elderly or disabled respondents are able to do so. In such cases, proxy responses are acceptable. The survey instrument instructs members who are unable to complete the survey to have a family member or other proxy complete the survey for them.
If a proxy completed the baseline survey, every effort is made to have the same proxy or, if possible, the sampled member, complete the follow-up survey. Having the same proxy complete both questionnaires minimizes bias. If the same proxy or the sampled member is unable to complete the follow-up survey, another proxy may be used to collect responses.
4) Methods to Maximize Response Rates
Medicare HOS is sampling a heterogeneous population that presents a unique challenge for maximizing response rates. The household survey will be approaching two groups--aged and disabled Medicare beneficiaries‑‑who have characteristics that often lead to refusals on surveys. Increasing age, poor health or poor health of a family member are prevalent reasons for refusal. On the other hand, older persons are the least mobile segment of the population and thus less likely to be lost due to failure to locate. Little is known about the disabled population in terms of survey response rates. It is anticipated that they will be most similar to the oldest old population because of their ill health.
Because this is a longitudinal survey it is essential that we maximize the response rates. In order to do so, CMS employs an extensive annual outreach effort. This includes the notification of government entities (CMS regional offices and hotline, carriers and fiscal intermediaries, and Social Security Offices), national organizations including the American Association of Retired Persons, the Association for Retarded Citizens and various community groups (e.g., mayor's offices, police, social service and health departments, home health agencies, state advocates for the elderly and area agencies on aging). These efforts are undertaken to increase the likelihood that respondents would answer the Medicare HOS questions and remain in the survey panel by: 1) informing authoritative sources to whom survey participants are likely to turn if they suspect the legitimacy of the Medicare HOS; 2) giving interviewers resources to which they can refer to reassure respondents of the legitimacy/importance of the survey; and 3) generally making information about Medicare HOS available through senior centers, other networks to which survey participants are likely to belong and through the CMS website.
In addition to the outreach efforts, the following efforts remain in place to maintain a sense of validity and relevance among the survey participants.
An advance letter is sent to potential respondents from CMS
Interviewer training emphasizes the difficulties in communicating with the older population and ways to overcome these difficulties.
Proxy respondents are sought for survey participants unable to participate for themselves.
Mail non-respondents are re-contacted by telephone.
A toll-free number is available to answer respondent's questions.
An E-mail address and website are available at CMS to answer respondent’s questions.
A shortened version of the survey instrument is administered to the frailest portion of the sample.
“Take my name off your list” refusals are flagged for exclusion from all future Medicare HOS activity.
The following disclosure statement is included with our survey instrument:
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0938-0701. The time required to complete this information collection is estimated to average 20 minutes per response, including the time to review instructions, search existing data resources, gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.
5) People responsible for statistical aspects of design.
Lewis Kazis, ScD William Rogers, PhD
Boston University 19 Rebecca Road
715
Albany Street (T-3W) Newton, MA 02465
Boston, MA
02118 Phone: (617) 332-1814Phone: (617) 414-1434
| File Type | application/msword |
| Author | CMS |
| Last Modified By | Chris Haffer - CMS |
| File Modified | 2007-01-24 |
| File Created | 2007-01-24 |