Medicare Health Outcomes Survey
Supporting Statement B
Contents
Collections of Information Employing Statistical Methods 3
1. Description of the Respondent Population and Sample 3
2. Procedures for the Collection of Information 4
3. Maximization of Response Rates, Nonresponse, and Reliability 5
5. Statistical Consultation and Information Analysis 8
The target population for the HOS is Medicare beneficiaries enrolled in Medicare Advantage Organizations (MAO). MAOs with more than 500 beneficiaries are required to administer the HOS Baseline annually.
The HOS sampling procedure is designed to reduce burden on survey respondents and to prevent MAOs from identifying beneficiaries selected for HOS participation. MAOs in effect on or before January 1 of the previous year with at least 500 beneficiaries as of February 1 of the reporting year are required to report HOS Baseline. CMS draws the HOS sample at the contract level using a random sampling methodology. MAOs with fewer than 500 beneficiaries are excluded from HOS Baseline. For MAOs with fewer than 1,200 beneficiaries, all eligible beneficiaries are sampled for HOS Baseline. For MAOs with 1,201 or more beneficiaries, a random sample of 1,200 beneficiaries is drawn for HOS Baseline.1
MAOs required to administer the Baseline survey two years ago are also required to administer a Follow-Up survey, regardless of enrollment size. All respondents with a valid Physical Component Summary (PCS) or Mental Component Summary (MCS) score calculated at Baseline survey (collected two years prior) are surveyed for Follow-Up. Beneficiaries who disenrolled from their MAO or died after the Baseline survey are not eligible for Follow-Up.
The target population for the HOS-M is the Program for All-Inclusive Care for the Elderly (PACE) population. All PACE organizations in effect on or before January 1 of the previous year with 30 or more beneficiaries are required to administer the HOS-M. The sample size is 1,200 for plans with at least 1,200 beneficiaries. The entire eligible enrollment is included in the sample for plans with less than 1,200 beneficiaries. PACE plans with less than 30 beneficiaries are excluded from the HOS-M.
A total survey response rate can be calculated for HOS (Baseline and/or Follow-Up) and HOS-M. The response rate is the total number of complete surveys divided by all eligible members of the sample. For the HOS, complete surveys are surveys that can be used to calculate physical or mental health summary scores. For the HOS-M, complete surveys are surveys with all ADL questions completed. Eligible sampled members include the entire random sample minus members assigned a disposition code of “Ineligible.” The total survey response rate is calculated as follows:
Number of Complete Surveys
Entire random sample – [Ineligible for any of the following reasons: Deceased, Not enrolled in MAO, Language barrier, Removed from sample, Duplicate, Beneficiary listed twice in the sample frame, Bad address and nonworking/unlisted phone number or person unknown at the dialed phone number, or Bad address and no available telephone protocol]
Refer to Attachment D for the number of entities covered in data collection and response rates from 2014 through 2019.
The HOS utilizes a mixed-mode approach for data collection that includes two mailings with telephone follow-up of non-respondents. MAOs contract with CMS-approved survey vendors who collect data on behalf of the MAO. This allows for protection of sampled beneficiary confidentiality, as well as a high level of quality in survey administration. CMS trains and oversees the survey vendors. To ensure comparability of survey results, survey vendors are not permitted to modify the wording of survey materials or to change any survey protocols. Therefore, all beneficiaries receive the exact same survey and survey outreach.
Sampled beneficiaries receive survey outreach in the form of a pre-notification letter, a questionnaire with a survey cover letter, and a reminder postcard in the mail. Beneficiaries are encouraged to complete the survey and mail it back (in a pre-paid business reply envelope) to their survey vendor. If a beneficiary does not respond to the first round of mail outreach, then they are sent an additional questionnaire with a survey cover letter five weeks after they receive the first questionnaire. This offers beneficiaries another opportunity to respond to the survey if they misplaced the first questionnaire. The language in the HOS mailing materials was designed to maximize respondent understanding while minimizing comprehension burden by using simple language. Attachment E and Attachment F include the templates for the mailing materials.
Beneficiaries who do not respond to the questionnaire via mail are also contacted via telephone. Survey vendors use specially trained telephone interviewers to contact nonrespondents and attempt to complete the survey with the beneficiary over the phone. Survey vendors are required to conduct at least six telephone attempts to try to reach the beneficiary. Survey vendors schedule call attempts at different times of the day, on different days of the week, and in different weeks to increase the chances of connecting with a beneficiary when they are available.
The HOS utilizes various strategies to maximize response rates. The mixed-mode protocol is a comprehensive strategy that avoids reliance on mail or telephone administration alone. Following the mail portion of the protocol, survey vendors identify beneficiaries who either did not return a questionnaire or who returned a blank or incomplete questionnaire (for operational purposes, an incomplete questionnaire has less than 80 percent of questions answered and is missing one or more Activities of Daily Living (ADL) questions) and triage them to telephone follow-up. Survey vendors are required to make at least six telephone attempts to these beneficiaries. The mixed-mode methodology (a pre-notification letter, two mailings of the questionnaire, and telephone follow-up for non-respondents) is a multi-pronged, comprehensive strategy that avoids the weaknesses of reliance upon mail or telephone administration alone.
Sampled beneficiaries are mailed a pre-notification letter signed by a CMS official one week before they receive the first questionnaire. This pre-notification letter includes information about the survey and highlights the importance of completing it. The letter also includes an assurance that the member’s information will be kept confidential. The intent of this letter is to increase the likelihood that sampled beneficiaries keep an eye out for the questionnaire and that they understand the critical role they play in the survey’s success. The HOS has been thoroughly reviewed and all of the mailing materials were designed to ease respondent comprehension and encourage participation by using plain language. Every effort is made to maximize response rates while retaining the voluntary nature of the HOS. All survey vendors are required to use current best practices for survey materials to enhance response rates. These best practices include using a simple font no smaller than 13-points on the survey cover letters, allowing ample white space between questions in the questionnaire, using bolded arrows to help guide the respondent over questions to be intentionally “skipped,”2 using a unique subject identification number on the questionnaire rather than printing the sampled member’s name, and displaying the OMB number and expiration date on the questionnaire.
If sampled beneficiaries have any questions about the survey, CMS and each survey vendor provide a website and email address to answer respondent questions and to attest to the validity of the survey. In addition, toll-free customer support numbers are included on all mailing materials so that sampled beneficiaries can easily determine who to contact with questions. Beneficiaries can also call the survey vendor’s toll-free phone number to complete the survey by phone prior to the start of the telephone follow-up period, if they feel more comfortable completing the survey on the phone rather than through the mail.
Survey vendors receive contact information (address and telephone numbers) for beneficiaries directly from their MAO contracts. The process of receiving contact information is structured to protect beneficiary confidentiality by preventing MAOs from identifying which beneficiaries have been selected for the survey. MAOs typically have up-to-date contact information for their beneficiaries, as many MAOs communicate with their beneficiaries often. In addition, to ensure that survey vendors have the most accurate and up-to-date contact information, they use the National Change of Address database to update beneficiary addresses. Similarly, survey vendors use telephone append services to obtain updated and/or corrected telephone information for each beneficiary.
To further maximize response rates, CMS provides the HOS in various languages to support the diversity of MA beneficiary response. The survey is offered in English, Spanish, Chinese, and Russian. CMS expanded the language options in 2019 to include Russian in response to stakeholder requests and routinely evaluates requests for new survey languages to best meet the needs of the Medicare population. Due to the wide variety of languages offered, less than 1% of survey responses are lost to language barriers.
Additionally, the HOS permits the use of proxies. Proxies can complete the survey on behalf of a beneficiary. This practice increases response rates, as some beneficiaries are unable to complete the survey on their own due to health reasons or various other circumstances.
Telephone interviewers are trained to seek proxies who can answer questions about the beneficiary’s health, so that the proxy is informed and can provide accurate information.
A shortened version of the survey instrument is administered to the frailest portion of the sample. The Medicare HOS-M survey is a modified version of the Medicare HOS (and remains unchanged). The instrument assesses the physical and mental health functioning of PACE and FIDE-SNP members to generate frailty information for payment adjustment. It includes 12 physical and mental health status questions, Activities of Daily Living (ADLs), one question about memory loss interfering with daily activities, and one question about urinary incontinence. The HOS-M also asks about the use of proxies.
HOS data collection is standardized across CMS-approved survey vendors to ensure comparability of survey results. CMS has designed technical specifications for the collection and submission of HOS data to further promote standardization and accuracy. Survey vendors submit data on behalf of MAOs required to report HOS data. All survey vendors are trained annually by NCQA on CMS’s technical specifications. Survey vendors are required to submit a Quality Assurance Plan, which details how the survey vendor will operationalize adherence to CMS’s technical specifications, to NCQA for evaluation. In addition to these established plans, NCQA provides survey vendor oversight throughout survey administration to verify that survey vendors comply with the established technical specifications.
The survey protocols have been rigorously tested over time. The mail and telephone surveys have been cognitively tested in different languages to ensure comparability of results across the survey translations and cultural competence. There are no plans to conduct field testing of the currently established procedures or methods. CMS considers web administration across all its various surveys and will submit a separate package for approval to test this mode of data collection should it proceed with testing it for this survey.
CMS receives ongoing input from statisticians in developing, designing, conducting, and analyzing the information collected from this survey. The HOS data analysis programs use multivariate analysis to control for differences in plan enrollments according to specific enrollee characteristics for which the plan has no control, such as age, education, marital status, and home ownership, and chronic conditions. By adjusting for these effects, the measures control for differences in the proportions of enrollees in each plan having these characteristics.
In the February 2020 Federal Register2 CMS proposed substantive updates to two measures from the Medicare HOS: Improving or Maintaining Physical Health measure and Improving or Maintaining Mental Health measure. The first proposed change is to the casemix adjustment (CMA) for the measures. CMS proposed to modify the current approach for adjusting for differences in the case mix of enrollees across contracts. The proposed approach improves the case-mix model performance and simplifies the implementation and interpretation of case-mix results when particular case-mix variables are missing. Now, when an adjuster is missing for a beneficiary, it would be replaced with the mean value for that adjuster for other beneficiaries in the same contract. This proposed approach has been used for the MA-PDP CAHPS surveys for many years. In simulation models, this approach either outperformed the current approach for predicting outcomes or matched the current approach. The second change involved increasing the minimum required denominator from 30 to 100 for the two measures. The proposed increase brings these measures into alignment with the denominator requirements for the HEDIS measures contained in the HOS survey and increases the reliability of these measures compared to the prior reporting minimum of 30.Statistical expertise will continue to be available from HSAG, NCQA, RAND, and Boston University.
Ongoing statistical consultation is provided by:
Doug Ritenour, MPH Health Services Advisory Group 3133 E. Camelback Rd., Suite 100 Phoenix, AZ 85016 |
Sarah Hudson Scholle, MPH, DrPH National Committee for Quality Assurance 1100 13th Street NW, Third Floor Washington, DC 20005 |
Marc Elliott, PhD RAND Health 1776 Main Street Santa Monica, CA 90401 |
Lewis Kazis, ScD Boston University 715 Albany Street (T-3W) Boston, MA 02118 |
NCQA manages data collection for HOS under contract to CMS. NCQA also calculates the HEDIS Effectiveness of Care measure results. HSAG conducts data cleaning, analysis, and data dissemination for HOS under contract to CMS.
1 Beneficiaries enrolled in an Institutional Special Needs Plan (I-SNP) are not sampled for HOS Baseline.
2 Federal Register /Vol. 85, No. 32 /Tuesday, February 18, 2020 / Proposed Rules posted athttps://www.federalregister.gov/d/2020-02085/page9002.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | OMB PRA Package - Statement B |
| Author | Centers for Medicare and Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2021-05-11 |