Response to Comments (60day)

CMS Response to Public Comments Received for CMS-10861

CMS received eight comments related to CMS-10861 (OMB control number 0938-New) for the Medicare Health Outcomes Survey Field Test during the 60-day comment period.

• Five commenters support CMS’s efforts to update the HOS survey instrument.

Response: CMS thanks the commenters for their support.

• Five commenters appreciate CMS’s proposed testing of PROMIS items as replacements for current functional status questions involving less inclusive activities. Some said these are more likely to resonate with most MA enrollees, but others expressed concerns about their applicability (e.g., they stated not everyone has regular access to 5 flights of stairs, some individuals do not move furniture, and some have permanent disabilities). Two commenters suggested CMS add “not applicable” as a response option.

Response: CMS thanks commenters for their feedback. We recognize that some MA enrollees cannot climb stairs or move furniture, but many can do these activities, and assessing the full range of physical functioning improves CMS’s ability to detect changes in functioning. The PROMIS item developers believe that even people without regular access to staircases are generally able to comprehend and assess their ability to engage in this activity. Since the items assess ability and not practice, a ‘not applicable’ response option is not appropriate.

• Five commenters support removing bowling and golf as examples of moderate activities. They note the activities are not relevant to certain groups, and many cannot afford them.

Response: CMS thanks the commenters for their support. Bowling and golf are examples from the original validated VR-12 item set. CMS recognizes these activities may not be applicable across the MA population and is therefore testing an alternate item (brisk walking) that covers a level of physical activity similar to bowling and golfing. Results of the field test will allow CMS to determine which items perform better empirically.

• Three commenters support adding PROMIS and GAD-2 questions to provide plans with better insights into their member’s mental and physical health. One asked CMS to clarify if the proposed questions will contribute to the Improving or Maintaining Physical Health (PCS) and Improving or Maintaining Mental Health (MCS) measures in 2025.

Response: CMS thanks commenters for their support. We proposed only to test alternatives for select survey items that could enhance and refine existing measures. No modifications to PCS or MCS have been proposed at this time. Any future proposals will be informed by analyses of field test data. Non-substantive changes to existing measures used in the Part C and D Star Ratings would be announced through the Advance Notice/Rate Announcement process. Substantive changes to existing or new measures for Part C and D Star Ratings would go through the Measures Under Consideration process, then be proposed through rulemaking.

• Four commenters recommended Health-Related Social Needs (HRSN) items not be added to the HOS, citing redundancy with Heath Risk Assessment (HRA) and NCQA’s Social Need Screening and Intervention (SNS-E) measure. One commenter supported adding the items but asked CMS to clarify the intent of the items and type of plan intervention CMS considers in the scope of this work. Another commenter noted other regulatory requirements require plans use validated screening tools, such as Protocol for Responding to & Assessing Patients' Assets, Risks & Experiences (PRAPARE) or Accountable Health Communities (AHC).

Response: CMS thanks commenters for this feedback. The approved screening tools for SNP HRAs are intended for use in medical and community-based settings, not a national survey. The intent of the proposed items is to measure ongoing unmet need, accountability, and plan performance in addressing their enrollees’ social needs. The proposed HOS items go beyond social risk screening and are not intended to replace the annual HRAs conducted by plans or the SNS-E measure. Rather, the items are intended to complement the electronic reporting of the SNS-E measure that assesses screening for unmet food, housing, and transportation needs and intervention referral if needed, by providing additional patient-reported data on ongoing unmet needs related to food insecurity, housing instability, and transportation availability in the MA population. The proposed questions about interventions are purposely broad to capture a variety of interventions. All questions will be evaluated using data from the field test to determine potential future use.

• Three commenters suggest retaining the proxy and living alone items, citing increasing rates of social isolation and lack of family care/partner support among older and disabled adults and the importance of comparing responses from paid professional caregivers.

Response: CMS thanks commenters for this feedback. We opted to remove the proxy item from the field test instrument because research indicates HOS proxy responses are quite similar to direct responses. Additionally, an indicator for proxy response contributes very little to baseline case-mix adjustment and is not used for performance measurement case-mix adjustment. Recent results show professional caregiver proxies account for fewer than 0.4% of HOS respondents. The living alone item was removed because some stakeholders and respondents have noted that the item makes them uncomfortable divulging information that may affect their personal security. The living alone item is not used for case-mix adjustment.

• One commenter suggested CMS retain six chronic conditions (CHF, myocardial infarction, other heart conditions, stroke, Crohn’s disease, and osteoporosis) and the current cancer treatment items (colon, lung, breast, prostate, and other cancer).

Response: CMS acknowledges that all HOS 3.0 items removed from the field test instrument have value, given the diversity of HOS respondents, but to limit burden and make room for testing items with potentially greater value for longitudinal quality measurement, CMS opted to remove items of relatively low prevalence. Removing the items from case-mix adjustment had a negligible impact on measure scores.

• One commenter asked CMS to add a question to allow respondents to self-report dual status (Medicaid enrollment) as a proxy for high social risk/low socio-economic status.

Response: An enrollee’s Medicaid status, as identified from CMS’s administrative data, has historically been used in the performance measurement case-mix adjustment (for details of the case-mix process and covariates, see Appendix A in the most recent Sample Performance Measurement Report: https://www.hosonline.org/globalassets/hos-online/survey-results/hos_samplepmr_c23.pdf). CMS sees no advantage in allowing respondents to self-report dual status.

• One commenter suggested adding questions about “Stroke or Progressive Neuro-Muscular Condition such as ALS, MS, or CMT” and “Alzheimer’s disease or dementia.”

Response: CMS will take this suggestion under consideration for the future. One concern would be overall survey length and burden to respondents.

• Three commenters said the HOS instrument needs more substantial revisions than those being proposed and suggested an additional review by stakeholders, including advocacy and beneficiary groups, special needs plans, researchers, and others, to guide revisions.

Response: CMS gathered stakeholder input regarding HOS enhancements through a Technical Expert Panel (TEP) in September 2022. Members included experts in geriatrics, mental health, survey methodology, patient reported outcomes, and SDOH, as well as health plans, academic researchers, advocacy groups, and health plan associations.

• Five commenters support proposed testing of a web mode for the HOS and CMS’s efforts to increase response rates. One commenter asked CMS to determine if adding a web-based mode would create any unintended survey bias. One said use of a web-based mode should be at the discretion of plans. Another asked that QR codes be added for convenience.

Response: CMS thanks the commenters for their support. CMS proposed testing a web-based mode as an addition to the existing mixed-mode protocol during the field test and will analyze results and the web mode’s impact on response rates before deciding whether to move forward with a web mode. We note that implementing a web mode for only some plans could create bias, and use of QR codes could reduce response rates since adults over age 65 are less likely to use them and 18% of seniors have never heard of QR codes.¹

• Two commenters agree that adding a web-mode may improve response rates but expressed concern that it would not increase representativeness and could increase the proportion of HOS respondents who are white, English speaking, and without social risk.

Response: Preliminary findings from other CMS implementations of web modes suggest a web-first mixed mode protocol may increase response rates and representativeness. Analyses of field test data will assess if that finding also applies for HOS. Per Supporting Statement B, page 7, field test analysis will include assessment of the representativeness of the population by respondent characteristics and by survey arm. While non-response varies by respondent characteristics for all surveys, case-mix adjustment helps to account for characteristics associated with non-response and improves the representativeness of the resulting scores.

• Three commenters recommended the field test sample size be substantially increased and diversified. Two said 340 individuals per contract is not sufficient to conduct valid statistical analyses. One suggested oversampling certain groups (e.g., persons of color, non-English-speaking persons, and individuals with condition complexity and social risk characteristics) to allow subgroup analysis. Another suggested stratifying by race, dual status, and spoken language. One commenter believed that voluntary oversampling for the field test would be allowed at an additional cost to plans.

Response: CMS appreciates these suggestions. As indicated in Supporting Statement B, page 9, the sample size for the field test is adequate for its intended purpose, including some analyses of the subgroups mentioned. CMS will bear all costs of the field test.

• Four commenters recommended CMS select contracts with a high proportion of diverse members for field testing (e.g., dual-eligible, non-White, non-English speaking, low literacy). One suggested selecting contracts that predominately serve frail elderly, those with high behavioral health needs, and those with physical disabilities. Another asked that the field test sample be pulled from the unused HOS and HOS-M sample frames. Another said that beneficiaries in the field test should be individuals who were not surveyed the previous year and they should be excluded from next year's HOS. Another said 50% of contracts be SNPs (e.g., FIDE-SNPs, HIDE-SNPs, I-SNPs, MMPs).

Response: The HOS field test sample will be representative of participating contracts. After determining a timeline, CMS will identify contracts available to participate. The purpose of the field test is to provide information on a diverse set of MA enrollees representing all those eligible to participate in the HOS. Because the changes proposed in the HOS field test are not suggested for application to the HOS-M, it is not appropriate to include the unused HOS-M sample frame. While CMS shares concern for respondent burden, restricting future HOS samples based on field test participation may have an adverse impact on the number of contract enrollees available for future sampling and could make obtaining reliable estimates infeasible for smaller MAOs. The field test sample will be representative of HOS participants but will not include I-SNP enrollees who have been excluded from the HOS since 2020.

• Four commenters noted blinded longitudinal data do not offer actionable information to plans because they are unable to retrieve specific respondents’ answers. Three asked that baseline data be returned to plans earlier for use in quality improvement.

Response: HOS is not designed to report individual outcomes or patient-level results, but rather to measure plan performance in addressing enrollees’ health needs. The purpose of blind data is to ensure data integrity to support objective, comparable assessment of plan performance. Comprehensive quality improvement approaches go beyond using HOS data to address concerns in specific enrollees, and instead use the information to devise approaches that improve health outcomes for all members. Plans are encouraged to use their aggregated Baseline results to identify contract-level priorities and their two-year Follow-Up results to track progress and improvement. Clinical data, including HRAs, are better used to screen for and address patient-level needs as part of an ongoing quality improvement process.

• Two commenters urged CMS to fully report all results from qualitative and quantitative examination of the HOS, including this field testing.

Response: CMS appreciates the feedback and will share relevant results when available.

• One commenter suggested CMS consider focus groups or key informant interviews for stakeholder input on additional revisions. Another commenter suggested stakeholder input as a means of understanding comprehension of languages other than English.

Response: CMS gathered stakeholder input on HOS enhancements through the TEP in September 2022, and this input is reflected in the field test instrument. We will test English and Spanish in their naturally occurring proportions. All future translations of the final instrument will undergo rigorous testing to ensure comprehension by intended audiences.

• Three commenters suggested CMS recalculate the burden estimate in the PRA Notice for this field test to increase completion time and costs. They suggested the HOS completion time is underestimated at 15 minutes and that a closer estimate would be 60-75 minutes. Two commenters asked CMS to add costs borne by others such as training of survey vendors.

Response: CMS thanks the commenters for bringing this issue to our attention. While the current request is to field test a briefer questionnaire with the goal of developing a shorter HOS instrument, the burden estimates in the initial package were not updated when additional questions were added to the field test instrument. As noted in the draft questionnaire, the estimated completion time for the HOS field test instrument is approximately 20 minutes. We have recalculated the burden estimates in Form 83 Part 11 and Supporting Statement A accordingly. HOS survey vendor training costs are always borne by CMS and were already included in the Costs to Government in Supporting Statement A.

• Two commenters suggested CMS test the whole instrument, not just specific questions.

Response: CMS notes the purpose of the field test is to test questionnaire Version A and Version B together to develop the most effective updated HOS instrument. Field test data will be compared to data collected using the current HOS 3.0 instrument during the national administration of the HOS. All versions (A, B, and 3.0) will be tested in their entirety.

• One commenter asked CMS to clarify if HOS will use a web protocol similar to CAHPS, and if CMS plans to compare web-only response rates to combined paper and telephone response rates per study group, or the total response rates per study group. Another asked if plans will be able to view the web templates (digital and non-digital) before they are distributed.

Response: CMS will build on best practices and lessons learns from recent web mode implementation, including CAHPS. Supporting Statement B, page 7 states that analysis of the web-based survey mode will include the review of response rates by mode. Once available, CMS will make web-mode templates available on the HOS website.

• One commenter asked when the field test is planned to launch.

Response: Ideally, the field test will launch in 2024, close to or concurrently with the annual administration of the HOS, since field test data will be compared to data collected during the national administration of HOS.

1 YouGov. (2021, June 28). Are QR codes leaving older Americans behind? https://today.yougov.com/topics/technology/articles-reports/2021/06/28/qr-codes-leaving-older-americans-behind

Page 4 of 4