60-day Response to Comment Document

CMS Response to Public Comments Received for CMS-10793

CMS received one comment related to CMS-10793 for the Medicare Advantage and Prescription Drug Plan Consumer Assessment of Healthcare Provider and Systems (CAHPS) Survey Field Test.

• The commenter stated that the survey documents make a number of significant changes to the current Medicare CAHPS survey. While the commenter supports many of these changes, they stated it is critical for CMS to consider the material effects these changes will have on responses – both the rate of responses and the actual responses. In addition, they state CMS should examine and solicit stakeholder input on how data from the survey might be used for Medicare Advantage (MA) and Prescription Drug Program (PDP) Star Ratings. The commenter also asks CMS to share the results of the field tests which will allow better review and input on the potential impact of survey changes.

Response: CMS thanks the commenter for their thoughtful comments on the MA & PDP CAHPS Survey measures. CMS plans to conduct thorough analyses of field test results and will carefully consider the utility of any questions added to MA & PDP CAHPS Survey. Additionally, CMS will solicit comment via the Advance Notice of any changes to the MA & PDP CAHPS Survey resulting from the field test and would propose any changes that affect Star Ratings through the regulatory process. Finally, CMS will share information on the field test results via the MA & PDP CAHPS Survey website.

• The commenter states they support the use of a web-based survey format to supplement the ability of individuals to complete the survey by mail or phone. The commenter recommends that CMS provide additional detail and guidance on how the web-based survey field test will be conducted including protections for the privacy and data security of protected health information included in the responses. In addition, they ask whether MA and PDP sponsors will be expected to provide member e-mail addresses to the survey vendor.

Response: CMS thanks the commenter for noting the importance of privacy and data security. The field test survey will be administered by a CMS contractor with extensive experience protecting the privacy of survey respondents while conducting surveys via web and other modes. Survey data collection procedures and data storage will protect confidentiality, privacy, and security as required by CMS and all Federal laws including provisions of the Health Insurance Portability and Accountability Act (HIPAA). Plans with beneficiaries sampled for the field test will be asked to provide e-mail addresses for the purposes of the field test, but will not be required to provide this information. CMS has not yet determined if web-based survey administration will be added to the MA & PDP CAHPS national implementation procedures, and whether plans would be required to provide e-mail addresses to their survey vendor if this mode is added.

• The commenter agrees with the addition of questions related to use of phone and video telehealth visits but states that longer surveys reduce response rates. As a result, they recommend that CMS select a subset of the questions to be included in the survey. For example, this section of the survey could be streamlined by removing Question 9 (In the last 6 months, how often did you get the instructions you needed to use phone or video for your visits?) as that topic is covered by Question 10 on the ease of use of the telehealth visit. In addition, Questions 12, 13, and 14 are indirectly covered by Questions 9 and 10 and should be removed.

Response: CMS thanks the commenter for these suggestions, and for noting the importance of a survey that is as brief as possible. CMS anticipates that not all of the questions included in the field test will be added to the MA & PDP CAHPS Survey. Statistical analyses of the field test results will inform CMS decision-making about adding any questions to the nationally administered survey. Additionally, CMS will solicit comment in advance of any changes to the MA & PDP CAHPS Survey.

• The commenter supports adding questions regarding blood tests, x-rays, and other tests. However, they recommend modifying Questions 20 and 21 since explanation of results may not always be needed directly from a doctor, nurse or other health care provider. In some cases, tests are explained electronically or by mail that meet the individual’s needs. They recommend an alternate question to replace Questions 20 and 21 – In the last 6 months, how often was it easy to understand your test result? In addition, they suggest removing Question 22 (In the last 6 months, how often did you get as much information as you needed about your test results?) as it appears to be duplicative of Questions 21 and 22.

Response: CMS appreciates this feedback about the proposed test result questions, and will carefully consider these comments. As indicated in the response above, CMS anticipates that not all of the questions included in the field test survey will be added to the MA & PDP CAHPS Survey. Statistical analyses of the field test results will inform CMS decision-making about adding any questions to the nationally administered survey.

• The commenter agrees with the additional questions concerning the individual’s personal physician and whether the doctor dismissed symptoms that were important (Question 31). However, they recommend removing or significantly rewriting the question regarding getting help managing care among different providers (Question 36). The question response has very low reliability and is often not reportable.

Response: Thank you for this feedback. Question 36 is an existing question that is currently included in the Care Coordination composite in the nationally administered MA & PDP CAHPS Survey. The Care Coordination composite as a whole has strong evidence of reliability and validity (Hays, et al., 2014) and has been found to be strongly predictive of better performance on HEDIS measures (Elliott, et al., 2021). The mean and median contract-level reliabilities for this composite measure exceed the conventional threshold of 0.7; it has very low reliability for fewer than 15% of contracts. The individual questions within the composite measure contribute positively to the measurement of contract performance at the measure (composite) level. CMS includes the reliability of the individual items within each composite in the annual reports to plans for informational purposes and for consideration in quality improvement activities. The reliability of items within composites is generally lower than the reliability of the composite as a whole, especially for items that are not applicable to all beneficiaries and thus have smaller sample sizes, and should not be interpreted as indicating that the composite itself is not reliable.

References

Elliott, M.N., Adams, J.L., Klein, D.J., Haviland, A.M., Beckett, M.K., Hays, R.D., Gaillot, S., Edwards, C.A., Dembosky, J.W. and Schneider, E.C., 2021. Patient-Reported Care Coordination is Associated with Better Performance on Clinical Care Measures. Journal of General Internal Medicine, 36(12), 3665-3671.

Hays, R. D., Martino, S., Brown, J. A., Cui, M., Cleary, P., Gaillot, S., & Elliott, M. (2014). Evaluation of a Care Coordination Measure for the Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Medicare Survey. Medical Care Research and Review, 71(2), 192-202.

• The commenter agrees that supporting individuals with Limited English Proficiency is a critical need within the health care system. The commenter recommends replacing the question on the individual’s primary spoken language (Question 50) with a question asking whether they needed language support or access – During the past 6 months did you need help getting access to services from a health care provider in a language other than English?

Response: CMS thanks the commenter for this suggestion. Question 50 is currently included in several CAHPS surveys and other federal surveys. The information captured by this question can provide CMS with information about beneficiaries receiving care from plans that may inform multiple aspects of CAHPS administration, including the development of additional survey translations.

• The commenter supports efforts to address discrimination and reduce health inequities. However, Question 55 about unequal treatment is overly complicated and could be simplified by asking the following – During the past 6 months did any health care provider treat you in an unfair or insensitive way due to your race, gender, religion, age or disability?

Response: CMS thanks the commenter for this feedback. CMS strives to collect information via the MA & PDP CAHPS Survey that measures aspects of care that are important to beneficiaries and also provide plans with actionable information for improving patient experience of care. CMS believes a general question about unfair or insensitive treatment will not meet this goal. However, CMS anticipates that not all of the questions included in the field test survey will be added to the MA & PDP CAHPS Survey.