60 Day Public Comment Response

CMS Response to Public Comments Received for CMS-10316 (OMB Control Number 0938-1113)

The Centers for Medicare & Medicaid Services (CMS) received comments from three organizations related to CMS-10316. CMS thanks the commenters for their feedback. This is the reconciliation of the comments.

Comment:

A health insurer commented about CMS’ proposal to remove a survey question (Did you ever need written information from the plan in Spanish? Yes/No) due to low endorsement and asked for further information including the percentage of beneficiaries responding Y/N.

Response:

This item appears only on the Spanish language survey, which is distributed almost exclusively in Puerto Rico where plans distribute written information to beneficiaries in Spanish. For the 2018 calendar year disenrollment survey, out of 509 Spanish-language respondents, only 64 (13%) responded “yes” to this question, 409 (80%) responded “no” and 36 (7%) did not answer the question.

Comment:

CMS received a comment from a health insurer to improve the clarity and utility of language for a proposed new survey item:

CMS proposed item: Did you leave your former plan because a change in your health meant the plan no longer met your needs? Yes/No.

Suggested Revision: Did you leave your former plan because another plan better met your needs after a change in your health? Yes/No

Response:

This item is designed to understand whether a beneficiary disenrolled because their health status changed and their former plan did not meet their health needs. CMS’ proposed phrasing leads with and focuses on whether the beneficiary changed plans because of a change in their health. In contrast, the commenter’s suggested revision emphasizes a new plan the beneficiary may have switched into, not the former plan. And, it places the change in health as the secondary part of the phrase. In addition, the proposed revision to the question would not allow for beneficiaries who disenroll from a health plan into Original Medicare (FFS coverage). CMS will stay with the currently proposed new item wording.

Comment:

CMS received a comment from a health insurer that a new item CMS proposes to add (Did you leave your former plan because it turned out to be more expensive than you expected?) may duplicate existing survey questions, and CMS should instead revise an existing, cost-related question (Did you leave your former plan because you found a health plan that costs less?)

Response:

The new question (Did you leave your former plan because it turned out to be more expensive than you expected?) is different than other existing survey questions on costs. The newly proposed question focuses on a beneficiary’s expectations and understanding about costs when they initially enrolled in a health or drug plan. The item provides information about how beneficiaries select a health or drug plan based on a set of cost assumptions at enrollment that may not match once they start using the plan. CMS will continue to include this question.

Comment:

CMS received a recommendation from a health insurer to shorten the survey to reduce “survey burnout.” First, they suggest CMS delete two sections about general beneficiary experience with the plan (Getting information or help from your former health plan; Getting health care and the prescription medicines you needed from your former health plan) noting items don’t relate to why the member left the plan, are duplicative of other surveys, or are leading. Also, they suggest CMS streamline items in the Reason you left your former health plan section with a “check all that apply” type question and add an open-ended item.

Response:

The beneficiary experience with the plan measures allow CMS to compare enrollees with disenrollees. There is an overlap between a subset of items on beneficiary experience items on this disenrollment survey and on the MA & PDP CAHPS enrollee survey. CMS analyses have found that disenrollees reported worse care experiences compared to the experiences reported by enrollees on the MA & PDP CAHPS survey. Further, there was a significantly lower probability of disenrollment associated with higher ratings of care experiences. These beneficiary experience items help give a greater understanding of why members leave the plan. Improving care experiences can help plans with beneficiary retention, and health plans receive these item-level frequencies for beneficiary experience items in an annual Excel spreadsheet.

Moving forward, CMS will drop the following 6 patient experience items that no longer are included on the MA and PDP CAHPS survey:

• “Did you ever try to get information from your former plan about which prescription medicines were covered?” (MA-PD Q5; PDP Q5)

• “How often did your former plan give you all the information you needed about which prescription medicines were covered?” (MA-PD Q6; PDP Q6)

• “Did you ever try to get information from your former plan about how much you would have to pay for a prescription medicine?” (MA-PD Q7; PDP Q7)

• “How often did your former plan give you information about how much you would have to pay for a prescription medicine?” (MA-PD Q8; PDP Q8)

• “Did you ever try to get any kind of care, tests, or treatment through your former plan?” (MA-PD Q9; MA-Only Q5)

• “Did a doctor ever prescribe a medicine for you that your former plan did not cover?” (MA-PD Q11; PDP Q9)

And, CMS will add two items included on the MA CAHPS survey to the MA-PD and MA-Only versions of the disenrollment survey to align with the MA CAHPS patient experience survey items:

• “In the last 6 months, did you make an appointment to see a specialist?”

• “In the last 6 months, how often did you get an appointment to see a specialist as soon as you needed?”

Items in these sections may help respondents feel comfortable and engage with the survey before answering questions about why they disenrolled from their health or drug plan.

CMS will also consider additional reporting of these items in future annual plan reports to allow plans to compare the care experiences of disenrollees and enrollees.

The suggestion to aggregate individual yes/no questions into a “single check all that apply” question was a problematic format. Respondents get confused when completing the question, frequently skipping items and leading to incomplete data.

Regarding adding an open-ended question, this has not proven to add new information and adds burden to the respondent and to the government to manually recode.

Comment:

CMS received a request from a health insurer to move disenrollment reasons sections immediately after the survey screener items.

Response:

The beneficiary experience items provide a gentle introduction into the survey that help respondents feel comfortable before answering questions about why they disenrolled from their health or drug plan.

Comment:

CMS received a request from a health insurer to provide to the plan the member level data of survey responses.

Response:

CMS does not share de-identified member level data with plans. CMS explains to beneficiaries in the survey cover letter that their responses will be kept private. The CMS Data Use Agreement for MA & PDP Disenrollment Survey prohibits the CMS contractor from sharing beneficiary-level data. These data could be used to identify an individual (either directly or indirectly), which would violate the guarantee of confidentiality that CMS provides all survey respondents. Confidentiality is necessary to ensure adequate response rates and accurate responses from beneficiaries. Confidentiality serves the purpose of obtaining accurate information from beneficiaries without fear of being identified when responding to questions where they may express dissatisfaction.

Comment:

CMS received a request from a health plan to increase the sample sizes instead of pooling two years to increase number of responses and make data usable in light of possible plan and disenrollment reasons year-to-year changes.

Response:

Disenrollment reasons change meaningfully year-to-year. All disenrollment composites are calculated annually, and most composites (all but two) can be measured using only the most recent year’s data. While larger sample sizes may allow all composites to be estimated using only current year data for larger contracts with high disenrollment rates, present resources don’t allow for such an increase. Current practices are valid and reliable.

In response to the commenter’s concern about reasons varying year-to-year, CMS adjusts the prior year’s score to account for the change in the national averages on the composite measure between years that are pooled. The adjustment is calculated by subtracting the prior year’s national average score from the current year’s national average score. This adjustment is then added to the prior year’s score. This adjusted prior year’s score is then averaged with the current year score to produce the final two-year composite score that is reported. National average one-year scores are calculated separately for MA-PD and PDP plans.

Comment:

CMS received a request from a health insurer to not use case-mix adjustment in calculating disenrollment reason scores. Alternatively, they propose CMS provide industry scores with and without case-mix adjustment.

Response:

Case-mix adjusted scores are the official scores because case-mix adjustment improves the validity of the measures. Adjustment of scores ensures that contract scores are not influenced by patient-level factors beyond the contract’s control. Unadjusted scores are not valid for comparisons between plans because the plan membership characteristics vary across plans. Those characteristics impact response tendencies; therefore, the scores need to be adjusted by those characteristics.

When the adjusted scores are different, that indicates a real difference in performance. Unadjusted scores could be different due to differences in the beneficiaries enrolled in each plan or differences in the plans. CMS provides plans with unadjusted response frequencies for informational purposes only, which are presented in the Excel file that accompanies the plan reports. Unadjusted scores are only provided for informational purposes and are not recommended for comparative quality differences.

Comment:

CMS received a request from a health insurer to provide separate results by whether the member stayed within the same parent organization or chose to go to a competitor.

Response:

CMS will not provide results stratified by whether a survey respondent switched contracts within the same or different parent organization. CMS is concerned with the small numbers of respondents available for analysis, making it very difficult to generate reliable estimates of disenrollment reasons to compare between the two strata.

Comment:

CMS received comments from a health insurer who recommended that in order to maximize response rates, CMS consider adding other modes of data collection for the survey such as a “digital survey” and email or text reminders in addition to phone follow-up.

Response:

CMS will explore the feasibility of implementing these changes in future years. In previous years, CMS has investigated the feasibility of using other survey administration options that rely on email and digital versions of the survey; however, it was determined that neither CMS nor the health plans had complete, reliable lists of email addresses that could be readily leveraged to use these other options and many beneficiaries would have challenges completing on-line surveys given mixed levels of internet and computer capabilities. CMS’s survey vendor conducted interviews with several plans, who indicated they did not maintain email contact information for enrollees, and that that information resided at the level of the physician practice. Despite these challenges, CMS is exploring adding the web mode of data collection to the various modes available to respondents across multiple surveys and will consider adding it for this survey in the future.

Comment:

CMS received comments from a health insurer suggesting survey items may be misleading (MAPD #24) unclear (MA #11, MAPD #22, MAPD#23, MAPD #26); or duplicative (MAPD #11 and #29).

Commenter’s Concern
Item Misleading	MAPD Question 24: Current version: Did you leave your former plan because you found a health plan that costs less? Yes/No Response: CMS does not agree this item is misleading and will retain original wording.
Item Unclear	MAPD Question 22/MA Question11: Current version: Some people have to pay their health plan a monthly fee (called a premium) out of their own pocket for health coverage. Plan suggested revision: Some people have to pay their health plan a monthly fee (called a premium, not to be confused with your Medicare Part B Premium which is paid to CMS) out of their own pocket for health coverage. Response: CMS will retain original wording that is clearer without referencing Medicare Part B Premium.
	MAPD Question 23: Current version: Health plans have a list of the prescription medicines they will cover. Did you leave your former plan because they changed the list of prescription medicines they cover? Yes/No Plan suggested revision: Health plans have a list of the prescription medicines they will cover (also known as a Formulary). Did you leave your former plan because they changed the list of prescription medicines they cover? Yes/No Response: CMS will retain current wording but consider adding a parenthetical with “formulary” in the future.
	MAPD Question 26: Current version: Did you leave your former plan because the plan refused to pay for a medicine your doctor prescribed? Yes/No Plan suggested revision: Did you leave your former plan because a doctor prescribed a medicine that your plan did not cover as it was not on their formulary and/or required prior authorization to be approved? Response: CMS will keep original wording because it is simpler for beneficiaries to understand.
Items Duplicative	Question 11: Current version: Did a doctor ever prescribe a medicine for you that your former plan did not cover? Yes/No Question 29: Current version: Did you leave your former plan because you were frustrated by the plan’s approval process for medicines your doctor prescribed? Yes/No Response: The two items are distinct and CMS will retain both items on the survey. Question 11 asks about medicines the plan did not cover at all, while Question 29 asks about the approval process for medicines a physician prescribed. Question 29 allows for instances where the former plan may have covered a prescription medicine (e.g., a brand name drug), but the approval process was so difficult that the beneficiary chose to leave the plan. CMS examined the correlation between the two items using a simple Pearson correlation statistic. If the correlation were high between the two items (e.g., 0.60 or higher), that would indicate strong overlap, suggesting the items are measuring the same underlying concept. The simple Pearson correlation among MA respondents is 0.34 (p<0.0001), while the correlation among PDP respondents is 0.33 (p<0.0001). The moderate correlation between the items indicates that the items are different enough in what they are measuring. CMS will keep both items on the survey.