CR_OMB 60-dayCommentSummary 100520

CR_OMB 60-dayCommentSummary 100520.pdf

Parts C and D Complaints Resolution Performance Measures

CR_OMB 60-dayCommentSummary 100520

OMB: 0938-1107

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0938-1107 can be found here:

Document [pdf]

Download: pdf | pdf

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
1

Organization
Aetna Inc.

Document
6401 OMB
Supporting
Statement A
100202

Item
Q9

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Question # 9, "Based on your recent experience with this plan, are you
planning to stay with this plan when you have the opportunity to switch
plans?"

N/A

CMS
ACTION
Accept

Statement A does not address the performance measure weight, if any.

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Add text to
Clarify text in the Supporting Statement A
regarding each question, its purpose, and Supporting
the indicator associated with it. In addition Statement
this question has been reworded to a more
positive tone.

Partial
Accept

Question #8, "Why are you dissatisfied with the way your complaint was
N/A
handled?" Check box #3, "Plan staff did not explain things in a way I could
understand"

Edit Q7
CMS has decided to add clarifying text to
this question asking the beneficiary to
disregard whether or not he/she is satisfied
with the final outcome itself and focus on
different aspects of the complaint process
and the overall handling of the complaint
(New questions Q2 and Q6).

Medicare is a complex product and in many situations the member may
not fully understand their plan benefit or may not agree with the resolution
and explanation. Therefore, members may not answer this question in the
context that was intended.
4

Need to:

N/A

Question #8, "Why are you dissatisfied with the way your complaint was
handled?" Check box #1, "It took too long for the plan to process my
complaint"
Concern how the retroactive enrollment/disenrollment process time
guidelines will be taken into account, if applicable, when tabulating this
response.

REASON FOR ACTION

Beneficiary Satisfaction with Complaint Handling Process:
Question #7, "Now, please indicate how satisfied you are with the way
your complaint was handled by the plan."

N/A

Although the purpose is to distinguish itself from Question #3, if a member
is dissatisfied with a correct resolution, there is a high probability that the
member will be dissatisfied with the handling as well.
5

Aetna Inc.

6401 OMB
Advance Letter
100202

General

In situations where the member has an appointed representative, how will Please clarify how address information is being identified Accept
this person be contacted for this survey.
for situations where the member appointed an authorized
representative as well as complaints filed by other parties,
such as SHIP counselors, providers, facility attendants,
etc. Will this type of complaint be omitted from the
sample selection?

Add text to
CMS considers an appointed
supporting
representative to be a valid respondent.
statement
Further clarification will be added to the
supporting statement regarding the
participation of appointed representatives in
the survey. This explanation will include
how representatives will be contacted
(through beneficiaries and/or CTM logs)
and how representative data can be used in
the survey data analysis.

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Repeat Complaints:
N/A
Question #5 "Did you have to make more than one attempt to resolve your
complaint before the plan contacted you?"

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Partial
Accept

Remove Q5
from survey
instrument

Statement indicates this question will be asked if member states they were
contacted by the plan- it is critical that members are appropriately
educated on the timeframes that have been established by CMS for plans
to respond to CTM complaints at the time they are filing their compliant
with 1-800 Medicare, otherwise the member may file their complaint
multiple times.

INFORMATION NOT RELEASABLE TO THE PUBLIC UNLESS AUTHORIZED BY LAW:
This information has not been publicly disclosed and may be privileged and confidential. It is for internal
government use only and must not be disseminated, distributed, or copied to persons not authorized to receive the
information. Unauthorized disclosure may result in prosecution to the full extent of the law.

1 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
7

Organization
Aetna Inc.

Document
6401 OMB
Survey
Instrument
100202

Item
Q9

N/A

CMS
ACTION
Accept

This question has a negative connotation to it and the member could
interpret it as a suggestion to change plans, when in all actuality their
complaint resolution may have no bearing on a future decision to change
their insurance carrier at the first opportunity.
8

Aetna Inc.

6401 OMB
Supporting
Statement A
100202

6401 OMB
Survey
Instrument
100202

Background

General

REASON FOR ACTION

Need to:

Edit Q9
CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan based on recent experience, which
provides a more comprehensive insight on
the beneficiaries perspective.

Partial
Last paragraph indicates survey sampling will be based on CTM closed in Recommend sampling period be a broader timeframe,
first qtr 2011.
keeping in mind that the first quarter of CY is a busier time Accept
period for MAO call volume and complaints due to annual
benefit changes taking effect and members who did not
read their plan materials may experience a problem and
require education on their plan benefits/coverage.

No action
CMS is interested in the months with the
needed
largest number of complaints in order to
achieve the most statistically valid sample.

Statement indicates final results will not be shared until 8/2011.

Please clarify if this is only being shared with CMS or will Partial
plans also receive a copy? Plans should be provided with Accept
dissatisfied response data as quickly as results are
tabulated, prior to July 2011 in order to allow sufficient
time for plans to develop and implement appropriate
action plans/training to address the low performance
areas identified prior to going into the next AEP, which
should be the main goal of this survey data.

CMS will follow a similar process for this
measure as it does for other measures. It
is undetermined what level of data will be
shared with plans.

Criteria selected for "major action" and "should be known Partial
by the complainant" should be developed and shared with Accept
plans in advance of finalizing and implementing the
survey.

CMS will review beneficiary responses with Add text to
HPMS CTM records for the veracity of the Supporting
complaint resolution. Additional information Statement
on the use of this data will be included in
the justification statement.

Veracity of Plan's Description of Resolution: Question #2 "What was the
resolution?"
The concern regarding this open forum question is that it will lead to the
beneficiaries' perception and potential lack of understanding of their plan
benefit because of the time period selected for review.

The sampling strategy will exclude
complaints that are outside of the scope of
the plan; particularly some complaints
associated with enrollment issues.

No action
needed

This data collection is a preliminary effort to
develop a satisfaction measure. CMS will
review the feasibility to develop a
performance measure in coming years.

CMS is well aware of other factors affecting
complaint outcomes and issues outside the
control of the plan. Approaches to control
for these issues need do be described in
more detail in the supporting statement.
CMS will assess the validity of complaints
against plans and screen complaints that
are included in analysis.

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Repeat Complaints:
Recommend asking "Did you contact the plan to attempt
Question #5 "Did you have to make more than one attempt to resolve your to resolve this complaint prior to contacting 800complaint before the plan contacted you?"
Medicare?" This question is important in order to
determine if the member gave the plan the opportunity to
Concern that this question could be interpreted and answered two different resolve their complaint prior to contacting 800-Medicare.
ways. Some beneficiaries may include calls made to plan prior to calling
800-Medicare and others may not have contacted the plan prior to filing a
complaint with 800-Medicare.

Partial
Accept

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Remove Q5
from survey
instrument

2 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
12

Organization
Aetna Inc.

Aetna Inc.

Document

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Item

6401 OMB
Survey
Instrument
100202

Beneficiary Satisfaction w/time until a resolution: Question #6 "How
satisfied are you w/the amount of time it took to resolve your complaint?"

CMS
ACTION
Partial
Recommend adding a question that asks the member if
the timeframes the plan has for resolving complaints was Accept
explained to them when they filed their initial complaint.

This measure is only appropriate, if the timeframes that CMS has
established for plans to resolve a complaint are clearly explained by the
staff at 800 Medicare when the member is filing their initial complaint so
that the appropriate expectations are established and 800-Medicare staff
are appropriately capturing and documenting all of the member's
complaint that requires resolution by the plan.
Beneficiary Satisfaction w/time until a resolution: Question #6 "How
satisfied are you w/the amount of time it took to resolve your complaint?"

Please clarify if actual CTM specified resolution
timeframes will be factored into the final survey results.
For example, beneficiary unhappy with time it took to
Agree w/CMS limiting sample selection to "Immediate Need" and "Urgent". receive resolution, however, plan resolved in less than
CMS required resolution timeframe. This response
should not reflect negatively in plan performance rating.

Partial
Accept

REASON FOR ACTION

Need to:

Remove Q6
Question 6 has been removed from the
survey. Some issues related to the amount from survey
of time it took to resolve a complaint have instrument
been incorporated in Q2 of the new survey
instrument.
CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Beneficiary Awareness of Resolution: Question #1 "According to our
records, the complaint you filed was recently closed by the plan. Was the
complaint resolved?"

N/A

Partial
Accept

Concern regarding this question is the beneficiary may not consider the
complaint resolved even though the resolution was correct based on their
benefit structure and/or CMS regulations/guidelines.

The supporting statement will clarify the
difference between the selected terms.
"Resolved" will be replaced with "settled" in
this question to prevent beneficiary bias. An
"I don't know" answer choice has been
added for beneficiaries who feel they do not
yet have a resolution or are unsure/do not
remember.

Add text to
Supporting
Statement
Edit Q1

Analyses will incorporate information about
resolutions in which plans are constrained
by CMS guidelines.

Aetna Inc.

6401 OMB
Survey
Instrument
100202

Q10

Beneficiary Consequences during Complaint Resolution Process:
Question #10 "During the complaint process, what problems did you
experience while you were waiting for your complaint to be resolved?".
General Comment

N/A

Partial
Accept

This question gathers data regarding the
Edit Q10
experiences beneficiaries may have during
the complaint resolution process. The
survey questions have also been revised to
reflect more neutral wording.

Please clarify that the definition of small businesses is
referring to the exclusion of 800 series members.

Reject

This survey gathers data regarding the
Add text to
experiences beneficiaries may have during Supporting
the complaint resolution process.
Statement A

Concern with the response available for member selection given that
some will be based on member perception and lack of understanding of
their plan benefit structure and/or applicable CMS guidelines.
16

Aetna Inc.

6401 OMB
Supporting
Statement A
100202

General

Small Businesses

We will clarify that contracts with only 800
series members will not be included.

3 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
17

Organization
Aetna Inc.

Document
6401 OMB
Survey
Instrument
100202

Item
Q10

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Beneficiary Consequences during Complaint Resolution Process:
Question #10 "During the complaint process, what problems did you
experience while you were waiting for your complaint to be resolved?".
Check box #2 "I did not receive my medications"

N/A

CMS
ACTION
Partial
Accept

Aetna Inc.

6401 OMB
Advance Letter
100202

General

Opening of letter repeatedly mentions Medicare program, however the
complaint is really associated to the plan contracted with Medicare.

Aetna Inc.

6401 OMB
Supporting
Statement A
100202

Background

Last paragraph indicates proposed surveys will occur within 21 calendars N/A
days of the complaint closure and will collect beneficiaries' opinions on the
complaint resolution process and their satisfaction with the resolutions.
Specific to all Immediate/Urgent need cases, plan attempts to contact the
complainant by telephone to provide resolution. If a minimum of 3
telephone contact attempts are unsuccessful, the plan mails a letter of
resolution to the address on file. Concern is that the proposed survey
could take place prior to receipt of a mailed resolution letter.

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

“How satisfied are you with the resolution of your complaint?” – Survey
Question #3 (Supporting Statement – Part A, pages 9-10).
CMS states that “to relay the extent to which complainants are satisfied
with the resolutions that plans have provided to their complaints” the
survey will ask (Survey Question 3), “How satisfied are you with the
resolution of your complaint?”
+The four options to respond to this question include “very satisfied,
satisfied, dissatisfied, and very dissatisfied,” but do not permit a neutral
response. We are concerned that the absence of a neutral option could
increase the potential for a negative response. For example, the
complaint may have been resolved according to CMS rules, and the
beneficiary may understand that this was the case but be reluctant to
indicate satisfaction with a resolution that was not the requested outcome.

Need to:

Timeframes will not be taken into account. No action
needed
This question gathers data regarding the
experiences beneficiaries may have during
the complaint resolution process.
CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

Please clarify how the Part D exceptions process timeframes will be taken
into account, if applicable, when tabulating this response.

REASON FOR ACTION

This should be more clearly explained to the member in
the letter to avoid confusion.

To reduce confusion about the topic and
Add text to
source of the complaints, the advance letter Advance
will make explicit reference of the MAO or Letter
Part D sponsor.

We understand the concern about allowing
sufficient time for beneficiaries to be notified
of the resolution. Due to the short
timeframe for completion of the surveys, we
can accommodate 7 days between
complaint closure and initial contact with
the beneficiary to allow time for
beneficiaries to receive notification of their
complaint resolution. This method will be
used for the third quarter pilot test.

Consistent with our comments above, we recommend
Partial
that an appropriate nationally recognized quality
Accept
measurement organization review the potential responses
and recommend an approach that is likely to provide the
most useful responses for performance measurement.
This comment also applies to other questions with
response choices that are similarly structured.

Add text to
Supporting
Statement
Adjust
sampling
plan

This data collection is a preliminary effort to Edit survey
develop a satisfaction measure. CMS will
instrument
review the feasibility to develop a
performance measure in coming years. At
this point a quality measurement
organization is not needed.
A neutral answer choice would be "neither
satisfied nor dissatisfied." However, CMS
decided against including a neutral answer
choice in order to encourage beneficiaries
to select an opinion one way or the other.
Instead, CMS will provide an "I Don't
Know/NA" answer choice for beneficiaries
who do not believe they have received a
final outcome or who do not remember the
resolution of their complaint.

4 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

Document
6401 OMB
Survey
Instrument
100202

6401 OMB
Survey
Instrument
100202

Item
Q4

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
“Did the plan contact you about your complaint? This includes contact by To address this concern, AHIP recommends that CMS
telephone, mail, email, or other means.” – Survey Question #4 (Supporting revise the response options offered to beneficiaries.
Statement – Part A, page 12).
CMS states that this measure will assess the frequency with which plans
contact complainants while handling the complaint to notify them of a
resolution by asking (Survey Question 4), “Did the plan contact you about
your complaint? This includes contact by telephone, mail, email, or other
means.” We understand that it is not uncommon for beneficiaries to forget
or be unaware that they have received a plan contact. However, the
survey instrument only allows beneficiaries to indicate “Yes” or “No” and
does not include the options for “I don’t know” or “I don’t remember” as
possible responses.

“Now, please indicate how satisfied you are with the way your complaint
was handled by the plan.” – Survey Question #7 and “Why are you
dissatisfied with the way your complaint was handled?” – Survey Question
#8 (Supporting Statement – Part A, page 11). CMS indicates that to learn
“the extent to which plans treated the complainant courteously, provided
the complainant with understandable explanations, and provided the
complainant with enough information,” the survey will ask (Survey
Question 7), “Now, please indicate how satisfied you are with the way your
complaint was handled by the plan.” The survey question does not
reference the three topics that CMS seeks to address with this question,
and we believe it is unlikely that beneficiaries will consistently provide the
information needed to address these three areas. In addition, it is not
clear that beneficiaries will distinguish between satisfaction with the
resolution and satisfaction with the handling of the complaint by the plan.

CMS
ACTION
Accept

Need to:

Edit survey
Q4 has been removed from the survey
instrument. Issues of plan communication instrument
with the beneficiary have been incorporated
into Q2 of the new survey instrument. The
new Q2 is a satisfaction question that
encompasses several aspects of the
complaint process such as courtesy of the
plan representative and time until contact
by the plan.
In addition, in good faith CMS added other
response options (Do Not Know, NA) for
beneficiaries to choose when they do not
have a clear response within the 4-likert
scale.

Accept
As an alternative approach, CMS could revise this
question to specifically ask beneficiaries how satisfied
they were with the plan’s courtesy, clarity of explanation,
and provision of sufficient information during the complaint
resolution process.

An alternative approach would be to eliminate Survey
“Now, please indicate how satisfied you are with the way your complaint
Question #7.
was handled by the plan.” – Survey Question #7 and “Why are you
dissatisfied with the way your complaint was handled?” – Survey Question
#8 (Supporting Statement – Part A, page 11). CMS also indicates that if
the beneficiary indicates dissatisfaction with complaint handling, the
surveyor will ask (Survey Question 8), “Why are you dissatisfied with the
way your complaint was handled?” to “provide CMS and plans with a
greater understanding of any low performance measure indicator values.”
The written survey instrument includes four specific choices and “other” for
the beneficiary or surveyor to check to indicate the reasons for
dissatisfaction. The reasons listed duplicate information gathered in
Survey Question #6 regarding the amount of time it took to handle the
complaint and the information that would be gathered in Survey Question
#7, if CMS adopts AHIP’s recommendation to reference the three areas of
courtesy, clarity of explanation, and provision of sufficient information.

REASON FOR ACTION

Edit survey
Clarifying text will be added to Q7 asking
the beneficiary to disregard whether or not instrument
he/she is satisfied with the final outcome
itself. The purpose of this question will be to
provide an overall satisfaction rating.
Q8 has been removed and the new Q2 is a
satisfaction question that encompasses
more aspects of the complaint process
such as coutesy of the plan representative
and time until contact by the plan.

Partial
Accept

Edit survey
Q8 has been removed. The new Q2 is a
instrument
satisfaction question that encompasses
more aspects of the complaint process
such as courtesy of the plan representative
and time until contact by the plan.
Clarifying text will be added to Q7 asking
the beneficiary to disregard whether or not
he/she is satisfied with the final outcome
itself. The purpose of this question will be to
provide an overall satisfaction rating.

5 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

Document
6401 OMB
Survey
Instrument
100202

Item
Q10

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

CMS
ACTION
We recommend that CMS use a uniform approach for the Partial
“During the complaint process, what problems did you experience while
Accept
surveys administered in writing and by telephone and
you were waiting for your complaint to be resolved?” – Survey Question
frame the question in a manner that avoid biasing the
#10 (Supporting Statement – Part A, pages 11-12).
Additionally, beneficiaries who receive a paper survey rather than a survey responses. This could be accomplished by removing the
checklist.
administered by telephone will receive a list of potential problems from
which to select. Their responses may be influenced by the checklist and
potentially biased to select more problem areas. It is not clear how CMS
will address this issue to differentiate and analyze the written and
telephonic responses.
Finally, CMS references a second question that will be used “to provide
more knowledge about the scale of the negative incidents experienced by
complainants.” However, neither the Supporting Statement nor the survey
materials include the question. We recommend CMS clarify whether there
is a second question and if so, how it will be used.

America's Health
Insurance Plans

6401 OMB
Supporting
Statement B
100202

We recommend that CMS provide a description of the
Respondent Respondent Universe and Sampling (pages 3-5).
manner in which the survey design will address these
Universe and Defining the Population:
issues.
CMS states that the survey population “is made up of beneficiaries with
Sampling
closed urgent or immediate need complaints that were filed against their
respective plans during the period covering the months of January and
February of the year 2011.” The Supporting Statement does not discuss
how the survey will be administered in the case of beneficiaries who have
an authorized representative who acts on their behalf or who may
otherwise have physical or cognitive impairments that are a barrier to
responding. The survey proposal also does not address the situation in
which a party other than the beneficiary reported the complaint to 1-800Medicare.

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

America's Health
Insurance Plans

6401 OMB
Supporting
Statement B
100202

Parts C and D Complaint Closure Beneficiary Survey

AHIP recommends that CMS either revise the question to Accept
ensure that it is neutral and explain its contribution to the
• Question 9 (page 2). The description of the performance measures that performance measures or remove it from the survey.
CMS intends to utilize, which appears on pages 8-12 of Supporting
Statement – Part A, contains no discussion of Survey Question #9, which
asks, “Based on your recent experience with this plan, are you planning to
stay with this plan when you have the opportunity to switch plans?” It is
unclear why CMS proposes to ask this question if it is not necessary for
construction of any of the proposed performance measures. Furthermore,
we are concerned that the wording of the question could potentially
influence the response, because it could be understood to imply that the
beneficiary should reconsider plan enrollment.

Respondent Sampling First Quarter Complaints – It is our understanding that a
Universe and significant proportion of calls received by 1-800 Medicare in the first
quarter of each year are related to beneficiaries seeking information and
Sampling
education about the structure or operation of their new plans. To the
extent that these complaints are categorized as immediate need/urgent
through the process discussed above, their inclusion in the survey
initiative is likely to make the results less useful to beneficiaries as an
indicator of plan performance.

We recommend that such complaints be excluded from
the sample or that CMS draw the sample across a
broader time frame than the first quarter of the year to
obtain a more representative cross-section of beneficiary
complaints.

Partial
Accept

REASON FOR ACTION
In review, CMS recognizes that there may
be a misunderstanding regarding the dual
survey formats and more clarifying text will
be added to the supporting statement
describing the uniformity of the telephone
and written survey approaches.

Need to:
Add text to
Supporting
Statement
Edit survey
instrument

In regards to Q10, the checklist is the most
appropriate method of gathering the
specified data, therefore this question's
structure will not be altered.

Add text to
CMS considers an appointed
Supporting
representative to be a valid respondent.
Statement
Further clarification will be added to the
supporting statement regarding the
participation of appointed representatives in
the survey. This explanation will include
how representatives will be contacted
(through beneficiaries and/or CTM logs)
and how representative data can be used in
the survey data analysis.

CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

Add text to
Supporting
Statement
Edit Q9

Further clarification on how this question
will be used in the development of the
performance measure will be provided.

No action
CMS is interested in the months with the
needed
largest number of complaints in order to
achieve the most statistically valid sample.
The sampling strategy will exclude
complaints that are outside of the scope of
the plan; particularly complaints associated
with enrollment issues.

6 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

Document
6401 OMB
Supporting
Statement B
100202

6401 OMB
Survey
Instrument
100202

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

CMS
ACTION
Partial
Accept

Respondent Categorization of Immediate Need/Urgent Complaint:
Universe and
Sampling
CMS’ decision to focus the survey on all immediate need/urgent
complaints raises significant concerns about the use of the survey data for
its intended purpose under the proposed Part C and D Complaint
Resolution Performance Measurement initiative – as an indicator of plan
performance. In contrast to the subset of complaints that involve access
to medical services or prescription drugs, many of the complaints involve
issues whose resolution is not within the sole, or in many cases principal
control, of MA and Part D sponsors. Through the categorization process
immediate need/urgent complaints may involve Social Security
Administration (SSA) premium withhold, Part D excluded drugs, Part D
eligibility date, or enrollment processes and policy. Expansion of MA and
Part D plan sponsor performance measures to assess beneficiary
satisfaction with the resolution of such complaints would not be
reasonable or appropriate, because the measures would be more likely to
reflect beneficiary understanding and satisfaction levels with CMS or SSA
policy, rather than the plan sponsor’s actions.

If CMS is considering evaluation of plan sponsor
performance spanning a sampling of beneficiaries with
closed complaints across the full spectrum of immediate
need and urgent complaints received through 1-800Medicare, we strongly recommend that the agency
reconsider this approach and revise the initiative to
ensure measures reflect actions within the control of MA
and Part D plan sponsors.

General

Reject
To address these concerns, we recommend that CMS
submit the proposed survey instrument and process for
evaluation by an appropriate nationally recognized quality
measurement organization, such as NCQA in consultation
with AHRQ, which develops and maintains the CAHPS
suite of surveys. Following this critical step in the
development process, we recommend that the final tool
should be reviewed for endorsement by a national, multistakeholder consensus entity to ensure that the
information provided to beneficiaries through this initiative
is reliable for use in their decision-making.

Survey content:
As discussed in more detail below, we have serious concerns that
features of the proposed survey could undermine its utility as an indicator
of plan sponsor performance. For example, we believe that the content of
the questions and the related telephone interview process have the
potential to elicit beneficiary responses that are more informative about the
beneficiary’s ability to provide clear and responsive answers than about
plan performance.

REASON FOR ACTION

Need to:

CMS is well aware of other factors affecting Add text to
complaint outcomes and issues outside the Supporting
Statement
control of the plan. Approaches to control
for these issues need do be described in
more detail in the supporting statement.
CMS will assess the validity of complaints
against plans and screen complaints that
are included in analysis.
Other CMS data will be used to control for
plan characteristics and beneficiary profiles.
The supporting statement will clarify
analyses that incorporate information about
resolutions in which plans are constrained
by CMS guidelines.

This data collection is a preliminary effort to No action
needed
develop a satisfaction measure. CMS will
review the feasibility to develop a
performance measure in coming years. At
this point a quality measurement
organization is not needed.

7 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
Organization

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

America's Health
Insurance Plans

6401 OMB
Supporting
Statement B
100202

Respondent Criteria for Inclusion of Beneficiaries/Complaints in the Survey Sample –
Universe and As noted above, resolution of many complaints received through 1-800Medicare is not within the control of MA and Part D plan sponsors, and
Sampling
resolution is in many cases the result of the application of CMS policy.

America's Health
Insurance Plans

Document

6401 OMB
Supporting
Statement A
100202

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Publication
Tabulation
Dates

CMS
ACTION
While this information may have some utility, Question #2 Partial
“What was the resolution?” – Survey Question #2 -- (Supporting
is unlikely to elicit information that is useful to beneficiaries Accept
Statement – Part A, page 10).
as they compare MA and Part D options. An alternative
CMS states that to “illustrate the accuracy of the plans’ descriptions of
approach would be for CMS to revise this question to
their complaint resolutions in the CTM” the survey will ask (Survey
Question 2), “What was the resolution?” CMS indicates that accuracy will focus on the beneficiary’s understanding of the plan’s
be determined through comparison of the complainants’ description of the response to his or her complaint and modify the stated
purpose of the question accordingly. If the list of “major
resolution and any “major action steps” “that should be known by the
action steps” is retained in conjunction with this question,
complainant” with the plan’s descriptions of the resolution. It is our
we also recommend that CMS provide an opportunity for
understanding that plan sponsors commonly record the calls made by
plan sponsor comment on the list to ensure that it
trained plan staff to notify members of the outcome of their complaints.
corresponds to plan sponsor operations.
We believe that the recording would be the most reliable source of
verification of the content of the plan sponsor’s description of their
complaint resolutions. Further, we are concerned that subjective factors
are likely to interfere with an effort to draw conclusions from any
discrepancy between the beneficiary’s response and the plan description
of the resolution. The discrepancy is likely to provide insight into such
areas as the accuracy of the member’s memory since the time of
resolution, the member’s ability to articulate the resolution, the surveyor’s
understanding of the beneficiary’s description, and the relevance of the
“major action step” categories to what the member states and recalls as
significant, rather than the accuracy of the plan sponsor’s description.

Timeframe:
The Supporting Statement indicates that the purpose of the proposed
expansion of Part C and D performance measurement is to add to the
data available to beneficiaries on Medicare Options Compare and the
Medicare Prescription Drug Plan Finder (MPDPF). The Supporting
Statement – Part A explains that CMS plans to publish the new
performance measures in November 2011.
-- We note that CMS’ past practice regarding the introduction of new
performance measures has been to utilize at least the initial year of data
collection and analysis to evaluate the validity and utility of the data for its
intended purpose, effect any needed modifications, and determine how the
information can be most effectively presented to the public. These steps
are critical to ensuring that information is not misleading or confusing, and
we urge CMS to follow this process for the proposed survey initiative.
-- Further, there is insufficient information to understand how CMS
envisions that the data may be incorporated into the existing performance
indicators (i.e., the Star Rating System) or whether it would be utilized
separately in some other manner.

REASON FOR ACTION

Need to:

CMS will review beneficiaries response with Add text to
HPMS CTM records for the veracity of the Supporting
complaint resolution. Additional information Statement
on the use of this data will be included in
the supporting statement.

Add text to
Supporting
Statement

We recommend that CMS establish criteria that exclude Accept
from the survey sample beneficiaries whose complaints
cannot be resolved solely by actions of the plan sponsor.
Examples of such complaints include:
o Complaints that must be resolved through a request for
retroactive disenrollment through the CMS Regional
Office;
o Complaints that do not include clear documentation of
the issue raised by the beneficiary and that the plan
sponsor is unable to clarify because the beneficiary does
not respond to requests for necessary information;
o Complaints expressing dissatisfaction with the decision
of the Independent Review Entity.

CTM categorization issues should have
minimal impact on effective and timely
complaint resolution. Complaint type will
also be taken into consideration and
excluded, if necessary.

Reject
Since CMS has stated the agency’s intent to consider
modifications to the Star Rating System in light of its role
in the MA payment methodology beginning in 2012, it
would be appropriate for CMS to consider addition of new
performance data as part of a comprehensive approach to
evaluating the agency’s performance measures, and we
recommend that these initiatives be coordinated.

No action
Efforts are being made to coordinate
changes to the Star Rating System with the needed
bonus payments. The process will be made
transparent to organizations and plans will
be made aware of CMS's strategy and
methodology.

Sampling strategy will take into
consideration complaints to be addressed
by CMS as well as complaints that are
challenging to identify as plan's
responsibility. Further details will be
included in the supporting statement.

8 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

Document
6401 OMB
Supporting
Statement B
100202

6401 OMB
Survey
Instrument
100202

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

CMS
ACTION
Reject

Respondent Respondent Universe and Sampling (pages 3-5).
Universe and Contracts with Low Enrollment or Minimal Complaints:
In the discussion of sampling, CMS notes that it plans to develop
Sampling
complaint resolution performance measures for “each contract” for a total
of 541 contracts (based on CY2009 data) and states that all MA and Part
D plans “will be surveyed regardless of their enrollment size.” CMS further
states that, “This distribution is expected to vary substantially from one
contract to another, with some small contracts having a total number of
closed complaints as low as one.” For some contracts the total number of
urgent and immediate need complaints and closed complaints is very
small so that the survey data, even if all beneficiaries in the sample
respond, is likely to produce results for the contract that would not be
statistically valid. The potential for this circumstance to arise appears
high, because the entire sample nationally will be 6,500 beneficiaries
drawn from the approximately 541 contracts.

AHIP strongly recommends that CMS ensure that the
project design addresses the issue of validity and
reliability of results based upon small sample size at the
contract level and that results are not publicly reported
that are not statistically valid.

“Did you have to make more than one attempt to resolve your complaint
before the plan contacted you?” – Survey Question #5 (Supporting
Statement – Part A, page 12). CMS states that this measure will
“demonstrate any patterns of delay by plans when contacting
complainants about their complaints” by asking (Survey Question 5), “Did
you have to make more than one attempt to resolve your complaint before
the plan contacted you?” We understand that beneficiaries often are
unaware of the timeframes CMS has established for plans to resolve
complaints and therefore beneficiaries may call multiple times within that
timeframe regarding the same issue. The receipt of such multiple calls
does not necessarily indicate that the plan sponsor is not compliant with
CMS timeframes.

AHIP recommends that CMS factor into the analysis of
responses to this question objective information about
whether the plan complied with CMS requirements and
timeframes for contacting the beneficiary and resolving
the complaint.

“How satisfied are you with the amount of time it took to resolve your
complaint?” – Survey Question #6 (Supporting Statement – Part A, pages
10-11).
CMS states that to determine beneficiary satisfaction with the time
between filing the complaint and the receipt of resolutions the survey will
ask, (Survey Question 6) “How satisfied are you with the amount of time it
took to resolve your complaint?” CMS has established timeframes that
plans must meet when resolving urgent and immediate need complaints.
However, as discussed above, the varied nature of the complaints in this
category raises significant concerns that in many cases, evaluation of
beneficiary satisfaction with the time required to resolve complaints is
likely to be a measure of satisfaction with the timeliness of CMS action to
effectuate resolution rather than with plan performance.

To address this issue, we recommend that CMS exclude
from the survey sample beneficiaries whose complaints
cannot be fully resolved through direct action by the plan
sponsor. We also recommend that CMS exclude or
otherwise make adjustments in the evaluation of survey
results when transmittal of a complaint to a plan sponsor
is delayed due to administrative factors inherent in the
CMS complaint processing system or the plan sponsor’s
ability to take timely action is hindered by other factors
outside of the plan sponsor’s control. Without these
changes, we believe that the information produced
through responses to this question will not provide useful
beneficiary information about plan performance.

REASON FOR ACTION
Statistically representative samples and
enrollment size variables will be taken in
consideration when developing measures.

Need to:
No action
needed

CMS is well aware of small sample size
issues and has decided to proceed with
collecting data for these contracts.

CMS has decided to drop questions of
repeat complaints/multiple attempts to
contact the plan.

Remove Q5
from survey
instrument

CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

Remove Q6
Question 6 has been removed from the
survey. Issues related to the amount of time from survey
instrument
it took to resolve a complaint have been
incorporated in Q2 of the new survey
instrument.

CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

9 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

Document

Item

Explanations for General
Inclusion of
Survey
Questions

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Indicators. This document contains a number of “performance measure
indicators” which, in most cases roughly correspond to the “performance
measures” discussed on pages 8-12 of Supporting Statement – Part A.
However, it is not clear how the indicators will be used and whether the
indicators are intended to be distinct from performance measures.

AHIP recommends that CMS revise this document to
clarify the relationship or interaction between the
“indicators” and the measures.

CMS
ACTION
Accept

REASON FOR ACTION

Need to:

The indicators provide information on a
particular area of interest and they will be
further analyzed and revised to develop the
performance measures.

Edit
Explanations
for Inclusion
of Survey
Questions
Add text to
Supporting
Statement

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

General

N/A
Review and testing of the survey and interview protocol:
Performance measures that utilize survey data typically undergo rigorous
review and testing similar for example, to the process utilized by the
Agency for Healthcare Research and Quality (AHRQ) to develop the
Consumer Assessment of Health Plans Survey (CAHPS). These efforts
are important to ensure that the survey questions and administration and
the analysis of responses yield reliable data. However, it does not appear
that CMS has utilized such an approach in the development of the survey.

Reject

Partial
Accept

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

As noted above, AHIP recommends CMS focus the
survey sample on complaints that the plan sponsor has
the ability to resolve directly, so that the beneficiary’s
expression of satisfaction or dissatisfaction will be
attributable to the plan sponsor’s action and therefore,
reflect plan performance.

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

Q10

“During the complaint process, what problems did you experience while
you were waiting for your complaint to be resolved?” – Survey Question
#10 (Supporting Statement – Part A, pages 11-12).
CMS states that to “assess how well plans are able to resolve complaints
before the beneficiary encounters a subsequent incident” the survey will
ask (Survey Question 10), “During the complaint process, what problems
did you experience while you were waiting for your complaint to be
resolved?” CMS states that this measure will “be constructed as a
percentage of complainants who indicate that they have experienced any
consequences due to their wait for a resolution.” However, the question
implies that the beneficiary experienced problems, and consequently,
AHIP believes that responses could be skewed towards the occurrence of
problems.

An alternative approach would be for CMS to revise this Partial
question to ask whether the beneficiary experienced any Accept
problems that resulted directly from how the plan handled
the complaint. Issues of this type illustrate the importance
of submitting the survey for review by an appropriate
nationally recognized quality measurement organization
as recommended above.

Analyses will incorporate information about Add text to
resolutions in which plans are constrained Supporting
Statement
by CMS guidelines.

The survey questions have been revised to Edit Q10
reflect more neutral wording.
Analyses will incorporate information about
resolutions in which plans are constrained
by CMS guidelines. CMS will assess the
validity of complaints against plans and
screen complaints that are included in
analysis.

Furthermore, the problem(s) a beneficiary may identify as having occurred
during the time they were awaiting resolution of their complaint may be out
of the plan’s control and therefore would not be an indicator of how well
the plan is able to resolve complaints. For example, if a beneficiary
indicates he or she “did not receive my medications,” this could be
because the medications were not covered Part D drugs, or if the
beneficiary “missed an opportunity to change plans,” this may be because
the beneficiary did not have an enrollment period available to change
plans rather than an indicator of the plan’s ability to resolve complaints. It
is not clear how such information would be useful in evaluating plan
performance.

10 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

America's Health
Insurance Plans

6401 OMB
Advance Letter
100202

General

America's Health
Insurance Plans

6401 OMB
Survey
Instrument
100202

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

CMS
ACTION
Plan Identification. The opening paragraph of the Advance Letter explains AHIP recommends that CMS revise the Advance Letter to Accept
explicitly reference the beneficiary’s MA or Part D plan.
that CMS is conducting a survey about beneficiary satisfaction with the
handling of a Medicare complaint, but does not clarify that the survey
concerns a complaint about the beneficiary’s Medicare health plan or
Medicare prescription drug plan. This is likely to cause confusion for
beneficiaries about the topic of the survey.
“According to our records, the complaint you filed was recently closed by
the plan. Was your complaint resolved?” -- Survey Question #1
(Supporting Statement – Part A, page 9).
CMS states that to “capture the percentage of a plan’s complainants who
agree that a resolution to their complaint has been implemented,” the
survey will ask, (Survey Question 1) “According to our records, the
complaint you filed was recently closed by the plan. Was your complaint
resolved?” AHIP is concerned that this question is unlikely to be specific
enough to elicit a beneficiary response that is reliably linked to plan
performance. For example, the question does not include any specific
reference to the nature of the complaint and asking whether the complaint
was “resolved” rather than for example, whether the beneficiary received
or understood the plan’s decision, may not be concrete enough for the
beneficiary to provide an information response.

REASON FOR ACTION

Need to:

Add text to
To reduce confusion about the topic and
source of the complaints, the advance letter Advance
will make explicit reference of the MAO or Letter
Part D sponsor.

For these reasons, we recommend that this survey
Partial
question be revised, and consistent with our
Accept
recommendations above, be reviewed by an appropriate
nationally recognized quality measurement organization to
ensure the questions will solicit useful and valid
information from the beneficiary. We have similar
concerns about other questions that are framed in a
similar manner, and if terminology/content (e.g.,
“resolved”) is changed in Survey Question 1, we
recommend that related changes be made to other
questions, as appropriate.

Add text to
Supporting
Statement

We recommend that CMS revaluate the error margin and Reject
confidence interval and make modifications as needed to
ensure they are consistent with well-established
parameters to performance measurement.

We cannot comply with this suggestion for No action
several reasons: a) it is cost-prohibitive for needed
the study, b) it would increase burden on
beneficiary respondents, and c) some
contracts may not have enough complaints
to receive a measure and this may count
negatively in the beneficiary's choice of a
plan.

Edit Q1

Analyses will incorporate information about
resolutions in which plans are constrained
by CMS guidelines.

AHIP also believes it is likely that beneficiaries could perceive a closed
complaint as unresolved if the resolution was contrary to what the
beneficiary requested, for example, denial of an enrollment or
disenrollment request or denial of coverage of an excluded drug,
consistent with CMS requirements. Furthermore, the cognitive status of
the beneficiary who is surveyed could be an additional complicating factor
in obtaining a answer that accurately reflects the responsiveness of the
plan.

America's Health
Insurance Plans

6401 OMB
Supporting
Statement B
100202

Respondent Respondent Universe and Sampling (pages 3-5).
Universe and Confidence Interval:
CMS indicates that the goal of the survey is to generate an error margin of
Sampling
10% for a minimum confidence level of 85% and that these parameters
depend on the size of the sample, which would be 6,500 complaints. We
are concerned that the specified confidence interval is inconsistent with
general design practices for performance evaluation. It is our
understanding that the commonly used margin of error for health plan
performance evaluation is 5.7percent for a minimum confidence interval of
95 percent. For example, this is the margin of error and confidence
interval used in CAHPS surveys where the minimum sampling size of 300
generates an error margin of 5.7 percent for the confidence interval of 95
percent (simple random sampling).

CMS seeks to collect information on all
contracts and this limits the number of
complainants to be surveyed.

11 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

America's Health
Insurance Plans

BlueCross
BlueShield of
Tennessee

Document
6401 OMB
Supporting
Statement B
100202

6401 OMB
Supporting
Statement A
100202

6401 OMB
Survey
Instrument
100202

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Item

CMS
ACTION
Partial
Accept

Methods to
Maximize
Response
Rates and
Data
Reliability

AHIP recommends that CMS address this issue through
Methods to Maximize Response Rates and Data Reliability (pages 8-9).
the survey design either by modifying the strategy for
To maximize response to the survey, the contractor will first send the
contacting beneficiaries or in some other manner.
Advance Letter, and then make up to 10 attempts over a three-week
period to contact the beneficiary via telephone. If the beneficiary does not
respond to the telephone survey by the three-week mark, the contractor
will send the beneficiary a paper survey and will follow up with a mailed
reminder to all non-respondents within one week after mailing the hard
copy survey. If we are correct that the interview scheduling process could
result in repeated messages for beneficiaries to call back the contractor,
we are concerned that beneficiaries could react negatively to the number
of attempts to contact them during a relatively short timeframe.
Dissatisfaction with the survey process itself could impact how
beneficiaries answer questions regarding plan activities, particularly if they
inadvertently associate the surveying contractor with their plan.

Background

Timing of Surveys
CMS indicates that the proposed surveys will occur within 21 calendar
days of closure of complaints which could mean that some surveys will be
administered soon after the start of this 21-day period. For example, if the
survey is administered within two – three days following closure of a
complaint, even though the plan may have resolved a complaint within the
CMS required timeframes, the beneficiary may not know of the resolution.
This could occur if the plan is unsuccessful in reaching the beneficiary by
telephone and must send a letter in accordance with CMS CTM Standard
Operating Procedures (SOP), released October 6, 2009.

Accept
AHIP recommends that when scheduling calls for the
survey, CMS wait to administer the survey until at least 10
days after the complaint is closed to ensure that the
beneficiary has been notified of its resolution.

Q8 - Rewrite

Partial
Why are you dissatisfied with the way the plan handled
your complaint? Mark all that apply [Same rationale for the Accept
change as above]

REASON FOR ACTION
We expect that the call center will adapt
survey schedules to obtain prompt
responses from beneficiaries.

Need to:
No action
needed

Contractor interviewers have a defined
introductory statement separating the
contractor from CMS and health plans and
will follow best practices for refusal cases.

Add text to
Supporting
Statement
Adjust
sampling
plan

Q8 has been removed and the new Q2 is a Edit survey
instrument
satisfaction question that encompasses
more aspects of the complaint process
such as coutesy of the plan representative
and time until contact by the plan.
Clarifying text will be added to Q7 asking
the beneficiary to disregard whether or not
he/she is satisfied with the final outcome
itself. The purpose of this question will be to
provide an overall satisfaction rating.

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

Introduction

The sentence that reads, “you recently filed a complaint with your
Medicare plan.”

The more accurate sentence would be, “you recently filed Accept
a complaint with us about your Medicare plan.” At the
plans, we find that a lot of members are quite confused
about the whole complaint process that they call in to
CMS but then hear back from the plan. Perhaps a brief
one-sentence explanation of that would be beneficial to
the Medicare beneficiaries receiving this survey.

CMS finds that it is not in the best interest
of the survey to use the suggested
sentence. The survey introduction will be
modeled on other similar beneficiary
correspondance and survey instructions.
More detail will be added to the survey
introduction regarding why beneficiaries are
being contacted and specifying terms such
as "Medicare," the MAO or Part D sponsor,
and the role of the contractor in conducting
the survey.

Edit
introduction
of survey
instrument

12 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Item

CMS
ACTION
Accept

REASON FOR ACTION

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

Q7 - Rewrite

Now, please indicate how satisfied you are with the way
the plan handled your complaint. [This makes it clear
CMS is asking about the plan performance and not the
complaint resolution. This is an example that despite the
accuracy of the plan’s response, the member will not be
satisfied. Hopefully, this re-write makes the focus on the
performance.]

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

Q9 - Rewrite

Based on your recent experience with the plan, are you
Accept
planning to stay with this plan when you have the
opportunity to switch plans during an Enrollment Period?
[We changed the first ‘this plan’ to ‘the plan.’ This question
does read as though CMS is encouraging members to
consider moving on to a different plan.]

CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

If yes, please mark the method you attempted: calling the Partial
plan’s customer service, writing a letter, calling 1-800Accept
Medicare

CMS has decided to drop questions of
repeat complaints/multiple attempts to
contact the plan.

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

Q5. Did you have to make more than one attempt to resolve your
complaint before the plan contacted you?

Need to:

Edit Q9

The placement of this question in the
survey (ordering) will also affect its
interpretation.
Remove Q5
from survey
instrument

CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.
50

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

General

N/A
Overall. When the members do not receive the answer that they want to
hear from the plans, they will not be satisfied, nor will they believe that the
complaint was resolved.

Partial
Accept

Add text to
The supporting statement will clarify the
Supporting
difference between "resolution" and "final
Statement
outcome." Analyses will incorporate
information about resolutions in which plans
are constrained by CMS guidelines.

BlueCross
BlueShield of
Tennessee

6401 OMB
Survey
Instrument
100202

General

For consideration.

Please consider asking a further question or including it
on Q10 or Q8: Did the complaint involve Social Security
(SSA) withhold?

Reject

This issue/question would only be relevant
to a small portion of the respondents and
would not improve the data gathered while
it would increase the burden (number of
questions) on the respondent.

Coventry Health
Care, Inc

6401 OMB
Survey
Instrument
100202

General

The Survey questions are set in a negative tone vs. being "Open ended
Questions

N/A

The survey questions have been revised to Edit survey
reflect more neutral wording.
instrument

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Repeat Complaints -- Calculation only conceders those that responded
that the plan did contact them. So the assumption is there are only repeat
complaints if the plan contacts a beneficiary?

To accurately measure repeat complaints the question
should be presented to each survey respondent.

Partial
Accept

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

N/A
Beneficiary Satisfaction with Time until a Resolution -- Complainant
responses may be very satisfied, satisfied, dissatisfied, very dissatisfied,
or not yet settled. The values assessed for 2 through – 2 do not include
not yet settled. How will a response of not yet settled be accessed? Will a
response of not yet settled be included in the calculation of the mean
value?

CMS has decided to drop questions related Remove Q6
from survey
to amount of time it took to resolve a
instrument
complaint.

No action
needed

Remove Q5
from survey
instrument

CMS will provide an "I Don't Know/NA"
answer choice for beneficiaries who do not
believe they have received a final outcome
or who do not remember the resolution of
their complaint.

13 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Organization

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Veracity of Plan's Description -- Calculation of measure is unclear uses
same logic as Beneficiary Awareness of Resolution. Additionally, a
dichotomous variable will be created for each included complaint and the
dichotomous variable will be used as the performance measure.

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Beneficiary Awareness of Resolution – calculation of measure is unclear. N/A
The denominator will be the count of sampled complaints where the
complainant indicates he/she is or is not aware of a resolution.” However,
exclusion criteria for the denominator will be responses of “Don’t know” or
“Refused”. How is don’t know different from ‘is not aware’? Unclear if
respondent is not aware of resolution whether it will be counted in
denominator or not.

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Beneficiary Satisfaction with Resolution -- what is the target threshold for
plans or the value assessed to accumulate to the star rating?

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Coventry Health
Care, Inc

CMS
ACTION
Reject

REASON FOR ACTION

Need to:

CMS will consider working toward this in
the future, but CMS is not standardizing
responses to beneficiaries regarding
complaint resolutions. CMS is looking for
the interventions that plans made. Plans
should continue following SOP guidelines.

No action
needed

In the explanation of the "Beneficiary
Awareness of Resolution" indicator, the
denominator will be the number of
complainants who are "not aware of
resolution." This language was intended to
indicate that the beneficiary gave a "no"
response to this question. The survey will
include the following response options:
"yes," "no," "I don't know," or no response.
The text of supporting statement A will be
further clarified with the calculation of the
denominator.

Add text to
Supporting
Statement

N/A

Reject

Star rating information is not yet available.

No action
needed

Beneficiary Satisfaction with Complaint Handling Process -- Unclear what N/A
target thresholds for plans to achieve are not clearly defined.

Reject

Star rating information is not yet available.

No action
needed

6401 OMB
Supporting
Statement B
100202

Respondent That given limited sample size (6,500),question whether the survey would N/A
Universe and accurately portray CTM satisfaction for any given health plan
Sampling

Partial
Accept

Statistically representative samples and
enrollment size variables will be taken in
consideration when developing measures.

No action
needed

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement B
100202

Methods to
Maximize
Response
Rates and
Data
Reliability

Need to ensure that the response rates equals a statistical valid sample

N/A

Partial
Accept

Statistically representative samples and
enrollment size variables will be taken in
consideration when developing measures.

No action
needed

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Information
Users

CMS may publish a performance metric using these data in November
2011. “May” is not a clear/defined affirmative statement. What would
prevent CMS from publishing?

Recommend change ‘may’ to ‘will’ publish.

Reject

This data collection is a preliminary effort to Add text to
develop a satisfaction measure. CMS will
Supporting
review the feasibility to develop a
Statement A
performance measure in coming years.

Need definitions of ‘major action’ terminology/phrasing
CMS looking for which will affect the dichotomous
variable. If a respondent needs to match to the ‘major
action’ plans need to know what these are to provide
same/similar language to the complainant providing a
consistency throughout the entire process.

The "may" refers to the fact that "CMS may
opt not to use the results of the survey for
performance measurement."

14 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
Organization

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

General

It stated that these interviews will not be done through a person but
through a telephonic process. This is nota good tool to use for our
Medicare population and most people might hang up in frustration

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement A
100202

Publication
Tabulation
Dates

Beneficiary Satisfaction with Complaint Handling Process -- Interpretation N/A
of satisfaction very subjective and measure should be more concrete. For
example, a member may not be satisfied because they still have out of
pocket expenses therefore dissatisfied with the process/response/
resolution.

Partial
Accept

Coventry Health
Care, Inc

6401 OMB
Supporting
Statement B
100202

Methods to
Maximize
Response
Rates and
Data
Reliability

The statistical Methods are very confusing to understand

Reject

Add text to
There were limited options for a detailed
narrative while still maintaining the technical Supporting
Statement
level required for the sampling approach.

6401 OMB
Survey
Instrument
100202

Q10

Beneficiary Consequences During Complaint Resolution Process -- Very
leading question assumes problems existed.

6401 OMB
Survey
Instrument
100202

General

Coventry Health
Care, Inc

Gateway Health
Plan (Medicare
Assured - H5932)

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

N/A

CMS
ACTION
Reject

REASON FOR ACTION

Need to:

The CATI system is a computerized system No action
needed
that a live interviewer uses to record the
responses from beneficiaries who take the
telephone interview. The beneficiary will be
speaking directly to the interviewer.

Adjust
sampling
plan
Perhaps following other survey examples such as CAHPS Accept
and conduct a leading question of Did you experience any
problems... yes no? Then if no obtain more detail.

Additionally, the question in the document states “Mark all that apply”
where is the listing of all that apply. How does the measure calculate no
problems?
N/A
Questions appear to assume that any resolution that is not in the
member’s favor, or that grants the member’s request, is an unsatisfactory
resolution. The complaint resolution may not be in the member’s favor,
but determined by other factors, such as CMS guidelines

Partial
Accept

Edit Q10
This question gathers data regarding the
experiences beneficiaries may have during
the complaint resolution process. The
survey questions have also been revised to
reflect more neutral wording.
CMS understands there is a possibility that
beneficiaries will associate dissatisfaction
with the resolution and the overall complaint
experience. In order to address this
concern, the supporting statement will
clarify the difference between "resolution"
and "final outcome." An emphasis on “final
outcome or decision” rather than
“resolution” puts the focus on the series of
actions the plan took, regardless of whether
the beneficiary believes his/her complaint
was resolved.

Add text to
Supporting
Statement
Edit survey
instrument

Analyses will incorporate information about
resolutions in which plans are constrained
by CMS guidelines. Survey questions will
be reworded to be more neutral.
Beneficiaries will be asked to provide an
opinion of the resolution regardless of
whether they agree with it.

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Advance Letter
100202

General

Is the interviewer going to call twice? Once to arrange a time to conduct
the interview and another to do the interview?

N/A

Add language to the Advance Letter to
Add text to
describe the calling process and how
Advance
beneficiaries should expect to be contacted. Letter

15 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Survey
Instrument
100202

This question is confusing. Did the member make more than one attempt N/A
to contact the plan to resolve this complaint? Or is it a combination of
contacting the plan and Medicare? Or, did the member need to contact
Medicare multiple times?

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Survey
Instrument
100202

Time to resolution may not be controlled by the plan. For example,
N/A
requests for retroactive disenrollment date changes must be routed to the
CMS contractor. The processing time is outside of the control of the plan.

CMS
ACTION
Accept

REASON FOR ACTION

Need to:

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Remove Q5
from survey
instrument

Question 6 has been removed from the
survey. Some issues related to the amount
of time it took to resolve a complaint have
been incorporated in Q2 of the new survey
instrument. However, approaches to control
for issues outside the plan's control will be
described in supporting statement.

Remove Q6
from survey
instrument

Edit Q9

Add text to
supporting
statement

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Survey
Instrument
100202

This may be confusing to dual-eligible beneficiaries, that have an on-going N/A
Special Election Period.

CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Survey
Instrument
100202

General

Questions do not consider that complaint resolution may be affected by
factors that are outside of the control of the plan. For example, loss of
Medicaid eligibility is not controlled by the plan.

N/A

Partial
Accept

CMS is well aware of other factors affecting Add text to
complaint outcomes and issues outside the Supporting
control of the plan. Approaches to control
Statement
for these issues need do be described in
more detail in the supporting statement.
CMS will assess the validity of complaints
against plans and screen complaints that
are included in analysis.

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Advance Letter
100202

General

The advance letter reading level is too high and the role of IMPAQ
International is not clear.

Recommend that the reading level of the language in the
letter be lowered.
Recommend to add the IMPAQ International is a
contractor being used by Medicare to conduct the survey

It is reasonable to lower the reading level of Add text to
the documents and add descriptive
Advance
language to the Advance Letter to further
Letter
identify the contractor.

Gateway Health
Plan (Medicare
Assured - H5932)

6401 OMB
Survey
Instrument
100202

General

The beneficiaries are being contacted because they filed a complaint with N/A
1-800-Medicare, but all questions are directed at the plan. No questions
with regard to the service the beneficiary received by Medicare.

Reject

Survey questions are designed to measure No action
plan performance rather than Medicare.
needed

Group Health
Cooperative and its
wholly owned
subsidiary, H2810
Group Health
Options, Inc.

6401 OMB
Survey
Instrument
100202

1-800 Medicare representatives do not appear to be adequately trained on
the use and purpose of the CTM and all need training on how to use it
correctly. Improper categorization of complaints leads to difficulties for the
plans attempting to resolve the complaints.

Evaluating the CTM would need to include whether or not Reject
the complaint was correctly categorized by 1-800Medicare and whether 1-800-Medicare delivered the
complaint to the correct party.

CTM categorization issues should have
minimal impact on effective and timely
complaint resolution. Complaint type will
also be taken into consideration and
excluded, if necessary.

No action
needed

Sampling strategy will take into
consideration complaints to be addressed.

Group Health
General
Cooperative and its
wholly owned
subsidiary, H2810
Group Health
Options, Inc.

General

The measure seems to be designed simply to lower plan star ratings.
N/A
Some plans receive very few complaints. The complaints that these plans
receive will all be subject to review whereas only a percentage of
complaints for other plans will be reviewed.

Reject

Statistically representative samples and
enrollment size variables will be taken in
consideration when developing measures.

No action
needed

16 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

CMS
ACTION
Reject

General
Group Health
Cooperative and its
wholly owned
subsidiary, H2810
Group Health
Options, Inc.

General

Plans are already measured by CMS on CAHPS surveys for beneficiary
perception of complaint resolution, and this is published to the public on
www.medicare.gov

Suggested not to use the proposed measure as a
performance measure for plans. Please use CAHPS
process for eliciting responses from beneficiaries.

Group Health
General
Cooperative and its
wholly owned
subsidiary, H2810
Group Health
Options, Inc.

General

Gathering information on beneficiary satisfaction with the CTM process is
not an accurate measurement of what plans must do to be in compliance
with CMS regulations to correctly resolve CTM complaint, and the role that
CMS plays in CTM resolutions.

An accurate assessment of whether the CTM process
Partial
works from the beneficiary perspective would need to
Accept
incorporate all the CMS rules plans must comply with, and
what role CMS itself plays in the CTM complaint
resolution decisions.

REASON FOR ACTION
The CAHPS surveys do not capture the
beneficiary's satisfaction with the plan's
handling of their complaint. This is a new
and necessary measure.

Need to:
No action
needed

The supporting statement will clarify the
Add text to
difference between "resolution" and "final
Supporting
outcome." Analyses will incorporate
Statement
information about resolutions in which plans
are constrained by CMS guidelines.
CMS will assess the validity of complaints
against plans and screen complaints that
are included in analysis.

Group Health
Cooperative and its
wholly owned
subsidiary, H2810
Group Health
Options, Inc.

6401 OMB
Survey
Instrument
100202

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

General

Why does CMS focus on such negative response? There are no
questions that positively measure helpfulness and courtesy of plans
regardless of what the complaint is. This survey is designed to elicit the
most negative response possible from beneficiaries.

N/A

The survey questions have been revised to Add text to
reflect more neutral wording.
Supporting
Statement
Edit survey
instrument

Question # 8 from the survey attempts to get to the core of why the person N/A
was dissatisfied but it is reasonable to expect that a person who did not
have a problem resolved favorably will take the opportunity to "kill the
messenger".

Q8 has been removed and the new Q2 is a Edit survey
satisfaction question that encompasses
instrument
more aspects of the complaint process
such as coutesy of the plan representative
and time until contact by the plan.
Clarifying text will be added to Q7 asking
the beneficiary to disregard whether or not
he/she is satisfied with the final outcome
itself. The purpose of this question will be to
provide an overall satisfaction rating.

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

General

The current survey gives the member an opportunity to bash their plan
without taking into account the terms and conditions of their contract or
whether the response to the complaint was within the appropriate time
frame.

Partial
The "new" survey should ask the member what it is that
could have been done differently to avoid the complaint in Accept
the first place or to help resolve the problem quicker once
it has been identified. It should also ask if the member has
filed complaints about this plan previously.

CMS is well aware of other factors affecting
complaint outcomes and issues outside the
control of the plan. Approaches to control
for these issues need do be described in
more detail in justification statement. Other
CMS data will be used to control for plan
characteristics and beneficiary profiles.

Add text to
Supporting
Statement A
Edit survey
instrument

CMS will assess the validity of complaints
against plans and screening complaints that
are included in analysis.
CMS will add an additional question to the
survey asking beneficiaries for feedback
such as, what it is that could have been
done differently to avoid the complaint in
the first place or how the complaints
process can be better handled by their plan.

17 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
81

Organization

Document

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

Item
Q7

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
(From question #7 on the survey) Now, please indicate how satisfied you N/A
are with the way your complaint was handled by the plan.
A complaint that is not resolved in the members favor has a very limited
chance of being reviewed favorably in the manner in which it was handled.
A complaint that was resolved in the members favor, has a better chance
of getting rated favorably even if the manner in which the problem was
handled was not in accordance with all of the guidance.

CMS
ACTION
Partial
Accept

REASON FOR ACTION
CMS understands there is a possibility that
beneficiaries will associate dissatisfaction
with the resolution and the overall complaint
experience. In order to address this
concern, the supporting statement will
clarify the difference between "resolution"
and "final outcome." An emphasis on “final
outcome or decision” rather than
“resolution” puts the focus on the series of
actions the plan took, regardless of whether
the beneficiary believes his/her complaint
was resolved.

Need to:
Add text to
Supporting
Statement
Edit survey
instrument

Independent Health 6401 OMB
(HP010)
Advance Letter
100202

General

The letter states that "answers will be kept strictly confidential and be used The letter should clarify and put emphasis on plan
only for research purposes."
monitoring. To simply state "research purposes" is
misleading/not clear on how the results will be used.

Add language to the Advance Letter to
Add text to
describe the intended use of collected data. Advance
The purpose of the data collection (to
Letter
improve how complaints are handled and to
inform the development of a plan rating
system) will be explained better.

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

(From question #6 on the survey) How satisfied are you with the amount
of time it took to resolve your complaint?
This doesn't measure the plans responsiveness. The plan could have
resolved the issue within 24 hours but if the member perceived that this
was too long, we fail. (i.e.: if the member spends more time in pain
because of the response time) This question does a better job of
measuring whether the regulation that defines the response time is
tracking with the member expectation of reaching a resolution to their
problem.

N/A

Question 6 has been removed from the
Remove Q6
survey. Some issues related to the amount from survey
of time it took to resolve a complaint have instrument
been incorporated in Q2 of the new survey
instrument.

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

General

When compiling a score from these surveys, CMS should take into
account whether the plan complied with the regulation.

Partial
If it did, the results should be reported separately from
Accept
those that failed to comply with the regulation. Plans
should not be penalized when they adhere to and execute
based on government regulations.

18 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
(From question # 10 from the survey) During the complaint process, what N/A
problems did you experience while you were waiting for your complaint to
be resolved? Then 11 options are given.
Option 4: "It caused me stress and anxiety" should be removed and the
member should be able to have that as a write-in on the "Other - please
specify" option. A complaint, regardless of whether it was justified or not,
handled properly or not; or handled timely or not is going to cause stress
and anxiety. Having this arbitrary option diminishes the value of the other
serious issues that the remaining options define, such as loss of coverage
or missing an opportunity to undergo a necessary procedure.

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

Q10

Independent Health 6401 OMB
(HP010)
Supporting
Statement B
100202

Respondent This section states that urgent and immediate need complaints will be
Universe and used.
Sampling

Independent Health 6401 OMB
(HP010)
Supporting
Statement B
100202

Methods to
Maximize
Response
Rates and
Data
Reliability

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

CMS would be able to increase the sample size and
accuracy of the study if this was also opened to 30 day
cases.

CMS
ACTION
Accept

REASON FOR ACTION

Need to:

Edit Q10
Experiencing stress and anxiety is
important and necessary to capture the
severity of their experience. However, CMS
will classify it as "extreme" stress and
anxiety.

Reject

Immediate and urgent complaints are the
primary concern of CMS.

No action
needed

This section states that "interviewers especially skilled at encouraging
cooperation will be available to persuade reluctant respondents to
participate and will be assigned to attempt conversions with respondents
who initially refuse (except for hostile refusals)."
This sounds like interviewers will be badgering and attempting to talk
potential respondents into participating until they become hostile. This
seems like the wrong approach. Also, if interviewers persuade
respondents to participate, these interviewers could potentially go too far
by trying to persuade specific responses (particularly if a respondent
sounds indecisive).

Respondents should be allowed to refuse initially, and
Reject
their refusal response should be accepted and respected.

It is standard procedure in call center
surveys. The call center staff have been
trained to encourage participation without
being forceful.

No action
needed

(From question #5 on the survey) Did you have to make more than one
attempt to resolve your complaint before the plan contacted you?

If the answer is "Yes", then the member should say "how Partial
many times".
Accept

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Remove Q5
from survey
instrument

General

Will CMS take into consideration instances in which members make
negative commentary even if the plan is expeditious and concise in its
response, particularly in cases in which the plan is not at fault for the
member’s situation?

N/A

CMS is well aware of other factors affecting Add text to
complaint outcomes and issues outside the Supporting
control of the plan. Approaches to control
Statement
for these issues need do be described in
more detail in the supporting statement.
CMS will consider assessing the validity of
complaints against plans and screening
complaints that are included in analysis.

Partial
Accept

19 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#
90

Organization

Document

Independent Health 6401 OMB
(HP010)
Survey
Instrument
100202

Item
Q3

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
(From question #3 on the survey) How satisfied are you with the resolution N/A
of your complaint? This is asking the member if the complaint was
resolved to their satisfaction, not necessarily to the letter (or spirit) of the
contract. It does nothing toward measuring whether the complaint was
justified, resolved in accordance with the contract or handled within a time
frame that meets the regulation. It actually measures how well the member
likes the complaint resolution process that CMS has defined

CMS
ACTION
Partial
Accept

REASON FOR ACTION

Need to:

Add text to
To ensure beneficiaries respond to the
question as intended, the word "resolution" Supporting
will be replaced with "final outcome" in Q3. Statement
An "NA" answer choice is available for
beneficiaries who do not believe they have
received a final outcome or who do not
remember the resolution of their complaint.
The supporting statement will clarify the
difference between "resolution" and "final
outcome." Analyses will incorporate
information about resolutions in which plans
are constrained by CMS guidelines. More
information will be included regarding
analysis plans and calculations of
satisfaction.

Independent Health 6401 OMB
(HP010)
Supporting
Statement B
100202

Procedures A minimum confidence level of 85% is not high enough.
for the
Collection of
Information

The confidence level should be at least 90%, preferably
95%.

Reject

Kaiser Foundation
Health Plan, Inc.

N/A

Partial
Accept

Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

Current language: “Now, please indicate how satisfied you are with the
way your complaint was handled by the plan.”
This question is problematic in a similar manner as question 2. The
question, as currently written, provides no context for the beneficiary as to
the rules and constraints the Part C sponsor has as a regulated entity by
CMS. As most beneficiaries do not fully understand the rules and
processes Medicare Advantage plans must use to resolve issues,
especially those related to enrollment/disenrollment which compromise
70% of Kaiser CTM cases, the concern is that this question may lead to
responses that relate directly to actions/timing of the health plan that are
not under the control of the Part C sponsor.

General

Kaiser urges CMS to reconsider and strengthen this
Kaiser has strong concerns about the proposed project. While Kaiser
project, as more fully discussed below.
supports and believes in beneficiary satisfaction with their Medicare
Advantage and Prescription Drug plans, Kaiser feels that this survey is not
an appropriate measure of the Part C sponsor’s performance or its
members’ satisfaction with the Part C sponsor.

Clarifying text will be added to Q7 asking
the beneficiary to disregard whether or not
he/she is satisfied with the final outcome
itself. The purpose of this question will be to
provide an overall satisfaction rating.

CMS will clarify that the proposed project is Edit survey
instrument
intended to explore the possibility for a
future performance measure. Based on
public comments, CMS has made
significant improvements to the survey and
study design and intends to continue
strengthening the project. After thorough
testing and analysis of all collected data,
CMS will decide if it will be possible to
formulate a performance measure.

20 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

Current language: “According to our records, the complaint you filed was
recently closed by the plan. Was the complaint resolved?”

Kaiser Foundation
Health Plan, Inc.

6401 OMB
Advance Letter
100202

General

The Advance Letter indicates that the survey will be asking Medicare
beneficiaries about “how well the [Medicare] program” responds to
concerns and how “satisfied” the beneficiary is with the outcome of their
complaint. This characterization of the survey, however, is not a truly
accurate representation of the survey. Rather, it is a survey of the
beneficiary's satisfaction with their Medicare plan's CTM resolution
process, which may or may not be under the complete control of the Part
C sponsor.

CMS
ACTION
We suggest alternative language for increased beneficiary Partial
understanding: “The complaint you filed with Medicare on Accept
xx/xx/xx was closed on xx/xx/xx by your Medicare health
plan. Did the health plan explain how your complaint was
handled and what the outcome was?”

Kaiser urges CMS to screen the CTM complaints to
determine if issues that are outside of the control of the
Part C sponsor (i.e. enrollment/disenrollment issues) are
at the core of the complaint and remove them from the
survey pool.

REASON FOR ACTION
CMS recognizes the merit in providing the
date of the complaint, however this
suggestion will be included in the survey
introduction as opposed to Q1.

Need to:
Edit
introduction
of survey
instrument

CMS is well aware of other factors affecting Add text to
complaint outcomes and issues outside the Supporting
control of the plan. Approaches to control
Statement
for these issues need do be described in
more detail in the supporting statement.
CMS will consider the validity of complaints
against plans and screen complaints that
are included in analysis.
However, CMS disagrees that the survey is
improperly characterized in the Advance
Letter.

Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

General

Current language: “What was the resolution?”

The survey, as currently structured, does not provide an unbiased and
quality framework to properly assess member satisfaction with Part C
sponsors. Rather, it is likely to result in distorted perceptions of health
plans and their complaint resolution processes with members rather than
meaningful data with which to assess Part C sponsors.

We suggest alternative language that is more specific in
terms of obtaining a defined, and useful response:
“Whether or not you agreed with the outcome, did you
understand the outcome of your complaint?”

N/A

Partial
Accept

While CMS does not accept the suggested
revision, we recognize the importance of
clarifying the difference between
"resolution" and "outcome." To this end, the
survey instrument and supporting
statement will be edited to reflect the
updated language.

Add text to
Supporting
Statement

CMS would like to clarify that this survey
gathers data regarding the experiences
beneficiaries may have during the
complaint resolution process.

Add text to
Supporting
Statement

Edit Survey
instrument

Edit survey
CMS does recognize the importance of
instrument
clarifying the difference between
"resolution" and "outcome." To this end, the
survey instrument and supporting
statement will be edited to reflect the
updated language.
In the development of the performance
measure, several factors will be considered
as to not rely on one single item such as
beneficiary's response of experiences of
stress and anxiety.

Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

Current language: “According to our records, the complaint you filed was
recently closed by the plan. Was the complaint resolved?”

In addition, it is suggested that if, according to the
response of the beneficiary, the complaint resolution was
not the desired outcome, CMS should consider it
“unresolved” and end the survey with the beneficiary.

Reject

An emphasis on “final outcome or decision” No action
rather than “resolution” puts the focus on
needed
the series of actions the plan took,
regardless of whether the beneficiary
believes his/her complaint was resolved.
Beneficiaries who feel their complaints have
not been resolved may still contribute their
opinions regarding how the complaints
were handled.

21 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Current language: “Based on your recent experience with this plan, are
It is suggested that this question be omitted from the
Partial
you planning to stay with this plan when you have an opportunity to switch survey as it does not provide feedback that is limited to
Accept
plans?”
the beneficiary’s experience with the complaint resolution
process.
In the context of a beneficiary that has disenrolled with the Part C sponsor
at the time of survey, this question is not applicable. In the context of
current members, however, this question is of great concern as it
suggests to members that they should consider switching plans.
Moreover, this question leads to the possibility that beneficiaries will
respond to factors outside the context of the complaint resolution itself
(e.g. change of provider, monthly premium, relocation, etc.).

CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

Edit Q9

Beyond the sampling set proposed for this survey, Kaiser has concerns
N/A
related to the timing of the survey itself. CMS proposes to conduct the
survey using January and February 2011 CMT complaints. During this
time period it is very likely that the complaints received by CMS and
logged into CTM will be heavily biased in terms of the substance of the
complaints as relating to plan/benefit changes which may be
disproportionate to other types of complaints received throughout the plan
year and may only be reflective of temporary issues caused by new CMS
requirements.

CMS is interested in the months with the
No action
largest number of complaints in order to
needed
achieve the most statistically valid sample.

Current language: “Did you have to make more than one attempt to
resolve your complaint before the plan contacted you?”

100 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

102 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Supporting
Statement A
100202

6401 OMB
Supporting
Statement B
100202

Need to:
Remove Q5
from survey
instrument

6401 OMB
Survey
Instrument
100202

101 Kaiser Foundation
Health Plan, Inc.

REASON FOR ACTION
CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Kaiser Foundation
Health Plan, Inc.

CMS
ACTION
Accept

The CTM module in HPMS has a repeat complainant
function; as such, it is suggested that this question be
eliminated.

This information is already captured by CMS.

Background

Respondent "All Medicare Advantage and Prescription Drug plans will be surveyed,
Universe and regardless of their enrollment size"
Sampling
There is no mention, however, of the possibility that Part C sponsors with
low volumes of complaints in CTM that are in the surveyed category may
be misrepresented in the survey.

Partial
Accept

CMS is not suggesting that beneficiaries
switch plans, but merely asking whether
beneficiaries are likely to do so.

CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

Kaiser suggests CMS implement a minimum threshold of Reject
CTM complaints during the survey time period in order for
a Part C sponsor to be included in the survey.

CMS is well aware of small sample size
issues and has decided to proceed with
collecting data for these contracts.

No action
needed

We suggest revision of language to following: “According
to our records, you recently filed a complaint through 1800-Medicare.”

CMS disagrees with the inclusion of 1-800Medicare in the survey. Not all complaints
are captured through 1-800-Medicare and
this may be confusing for some
beneficiaries. However, the survey
introduction will be reworded to improve
beneficiary understanding.

Edit
introduction
of survey
instrument

This possibility that the variation in distribution of complaint types will likely
vary significantly between Part C sponsors with large enrollments (and
likely larger numbers of complaints) and Part C sponsors with smaller
enrollments (and likely fewer numbers of complaints) is of concern to
Kaiser as it may lead to misrepresentations of Part C sponsors
irrespective of the actual sponsors’ overall efforts at complaint resolution.

103 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

Introduction

The introduction indicates that beneficiary “recently filed a complaint with
their Medicare plan”, but actually the beneficiary filed a complaint with
CMS through the CTM system.

Partial
Accept

22 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

104 Kaiser Foundation
Health Plan, Inc.

Document
6401 OMB
Supporting
Statement A
100202

Item
Information
Users

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Kaiser has concerns as to the ultimate use of the information collected by
the survey. It is unclear based on the documents describing this survey if
or when Part C sponsors would receive a detailed report of the results of
the survey.

Kaiser recommends that CMS provide detailed survey
results to Part C sponsors so that they can implement
process improvements as necessary. Moreover, the
release of this report to Part C sponsors would further
CMS’ goal of transparency within the Medicare program.

CMS
ACTION
Partial
Accept

REASON FOR ACTION
CMS will follow a similar process for this
measure as it does for other measures. It
is undetermined what level of data will be
shared with plans.

Need to:
No action
needed

This data collection is a preliminary effort to
develop a satisfaction measure. CMS will
review the feasibility to develop a
performance measure in coming years.
105 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Supporting
Statement B
100202

Respondent Kaiser is concerned that CTM cases may not be appropriately categorized N/A
Universe and and the beneficiary issue is not fully and accurately reflected in the notes
that the Part C sponsor receives. If there is less than accurate information
Sampling
transmitted to the Part C sponsor, it is to the Part C sponsor’s
disadvantage in terms of resolution of the issue in a manner optimal to the
beneficiary as well as the Part C sponsor.

106 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

107 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Supporting
Statement B
100202

Respondent Kaiser questions the value of the beneficiary’s response to the closure of
Universe and his/her CTM complaint given that the beneficiary likely will not have a full
Sampling
understanding of the rules Medicare Advantage plans must follow to
resolve system discrepancies.

Current language: “How satisfied are you with the amount of time it took
to resolve your complaint?”

Reject

CTM categorization issues should have
minimal impact on effective and timely
complaint resolution.

No action
needed

Partial
Suggested language: “Health Plans have xx days to
resolve complaints like the one you filed. Your complaint Accept
was resolved in xx days. How satisfied are you with that
time frame?” The rephrasing of the question to include
the context in which Part C sponsors must operate helps
to provide the beneficiary with a reasonable expectation of
what is an “acceptable” response time.

N/A

Partial
Accept

While Kaiser is sympathetic to the beneficiary’s frustration and will do what
it can to assist the beneficiary, the ultimate resolution of the issue is
beyond Kaiser’s immediate control. However the survey of the beneficiary
based on this circumstance will not take into account this context for the
Part C sponsor.
108 Kaiser Foundation
Health Plan, Inc.

6401 OMB
Survey
Instrument
100202

We suggest language that would indicate that there may Partial
Accept
be specific CMS rules that the Part C sponsor had to
follow in terms of resolution of the complaint: “Was it
As mentioned above, without a clear understanding of Medicare rules, the explained to you how the plan reached its resolution of
beneficiary may not be satisfied with the outcome, and the Part C sponsor your complaint? With that information, were you satisfied
with the explanation?”
may be unable to take any different action.
Current language: “How satisfied are you with the resolution of your
complaint?”

To ensure beneficiaries respond to the
question as intended, the word "resolution"
will be replaced with "final outcome" in Q3.
An "NA" answer choice is available for
beneficiaries who do not believe they have
received a final outcome or who do not
remember the resolution of their complaint.

Add text to
Supporting
Statement
Edit Q3

The supporting statement will clarify the
difference between "resolution" and "final
outcome." Analyses will incorporate
information about resolutions in which plans
are constrained by CMS guidelines. More
information will be included regarding
analysis.

23 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

Document

Item

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Need to:

Q10

110 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Introduction

With regards to the introductory statement, “According to our records, you We recommend being more specific here, as in the
recently filed a complaint with your Medicare plan.”
introductory letter, and including reference to 1-800Medicare.

Partial
Accept

Edit
introduction
of survey
instrument

111 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Q4. Did the plan contact you about your complaint? This includes contact
by telephone, mail, email, or other means.

Question 4 has been removed and issues
of plan communication with the beneficiary
have been incorporated in Q2 of the new
survey instrument.

Edit survey
instrument

As written, this is a leading question.

It is suggested that the question be revised as follows:
“During the complaint process did you experience any
problems while you were waiting for your complaint to be
resolved?” If yes, then the surveyor can ask further
details. If the answer is no, the list of items will not be
provided.

REASON FOR ACTION

6401 OMB
Survey
Instrument
100202

109 Kaiser Foundation
Health Plan, Inc.

Current language: “During the complaint process, what problems did you
experience while you were waiting for your complaint to be resolved?”

CMS
ACTION
Accept

• We suggest adding "don't know" and "don't remember"
to the list since it may be a common response to this
question.
 Yes
 No
 Don't Know/Don't remember

112 Medco Health
Solutions

113 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Q8. Why are you dissatisfied with the way your complaint was handled?

• Suggest adding additional options to the list:

"I don't know/NA" answer choices will also
be offered to the respondents.
Partial
Accept

N/A

Partial
Accept

A neutral answer choice would be "neither
satisfied nor dissatisfied." However, CMS
decided against including a neutral answer
choice in order to encourage beneficiaries
to select an opinion one way or the other.
Instead, CMS will provide an "I Don't
Know/NA" answer choice for beneficiaries
who do not believe they have received a
final outcome or who do not remember the
resolution of their complaint.

Edit survey
instrument

A softer tone should be considered to persuade
responders to complete the telephone surveys.

Partial
Accept

CMS understands the concern with
protecting beneficiaries, but surveyors will
be complying with call center protocols.
This approach will be clarified in the
supporting statement. Text in the advance
letter and survey introduction will also be
revised.

Edit
introduction
of survey
instrument

 Plan staff did not provide me with any alternatives
 I was not happy with the outcome

Q7. Now, please indicate how satisfied you are with the way your
complaint was handled by the plan.

Same concern with a 4 point scale.

114 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

General

The approach that will be used for training may result in upset/irritated
responders which could negatively impact results.

Add text to
supporting
statement
Edit Advance
Letter

24 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

115 Medco Health
Solutions

Document
6401 OMB
Survey
Instrument
100202

Item
Q10

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Q10. During the complaint process, what problems did you experience
while you were waiting for your complaint to be resolved?
We find this question to be negatively worded.

We suggest a revision:

CMS
ACTION
Partial
Accept

Q: During the complaint process, did you experience any
problems during the resolution of your complaint?
– Yes/No
– If yes, please mark all that apply.

REASON FOR ACTION

Need to:

Edit survey
CMS did not agree with the suggested
instrument
revisions, however this question will be
revised to incorporate a more neutral tone.
This question gathers data regarding the
experiences beneficiaries may have during
the complaint resolution process.

For Phone surveys:
• Since the list is lengthy, we recommend creating a
predefined list for the surveyor to select from with an
option for "other/free form text".
• If the direction is to read the list then we suggest rotating
the list for each survey to ensure that results aren't
skewed toward those on the top of the list.

For Mail surveys:
• Several of the options do not apply to PDP plans. Our
suggestion is to suppress for the survey. If this can't be
done then our suggestion is to add "(if applicable)" or
"(applies tot MAPD Plans only)" to the option:
– Option 6 is specific to MAPD
– Option 7 should be "Out of Network" for PDP plans (Out
of Plan indicates MAPD)
– Option 9 is specific to MAPD
– Option 10 is specific to MAPD

116 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Q3. How satisfied are you with the resolution of your complaint?
We are concerned about the use of a 4 point scale for the survey. It is
common for people to be dissatisfied with the handling of the complaint
because they won't be able to differentiate between the complaint itself
and the complaint handling. The likelihood of anyone responding
"satisfied" on a complaint survey is low. A 5 point scale would allow for
the option to respond "neutral."

We suggest tightening the question to ensure the
respondent is rating the overall experience and not one
specific area.

Partial
Accept

For example, "Taking all aspects of the complaint
resolution process into consideration, how satisfied are
you with the process?”

To ensure beneficiaries respond to the
Add text to
question as intended, the word "resolution" Supporting
will be replaced with "final outcome" in Q3. Statement
An "NA" answer choice is available for
beneficiaries who do not believe they have
received a final outcome or who do not
remember the resolution of their complaint.
The supporting statement will clarify the
difference between "resolution" and "final
outcome." Analyses will incorporate
information about resolutions in which plans
are constrained by CMS guidelines. In the
development of the performance measure,
several factors will be considered as to not
rely on one single item such as
beneficiary's response of experiences of
stress and anxiety.

117 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Q6. How satisfied are you with the amount of time it took to resolve your
complaint?
Same concern with a 4 point scale.

N/A

Partial
Accept

25 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

118 Medco Health
Solutions

Document
6401 OMB
Survey
Instrument
100202

Item
Q9

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Q9. Based on your recent experience with this plan, are you planning to
stay with this plan when you have the opportunity to switch plans?
• We find this question may lead the person to respond negatively

We recommend rewording it. For example:

CMS
ACTION
Accept

Q: How likely are you to stay with in the
future?"

REASON FOR ACTION
CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

Need to:
Edit Q9

CMS is not suggesting that beneficiaries
switch plans, but merely asking whether
beneficiaries are likely to do so.

• Additionally, being able to differentiate responses by new
vs. existing members would be beneficial. We suggest
adding two demographic questions at the end of the
survey to assist with results analysis:
Q: Are you new to Medicare?
Q: (If not new to Medicare) Are you new to the
119 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

• We suggest using pre-defined areas for this question,
with the option for "other" to capture free form text. This
would allow specific areas of focus with sufficient sample
The open-ended responses to this survey question will be used in the
“Resolution Veracity” indicator. The survey responses will be compared to to draw a conclusion.
the resolution summary provided by the plan in the CTM. Thereby, the
For phone surveys,
indicator will indicate what percentage of a plan’s CTM resolution
– recommend the surveyor fits the response into predescriptions agree with the experience described by the beneficiary.
defined areas with an option for free-form versus reading
each area. (see below).
Q2. What was the resolution?

Partial
Accept

For mail surveys,
– recommend providing a list of pre-defined areas with an
option for free form responses.
• Based on an analysis of previous complaint types, we
would like to suggest "pre-defined areas" as follows:
– Enrollment correction
– Disenrollment submitted
– Prescription issued
– Temporary supply provided
– Coverage Determination offered
– Explanation provided
– Other (with free form text box)

26 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

120 Medco Health
Solutions

Document
6401 OMB
Survey
Instrument
100202

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision

Item
Q5

Q5. Did you have to make more than one attempt to resolve your
complaint before the plan contacted you?

We suggest expanding on this question so it can be
determined 1) who was contacted first (Medicare or the
Plan), 2) And the number of contacts made prior to
speaking with the Plan. It would be beneficial to know if
the member attempted to resolve with the Plan before
calling 1-800-Medicare, or if the issue went straight to
CMS. For example:

CMS
ACTION
Partial
Accept

REASON FOR ACTION

Need to:

CMS has decided to drop questions of
repeat complaints or multiple attempts to
contact the plan.

Remove Q5
from survey
instrument

Add text to
Supporting
Statement

Q: Prior to filing your complaint with CMS, how many
times did you attempt to resolve your issue directly with
?
–0
–1
–2
– 3 or more
– Don't know/Don't remember
Q: After filing your complaint with CMS, how many
additional times did you attempt to resolve your issue?
–0
–1
–2
– 3 or more
– Don't know/Don't remember
121 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

Q1. According to our records, the complaint you filed was recently closed
by the plan. Was the complaint resolved?
• This question is critical to distinguish responder perception of complaint
completion since satisfaction can greatly differ based on this.

Therefore we suggest a more specific line of questioning Partial
Accept
so the results can be differentiated between responders
who believe the complaint is resolved vs. responders who
feel it's not resolved. For example:
Q: According to our records the complaint that you filed
with CMS was recently closed by . Is this
your understanding?

Edit Q1

The survey will also be revised to
distinguish responders who believe the
complaint is resolved vs. responders who
feel it is not resolved.

Q: If yes - Was your complaint fully resolved?
 Yes - go to Q2
Q: If No - Please explain why.
 Skip to Q4.
 For responders that don't think it's complete the
questions should end after Q5 or differ then the questions
for "yes".
Q: Don't know

122 Medco Health
Solutions

General

Has a satisfaction goal been established? And will a goal be set differently N/A
for mail vs. phone results?

123 Medco Health
Solutions

6401 OMB
Survey
Instrument
100202

General

For consideration.

Reject

Recommend including the plan name & type within the
Accept
survey so it is clear to the responder what plan the survey
is for, and not for Medicare in general.

The survey instrument is consistent for both No action
implementation approaches. Furthermore, it needed
is expected that most surveys will be
answered by phone and a small proportion
of surveys will be answered via mail.

To reduce confusion about the topic and
Add text to
source of the complaints, the advance letter Advance
will make explicit reference of the MAO or Letter
Part D sponsor.

27 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

124 SilverScript
Insurance
Company and
RxAmerica

Document
6401 OMB
Survey
Instrument
100202

Item
Q8

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Why are you dissatisfied with the way your complaint was handled?

We suggest the following modification to the question
being asked: "Please indicate how satisfied you are with
the following support you received from the plan

CMS
ACTION
Accept

Length of time to process my complaint (Very Satisfied,
Satisfied, Dissatisfied, Very Dissatisfied)
Treated with courtesy and respect (Very Satisfied,
Satisfied, Dissatisfied, Very Dissatisfied)
Staff explained things in a way that was easy to
understand (Very Satisfied, Satisfied, Dissatisfied, Very
Dissatisfied)
Staff provided enough information (Very Satisfied,
Satisfied, Dissatisfied, Very Dissatisfied)
Other comments (specify)"
125 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Survey
Instrument
100202

Why are you dissatisfied with the way your complaint was handled?

REASON FOR ACTION

Need to:

We recommend that this question should be skipped if the Partial
Accept
member indicates that they are satisfied with how the
complaint was handled. (Response to Q7)

126 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Survey
Instrument
100202

Now, please indicate how satisfied you are with the way your complaint
was handled by the plan.

We suggest the following modification to the question
being asked: "Please indicate how satisfied you are
overall with the way your complaint was handled by the
plan. (Very Satisfied, Satisfied, Dissatisfied, Very
Dissatisfied)"

127 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Supporting
Statement A
100202

General

Also, plans with a higher number of mentally challenged members would
likely get lower marks, simply because of the demographics of their
populations.

N/A

Beneficiary characteristics will be taken into Add text to
consideration in the analysis of survey data. Supporting
Further details regarding analyses will be
Statement
included in supporting statement A.

128 SilverScript
Insurance
Company and
RxAmerica

General

How will CMS determine who to survey? We have complaints filed by
outside agencies, pharmacies, appointed representatives and
congressional offices. The member may not always be involved in the
resolution, but the complaint is resolved and handled appropriately.

N/A

28 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

129 SilverScript
Insurance
Company and
RxAmerica

Document
6401 OMB
Survey
Instrument
100202

Item
Q5

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
Did you have to make more than one attempt to resolve your complaint
before the plan contacted you?

N/A

CMS
ACTION
Partial
Accept

Is there any analysis planned to identify situations where enrollees make
repeat requests hoping to obtain a different outcome?

REASON FOR ACTION
CMS has decided to drop questions of
repeat complaints/multiple attempts to
contact the plan.

Need to:
Remove Q5
from survey
instrument

CMS will focus on complaints as the unit of
analysis. No analysis will be conducted for
the small number of repeated complaints.

Accept
We recommend that this question should not be asked.
Members should be encouraged to evaluate all aspects of
the plan's performance, and not encouraged to focus on a
single event when determining which plan best fits his/her
needs.

CMS has decided to reword this question
to be more neutral and to ask about the
likeliness of the beneficiary to stay with the
plan.

Edit Q9

130 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Survey
Instrument
100202

Based on your recent experience with this plan, are you planning to stay
with this plan when you have the opportunity to switch plans?

131 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Supporting
Statement A
100202

Background

The time frame for the survey of members in immediately after the
CMS should consider selecting a different time period.
January 1st plan year start. This is a time of year when members are just
becoming familiar with the elements of their new plan, and yet in many
cases, because of transition policies, may not be experiencing the full
impact of others.

Reject

CMS is interested in the months with the
No action
largest number of complaints in order to
needed
achieve the most statistically valid sample.

132 SilverScript
Insurance
Company and
RxAmerica

General

Will CMS provide specifics on how plans will be evaluated?

N/A

Reject

CMS will follow a similar process for this
measure as it does for other measures.
Technical specifications will be provided.

No action
needed

133 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Supporting
Statement A
100202

General

Is there any implication of the "Do Not Call List." How will CMS gather
data on members residing in LTC or other situations where access to a
phone is not readily available?

N/A

Phone numbers are included in CMS
datasets including the CTM records.

No action
needed

134 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Supporting
Statement A
100202

General

There have been cases that were our members have complained that they A plan should be able to provide supportive data showing Partial
never received promised reimbursement, yet we are able to show
that they took the actions required.
Accept
canceled checks where they did.

The purpose of the survey is to ascertain
No action
the satisfaction of the beneficiaries, not the needed
appropriateness of the plan's actions.

CMS will incorporate several data into the
development of the preliminary measures
including complaint categories, CMS
guidelines, beneficiary and plan
characteristics.

135 SilverScript
Insurance
Company and
RxAmerica

6401 OMB
Survey
Instrument
100202

According to our records, the complaint you filed was recently closed by
the plan. Was the complaint resolved?

Please clarify what the definition of "resolved" means. We Partial
ask because resolution of an issue may not result in the
Accept
enrollee being satisfied in the event the enrollee doesn't
understand the requirements of the program that the Part
D plan must comply with. As such, the enrollee might not
conclude that the issue was resolved even after the Part
D plan has done all it can to resolve it.

The supporting statement will clarify the
Add text to
difference between the selected terms.
Supporting
"Resolved" will be replaced with "settled" in Statement
this question to prevent beneficiary bias. An
"I don't know" answer choice has been
added for beneficiaries who feel they do not
yet have a resolution or are unsure/do not
remember."resolution" and "final outcome."
Analyses will incorporate information about
resolutions in which plans are constrained
by CMS guidelines.

29 of 30

Part C and D Complaints Resolution Performance Measure
60-Day Comment Summary
#

Organization

136 SilverScript
Insurance
Company and
RxAmerica

Document
6401 OMB
Survey
Instrument
100202

Item
Q10

Parts C and D Complaints Resolution Performance Measure OMB PRA Package
Description of Issue or Question
Suggested Revision
During the complaint process, what problems did you experience while
you were waiting for your complaint to be resolved?
There are concerns that Q10 is ambiguous as far as evaluating the plan's
handling of requests. Even if the plan resolved the request in a very timely
fashion, perhaps providing satisfaction to the member, that member may
experience problems (e.g. stress and anxiety, out-of-pocket expenses,
etc.) due to the event that raised the grievance, not necessarily a result in
the delay of the resolution of that grievance. How will the person filling
out the survey interpret this question?

N/A

CMS
ACTION
Partial
Accept

REASON FOR ACTION

Need to:

Edit Q10
This question gathers data regarding the
experiences beneficiaries may have during
the complaint resolution process. The
survey questions have also been revised to
reflect more neutral wording.
In the development of the performance
measure several factors will be considered
as to not rely on one single item such as
beneficiary's response of experiences of
stress and anxiety.

30 of 30

File Type	application/pdf
Author	CMS
File Modified	2010-05-20
File Created	2010-05-19