Response to Comments

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

general

None
Analysis of Raw Data: While we appreciate
receiving the aggregate data from the survey, we
believe plan analysis of the survey raw data would
allow for improvements that would prove beneficial
to both plans and beneficiaries. To promote
transparency in the process and allow for more
beneficial plan improvements we request that CMS
provide the survey response raw data to plans for
further analysis.

No revisions The disenrollment survey is an initial effort to assess beneficiary
made to
experience with this new Medicare benefit. CMS is conducting the
item
survey to better understand what are the key issues/challenges that
beneficiaries are facing, and this information will be used to guide
decisions about the administration of the Part D program.
Additionally this survey will inform the design of subsequent
surveys should CMS decide to continue this type of surveying in the
future. CMS will further consider this comment for future surveying
efforts, in the event that CMS receives funding to continue periodic
surveying of Part D beneficiaries. CMS is not intending to make
plan-specific results available to the public or to plans during this
initial round. At this time, CMS has not determined whether it will
make publicly available a national aggregated data file. CMS will
be producing a final report that summarizes the key findings from
the survey, and this report will be publicly available.

general

None
Response Rate Calculation: It is unclear what
factors will be utilized in determining which survey
responses may be disqualified and whether or not
disqualified responses and incomplete surveys will
be counted in the response rate.
Recommendation: We recommend providing
greater transparency regarding the specifics that
yield the expected response rate. For example,
whether or not disqualified responses and
incomplete surveys will be factored into the rate.
Further, we recommend that the disqualified
responses and incomplete surveys not be factored
into the response rate.

No revisions CMS intends to follow the same rules that are applied for the
made to
Medicare CAHPS survey. We will count any survey with a
item
reportable item as a response.

general

Live calls: Commenter suggests that CMS
consider doing live calls, as proposed for CTM
follow-up

None

general

Languages: In addition to the specific
recommendations outlined below, we request that
CMS clarify in which languages the survey will be
made available.

None

No revisions CMS's approach to conducting the survey involves a mail survey
made to
with phone follow-up for non-respondents. CMS believes this
item
blended approach will work to maximize response rates and allow
for a larger number of beneficiaries to be surveyed in a costeffective manner.
No revisions The survey will be available in English and Spanish.
made to
item

1

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
general
(b3, b4 and
c1, D1 go
together)

general

Comment

Survey Question Referenced
in Comment

None
Sample Selection: The sampling timeframe of
July 2010-March 2011 will capture those
beneficiaries that are experiencing plan changes
due to recent CMS mandated changes regarding
limited plans per market. These beneficiaries are
experiencing an involuntary disenrollment and are
not choosing to leave a plan. The survey is
designed to capture why beneficiaries leave or
switch plans, so should only include those that
voluntarily exit a plan; or at a minimum, exclude
those that are experiencing plan changes due to
the termination of plans per CMS policy or
changes in guidelines. It is unclear what
populations CMS intends to target (voluntary vs.
involuntary disenrollments). We recommend the
exclusion of beneficiaries that were involuntarily
disenrolled from the plan or at a minimum, those in
plans that were terminated due to CMS changes in
policy.
Sample Size: In the sample design, small samples None
will still yield a higher percentage of margins of
error regardless of weighting. We recommend
plans with small sample sizes either not be
reported, or be oversampled to get a readable
sample size. In the alternative, we recommend
CMS report non-weighted results along with
weighted results with base sizes for review.

Revision
Status

Response

No revisions The survey will focus only on beneficiaries who voluntarily disenroll
made to
from their Part C or D plan during the Annual Election Period (AEP),
item
the Medicare Advantage Open Enrollment Period (OEP) or the
Special Election Periods (SEPs). The survey will exclude
beneficiaries who disenroll from plans because of eligibility
reasons, movement out of the service area and deaths. CMS
maintains a set of codes that indicate whether a beneficiary is
voluntarily disenrolling, and all beneficiaries who disenroll for nonvoluntary reasons will be excluded from the population from which
the sample will be drawn. Note: some of the disenrollment during
the SEP period is "voluntary" disenrollment (such as among dually
eligible beneficiaries). It is anticipated that the time from
disenrollment to receipt of a survey will be approximately 3 months.
This survey has been funded for a single fielding at this point in
time. CMS has not yet decided whether the survey will be repeated
and any future surveying is contingent on receipt of funding. There
is interest at CMS in periodic surveying of this population for
program improvement purposes. CMS intends to survey voluntary
disenrollees as close as possible to the time of their disenrollment
No revisions This is the first time this survey will be conducted, and CMS does
made to
not intend to report publicly any results. It is possible that in the
item
future, CMS may repeat the survey and may adjust the sampling
approach to generate robust estimates for plans of varying sizes
and/or adjust the results to account for larger standard errors in the
estimates.

2

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
general

general

Comment

Timing of Survey: It is unclear in the
documentation what the timeframe will be between
the beneficiary's disenrollment and when the
survey will be sent to them. We agree with CMS
that is important to survey disenrollees relatively
soon after their disenrollment experience. A
timeframe of several months between
disenrollment and surveying will severely hinder
the reliability of the data based on respondent
recollection. We recommend CMS reduce the
timeframe between the sample extract and data
collection as much as possible. We recommend no
more than 3 months from time of disenrollment to
the date the beneficiary receives the survey. In
addition, we request CMS update the
documentation regarding when the survey will be
sent to beneficiaries in relation to their
disenrollment date.
Response ordering: Suggestion was made to
revise the satisfaction scale and start with the most
positive choice in the order of options for the
member to choose from.

Survey Question Referenced
in Comment
None

Revision
Status

Response

No revisions CMS intends to survey voluntary disenrollees as close as possible
made to
to the time of their disenrollment to minimize recall issues. It is
item
anticipated that the time from disenrollment to receipt of a survey
will be approximately 3 months.

None

No revisions The proposed response ordering on the Part C and D disenrollment
made to
survey is consistent with the ordering of responses on the Medicare
PDP4/MA-PD4 - How often did item
CAHPS survey. CMS seeks equivalency in the ordering of
the plan’s customer service
response options across the disenrollment survey and the Medicare
give you the information or
CAHPS survey to enable comparison of scores on comparable
help you needed?
items among those who do not disenroll (from the Medicare CAHPS
This scale throughout should be flipped so the
survey). Prior CAHPS survey testing work revealed that the
most positive point is listed first: Very Satisfied,
ordering of response choices matters in terms of results (refer to
Satisfied. Neither sat/nor dis sat, Dissatisfied, Very
the paper by M. Elliott, A. Zaslavsky et al. in the 2008 issue of
dissatisfied.
Health Services Research for more details).
Many of the questions use the phrase “try to get”.
Recommends “seek” vs. “try to get”. “try” has a
more negative connotation.

3

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

general

Survey Length: Commenter is concerned with
the length of the proposed survey.

None

No revisions Based on standard metrics for survey administration time we
made to
estimate it will take Medicare beneficiaries about 20 minutes to
item
complete this survey. This length is shorter than the current
Medicare CAHPS surveys, which achieved an overall response rate
in 2009 of 61.8%. Separately, the response rates were: Medicare
Advantage = 64.1%; the fee-for-service (FFS)-prescription drug =
57.7%; fee-for-service-only = 58.3%. Previous tests of different
length versions of CAHPS surveys found that response rates were
not sensitive to survey length. Should the survey be used beyond
the currently funded data collection effort, content may be refined to
meet the agency's information needs.

general

None
Question Addition: Identify reason for leaving
plan and have a choice model. i.e. leaving out “did
you leave because plan communications were
confusing?” If answered yes, list types of
communications.. list CMS requiring
communications, i.e.. Annual notice of change,
welcome materials.

general

Streamlining: simplify and streamline this whole
survey. If laid out properly, would flow better.

No revisions In previous survey work, the CAHPS Consortium has found that
made to
some survey participants have difficulty navigating grid items (items
item
with lists of options to mark yes or no) due to the cognitive
complexity of the task. We have selected the existing question
format to minimize errors in marking responses, and to simplify the
cognitive response task. This initial disenrollment survey is
attempting to identify the primary reasons for disenrollment in an
effort to better understand the operation of this new Medicare
benefit. CMS anticipates that the information from this first
administration will inform the development of other survey items
related to reasons for disenrollment which may provide useful
quality improvement information for any future survey
administrations.
No revisions Careful thought was given to the layout of the survey to facilitate
made to
beneficiaries' ability to comprehend and follow the survey flow logic.
item
In the process of designing the survey, an alternative survey layout
was evaluated and was subsequently modified and streamlined to
facilitate the beneficiary's ability to respond to the survey.

general

Wording suggestion: Many of the questions use None
the phrase “try to get”. Recommend changing
wording to “seek” vs. “try to get”. “try” has a more
negative connotation.
Underlining: Suggestion made to underling key None
points in questions for better emphasis (see
question Q5, Q6).

general

None

No revisions We strive to produce surveys at the lowest reading and
made to
comprehension level possible. While "try" and "seek" are
item
synonyms, "try to get" is lower literacy, easier to comprehend
language than "seek."
No revisions In developing questionnaires to assess consumer experience with
made to
care (i.e., the CAHPS family of surveys), underlining is used as a
item
tool to alert survey participants to differences between two adjacent
questions with similar wording.
4

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment
None

Revision
Status

Response

general

Question Addition: We suggest adding a future
likelihood question e.g. likelihood to consider
using plan again/recommending plan to others.

No revisions This item would not be appropriate for a survey of disenrollment, so
made to
it was not considered for inclusion. This is an item on the Medicare
item
CAHPS survey for drug plans.

general

Numerous items
Streamlining and Question reformatting: The
'REASONS YOU LEFT YOUR FORMER
PRESCRIPTION DRUG PLAN' section could be
reworked to be more efficient. Was a contributing
reason for your leaving the plan related to a grid
type response set: Someone enrolled you with out
your knowledge? (Yes No) An error was made and
ended your coverage accidentally (Yes No) A
change in the prescription medications that were
covered (i.e. a change in formulary) (Yes No) Plan
required you to take generic vs. name brand
medications (Yes No), Etc…

No revisions Prior experience with more than a decade of work on the CAHPS
made to
surveys indicates that some survey participants have difficulty
item
navigating grids items (items with lists of options to mark yes or no)
due to the cognitive complexity of the task. We have selected the
existing question format to minimize error in marking responses to
simplify the cognitive response task.

general

None
Plan Identification: Given that members
sometimes switch plans within an organization,
there may be beneficiary confusion when
completing the survey as to which of the
organization's plans is the subject of the survey.
We recommend CMS clearly identify both the
organization and the plan name that is the subject
of the survey on both the survey and in the results
that are shared with each plan.

Item has
been
revised

The survey will be customized for each individual, and it will
reference at the beginning and at multiple places throughout the
survey which MA-PD or PDP is the reference plan the beneficiary is
being asked to comment on. The name that will be used is the
name as published to beneficiaries (i.e., the marketing name),
rather than the plan's contract name. We have expanded the first
set of questions (previously Q1 & Q2) to 5 questions to ensure plan
name recognition, allow beneficiaries who have not disenrolled or
who moved to self-identify (as a check against the administrative
data from CMS that will be used to identify the candidate sample of
beneficiaires), and allow beneficiaires to write in the plan name that
they dropped/switched away from. This modification does not add
additional burden to respondents of the survey.

5

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
general

Comment

Survey Question Referenced
in Comment

Response

No revisions The phrasing of this question is meant to be inclusive of terms a
made to
disenrollee might use. The survey will focus only on those who
item
voluntarily disenroll from their Part D or C plans. We will rely on the
CMS disenrollment database to determine eligibility for the survey,
and only include those individuals who voluntarily disenroll.

Wording Suggestion: PDP/MA-PD Introductory
Paragraph (PDP – page 10; MA-PD – page 24
refers to beneficiaries who “leave, switch or drop”
prescription drug coverage. As drafted, we believe
that the distinction between “leaving” and
“dropping” prescription drug coverage is likely to
be unclear to beneficiaries. It appears that the
references to “dropping” a plan are intended to
mean that the beneficiary was dropped due to
involuntary disenrollment. We believe that CMS
should survey only beneficiaries who voluntarily
disenroll, and therefore, we recommend that CMS
revise the draft by striking references to “dropping”
a Part D plan. If the agency decides to include in
the survey sample beneficiaries who have been
involuntarily disenrolled, we recommend that the
agency revise this question to clarify the distinction
between “leaving” and “dropping” a plan.

general (A6 Question Addition: CMS should add a question
and A8)
about Part D members disenrolling from a plan
because they required a combined prescription
and medical benefit?

Revision
Status

None

No revisions Using CMS administrative enrollment information, we can
made to
determine what proportion of individuals with PDP and FFS medical
item
coverage switch to an MA-PD plan. Additionally, we have included
an open ended question on the disenrollment survey that asks the
disenrollee what was the "one most important reason" they left the
plan, which should provide an opportunity to identify this type of
issue if it is indeed an important issue. If, through this initial survey,
it is determined that this is a common reason for disenrolling
expressed by respondents, CMS will consider adding this in future
disenrollment surveys should those be approved for funding.

6

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
MA-PD11

Comment

Wording Suggestion: (The MA-PD 11 and 12)
Members of staff/group model HMOs do not
generally differentiate between the health plan and
the care delivery systems. Members of staff/group
model HMOs are more likely to think of these two
questions as typical access questions (e.g.
difficulty making appointments and wait times).
Members of network/IPA model HMOs are more
likely to interpret these questions as the health
plan creating administrative barriers (e.g. forms to
obtain and have doctors complete and fax to
health plan, phone calls to be made by doctor to
health plan) to members getting care from the
respondents’ doctors. The differences in
interpretation of these questions may lead to
inconsistent results. Suggest Q12 be revised as
follows: “How often did the health plan make it
difficult for you (e.g. forms to fill out, phone calls to
be made by your doctor to the health plan, etc.) to
get the care, tests, or treatment you thought you
needed through the plan?”

Survey Question Referenced
in Comment

Revision
Status

MA-PD11 - Did you ever try to No revisions
get any kind of care, tests, or made to
treatment through the plan?
item
MA-PD12 - How often was it
easy to get the care, tests or
treatment you thought you
needed through the plan?

Response

This question is part of the Prescription Drug Plan (PDP) item set in
Medicare CAHPS, and is based on the CAHPS work. The item
performs as intended with beneficiaries and exhibits good
psychometric properties. We want to retain consistency in
language between the disenrollment survey items that map to the
Medicare CAHPS items to allow comparison in the ratings between
those who disenroll and those who do not.

7

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
MA-PD19

PDP1

Comment

Question order: The placement of Q19 seems
problematic. It follows a string of questions about
prescription drug benefits and services. The
responses to this question will be heavily
influenced and highly correlated to responses to
the preceding prescription questions. But the
respondent may also think of other aspects of the
health plan. If CMS wants respondents’ overall
impressions of the health plan this question should
be placed at the front of the questionnaire, before
Q3. If, however, the intent is to obtain respondents’
impressions of their former plans’ Part D benefits
and services, then suggest that the question be
revised to instruct respondents to think about Part
D benefits and services when answering the
question.
Plan identification: We suggest that CMS merge
this question into the introduction.

Survey Question Referenced
in Comment

Question Addition: Consider: How often did the
plan provide you with the written information the
language you requested? (satisfied) Ask what
language?

Response

MA-PD19 - Using any number No revisions This item will allow comparison of plan ratings by disenrollees to
from 0 to 10, where 0 is the
made to
plan ratings by enrollees (via the Medicare CAHPS survey). The
worst health plan possible and item
item placement is similar to that of the Medicare CAHPS survey.
10 is the best health plan
possible, what number would
you use to rate the plan?

PDP1/MA-PD1 - Our records
show that you used to belong
to [PLAN NAME], but no
longer belong to that plan. Is
that right?
PDP1,
Question Addition: Suggestion that on questions None
PDP4, MA- that ask specifically about customer service that
PD3, MA- you add attributes of behavior to this listing.
PD4

PDP10

Revision
Status

No revisions This question is used to orient the survey participant to the sampled
made to
plan. If the participant does not recognize the sampled plan as a
item
plan that he or she has recently left, we do not want them to
complete the survey.
No revisions This initial disenrollment survey is attempting to identify the primary
made to
reasons for disenrollment in an effort to better understand the
item
operation of this new Medicare benefit. CMS anticipates that the
information from this first administration will inform the development
of other survey items related to reasons for disenrollment. This
may include more detailed examination of specific attributes of
customer service or plan operations. These issues will be
considered based on the findings of this initial survey, and future
survey work is contingent on receipt of funding.

PDP10/MA-PD10 - How often No revisions
did the plan give you written
made to
information in a language
item
other than English?

The purpose of this question is to understand, when needed (based
on previous question), how often a beneficiary received information
in a language other than English. We believe the item as worded
gets at how often this may/may not have been a problem for a
beneficiary. We appreciate the desire for a given plan sponsor to
know the specific language that is needed by a beneficiary, but that
is outside the scope of this initial survey effort. Medicare Part D
plan providers may wish to conduct their own survey's and quality
improvement interventions with plan subscribers.

8

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

PDP11

Wording Suggestion: (PDP11) "During the time PDP11 - Did a doctor ever
you belonged to (name), were you ever prescribed prescribe a medicine for you
that the plan did not cover?
a medication that the plan did not cover?"

PDP12

Wording Suggestion: We suggest that CMS
clarify this question, “How often was it easy to
obtain the medications that your doctor
prescribed?”

Revision
Status

Response

No revisions This question is part of the Prescription Drug Plan (PDP) item set in
made to
Medicare CAHPS and we are trying to ensure consistency with
item
Medicare CAHPS where possible to allow comparisons in their
ratings between those who disenroll and those who do not.
Experience from the Medicare CAHPS survey shows that the
existing wording of the item performs as intended and exhibits good
psychometric properties. The referent for this question is the plan
from which the beneficiary has voluntarily disenrolled, which
addresses the issue identified by the commenter.
PDP12 - How often was it
No revisions The wording modification proposed by the commenter gets at a
easy to use the plan to get the made to
different issue than the issue that the survey question is seeking to
medicines your doctor
item
address. The survey question is focused on the ease/difficulty of
prescribed?
using the plan to get the medicines prescribed by their doctor, as
opposed to whether it was easy/difficult to obtain medications
prescribed by their physician. The two questions are substantively
quite different. CMS' interest is understanding beneficiary issues
related to using the plan to get their medications. Furthermore, this
question is part of the Prescription Drug Plan (PDP) item set in
Medicare CAHPS and we are trying to ensure consistency with
Medicare CAHPS where possible to allow comparisons in their
ratings between those who disenroll and those who do not.

9

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP13

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

PDP13/MA-PD15 - Did you
ever use the plan to fill a
prescription at a local
pharmacy?

No revisions This questions are part of the Prescription Drug Plan (PDP) item set
made to
in Medicare CAHPS, and based on the CAHPS work. The item
item
performs as intended with beneficiaries and exhibits good
psychometric properties. We want to retain consistency in
language between the disenrollment survey items that map to the
Medicare CAHPS items to allow comparison in the ratings between
those who disenroll and those who do not. Experience from the
Medicare CAHPS survey shows that the existing wording of the
items perform as intended and exhibits good psychometric
properties.

PDP15

Wording suggestion: Did you ever use the plan’s PDP15/MA-PD17 - Did you
ever use the plan to fill any
mail order pharmacy to fill a prescription?
prescriptions by mail?

PDP16

Wording Suggestion: Response Option 5 in Q18
(MA-PD version) states that “I did not use the plan
to fill a prescription by mail.” Suggest that this
response option be deleted altogether. It is not
needed because Q17 instructs respondents to skip
Q18 if they did not use the plan to fill a prescription
by mail.

No revisions We sought simplicity in wording in this item and throughout the
made to
survey. The modified wording offered by the commenter does not
item
improve the readability and we will retain the original item.
Additionally, this question is part of the Prescription Drug Plan
(PDP) item set in Medicare CAHPS and we are trying to ensure
consistency with Medicare CAHPS where possible to allow
comparisons in their ratings between those who disenroll and those
who do not. Experience from the Medicare CAHPS survey shows
that the existing wording of the item performs as intended and
exhibits good psychometric properties.
No revisions We have found from our previous survey work that a small number
made to
of beneficiaries do not follow the appropriate skip logic. Thus we
item
provide what is in effect a "not applicable" response.

Wording Suggestion: Suggest this wording
change for PDP13/MA_PD15 items: "Did you ever
fill a prescription at a local pharmacy which utilized
this plan?" Response Option 5 in PDPQ16 states
that “I did not use the plan to fill a prescription at a
local pharmacy.”
Suggest that this response option be deleted
altogether. It is not needed because Q15 instructs
respondents to skip Q16 if they did not use the
plan to fill a prescription at a local pharmacy.
Moreover, there may be some confusion with the
term “local” pharmacy. It is not clear what
distinction is being made by classifying a
pharmacy as “local”. What are “non-local”
pharmacies? Does “local” depend on how far away
they are from the respondents’ homes? It is
suggested that the term “local pharmacy” be
defined for the respondent to provide a clearer
question to which he/she can respond.

PDP15/MA-PD17 - Did you
ever use the plan to fill any
prescriptions by mail?
PDP16/MA-PD18 - How often
was it easy to use the plan to
fill prescriptions by mail?

10

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

PDP17

Wording suggestion: Consider using
“satisfaction with plan” vs. worst prescription drug
plan possible. Flip scale to positive on top. Add
neutral point. Throughout the rest of the survey
references are made to “the plan” vs. “prescription
drug plan”. Make consistent. Reword “worst.”
2nd comment: These questions ask the
beneficiary to rate the plan on a scale where 0 is
the worst possible plan and 10 is the best possible
plan. The addition of a free form follow-up
question asking the beneficiary what aspects of
the plan were most important to his/her rating
would provide information that would be useful to
the plan in evaluating areas where performance
could be strengthened or modifications could be
made to meet beneficiary needs or preferences.
We recommend that CMS add such a question.

PDP17/MA-PD19 - Using any No revisions
number from 0 to 10, where 0 made to
is the worst prescription drug item
plan possible and 10 is the
best prescription drug plan
possible, what number would
you use to rate the plan?

PDP18

Wording suggestion: Suggestion that the survey
be modified to clarify if members were autofacilitated, this question is meant to target agent
and brokers and from our experience members
who are facilitated into a plan often have this
belief.
Wording suggestion: In the introductory
paragraph before Q20, suggest the addition of a
statement explaining that respondents might have
multiple reasons for switching and/or dropping
their former plan and that CMS is interested in
capturing all their reasons.
Question reformatting: Financial reasons should
be bundled together. Perhaps ask, did your leaving
the plan have anything related to costs/prices,
affordability or anything financial? Y/N -> if no skip
out of section. A "premium"...

PDP18/MA-PD20 - Did you
leave the plan because you
found out that someone had
signed you up for the plan
without your permission?

No revisions Using CMS administrative enrollment information, we can
made to
determine which individuals are autoenrolled because their plan is
item
now above the LIS benchmark. These individuals will be excluded
at the front end of the survey process and would not be included in
the sample frame.

PDP18/MA-PD20 - Did you
leave the plan because you
found out that someone had
signed you up for the plan
without your permission?

No revisions We believe the recommended wording change is unnecessary, as
made to
the introductory text indicates that people leave for different
item
reasons (and then we proceed with asking about an array of
possible reasons that a beneficiary can check).

PDP18

PDP20

This question is part of the Prescription Drug Plan (PDP) item set in
Medicare CAHPS, and have been well tested in prior CAHPS work.
The item performs as intended with beneficiaries and exhibits good
psychometrics properties. We want to retain consistency in
language between the disenrollment survey items that map to the
Medicare CAHPS items to allow comparison in the ratings between
those who disenroll and those who do not. Because of the
substantial expense associated with entering, coding and analyzing
open-ended responses, CMS has sought to minimize open-ended
questions in the survey. The single open-ended question on the
survey focuses on understanding what was the most important
reason for disenrollment. We also believe that the "most important
reason" for disenrollment will be highly correlated with the overall
rating item, and will in effect, serve to address the issue raised by
the commenter.

PDP20 - A premium is the
No revisions We appreciate the suggested alternative but perceive that it would
amount that you pay to have made to
result in a more complex survey question structure than currently
prescription medicine
item
exists.
coverage from a prescription
drug plan. Some prescription
drug plans charge a premium
to people on Medicare who
are enrolled in that
prescription drug plan. Did
you leave the plan because
the monthly premium for
prescription medicine
coverage went up?

11

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP21

Comment

Survey Question Referenced
in Comment

Revision
Status

Wording suggestion: We suggest that CMS
rephrase this question or remove entirely. Were
you disenrolled from the plan because you failed to
pay your premium, members do not opt into
disenrolling if they are not paying their Part D
premiums.
Highlighting text: A "formulary" is the…

PDP21/MA-PD23 - Did you
leave the plan because you
stopped paying the monthly
premium for the plan?

PDP24

Wording suggestion: Replace “temporary limit”
with “donut hole”.

PDP24/MA-PD26 - Did you
Item has
leave the plan because you hit been
the temporary limit (also called revised
the "coverage gap") when you
had to pay all of the costs of
your prescription medicines up
to a yearly limit?

PDP25

Wording suggestion: Question 25 asks, “Did you
leave the plan because the dollar amount you had
to pay each time you filled or refilled a prescription
went up?” We believe that the reference to a
prescription the enrollee “filled or refilled” is likely
to be confusing to beneficiaries. Asking about
“refilling” the same prescription would yield
information about the beneficiary’s concerns with
the cost for the same drug dosage, strength, and
supply subsequent to the initial fill and could be
useful. However, the reference to “filling”
prescriptions may appear to ask the beneficiary to
compare the cost of prescriptions for different
drugs. As a result beneficiary responses to the
question may not be either informative or
actionable. Recommend that CMS revise the
question by deleting the words “filled or”.

PDP25/MA-PD27 - Did you
leave the plan because the
dollar amount you had to pay
each time you filled or refilled
a prescription went up?

PDP23

Response

No revisions This item is a quality check and that is the reason for its inclusion.
made to
Beneficiaries who failed to pay their premiums would not be in the
item
sample, and this is a means to verify the quality of the
administrative data used to generate the sample frame.

PDP23 - A formulary is the list No revisions We appreciate the desire to highlight the word formulary, but
of prescription medicines
made to
believe the quotation marks are not necessary.
covered by a prescription drug item
plan. Did you leave the plan
because of a change in the
formulary?

Did you leave [PLAN NAME] because you hit the temporary limit
(also called the “coverage gap” or “donut hole”) when you had to
pay all of the costs of your prescription medicines up to a yearly
limit?

No revisions The goal of this item is to understand whether a beneficiary left a
made to
plan because the amount they had to pay (i.e., the copayment) for
item
prescriptions went up, regardless of whether it was an initial fill of a
medication or a refill. We do not wish to exclude respondents who
might have experienced a copay increase when they went in with a
new order for a previous drug. It is not the intent of this question to
ask the beneficiary to compare the cost of prescriptions for different
drugs.

12

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP28

Comment

Question Addition: What is this referring to?
Add a question. Did the plan offer you an
alternative med to the one not on the formulary?
Did we actively try to get a coverage med…

PDP29

PDP3

Survey Question Referenced
in Comment
PDP28/MA-PD31 - Did you
leave the plan because the
plan refused to pay for a
medicine your doctor
prescribed?

Revision
Status

Response

No revisions The modified wording offered by the commenter would raise the
made to
complexity of the question being asked and we seek to maintain
item
simplicity to facilitate understanding. The survey item seeks to elicit
a potential reason why the beneficiary disenrolled, in terms of not
being able to get their medicine (irrespective of whether the plan
offered an alternative). The purpose of this question is not to
evaluate the +/- of any given plan's formulary.

• PDP Questions #28, #29 and #31 (page 10) and
MA-PD Questions #31, #32, and #34 (page 26).
The survey asks three questions about problems
the enrollee experienced in obtaining coverage of
medicines his/her doctor prescribed. These
questions appear to address related and
overlapping issues. For clarity and to provide
additional information about the beneficiary’s
experience, we recommend that CMS make the
following revisions:
+ Retain PDP question #29 and retain MA-PD
question #32, “Did you leave the plan because you
had problems getting the medicines your doctor
prescribed?”
+ Add a note to indicate that if the answer is no,
the beneficiary should skip the following question;
and
+ Combine PDP questions #28 and #31 and
combine MA-PD questions #31 and #34 into a set
of questions organized in a check list/grid
Clarification requested: Please clarify the intent
of this question, did you leave your plan because
the medications that your doctor prescribed were
not on the plan’s formulary? The question as is
has many different interpretations.

PDP31/MA-PD34 - Did you
leave the plan because you
were frustrated by the plan's
approval process for
medicines your doctor
prescribed that were not on
their formulary?

PDP29/MA-PD32 - Did you
leave the plan because you
had problems getting the
medicines your doctor
prescribed?

No revisions The modified wording offered by the commenter would raise the
made to
complexity of the question being asked. The survey item seeks to
item
elicit a potential reason why the beneficiary disenrolled, in terms of
not being able to get the medicines their doctor prescribed.

Wording Suggestion: Change to: Did you ever
seek customer service from [PLAN NAME]'s for
any reason? "Customer service" refers to
information / assistance provided from staff about
what is covered, how to use the plan, etc.

PDP3/MA-PD3 - Customer
service is information you get
from staff about what is
covered and how to use the
plan. Did you ever try to get
information or help from
[PLAN NAME]'s customer
service?

No revisions The Survey Design Project conducted by RTI in 1995 explored the
made to
placement of definitions within survey questions. The project found
item
that the most effective placement is for definitions to appear first.

Prior experience over the past decade with the CAHPS survey
indicates that some survey participants have difficulty navigating
grids items (items with lists of options to mark yes or no) due to the
cognitive complexity of the task. We have selected the existing
question format to minimize error in marking responses to simplify
the cognitive response task.

13

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP3

PDP3

Comment

Plan Identification: Statement Preceding
Question 3, pp 5 & 20 Issue: In the statement that
immediately precedes question 3 in both surveys,
it is explained to beneficiaries that the questions
are about their "former" health plan. However, the
beneficiary may not recall the former plan,
especially if the time from the actual disenrollment
to the survey is more than several weeks.
Recommendation: To assure comprehension
regarding what plan the questions refer to, we
recommend inserting the drug or health plan name
into the sentence that immediately precedes
question 3.
Wording suggestion: The definition of “customer
service” in this question does not seem adequate
and does not reflect what people think of when
they refer to “customer service.” Customer service
departments provide more services and functions
than simply handing out information. As such, the
provided definition seems confusing.

Survey Question Referenced
in Comment

Revision
Status

Response

PDP3/MA-PD3 - Customer
service is information you get
from staff about what is
covered and how to use the
plan. Did you ever try to get
information or help from
[PLAN NAME]'s customer
service?

No revisions We appreciate the suggestion but perceive that the insertion of the
made to
former plan name in question 3 will promote recognition of the plan.
item

PDP3/MA-PD3 - Customer
service is information you get
from staff about what is
covered and how to use the
plan. Did you ever try to get
information or help from
[PLAN NAME]'s customer
service?

No revisions This question is in the Medicare CAHPS survey and will allow
made to
comparison of survey participants across the various surveys.
item

Suggest the revision of the definition of customer
service in Q3 to provide the respondents with a
more accurate description of customer service
such as the following: “Customer service is a
department within your former health plan
responsible for answering questions about your
membership, benefits, grievances and appeals.”

14

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP30

Comment

Wording suggestion: This question does not
follow current guidance, possible misinterpretation
from beneficiaries that plans make them take a
generic because they are cheaper. Plans are not
allowed to mandate the use of generics for
members, suggest removing this question.

Survey Question Referenced
in Comment

Revision
Status

Response

PDP30/MA-PD33 - Did you
leave the plan because the
plan required you to take a
generic medicine when you
preferred the brand name
medicine?

Item has
been
revised

Did you leave the plan because it was difficult to get brand name
medicines?

PDP31/MA-PD34 - Did you
leave the plan because you
were frustrated by the plan's
approval process for
medicines your doctor
prescribed that were not on
their formulary?

No revisions The "coverage determination process" language (i.e., language
made to
typically used by the health plans) is complex and not language that
item
most Medicare beneficiaries would understand. The existing
question is written in simple, easy to understand language, and its
goals is to elicit whether beneficiaries disenrolled due to difficulties
they experienced with the plan approval processes for medications
not on the formulary.

Questions 30 (PDP) & 33 (MA-PD), pp 10, 26.
Issue: The question asks if the beneficiary left the
plan because the plan "required" them to take a
generic medicine. However, plans cannot require
beneficiaries to take a generic medicine.
Recommendation: We recommend revising the
question by replacing the word "required" with
"wanted."
PDP31,
PDP32

Wording suggestion: Clarify and use member
materials…i.e. coverage determination process.
(questions PDP31/MA-PD34 & PDP32/MA-PD35)

PDP32/MA-PD35 - Did you
leave the plan because you
did not know whom to contact
when you had a problem filling
or refilling a prescription
PDP34

Item deletion: Redundant to #36.

PDP34/MA-PD43 - Did you
leave the plan because you
were unhappy with how the
plan handled a question or
complaint?

No revisions This question is measuring a separate dimension (handling of
made to
questions/complaints) vs. #36 which asks about whether the
item
customer service staff treated the beneficiary with courtesy and
respect. We do not feel the two items are repetitive.

15

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP35

PDP36

PDP37

PDP37

Comment

Question Addition: Questions 35 (PDP) & 44
(MA-PD), pp 11, 27
Issue: It would be beneficial for plans understand
the method of contact the beneficiary used.
Improvements could then be targeted to
appropriate staff.
Recommendation: We recommend capturing the
method of contact between the beneficiary and the
plan. For example, by phone to plan customer
service, website, via a sales agent, etc.
Question addition: At the end of the section
entitled “Other Reasons for Leaving Your Former
Health Plan”, the survey asks the respondent in
Q53 “What was the one most important reason
you left [PLAN NAME]?”
Suggest that a similar question should be asked in
the section entitled “Reasons You Left Your
Former Health Plan” after Q45: “Which of the
above is the most important reason you left [PLAN
NAME]?”
Question ordering: Moved as first question in
section

Survey Question Referenced
in Comment
PDP35/MA-PD44 - Did you
leave the plan because you
could not get the information
or help you needed from the
plan?

Revision
Status

Response

No revisions This initial disenrollment survey is attempting to identify the primary
made to
reasons for disenrollment in an effort to better understand the
item
operation of this new Medicare benefit. CMS anticipates that the
information from this first survey administration will inform the
development of other survey items related to reasons for
disenrollment. This may include more detailed examination of
specific attributes of customer service or plan operations. These
issues will be considered based on the findings of this initial survey,
and future survey work is contingent on receipt of funding.

PDP36/MA-PD45 - Did you
No revisions We perceive that adding such a question would be duplicative of
leave the plan because their made to
the existing question.
customer service staff did not item
treat you with courtesy and
respect?

PDP37/MA-PD46 - Did you
No revisions We appreciate the suggested alternative location but think the
leave [PLAN NAME] because made to
current placement of the item is satisfactory.
it wasn't what you expected? item

PDP37/MA-PD46 - Did you
No revisions
Plan Identification: throughout survey, e.g.,
leave [PLAN NAME] because made to
Questions 37 (PDP) & 46 (MA-PD), pp 12, 28.
Issue: It is unclear whether CMS will be populating it wasn't what you expected? item
the plan name by using the plan's contract name
or the name of the plan as published to
beneficiaries. Beneficiaries will not understand
which plan is referred to if the CMS contract name
is used.
Recommendation: We strongly recommend the
use of the plan name that is published to, and
recognized by, the beneficiary. In addition, we
request that the documentation be clarified as to
which name is used.

The survey will be customized for each individual, and it will
reference at the beginning and at multiple places throughout the
survey which MA-PD or PDP is the reference plan the beneficiary is
being asked to comment on. The name that will be used is the
name as published to beneficiaries (i.e., the marketing name),
rather than the plan's contract name.

16

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP37

PDP39

PDP49

PDP4

Comment

Text Modification: Before Q46, suggest that an
introductory paragraph similar to that located
before Q20 be added. The introductory paragraph
should contain a statement explaining that
respondents might have multiple reasons for
switching and/or dropping their former plan and
that CMS is interested in capturing all their
reasons.
Streamlining and Question reformatting: List
for the reasons they left…instead of having so
many questions.

Survey Question Referenced
in Comment

Revision
Status

Response

PDP37/MA-PD46 - Did you
No revisions We have reviewed this comment and believe that the suggested
leave [PLAN NAME] because made to
change is unnecessary and does not further clarify or provide
it wasn't what you expected? item
additional assistance to the individual completing the survey

PDP39/MA-PD48 - Did you
leave the plan because a
family member or friend told
you that another prescription
drug plan was a better plan?
PDP49/MA-PD59 - In general,
how would you rate your
overall health?

No revisions Prior CAHPS experience indicates that some survey participants
made to
have difficulty navigating grids items (items with lists of options to
item
mark yes or no) due to the cognitive complexity of the task. We
have selected the existing question format to minimize error in
marking responses to simplify the cognitive response task.
No revisions This is a standardized measure, used across several CMS and
made to
CAHPS surveys and will allow CMS to compare responses across
item
the various surveys. The variable is used to casemix adjust the
patient populations managed by different providers/plans.

Wording Suggestion and clarification on
question purpose: In general, how would you
rate your overall physical health?
Personal questions. Are these necessary for this
type of survey.
Wording suggestion: Many of the questions use PDP4/MA-PD4 - How often did No revisions
made to
the phrase “try to get”. Recommends “seek” vs. “try the plan’s customer service
give you the information or
item
to get”. “try” has a more negative connotation.
help you needed?

We strive to produce surveys at the lowest reading and
comprehension level possible. While "try" and "seek" are
synonyms, "try to get" is lower literacy, easier to comprehend
language than "seek." Additionally, based on prior experience with
the CAHPS surveys, the rationale for the ordering of response
items is to promote greater variation in responses.

17

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP4

PDP41

Comment

Question addition: Question 4. Issue: We
believe the definition of customer service as
defined in question 3, "information you get from
staff" is rather broad. The responses then, to
question 4 are not as helpful to plans as they could
be. It is difficult to make needed improvements if
the plan cannot identify the staffing area that
responded to the inquiry. We recommend either
capturing the method of contact between the
beneficiary and the plan or designing the
responses to be more specific about what
staff/area was contacted. For example, sales
agent, plan customer service, etc.

Survey Question Referenced
in Comment

PDP4/MA-PD4 - How often did No revisions
the plan’s customer service
made to
give you the information or
item
help you needed?

The definition of “customer service” as set forth in
Q3 is problematic when combined with the wording
of Q4. Substituting in the definition from Q3 gives
you the following: “How often did information you
get from the plan’s staff give you the information or
help you needed?” Suggest that the definition in
Q3 be revised as set explained above in Comment
3 to provide respondents with a more precise
definition and as such, more logical language in
Q4
PDP51/MA-PD61 - In the last
Wording suggestion: This is unclear (are you
referring to unique Rxs or how many transactions? 12 months, how many
different prescription
and scale is too limited.
medicines did you fill or have
refilled?
Also need to specify if Long term only or all?
Perhaps do as write in and limit time frame (past
3/or in typical month).
In the past 3 months, thinking about all the
prescriptions you have filled for your own personal
use, how many have you filled. In your answer
please include all prescription medications you
take on an on-going or long term basis as well as
those taken for a short period.

Revision
Status

Response

This initial disenrollment survey is attempting to identify the primary
reasons for disenrollment in an effort to better understand the
operation of this new Medicare benefit. CMS anticipates that the
information from this first administration will inform the development
of other survey items related to reasons for disenrollment which
may provide useful quality improvement information in future survey
administrations. We appreciate the desire of Part D plans to obtain
drill down information which we are unable to accommodate in this
first survey. The results of this question will allow plans to identify
where further investigation may be needed by QI or QA staff to
inform improvements to customer service.

This question is in the Medicare CAHPS survey and will allow
comparison of survey participants across the various surveys.

No revisions The purpose of this item is to enable analyses of the results by
made to
different subpopulations of beneficiaries (e.g., those with complex
item
health needs as compared to relatively simple health needs); as
such it is an analytic variable that will allow us to stratify the results.
Because we are looking for "orders of magnitude" differences in the
number of medications a beneficiary is taking, we believe that the
proposed item can be reliably reported by a beneficiary and is
suitable for the proposed "use" purposes.

18

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP44

Comment

Analytic issue: If the respondent was a long-term
member of the plan (e.g., joined the health plan
five or more years ago), then the information
collected from this member will not reflect current
attributes of the health plan. Analysis of these
questions should be stratified by respondents’
length of membership in the plan. For example,
scores should only be reported for respondents
who have been with the health plan two years or
less.

Survey Question Referenced
in Comment

Revision
Status

PDP44/MA-PD54 - An
Item has
insurance agent or broker
been
sells insurance for your health, revised to
your home, or your car. Did
clarify the
an insurance agent or broker terminology
ever call you without your
regarding
asking them to, to tell you
agents
about insurance for
prescription medicines?
PDP45/MA-PD55 - Did an
insurance agent or broker
ever visit your home without
your asking them to, to tell you
about insurance for
prescription medicines?
PDP46/MA-PD56 - Did you
decide to leave [PLAN NAME]
because of information you
got from an insurance agent
or broker?
PDP47/MAPD57 - Did an insurance
agent or broker give you any
information that was not
correct?
PDP48/MA-PD58 - What kind
of information was not
correct?

Response

CMS's goal for this group of questions is to understand the
experiences of the overall population of disenrollees. CMS is
interested in knowing whether beneficiaries were inappropriately
contacted or given information about a prescription drug plan that
was not correct at any point during their enrollment with the plan
from which they voluntarily disenrolled. This issue is relevant
regardless of the length of time a beneficiary was enrolled with their
Part C or D plan. We are not modifying our analytic approach to
limit analyses to the subset of beneficiaries who have been with a
Part C or D plan two years or less.
We did modify the wording on the introduction to clarify
terminology: "Different kinds of people sell health insurance.
Insurance may be sold by independent insurance agents or brokers
who don’t work for the health plan OR by plan representatives who
work directly for the plan. Did an insurance agent, broker, or plan
representative ever call you without your asking them to, to tell you
about insurance for prescription medicines?"

19

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP44

Comment

Clarification: There is a section containing five
questions (#44-48) about agent/broker activity that
may not be relevant or valid for all plans. Many
PDPs don't use agents and even if an agent
contacted a member, it does not mean that the
agent was affiliated with the member's plan.

Survey Question Referenced
in Comment

Revision
Status

PDP44/MA-PD54 - An
Item has
insurance agent or broker
been
sells insurance for your health, revised to
your home, or your car. Did
clarify the
an insurance agent or broker terminology
ever call you without your
regarding
asking them to, to tell you
agents
about insurance for
prescription medicines?
PDP45/MA-PD55 - Did an
insurance agent or broker
ever visit your home without
your asking them to, to tell you
about insurance for
prescription medicines?
PDP46/MA-PD56 - Did you
decide to leave [PLAN NAME]
because of information you
got from an insurance agent
or broker?
PDP47/MAPD57 - Did an insurance
agent or broker give you any
information that was not
correct?
PDP48/MA-PD58 - What kind
of information was not
correct?

Response

These items do not link agents or brokers to a specific plan. They
provide information that may inform further follow-up effort by CMS
as the items will yield information on the role and interaction with
agents and brokers which may or may not vary by geographic
region. CMS is concerned about incorrect information being
provided by sales representatives from the beneficiary's current
plan as well as from sales representatives from competing plans
who may be soliciting new members.
Modified wording on the introduction (PDP44/MA-PD55) to this
set of questions to clarify terminology: "Different kinds of people
sell health insurance. Insurance may be sold by independent
insurance agents or brokers who don’t work for the health plan OR
by plan representatives who work directly for the plan. Did an
insurance agent, broker, or plan representative ever call you
without your asking them to, to tell you about insurance for
prescription medicines?"

20

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP45

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

Wording suggestion: Questions 45 (PDP) & 55
(MA-PD), pp 13, 29
Issue: We believe the location of the underlining
does not emphasize the key point of the question.
CMS does not prohibit visits to the home, but
prohibits visits to the home without the beneficiary
asking the agent or broker to do so.
Also, the questions contain extraneous language-the word "you" after "visit your home."
Recommendation: We recommend removing the
underlining from "visit your home" and instead,
underlining "without your asking." In addition, we
suggest removing the word "you" from the
question. The question would then read:
“Did an insurance agent or broker ever visit your
home without your asking them to, to tell you about
insurance for prescription medicines?”
Wording suggestion: Did you ever try to get
(seek) information from the plan about which
prescription medicines were covered?

PDP45/MA-PD55 - Did an
No revisions The underlining is designed to highlight the specific wording
insurance agent or broker
made to
differences between Questions 45 and 55 and the similarly worded
ever visit your home without
item
Questions 44 and 54.
your asking them to, to tell you
about insurance for
prescription medicines?

PDP5/MA-PD5 - Did you ever No revisions
try to get information from the made to
plan about which prescription item
medicines were covered?

We strive to produce surveys at the lowest reading and
comprehension level possible. While "try" and "seek" are
synonyms, "try to get" is lower literacy, easier to comprehend
language than "seek."

PDP51

Wording suggestion: In the past 12 months,
have you seen a doctor or other health provider 3
or more times for the same (a certain) condition or
problem?

PDP52/MA-PD62 - In the past No revisions
12 months, have you seen a made to
doctor or other health provider item
3 or more times for the same
condition or problem?

This is a standard CAHPS question and we seek to maintain
comparability to allow CMS to compare/contrast the results from
this survey of disenrollees with the Medicare CAHPS results for
beneficiaries who do not disenroll.

PDP53

Wording suggestion: Consider: Thinking of the PDP53/MA-PD63 - Is this a
condition for which you’ve seen a doctor 3 or more condition or problem that has
lasted for at least 3 months?
times in the past 12 months, has this condition
lasted for 3 months or longer?

PDP5

No revisions This is a standard CAHPS question and we seek to maintain
made to
comparability to allow CMS to compare/contrast the results from
item
this survey of disenrollees with the Medicare CAHPS results for
beneficiaries who do not disenroll. Additionally, prior CAHPS
experience shows this items performs well.

21

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

PDP54

Clarification: For the condition above? "Has your PDP54/MA-PD64 - Do you
now need or take medicine
doctor prescribed medication for this condition?"
prescribed by a doctor?

PDP56

Question addition: The list should be expanded

PDP56/MA-PD66 - Has a
doctor ever told you that you
had any of the following
conditions?

PDP57

Wording suggestion: Consider, in what year
were you born?

PDP57/MA-PD67 - What is
your age?

PDP59

Wording suggestion: Some college or 2-year
degree or other trade school

PDP59/MA-PD69 - What is
the highest grade or level or
school that you have
completed?

Revision
Status

Response

No revisions This item is not intended to reference the prior two questions about
made to
whether the person has a condition for which they've seen their
item
doctor 3 or more time and whether the condition has lasted for at
least 3 months (questions 52 and 53 on the PDP version and 62
and 63 on the MA-PD version). Those two questions are used as
analytic variables to stratify results between those beneficiaries with
more complex medical needs and those without. Question PDP52/MA-PDP64 asks the beneficiary a different question about
whether they now need to take a prescription medication. This is a
standard CAHPS question and we seek to maintain comparability to
allow CMS to compare/contrast the results from this survey of
disenrollees with the Medicare CAHPS results for beneficiaries who
do not disenroll.
No revisions The purpose of this item is to enable analyses of the results by
made to
different subpopulations of beneficiaries (e.g., those with few/no
item
comorbid conditions vs. those with many). We have focused on
identifying conditions that are widespread in the population and
have significant chronic impact. This variable will be used to stratify
the results of the survey.
No revisions We capture age as a categorical response to minimize missing
made to
data due to nonresponse.
item

No revisions This is a standardized measure, used across several CMS and
made to
CAHPS surveys and will allow comparison of survey participants
item
across the various surveys.

22

Response to Public Comments (Version 9_02_2010)
Question #
for sorting
PDP6

Comment

Survey Question Referenced
in Comment

Revision
Status

Response

Wording suggestion: How often did the plan give
you all the information you needed about which
prescription medicines were covered?
5 N/A - I did not seek information about which
prescription medicines were covered

PDP6/MD-PD6 - How often
No revisions
did the plan give you all the
made to
information you needed about item
which prescription medicines
were covered?

We strive to produce surveys at the lowest reading and
comprehension level possible. While "try" and "seek" are
synonyms, "try to get" is lower literacy, easier to comprehend
language than "seek."

Response Option 5 in both questions starts as
follows: “I did not try and get information about ….”
Suggest that language be revised as follows: “I did
not try to get .…” This revision is a grammatical
improvement and agrees with the text of the
question. In the alternative, suggest that this
response option be deleted altogether. It is not
needed because Q5 and Q7 instruct respondents
to skip Q6 or Q7 if they did not try to get
information about prescription medicines.

PDP8/MA-PD8 - How often did
the plan give you all the
information you needed about
how much you would have to
pay for a prescription
medicine?

PDP60

Question addition: ASK INCOME?
EMPLOYMENT/MARITAL STATUS?

PDP60/MA-PD70 - Are you of No revisions Income and education are highly correlated, and given the
Hispanic or Latino origin or
made to
sensitivity of asking about income, we have focused on education
descent?
item
(which proxies for income). Additionally, income is not a reliable
indicator for retired people as the computation of this is more
complex regarding what assets to include (e.g., what's in their bank
account, their home value, etc.). We are able to bring in income
via census data imputation based on the beneficiaries geographic
region. We do not believe that marital status or employment
provide additional useful information about understanding
differences in disenrollment for various sub-populations.

PDP61

Wording suggestion: "1= Caucasian "

PDP61/MA-PD71 - What is
your race? Please mark one
or more.

PDP62

PDP62/MA-PD72 - What
No revisions
Wording suggestion and ordering: Suggest
items PDP62/MA-PD72 be moved up to language language do you mainly speak made to
item
section. Do these choices reflect the population? at home?
Suggest re ordering based on frequency (e.g. Eng,
Spanish…). Suggest making the list more
complete. Missing Portuguese, French, Japanese.

We have found from our previous survey work that a small number
of beneficiaries do not follow the appropriate skip logic. Thus we
provide what is in effect a "not applicable" response. Question
wording has been modified to read: "I did not try to get information
about which prescription medicines were covered"

No revisions The survey uses the classifications for race approved and required
made to
by OMB.
item
The purpose of this question is to understand whether the
beneficiary is English speaking or not, for analysis purposes. At the
end of this question we provide an open-ended response to allow
the beneficiary to write in other languages than what appear on the
list. The list is organized alphabetically.
23

Response to Public Comments (Version 9_02_2010)
Question #
for sorting

Comment

Survey Question Referenced
in Comment

Revision
Status

PDP64

Question addition: Questions 64 (PDP) & 74
(MA-PD). Issue: It would be helpful for plans to
understand who took the survey, for example, if
the respondent is a care provider, the beneficiary,
a family member, etc. We recommend asking
what role the respondent fills, for example, if the
respondent is a care provider, the beneficiary, a
family member, etc.

PDP64/MA-PD74 - How did
that person help you? Please
mark one or more.

PDP7

Wording suggestion: Did you ever try to get
information from the plan about pricing or how
much you would have to pay for a prescription
medicine?

PDP7/MA-PD7 - Did you ever No revisions
try to get information from the made to
plan about how much you
item
would have to pay for a
prescription medicine?

PDP8

Wording suggestion: How often did the plan give
you all the information you needed about pricing or
how much you would have to pay for a prescription
medicine?
5
N/A - did not seek information about pricing/how
much I would have to pay for a prescription
medicine
Wording suggestion: Did you ever need written
information from the plan in a language other than
English? (They might need it but didn't ask.).

PDP9

Response Option 5 in Q10 states that “I did not
need written information in a language other than
English.” Recommend that this response option
be deleted altogether. It is not needed because Q9
instructs respondents to skip Q10 if they did not
need written information from the plan in a
language other than English.

Response

No revisions The Medicare CAHPS survey contains two questions that seek to
made to
determine "whether the beneficiary had any assistance completing
item
the survey" and "how this person helped." We have included these
same questions on the disenrollment survey for consistency. The
questions will be used to examine whether there are differences in
the case mix of patient populations served by the Part C and D
plans that should be considered in generating scores. As
background, in the Medicare CAHPS work, this variable has not
proven to be a major case mix adjustment variable, as there is little
variation plan-to-plan in responses to this set of variables. The
"who" helped the beneficiary complete the survey has been retired
from Medicare CAHPS and was not considered here given space
constraints and other items that were deemed higher priority.

This question is part of the Prescription Drug Plan (PDP) item set
in Medicare CAHPS and we are trying to ensure consistency with
Medicare CAHPS where possible to allow comparisons in their
ratings between those who disenroll and those who do not.
Experience with the PDP item set in Medicare CAHPS shows the
time wording performs as intended and exhibits good psychometric
properties.
PDP8/MA-PD8 - How often did No revisions This question is part of the Prescription Drug Plan (PDP) item set in
the plan give you all the
made to
Medicare CAHPS and we are trying to ensure consistency with
information you needed about item
Medicare CAHPS where possible to allow comparisons in their
how much you would have to
ratings between those who disenroll and those who do not.
pay for a prescription
Experience with the PDP item set in Medicare CAHPS shows the
medicine?
time wording performs as intended and exhibits good psychometric
properties.
PDP9/MA-PD9 - Did you ever No revisions We appreciate the comment. We believe that the item as worded
need written information from made to
gets at whether the beneficiary needed information from the plan in
the plan in a language other
item
a language other than English. This item is a "gate" item for the
than English?
subsequent question that asks whether the plan provided the
beneficiary with information in a different language.
PDP10/MA-PD10 - How often
did the plan give you written
information in a language
other than English?

We have found from our previous survey work that a small number
of beneficiaries do not follow the appropriate skip logic. Thus we
provide what is in effect a "not applicable" response.

24