Download:
pdf |
pdfJuly 23, 2010
Cognitive Testing of the Medicare
Health Outcomes Survey
Final Report
Prepared for
Janet Holzman
National Committee for Quality Assurance
1100 13th Street, NW
Washington, DC 20005
Prepared by
Claudia Squire
Cindy Soloe
Peyton Williams
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
RTI Project Number 0212426.000.002
RTI Project Number
0212426.000.002
Cognitive Testing of the Medicare
Health Outcomes Survey
Final Report
July 23, 2010
Prepared for
Janet Holzman
National Committee for Quality Assurance
1100 13th Street, NW
Washington, DC 20005
Prepared by
Claudia Squire
Cindy Soloe
Peyton Williams
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
_________________________________
RTI International is a trade name of Research Triangle Institute.
CONTENTS
Section
Page
1.
Introduction................................................................................................................... 1-1
2.
Round 1 ......................................................................................................................... 2-1
2.1 Research Design................................................................................................... 2-1
2.2 Recruitment Procedures and Eligibility Requirements ........................................ 2-1
2.3 Data Collection .................................................................................................... 2-1
2.4 Cognitive Interviews ............................................................................................ 2-1
2.4.1 Respondent Characteristics ...................................................................... 2-1
2.4.2 Results ...................................................................................................... 2-2
3.
Round 2 ......................................................................................................................... 3-1
3.1 Research Design................................................................................................... 3-1
3.2 Recruitment Procedures and Eligibility Requirements ........................................ 3-1
3.3 Data Collection .................................................................................................... 3-2
3.4 Cognitive Interviews ............................................................................................ 3-2
3.4.1 Respondent Characteristics ...................................................................... 3-2
3.4.2 Results ...................................................................................................... 3-3
Appendixes
A
Round 1 Recruitment Screener ........................................................................... A-1
B
Round 1 Discussion Guide...................................................................................B-1
C
Round 2 Recruitment Screener ............................................................................C-1
D
Round 2 Discussion Guide.................................................................................. D-1
iii
LIST OF TABLES
Number
Page
2-1. Interview Respondent Characteristics: Round 1 ........................................................... 2-2
3-1. Interview Respondent Characteristics: Round 2 ........................................................... 3-3
iv
SECTION 1
INTRODUCTION
This report includes the results from Rounds 1 and 2 of the Cognitive Testing for the
Medicare Health Outcomes Survey (HOS). In the first round, conducted in March 2010, we
tested the Medicare HOS to assess whether respondents were able to understand and answer
survey questions. In the second round, conducted in May 2010, we tested the series of mailings
that are sent to respondents about the survey. RTI International is conducting the study for the
National Committee for Quality Assurance (NCQA).
1-1
SECTION 2
ROUND 1
This section summarizes findings from Round 1 of the Cognitive Testing for the
Medicare Health Outcomes Survey (HOS). The research design for Round 1 consists of testing
the English version of the Medicare HOS (during the previous contract year, we tested the
Spanish and Chinese versions of the HOS). This section presents findings from the first round of
cognitive interviews conducted in March 2010. A second round of testing took place in May
2010; results from Round 2 are presented in Section 3.
2.1
Research Design
We conducted nine cognitive interviews with adults aged 65 or older to test their
understanding of the survey items. The interviews focused on respondents’ understanding of the
questions and their ability to choose an appropriate answer category.
2.2
Recruitment Procedures and Eligibility Requirements
Respondents were recruited by First in Focus Research, Inc., in Raleigh, North Carolina.
To be eligible for the study, respondents had to be age 65 or older. Specifications were given to
the recruiter to achieve diversity in gender, education, and region (respondents were asked what
state[s] they lived in before moving to North Carolina). A copy of the Round 1 recruitment
screener is included in Appendix A.
2.3
Data Collection
All interviews were conducted at RTI’s offices in Research Triangle Park, North
Carolina. A trained moderator conducted the interviews using a semistructured interview guide
developed by RTI in consultation with NCQA and the Centers for Medicare & Medicaid
Services (CMS) (see Appendix B). Each interview lasted approximately 90 minutes. After the
interview, respondents were provided with an honorarium of $75.
2.4
Cognitive Interviews
2.4.1
Respondent Characteristics
A total of nine respondents were recruited for Round 1 interviews. Table 2-1 shows the
specific breakdown of the respondents’ characteristics. Respondents ranged in age from 65 to 87.
The majority of respondents had less than a college degree (four were high school graduates, and
2-1
Table 2-1. Interview Respondent Characteristics: Round 1
Characteristic
Number
Gender
Male
5
Female
4
Education
Less than high school
0
High school graduate
4
Some college/post high school education
4
College graduate of higher
1
Age
65–74
5
75–80
3
81 and over
1
Race
American Indian or Alaska Native
0
Asian
0
Black or African American
1
Native Hawaiian or Other Pacific Islander
0
White
8
Other
0
four completed some college or post high school training). Respondents were recruited to
achieve diversity in state/regional background. All but one reported living in a state other than
North Carolina for a substantial portion of their life. States mentioned by respondents include
California, Connecticut, Georgia, Minnesota, New Jersey, New York, Pennsylvania, and
Washington.
2.4.2
Results
2.4.2.1 General Issues
Time Periods. For the most part, respondents noticed the changing time periods and
attempted to answer questions accordingly. However, a few respondents ignored time periods
when they felt there was something different about that time period that would affect their
answer. For example, when answering Q35 (In the past 4 weeks, how often has low back pain
2-2
interfered with your usual daily activity), one respondent said he normally does not have low
back pain, but had hurt his back moving furniture a few weeks ago. Since he did not view the
past 4 weeks as typical, he decided to answer the question based on his typical experience of not
having back pain. Another respondent, who was recovering from knee surgery, excluded
complications from his surgery when answering questions about the past 4 weeks because he did
not think it would provide an accurate picture of his usual state.
Skip Pattern Compliance. Following skip patterns was difficult for most of the
respondents. Most had trouble following at least some of the skip instructions. The way skip
instructions were presented for Q33 (with words) seemed to be more confusing for respondents
than how they were presented for Q42 through Q46 (arrows). These findings are similar to what
we found during testing of the Spanish and Chinese versions of the survey.
2.4.2.2 Item-Specific Findings
In this section, we discuss survey items that were problematic and our suggested
revisions.
Q2.
The following items are about activities you might do during a typical day. Does your
health now limit you in these activities? If so, how much? a. Moderate activities, such
as moving a table, pushing a vacuum cleaner, bowling, or playing golf; b. Climbing
several flights of stair [Yes, limited a lot; Yes, limited a little; No, no limited at all]
Findings for Question 2a
All respondents were able to answer this question without difficulty. Respondents
thought that the examples of moderate activities worked well even if they did not
personally participate in the example activities. As one respondent explained, “I don’t
do housework anymore or bowl or play golf, but I think they are good examples. I
think it’s good to include something like vacuuming that is an everyday type of
activity and something that is more of a sport like golf.”
When asked about other examples of moderate activities, a few respondents
suggested walking (one respondent suggested walking a pet). Other suggestions
included doing laundry, making beds, and practicing yoga.
Recommendation for Question 2a
If adding another example activity is desirable, “walking at a moderate pace” would
be a good option to test. However, because the question worked well with the current
set of examples, we do not think it is necessary to make this change.
2-3
Finding for Question 2b
Although respondents were able to answer this question without difficulty, definitions
of “several flights of stairs” varied. Most respondents defined “several” as being
between two and five flights of stairs. However, one respondent said “seven or eight”
flights and another said up to fifteen. These respondents seemed to be thinking about
actual steps/stairs as opposed to flights of stairs.
Recommendations for Question 2b
Q3.
No changes are recommended at this time.
During the past 4 weeks, have you had any of the following problems with your work or
other regular daily activities as a result of your physical health? a. Accomplished less
than you would like; b. Were limited in the kind of work or other activities. [No, none of
the time; Yes, a little of the time; Yes, some of the time; Yes, most of the time; Yes, all
of the time]
Findings for Question 3, 3a, and 3b
Five of the nine respondents had some difficulty understanding and answering these
questions. Two respondents thought that Q3a and Q3b were asking the same thing.
Other respondents thought the questions were confusing but were unable to articulate
a specific reason. (All but one respondent chose “No, none of the time” for Q3a and
Q3b. One respondent chose “Yes, a little of the time” for both because he said he
could do more things when he was younger.)
Respondents understood what “work or other regular daily activities” meant and were
able to provide examples:
– “Work means a regular paid job, but if you’re retired it can mean housework,
taking care of bills, etc.”
– “For work I think of an office job, regular activities can be anything that you do
every day.”
Recommendation for Question 3, 3a, and 3b
No changes are recommended at this time.
2-4
Q4.
During the past 4 weeks, have you had any of the following problems with your work or
other regular daily activities as a result of any emotional problems (such as feeling
depressed or anxious)? a. Accomplished less than you would like; b. Didn’t do work or
other activities as carefully as usual. [No, none of the time; Yes, a little of the time; Yes,
some of the time; Yes, most of the time; Yes, all of the time]
Finding for Question 4, 4a, and 4b
Similar to Q3, several respondents either had difficulty understanding these questions
or explaining the difference between Q4a and Q4b. However, all but one respondent
quickly identified the difference between Q3 and Q4 (physical problems vs.
emotional problems). All were able to define the term “emotional problems.” (All
respondents chose “No, none of the time” for Q4a and Q4b.)
Recommendation for Question 4, 4a, and 4b
Q5.
No changes are recommended at this time.
During the past 4 weeks, how much did pain interfere with your normal work (including
both work outside the home and housework)? [Not at all, A little bit, Moderately, Quite a
bit, Extremely]
Finding for Question 5
Although respondents understood the intent of this question, some noted that the term
“work” sounds like a paid job. One respondent suggested using the term “activities”
instead.
Recommendation for Question 5
Q6.
Consider replacing “normal work (including both work outside the home and
housework” with “work or other regular activities.” This phrase tested well in earlier
questions. In addition, using the same language across questions when possible is
recommended for consistency.
How much of the time during the past 4 weeks: a. Have you felt calm and peaceful? b.
Did you have a lot of energy? c. Have you felt downhearted and blue? [All of the time,
Most of the time, A good bit of the time, Some of the time, A little of the time, None of
the time]
Finding for Question 6, 6a, and 6b
No problems noted. All respondents understood this question and were able to answer
appropriately.
2-5
Q7.
During the past 4 weeks, how much of the time has your physical health or emotional
problems interfered with your social activities (like visiting with friends, relatives, etc.)?
[All of the time, Most of the time, Some of the time, A little of the time, None of the
time]
Finding for Question 7
Although respondents understood and were able to answer this question without
difficulty, some were uncomfortable with the term/talking about “emotional
problems.”
Recommendation for Question 7
Q8.
Consider replacing the term “emotional problems” with “emotional health” to see if
respondents have a better reaction to it (this would also make it consistent with the
term “physical health”).
Compared to one year ago, how would you rate your physical health in general now?
[Much better, Slightly better, About the same, Slightly worse, Much worse]
Finding for Question 8
Q9.
No problems were noted. All respondents understood this question and were able to
answer appropriately.
Compared to one year ago, how would you rate your emotional problems (such as
feeling anxious, depressed, or irritable) in general now? [Much better, Slightly better,
About the same, Slightly worse, Much worse]
Finding for Question 9
As with Q8, some respondents disliked the term “emotional problems.” In addition,
one respondent noted that the question wording and answer categories imply that the
respondent has emotional problems.
Recommendation for Question 9
Consider one of the following options:
1. Replace the term “emotional problems” with “emotional health.”
2. Add a “Not Applicable” answer category for those who do not think they have any
emotional problems.
2-6
Q10. Because of a health or physical problem, do you have any difficulty doing the following
activities without special equipment or help from another person? a. Bathing;
b. Dressing; c. Eating; d. Getting in or out of chairs; e. Walking; f. Using the toilet [No, I
do not have difficulty; Yes, I have difficulty; I am unable to do this activity]
Findings for Question 10a–f
Respondents understood this question and were able to provide examples of special
equipment, including adaptations for showers like bath seats and handle bars, a cane,
and reach bars.
Several respondents suggested adding an answer category such as “Yes, I have some
difficulty” between “No, I do not have difficulty” and “Yes, I have difficulty.” One of
these respondents said that since he does not have difficulty with any of the activities
asked about, he would not have chosen “some difficulty” but thought that such an
answer would allow for “leeway.” Three other respondents said they would have
chosen such an answer if it were available for at least one of the items (one
respondent did not indicate which one[s] she would have answered differently, one
respondent said she would have chosen “some difficulty” for 10d [getting in or out of
chairs] and 10e [walking], and the third respondent said he would have chosen “some
difficulty” for 10a [bathing] and 10e [walking]). When asked to answer this set of
questions using the existing answer choices, all three chose “No, I do not have
difficulty” for each one.
Recommendation for Question 10a–f
Consider adding “Yes, I have some difficulty” as an answer category.
2-7
These next questions ask about your physical and mental health during the past 30 days.
Q11. Now, thinking about your physical health, which includes physical illness and injury,
for how many days during the past 30 days was your physical health not good?
Please enter a number between “0” and “30” days. If no days, please enter “0” days.
days
Q12. Now, thinking about your mental health, which includes stress, depression, and
problems with emotions, for how many days during the past 30 days was your mental
health not good?
Please enter a number between “0” and “30” days. If no days, please enter “0” days.
days
Q13. During the past 30 days, for about how many days did poor physical or mental health
keep you from doing your usual activities, such as self-care, work, or recreation?
Please enter a number between “0” and “30” days. If no days, please enter “0” days.
days
Findings for Questions 11, 12, and 13
The revised format (separating the instruction on how to record an answer from the
question and providing boxes to record the number of days) seemed to work well.
None of the respondents had difficulty recording their answers.
Although most respondents understood this set of questions without difficulty, one
respondent was not sure how to define “physical illness” (Q11). He was not sure if
this included only major conditions or temporary illnesses such as a cold. The same
respondent was not sure how to define “not good” in Q12. He questioned whether a
fight with his wife should be considered a part of mental health (this respondent said
“0” days for both questions). Another respondent noted that Q12 “sounded
awkward.”
Recommendation for Questions 11, 12, and 13
No changes are recommended at this time.
Q14. During the past 4 weeks, how often have you had any of the following problems?
a. Chest pain or pressure when you exercise; b. Chest pain or pressure when resting [All
of the time, Most of the time, Some of the time, A little of the time, None of the time]
Finding for Question 14a–b
No problems were noted.
2-8
Q15. During the past 4 weeks, how often have you felt short of breath under the following
conditions? a. When lying down flat; b. When sitting or resting; c. When walking less
than one block; d. When climbing one flight of stairs [All of the time, Most of the time,
Some of the time, A little of the time, None of the time]
Finding for Question 15a–d
One respondent, who uses a wheelchair, was not sure how to answer Q15c (walking
less than one block) and Q15d (climbing one flight of stairs) because he is unable to
do either. He asked whether he should say “All of the time” or “None of the time”
(this respondent left the question blank).
Recommendation for Question 15
Depending on population estimates for the number of potential respondents who are
in a wheelchair or cannot walk for some other reason, we suggest considering adding
a “Not Applicable” response option.
Q16. During the past 4 weeks, how much of the time have you had any of the following
problems with your legs and feet? a. Numbness or loss of feeling in your feet; b.
Tingling or burning sensation in your feet especially at night; c. Decreased ability to feel
hot or cold with your feet; d. Sores or wounds on your feet that did not heal [All of the
time, Most of the time, Some of the time, A little of the time, None of the time]
Finding for Question 16a–d
No problems were noted.
Q17. During the past 4 weeks, how would you describe any arthritis pain you usually had?
[None, Very mild, Mild, Moderate, Severe]
Findings for Question 17
All respondents understood what this question was asking, although one respondent
had some difficulty transitioning from previous questions that asked about the
duration of different issues to this question that asks about intensity. He also was not
sure how to answer the question because he had differing levels of pain in different
body parts (this respondent chose “Moderate” because it seemed like the “average”
pain he experienced).
One respondent thought that the question wording implies that someone has arthritis.
Because he does not have arthritis, he was not sure how to answer (he ended up
choosing “None”).
Recommendation for Question 17
No changes are recommended at this time.
2-9
Q18. Can you see well enough to read newspaper print (with your glasses or contacts if that’s
how you see best)? [Yes, No]
Finding for Question 18
One respondent noted this question might be confusing to someone who wears
glasses. However, we did not find this to be the case. All respondents, including three
who wore glasses, were able to answer this question without difficulty.
Recommendation for Question 18
No changes are recommended at this time.
Q19. Can you hear most of the things people say (with a hearing aid if that’s how you hear
best)? [Yes, No]
Q19. (Alternate version) Can you hear most of the things people say, regardless of whether
you need a hearing aid or not? [Yes, No]
Findings for Question 19
All respondents were able to understand Q19 (original wording) without difficulty.
Five respondents found the alternate wording for Q19 confusing. The word
“regardless” seemed to be what confused some people (noted by three respondents).
When asked which version they preferred, most said they liked the original wording
better: “The first one is clearer and shorter. It gets to the point.” However, two
respondents said that they preferred the alternate wording.
Recommendation for Question 19
No changes are recommended at this time.
2-10
Has a doctor ever told you that you had:
Yes
No
Q20. Hypertension or high blood pressure
Q21. Angina pectoris or coronary artery disease
Q22. Congestive heart failure
Q23. A myocardial infarction or heart attack
Q24. Other heart conditions, such as problems with heart valves or the
rhythm of your heartbeat
Q25. A stroke
Q26. Emphysema, or asthma, or COPD (chronic obstructive pulmonary
disease)
Q27. Crohn’s disease, ulcerative colitis, or inflammatory bowel disease
Q28. Arthritis of the hip or knee
Q29. Arthritis of the hand or wrist
Q30. Osteoporosis, sometimes called thin or brittle bones
Q31. Sciatica (pain or numbness that travels down your leg to below
your knee)
Q32. Diabetes, high blood sugar, or sugar in the urine
Finding for Questions 20 through 32
All respondents understood these questions and were able to answer without
difficulty. However, two respondents answered at least some of the questions based
on their own diagnosis and not that of a doctor.
Recommendation for Questions 20 through 32
Although this was not a significant problem, you may want to consider bolding the
word “doctor” for emphasis.
Q33. Has a doctor ever told you that you had: Any cancer (other than skin cancer) [Yes, No]
Finding for Question 33
No problems were noted.
2-11
If you answered “yes” to Question 33 above (that you have had cancer),
Q34. Are you currently under treatment for: a. Colon or rectal cancer; b. Lung cancer;
c. Breast cancer; d. Prostate cancer [Yes, No]
Findings for Question 34a–d
Most respondents were able to answer this set of questions without difficulty.
However, one respondent, who was cancer free but receiving follow-up treatment,
was unsure whether such follow-up care should be included (this respondent decided
not to include follow-up treatment in her answer and said “No”).
Two respondents had difficulty with the skip instruction before Q34. Other
respondents were able to follow it successfully but had to reread the instructions at
least once.
Recommendations for Question 34a–d
If the intent of this question is to include both treatment and follow-up care, consider
asking about “treatment or follow-up care.”
One respondent suggested modifying the skip instructions so that respondents are
directed where to go if they say “yes” or “no.” For example:
If you answered “yes” to Q33 Æ Go to Q34
If you answered “no” to Q33 Æ Go to Q35
We think this is reasonable to consider, although we advise more testing.
Q35. In the past 4 weeks, how often has low back pain interfered with your usual daily
activities (work, school or housework)? [All of the time, Most of the time, Some of the
time, A little of the time, None of the time]
Finding for Question 35
All respondents understood this question without difficulty, although one respondent
suggested adding “etc” after the examples to indicate that the question is asking about
all daily activities and not just the ones listed.
Recommendation for Question 35
Although this is not a significant issue, you may want to add “for example” to the
beginning of the list of examples or “etc.” at the end.
2-12
Q36. In the past year, have you had 2 weeks or more during which you felt sad, blue, or
depressed; or when you lost interest or pleasure in things that you usually cared about or
enjoyed? [Yes, No]
Finding for Question 36
One respondent was not sure whether the question was asking about 2 consecutive
weeks or a total of 2 weeks throughout the year (the respondent answered “No” and
said that would have been his answer under each interpretation).
Recommendation for Question 36
No changes are recommended at this time.
Q37. In the past year, have you felt depressed or sad much of the time? [Yes, No]
Finding for Question 37
No problems were noted.
Q38. Have you ever had 2 years or more in your life when you felt depressed or sad most
days, even if you felt okay sometimes? [Yes, No]
Finding for Question 38
No problems were noted.
Q39. How much of the time in the past week did you feel depressed? [Less than one day, One
or two days, Three or four days, More than four days]
Finding for Question 39
Although all respondents understood this question, three had trouble choosing an
answer because they said they were never depressed and did not feel like “less than
one day” fit their situation (one respondent chose “Less than one day,” while the
other two did choose an answer).
Recommendation for Question 39
Consider adding a category for “Zero days or less than one day.”
Q40. In general, compared to other people your age, would you say that your health is:
[Excellent, Very good, Good, Fair, Poor]
Finding and Recommendations for Question 40
All respondents understood this question and were able to answer appropriately.
2-13
Q41. Do you now smoke every day, some days, or not at all? [Every day, Some days, Not at
all, Don’t know]
Finding for Question 41
No problems were noted.
Q42. Many people experience problems with urinary incontinence, the leakage of urine. In the
past 6 months, have you accidentally leaked urine? [Yes, No]
Finding for Question 42
Most respondents understood the term “urinary incontinence” as intended (that urine
leakage can be in any amount and episodic). However, two respondents interpreted it
to mean having no control at any time.
Recommendation for Question 42
No changes are recommended at this time.
Q45. There are many ways to treat urinary incontinence including bladder training, exercises,
medication and surgery. Have you received these or any other treatments for your
current urine leakage problem? [Yes, No]
Finding for Question 45
Although some respondents had not heard the term “bladder training” before the
interview, all were able to understand the question and answer appropriately.
Recommendation for Question 45
No changes are recommended at this time.
Q46. In the past 12 months, did you talk with a doctor or other health provider about your
level of exercise or physical activity? For example, a doctor or other health provider may
ask if you exercise regularly or take part in physical exercise. [Yes, No, I had no visits in
the past 12 months]
Findings for Question 46
All respondents understood this question and were able to answer appropriately.
Definitions of “physical activity” ranged from just moving around to formal exercise
programs:
– Not sitting on the couch and watching TV
– Going for a walk
– Riding a bike
– More than sitting and watching TV—getting up and doing something
2-14
– Doing cardio
– Taking an exercise class
Recommendation for Question 46
No changes are recommended at this time.
Q47. In the past 12 months, did a doctor or other health provider advise you to start, increase,
or maintain your level of exercise or physical activity? For example, in order to improve
your health, your doctor or other health provider may advise you to start taking the stairs,
increase walking from 10 to 20 minutes every day, or to maintain your current exercise
program. [Yes, No]
Finding for Question 47
All but one respondent understood this question as intended. The one respondent who
did not thought this question was the same as Q46, but “for people who were told to
increase their physical activity due to weight problems.”
Recommendation for Question 47
No changes are recommended at this time.
Q48. A fall is when your body goes to the ground without being pushed. In the past 12
months, did you talk with your doctor or other health provider about falling or problems
with balance or walking? [Yes, No, I had no visits in the past 12 months]
Finding for Question 48
All respondents understood this question and were able to answer appropriately.
Q49. Did you fall in the past 12 months? [Yes, No]
No problems were noted.
Q50. In the past 12 months, have you had a problem with balance or walking? [Yes, No]
Finding for Question 50
No problems were noted.
2-15
Q51. Has your doctor or other health provider done anything to help prevent falls or treat
problems with balance or walking? Some things they might do include: Suggest that you
use a cane or walker. Check your blood pressure lying or standing. Suggest that you do
an exercise or physical therapy program. Suggest a vision or hearing testing. [Yes, No, I
had no visits in the past 12 months]
Findings for Question 51
All respondents understood this question and were able to answer appropriately.
Respondents were asked to review the list of examples. All thought they covered the
main things a provider would suggest. One respondent also suggested wearing “good”
shoes for walking and walking slowly.
One respondent thought some people might be embarrassed or defensive about
falling. He thought it was important to distinguish between an accidental fall and
falling because one is infirm.
Recommendation for Question 51
No changes are recommended at this time.
Q52. Have you ever had a bone density test to check for osteoporosis, sometimes thought of
as “brittle bones”? This test may have been done to your back, hip, wrist, heel or finger.
[Yes, No]
Finding for Question 52
No problems were noted. All respondents had heard the terms “bone density test” and
“osteoporosis” before the interview.
Q53. How much do you weigh in pounds (lbs.)? [90 lbs. or less, 91–100 lbs., 101–110 lbs.,
111–120 lbs., 121–130 lbs., 131–140 lbs., 141–150 lbs., 151–160 lbs., 161–170 lbs.,
171–180 lbs., 181–190 lbs., 191–200 lbs., 201–210 lbs., 211–220 lbs., 221–230 lbs.,
231–240 lbs., 241–250 lbs., 251–260 lbs., 261–270 lbs., 271–280 lbs., 281–290 lbs.,
291–300 lbs., 301–310 lbs., 311–320 lbs., 321 lbs. or more]
Finding for Question 53
No problems were noted.
Q54. How tall are you without shoes on in feet (ft.) and inches (in.)? (If 1/2 in., please round
up.) [5 ft. 00 in. or less, 5 ft. 01 in., 5 ft. 02 in., 5 ft. 03 in., 5 ft. 04 in., 5 ft. 05 in., 5 ft. 06
in., 5 ft. 07 in., 5 ft. 08 in., 5 ft. 09 in., 5 ft. 10 in., 5 ft. 11 in., 6 ft. 00 in., 6 ft. 01 in., 6 ft.
02 in., 6 ft. 03 in. or more]
Finding for Question 54
No problems were noted.
2-16
Q55. In what year were you born? Please provide your year of birth only. [Year]
Finding and Recommendation for Question 55
Although none of the respondents had difficulty with this item, we suggest
considering the addition of 19 in the first two boxes:
1
9
Q56. Are you male or female? [Male, Female]
Finding for Question 56
No problems were noted.
Q57. Are you of Hispanic or Latino origin or descent? [Yes, Hispanic or Latino; No, not
Hispanic or Latino]
Finding for Question 57
No problems were noted.
Q58. How would you describe your race? Please mark one or more. [American Indian or
Alaska Native, Asian, Black or African American, Native Hawaiian or Other Pacific
Islander, White, Another race]
Finding for Question 58
No problems were noted.
Q59. What is your current marital status? [Married, Divorced, Separated, Widowed, Never
married]
Finding for Question 59
No problems were noted.
Q60. What is the highest grade or level of school that you have completed? [8th grade or less;
Some high school, but did not graduate; High school graduate or GED; Some college or
2 year degree; 4 year college graduate; More than a 4 year college degree]
Finding for Question 60
No problems were noted.
2-17
Q61. Is the house or apartment you currently live in: [Owned or being bought by you, Owned
or being bought by someone in your family other than you, Rented for money, Not
owned and one in which you live without payment of rent, None of the above]
Finding for Question 61
No problems were noted.
Q62. Who completed this survey form? [Person to whom survey was addressed, Family
member or relative of person to whom the survey was addressed, Friend of person to
whom the survey was addressed, Professional caregiver of person to whom the survey
was addressed]
Finding for Question 62
Respondents were asked whether they preferred the current option “Person to whom
survey was addressed” or “Person selected for the survey.” Six of the nine
respondents preferred the original wording (“Person to whom the survey was
addressed”). All thought the answers meant the same thing.
Recommendation for Question 62
No changes are recommended at this time.
2-18
SECTION 3
ROUND 2
This section summarizes findings from Round 2 of the Cognitive Testing for the
Medicare Health Outcomes Survey (HOS). The research design for this round consists of testing
mailing materials for the baseline and follow-up HOS. This section presents findings from the
second round of cognitive interviews conducted in May 2010.
3.1
Research Design
We conducted nine cognitive interviews with adults aged 65 or older to test respondents’
understanding of the mailings survey participants receive about the HOS. The following
materials were tested:
Baseline Survey Mailings
Prenotification post card
Cover letter for first survey mailing
Reminder/thank you post card
Cover letter for second survey mailing
Follow-Up Survey Mailings
3.2
Prenotification post card
Cover letter for first survey mailing
Reminder/thank you post card (this is the same as the baseline reminder/thank you
card, so it was not tested with the follow-up materials)
Cover letter for second survey mailing
Recruitment Procedures and Eligibility Requirements
Respondents were recruited by First in Focus Research, Inc., in Raleigh, North Carolina.
To be eligible for the study, respondents had to be age 65 or older. Specifications were given to
the recruiter to achieve diversity in gender, education, race, and region (respondents were asked
what state[s] they lived in before moving to North Carolina). See Appendix C for a copy of the
Round 2 recruitment screener.
3-1
3.3
Data Collection
All interviews were conducted at RTI’s offices in Research Triangle Park, North
Carolina. A trained moderator conducted the interviews using a semistructured interview guide
developed by RTI in consultation with NCQA and the Centers for Medicare & Medicaid
Services (CMS) (see Appendix D for a copy of the Round 2 interview guide). Each interview
lasted approximately 90 minutes. After the interview, respondents were provided with an
honorarium of $75.
3.4
Cognitive Interviews
3.4.1
Respondent Characteristics
A total of nine respondents were recruited for Round 2 interviews. Table 3-1 shows the
specific breakdown of the respondents’ characteristics. Respondents ranged in age from 66 to 78.
All of the respondents had less than a college degree (five were high school graduates, and four
completed some college or post high school training). Respondents were recruited to achieve
diversity in state/regional background. All reported living in at least one other state for a
substantial portion of their life before moving to North Carolina. States mentioned by
respondents include California, Florida, Massachusetts, New Hampshire, New Jersey, Virginia,
Washington, and Wisconsin.
3-2
Table 3-1. Interview Respondent Characteristics: Round 2
Characteristic
Number
Gender
Male
3
Female
6
Education
Less than high school
0
High school graduate
5
Some college/post high school education
4
College graduate of higher
0
Age
65–74
6
75–80
3
81 and over
0
Race
American Indian or Alaska Native
1
Asian
1
Black or African American
4
Native Hawaiian or Other Pacific Islander
0
White
3
Other
0
3.4.2
Results
3.4.2.1 General Issues
Overall respondents understood the main points of each mailing and were able to list
them in their own words.
In most cases, respondents thought the mailing materials included the right amount of
information (in some cases, respondents suggested adding information to some of the
mailings, as detailed in Section 3.4.2.2). However, in real-life situations, people may
not take the time to read the material as closely.
The terms “survey” and “questionnaire” are used interchangeably throughout the
mailings. You may want to consider using one term throughout for consistency and to
avoid confusion.
Our editorial staff reviewed all materials and noted that there are some minor editorial
changes that can be made to improve the readability (e.g., breaking longer sentences
3-3
into multiple sentences, using less formal language). If this is of interest, they can do
a more thorough review and provide a revised set of the materials.
3.4.2.2 Item-Specific Findings
In this section, we discuss findings and our suggested revisions for each of the items
tested.
Baseline Materials
Prenotification Post Card
Findings
All of the respondents were able to describe the main points of the post card. Answers
included the following:
– The mailing is being sent by the Federal government.
– They were selected at random for a survey they would receive in a few days.
– They were selected because they have Medicare.
– The survey is about the health care that people with Medicare receive (two
respondents assumed the survey would ask about their satisfaction with Medicare
as well).
A few respondents said they would have liked more information on what the survey is
about. Although all respondents understood that the survey is about their health care,
some thought the goal of the survey is to improve Medicare, whereas others thought
the goal of the survey is to improve people’s health.
When asked if there was anything that they thought should be included in the post
card that was not, respondents offered the following suggestions:
– A telephone number to call for more information
– The number of questions on the survey
– A timeline for responding to the survey
– A statement saying that participation will not affect Medicare benefits
When asked if there were changes that could be made to the design or format of the
post card to make it easier to read, a couple of respondents suggested using larger or
darker font.
Recommendation
You may want to consider adding a telephone number for people to call if they have
any immediate questions.
3-4
Cover Letter for First Survey Mailing
Findings
All of the respondents were able to describe the main points covered in the letter.
Answers included the following:
– The letter is from a Federal agency.
– A survey is included.
– The recipient was selected by random (a couple of respondents wanted more
information on the selection process).
– Completing the survey is voluntary, and participation will not affect Medicare
benefits (one respondent said that not participating would not mean Medicare
costs would go up).
– Another survey will be sent in 2 years.
– The results of the survey will be used to improve health care.
Although respondents understood that the results of the survey would be used to
improve health care, there was some confusion on how that would be done. Some
respondents thought that things would be done to make them healthier, whereas
others thought the results would be used to help plans and doctors administer care or
to help get them in the “right” plan.
Several respondents were concerned about how their responses would be shared with
their health plan. Most thought this meant that their individual responses would be
given to their plan. One respondent thought that this also meant that her answers
would be shared with her provider.
A couple of respondents suggested providing a timeline for returning the survey.
Recommendations
Consider adding a sentence or two to clarify how survey results will be used to
improve health.
Consider adding a sentence asking respondents to return the survey within 1 week.
(After reading the reminder/thank you post card, a couple of respondents were
surprised that it would be sent after only 1 week. They said they would want to know
that they were expected to return the survey that quickly when it was received.)
Reminder/Thank You Post Card
Findings
All respondents understood that the point of the post card is to remind people to
complete and return the survey. Respondents also noted that the post card emphasized
the importance of the survey and included information on how to request another
copy.
3-5
Respondents provided a few suggestions for improving the post card:
– A couple of respondents had forgotten who Datastat was and suggested that their
role be defined (a third respondent remembered that Datastat was the organization
administering the survey, but thought it should be stated again so that there was
no confusion between CMS and Datastat).
– A couple of respondents suggested either adding color to the post card or
incorporating other design elements so that it would stand out.
– Some respondents suggested adding CMS’ logo and/or Walter Stone’s signature
to the post card so that recipients would immediately know the mailing was from
CMS (some respondents overlooked the CMS logo on the front of the post card).
Recommendations
Consider noting Datastat’s role as is done in the first cover letter.
Consider adding the Privacy Officer’s signature to the post card (respondents noted
that seeing “CMS Privacy Officer” on the other mailings made them feel like the
survey was “safe” and that someone was “overseeing things.”
Cover Letter for Second Survey Mailing
Findings
All of the respondents understood that this letter would be sent to people who had not
returned the initial survey that was sent. Respondents were able to list the main points
of the letter and noted they were the same or very similar to the information included
in the first cover letter. However, a few respondents noted that this letter referred to
“managed care plans,” whereas the first letter did not. One respondent found this
confusing.
As with the first letter, some respondents expressed concerns about how their
information would be shared. Several assumed that their individual answers were
going to be shared with their health plans or doctors.
The issue of how the survey data will be used to improve health or health care was
raised again. Respondents wanted to know how the survey results could improve their
health.
Recommendations
For consistency, either refer to Medicare managed care plans in both cover letters or
in neither letter.
Consider adding a sentence or two to clarify how survey results will be used to
improve health.
3-6
Follow-Up Materials
Prenotification Post Card
Findings
All of the respondents understood and were able to list the main points of the post
card. Answers included the following:
– The post card is about a survey that we filled out 2 years ago.
– The follow-up survey will be received in a few days.
– The survey is sponsored by Medicare.
– The survey is important—it will be used to improve the quality of care people on
Medicare receive.
A couple of respondents suggested including something in the title to note that this is
a follow-up survey.
Recommendations
None noted.
Cover Letter for First Survey Mailing
Findings
All respondents understood the intent of this letter and were able to list the main
points. Answers included the following:
– The letter is reminding people that they participated in a survey 2 years ago and
this is a follow-up.
– There is a postage-paid envelope that can be used to return the survey.
– The goal of the survey is to improve health.
– Answers will be shared with health plans.
– There is a phone number to call if needed.
Respondents offered a few suggestions for improving the letter:
– Two respondents noted that this letter did not include information about what
CMS is (referred to in the first sentence of the baseline cover letter). One
respondent explained that even if someone remembers what CMS is, it is
“reassuring to see a reminder that it is a government agency.”
– Add a phone number for Walter Stone.
– Clarify how information will be shared with health plans (again, respondents were
concerned that their individual answers would be provided to health plans).
3-7
– A few respondents said that it was not clear what the sentence “Your responses to
this follow-up survey will also help the Centers for Medicare & Medicaid
Services (CMS) to determine if the care you receive is keeping you as healthy as
possible” really means. As they noted previously, they wanted to know how the
results of the survey will be used to keep them “as healthy as possible.”
Recommendations
You may want to consider adding the first sentence from the baseline cover letter that
explains what CMS is.
Consider previous recommendations to add a sentence or two to clarify how results of
the survey will be used to improve health.
Cover Letter for Second Survey Mailing
Findings
All respondents understood the intent and main points of this mailing. Respondents
said that the letter included information similar to the first cover letter, but this letter
was for people who did not return the first survey that was sent.
Respondents expressed concerns about how their data would be shared with their
health plan. They also wanted clarification on how the survey results would be used
to improve their health.
Recommendations
Consider previous recommendations to add a sentence or two to clarify how results of
the survey will be used to improve health.
3-8
APPENDIX A:
ROUND 1 RECRUITMENT SCREENER
Medicare Health Outcomes Survey
Survey Testing
Recruitment Screener: Round 1
Introduction
Ask to speak to an adult male or female age 18 or older living in the household:
Hello, my name is _______________ and I’m from (name of company). We are calling
on behalf of RTI International, a nonprofit research organization.
We are not selling or promoting any product. We are calling to recruit people to take part
in a research study about a health survey. The purpose of the research is to get feedback
from people about the survey questions. The study is sponsored by the Centers for
Medicare & Medicaid Services (CMS), the Federal government agency that operates the
Medicare and Medicaid programs.
We will be conducting interviews with several people and we would like to conduct one
with you. We will be conducting interviews on [DATES]. We have various time slots
available and will work with you to find a time that fits your schedule. The interview will
last up to 1 ½ hours. In appreciation for your participation, you will be reimbursed for
your time, effort, and travel expenses.
To see if you are eligible for this study, I need to ask you a few questions. All of your
comments will be kept private. My questions will only take a few minutes. May I
proceed?
Yes – CONTINUE
No – Thank you and end call
1. RECORD GENDER (Ask if necessary)
1. Female
2. Male
[Attempt Mix]
2. How old were you on your last birthday? ______________
CODE AGE RANGE BELOW
1. 18–64 ÆTerminate
2. 65 or over
A-1
3. What is the highest level of education you have completed?
1. Less than high school
2. High school graduate
3. Some college or post-high school education
4. College (university) graduate or higher (See note below)
Note: No more than 3 Rs per round in category 4
4. How long have you lived in North Carolina?
_______________________________________
5.
[If Q4 - not whole life] Where did you live before moving to North Carolina?
_______________________________________
[Attempt mix from different states and regions]
Invitation:
Thank you for answering all of my questions. As I mentioned earlier, we are conducting a
research study, sponsored by CMS, to get feedback on a health survey. We would like to
invite you to take part in an informal, personal interview. The interviews are being
scheduled on [DAYS/DATE] at a time that would be convenient for you [GIVE
AVAILABLE TIMES]. The discussion will last about 90 minutes. No one will attempt to
sell you anything and no one will call on you for other studies as a result of your
participation in this study. To help repay you for your time, effort, and travel expenses,
you will receive $75 at the time of the interview. This is an important research effort and
we hope that you will be part of it. Can we schedule your attendance?
Facility to provide:
•
Directions/map
•
Interviewer name and phone number
Closing for Ineligible Participants:
Thank you for answering my questions. At this time you are not eligible to be in this
study because [R is either under age 65 or is in a category that is full]. We value your
interest in this research study. Thank you for your willingness to help us.
A-2
Information Sheet:
We are asking for your contact information only for the purpose of sending you a
reminder letter and giving you a call to remind you of your interview. We will destroy all
contact information upon conclusion of the interviews.
NAME: ___________________________________________________________
ADDRESS: _______________________________________________________
CITY: ____________________________________________________________
ZIP CODE: ________________________________________________________
EMAIL: __________________________________________________________
What is the best time to reach you? What is the best telephone number to reach you at
that time?
BEST TIME TO BE REACHED: ____________________________________________
BEST PHONE NUMBER: _________________________________________________
Is there another time to call or phone number that we may try if we miss you?
ALTERNATIVE TIME: ___________________________________________________
ALTERNATE PHONE NUMBER: __________________________________________
Your participation in this study is very important. If for some reason you will not be able
to attend, please let us know right away. You can call us anytime at [INSERT PHONE
NUMBER], and if we are not here, please leave a message.
Interviewer: _____________________________________________________________
Supervisor Confirm: _______________________________________________________
A-3
APPENDIX B:
ROUND 1 DISCUSSION GUIDE
Medicare Health Outcomes Survey
Questionnaire (English)
2009
(English)
Discussion Guide
B-1
Interviewer: Read/Paraphrase this Introduction
Hi, my name is ________. I want to thank you for coming today. Before we get started,
I’m going to give you some information about the study to read.
Give R the Informed Consent to read. Ask if R has any questions. Give R a copy of
the Informed Consent form and note phone numbers if they have follow-up questions.
Today we’re going to be talking about the Medicare Health Outcomes Survey (HOS).
The HOS is a large survey conducted by the Centers for Medicare & Medicaid Services (CMS),
the Federal agency that runs Medicare. The goal of the Medicare HOS program is to find out
more about the health care that is provided to people with Medicare. The survey includes
questions about your health and its impact on your daily activities. The results of the survey are
used by CMS to determine if the care people with Medicare receive is keeping them as healthy
as possible.
We are conducting this study on behalf of CMS to see how well some of the questions in
the Medicare HOS are working. Specifically, we want to know: Do the questions make sense?
Are they being understood as intended? How easy or difficult are they to understand and answer?
As we go through the survey, I’d like you to read each question out loud and then “think
out loud” as you answer each question. For example, if the question asked how many times you
cooked dinner at home during the past week, rather than tell me that you made dinner five times,
I want to know how you came up with that answer. It may feel a little unnatural to do this, but
remember, there are no right or wrong answers. We just want to understand how you decide on
your answer and what you think of the question.
Here’s an example:
Question: In the past month, how many days has pain interfered with your ability to sleep
well at night?
Answer 0 to 30 days
Response: Well, I usually don’t have pain, but I twisted my ankle a couple of week ago.
The first two nights I woke up during the night because of the pain. It still hurt for about a week
after that, but I’d say two, because it only affected sleep the first two nights.
B-2
Do you have any questions about the example?
After some of the survey questions, I’ll ask you some additional questions. You should
also feel free to say what you think about the questions at any time. We want to find the best way
to ask each question, and your comments will help us do that.
The most important thing I want you to know is that there are no right or wrong answers.
Your participation in this interview is very important because it will help improve the
questionnaire. When we are done, I will give you $75 for your time, effort, and travel expenses. I
will ask you to sign a receipt to document that you have received it.
B-3
MEDICARE HEALTH OUTCOMES SURVEY INSTRUCTIONS
This survey asks about you and your health. Answer each question thinking about
yourself. Please take the time to complete this survey. Your answers are very important
to us. If you are unable to complete this survey, a family member or “proxy” can fill out
the survey about you.
Please return the survey with your answers in the enclosed postage-paid envelope.
Sample Questions:
¾ Answer the questions by putting an ‘X’ in the box next to the appropriate answer
category like this:
56. Are you male or female?
1
2
Male
Female
¾ Be sure to read all the answer choices given, before marking a box with an ‘X.’
¾ You are sometimes told to answer some questions in this survey only when you
have answered a previous question. When this happens, you will see an italicized
instruction like the one below:
If you answered "yes" to question 33 above (that you have had cancer),
INTERVIEWER: Note whether R completes the survey according to the instructions.
All information that would permit identification of any person who completes this
survey will be kept strictly confidential. This information will be used only for the
purposes of this study and will not be disclosed or released for any other purposes
without your permission.
If you have any questions or want to know more about the study, please call [vendor
name] at [toll-free number].
Probe: What does the statement in the box mean to you?
Probe: Does the statement alleviate your concern about confidentiality?
OMB 0938-0701 Version 02-1
© 2009 by the National Committee for Quality Assurance (NCQA). This survey instrument may not be reproduced or
transmitted in any form, electronic or mechanical, without the express written permission of NCQA. All rights
reserved.
Items 1–9: The VR-12 Health Survey item content was developed and modified from a 36-item health survey.
B-4
MEDICARE HEALTH OUTCOMES SURVEY
1.
In general, would you say your health is:
Excellent
1
2.
Very good
Good
2
Fair
3
Poor
4
5
The following items are about activities you might do during a typical day. Does your
health now limit you in these activities? If so, how much?
Yes,
limited
a lot
ACTIVITIES
a.
b.
Moderate activities, such as moving a table,
pushing a vacuum cleaner, bowling, or playing golf ....
Climbing several flights of stairs..................................
Yes,
limited
a little
No, not
limited
at all
1
2
3
1
2
3
Probe: Please tell me how you decided on your answer to Q2a.
• What does “moderate activities” mean to you? What do you think about the examples
used? Do you bowl or play golf? Are there other examples of moderate activities that
you can think of?
• What does “several” flights of stairs mean (how many is several)?
• [IF YES TO QUESTION] How has your health limited your participation in moderate
activities?
Probe: In your opinion, what is the difference between “limited a lot” and “limited a little”?
3.
During the past 4 weeks, have you had any of the following problems with your work or
other regular daily activities as a result of your physical health?
No,
Yes,
Yes,
Yes,
Yes, all
none of a little of some of most of of the
the time the time the time the time time
a.
b.
Accomplished less than you would
like .......................................................
1
2
3
4
5
Were limited in the kind of work or
other activities ......................................
1
2
3
4
5
Probe: In your own words, please tell me what this question is asking (ask for a and b—probe
further if necessary to determine whether R understands the difference between a and
b)?
Probe: What does the term “work” mean to you in this context? [INTERVIEWER: probe to
determine if the term “work” is relevant to R’s situation]
B-5
• [IF YES] Please tell me about how you accomplished less/were limited in the kind of
work or other activities as a result of your physical health?
4.
During the past 4 weeks, have you had any of the following problems with your work or
other regular daily activities as a result of any emotional problems (such as feeling
depressed or anxious)?
No,
Yes,
Yes,
Yes,
Yes,
none of a little of some of most of all of the
the time the time the time the time time
a.
b.
Accomplished less than you would
like .......................................................
1
2
3
4
5
Didn't do work or other activities as
carefully as usual ................................
1
2
3
4
5
Probe: In your own words, please tell me what this question is asking?
• What is the difference between this question and Q3?
• What does “emotional problems” mean to you in this context?
5.
During the past 4 weeks, how much did pain interfere with your normal work (including
both work outside the home and housework)?
Not at all
1
A little bit
Moderately
2
Quite a bit
3
Extremely
4
5
These questions are about how you feel and how things have been with you during the past 4
weeks. For each question, please give the one answer that comes closest to the way you have
been feeling.
Probe: What did you have to think about in order to answer this question? Was it easy or
difficult for you to remember what happened during the past 4 weeks?
6.
How much of the time during the past 4 weeks:
All
of the
time
a.
Have you felt calm and
peaceful? ..................................
b.
Did you have a lot of energy? ...
c.
Have you felt downhearted
and blue? ..................................
Most A good
of the bit of the
time
time
Some
of the
time
A little
of the
time
None
of the
time
1
2
3
4
5
6
1
2
3
4
5
6
1
2
3
4
5
6
B-6
7.
During the past 4 weeks, how much of the time has your physical health or emotional
problems interfered with your social activities (like visiting with friends, relatives, etc.)?
All of
the time
1
Most of
the time
Some of
the time
2
3
A little of
the time
None of
the time
4
5
Now, we’d like to ask you some questions about how your health may have changed.
8.
Compared to one year ago, how would you rate your physical health in general now?
Much better
1
9.
Slightly better
About the same
2
3
Slightly worse
4
Much worse
5
Compared to one year ago, how would you rate your emotional problems (such as
feeling anxious, depressed, or irritable) in general now?
Much better
1
Slightly better
About the same
2
3
Slightly worse
4
Much worse
5
Probe: What did you have to think about in order to answer Q8 and Q9? (INTERVIEWER: Did R
compare health to one year ago)?
Earlier in the survey you were asked to indicate whether you have any limitations in your
activities. We are now going to ask a few additional questions in this area.
10. Because of a health or physical problem, do you have any difficulty doing the following
activities without special equipment or help from another person?
No, I do not
have difficulty
a.
Bathing ..............................................
b.
Dressing ............................................
c.
Eating ................................................
d.
Getting in or out of chairs ..................
e.
Walking .............................................
f.
Using the toilet ..................................
B-7
Yes, I have
difficulty
I am unable to
do this activity
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
1
2
3
Probe: In your own words, please tell me what this question is asking?
Probe: What does “special equipment” mean in this context? Can you provide an example of
special equipment that can be used to complete one of these activities?
Probe: Were you able to choose an answer category that “fit” your situation (Note in previous
testing some R’s wanted an answer b/w no “I do not have difficulty” and “I have
difficulty”)?
These next questions ask about your physical and mental health during the past 30 days.
11. Now, thinking about your physical health, which includes physical illness and injury, for how
many days during the past 30 days was your physical health not good?
Please enter a number between "0" and "30" days. If no days, please enter “0” days.
days
12. Now, thinking about your mental health, which includes stress, depression, and problems
with emotions, for how many days during the past 30 days was your mental health not
good?
Please enter a number between "0" and "30" days. If no days, please enter “0” days.
days
13. During the past 30 days, for about how many days did poor physical or mental health keep
you from doing your usual activities, such as self-care, work, or recreation?
Please enter a number between "0" and "30" days. If no days, please enter “0” days.
days
Probe: [IF R ANSWERED 1 TO 30 DAYS] How did you calculate your answer – e.g., did you
think of specific days, estimate the number of days, etc.?
Probe: [IF R ANSWERED 1 TO 30 DAYS] Did you have any difficulty remembering the specific
problems in the last 30 days?
B-8
Now we are going to ask some questions about specific medical conditions.
14. During the past 4 weeks, how often have you had any of the following problems?
All of
the time
a.
b.
Most of
the time
Some of
the time
A little of
the time
None of
the time
Chest pain or pressure when
you exercise ...........................
1
2
3
4
5
Chest pain or pressure when
resting ....................................
1
2
3
4
5
15. During the past 4 weeks, how often have you felt short of breath under the following
conditions?
All of
the time
a.
When lying down flat ..............
b.
When sitting or resting ...........
c.
d.
Most of
the time
Some of
the time
A little of
the time
None of
the time
1
2
3
4
5
1
2
3
4
5
When walking less than
one block ...............................
1
2
3
4
5
When climbing one flight
of stairs ..................................
1
2
3
4
5
16. During the past 4 weeks, how much of the time have you had any of the following
problems with your legs and feet?
All of
the time
a.
b.
c.
d.
Most of
the time
Some of
the time
A little of
the time
None of
the time
Numbness or loss of feeling
in your feet .............................
1
2
3
4
5
Tingling or burning sensation
in your feet especially at
night .......................................
1
2
3
4
5
Decreased ability to feel hot
or cold with your feet ..............
1
2
3
4
5
Sores or wounds on your feet
that did not heal .....................
1
2
3
4
5
B-9
17. During the past 4 weeks, how would you describe any arthritis pain you usually had?
None
1
Very mild
2
Mild
3
Moderate
Severe
4
5
Yes
No
18. Can you see well enough to read newspaper print (with your glasses or
contacts if that's how you see best)? .......................................................
1
2
19. Can you hear most of the things people say (with a hearing aid if that's
how you hear best)? ................................................................................
1
2
Probe Q18: In your own words, what does this question mean to you?
Probe Q19: In your own words, what does this question mean to you?
INTERVIEWER: To test alternate wording for Q19, show R paper with both versions of question
(cover current wording). Probe for understanding of alternate wording. Then ask R to look at
both versions and probe for preference.
Has a doctor ever told you that you had:
INTERVIEWER: Probe these terms as necessary
20. Hypertension or high blood pressure .......................................................
Yes
No
1
2
1
2
1
2
1
2
1
2
1
2
26. Emphysema, or asthma, or COPD (chronic obstructive pulmonary
disease) ...................................................................................................
1
2
27. Crohn’s disease, ulcerative colitis, or inflammatory bowel
disease.....................................................................................................
1
2
1
2
1
2
1
2
21. Angina pectoris or coronary artery disease .............................................
22. Congestive heart failure ...........................................................................
23. A myocardial infarction or heart attack.....................................................
24. Other heart conditions, such as problems with heart valves or the
rhythm of your heartbeat ..........................................................................
25. A stroke ....................................................................................................
28. Arthritis of the hip or knee ........................................................................
29. Arthritis of the hand or wrist .....................................................................
30. Osteoporosis, sometimes called thin or brittle bones ..............................
B-10
Has a doctor ever told you that you had:
INTERVIEWER: Probe these terms as necessary
Yes
31. Sciatica (pain or numbness that travels down your leg to below your
knee) ........................................................................................................
1
2
1
2
1
2
32. Diabetes, high blood sugar, or sugar in the urine ....................................
33. Any cancer (other than skin cancer) ........................................................
No
If you answered "yes" to question 33 above (that you have had cancer),
INTERVIEWER: Probe if R does not follow skip appropriately
34. Are you currently under treatment for:
INTERVIEWER: Probe these terms as necessary
Yes
a.
Colon or rectal cancer .......................................................................
b.
Lung cancer ......................................................................................
c.
Breast cancer ....................................................................................
d.
Prostate cancer .................................................................................
No
1
2
1
2
1
2
1
2
35. In the past 4 weeks, how often has low back pain interfered with your usual daily activities
(work, school or housework)?
All of
the time
1
Most of
the time
2
Some of
the time
3
A little of
the time
None of
the time
4
5
Yes
36. In the past year, have you had 2 weeks or more during which you felt
sad, blue or depressed; or when you lost interest or pleasure in things
that you usually cared about or enjoyed? ...................................................
37. In the past year, have you felt depressed or sad much of the time? ..........
38. Have you ever had 2 years or more in your life when you felt depressed
or sad most days, even if you felt okay sometimes? ..................................
B-11
No
1
2
1
2
1
2
39. How much of the time in the past week did you feel depressed?
Less than
one day
1
Three or
four days
One or two days
2
3
More than
four days
4
INTERVIEWER: Does changing time periods in Q35–Q39 cause confusion?
Probe: When answering questions 35–39, did you think about specific time periods, or did you
answer “in general.”? Did you notice the change in time periods between questions 36
and 39?
40. In general, compared to other people your age, would you say that your health is:
1
2
3
4
5
Excellent
Very good
Good
Fair
Poor
Probe: What did you have to think about in order to answer this question (who are you
comparing yourself to)?
41. Do you now smoke every day, some days, or not at all?
1
2
3
4
Every day
Some days
Not at all
Don’t know
42. Many people experience problems with urinary incontinence, the leakage of urine. In the
past 6 months, have you accidentally leaked urine?
1
2
Yes
ÎGo to Question 43
No
ÎGo to Question 46
Probe: In your own words, please tell me what this question is asking
Probe: What does the term “urinary incontinence” mean to you?
INTERVIEWER: Does R follow skip appropriately?
B-12
43. How much of a problem, if any, was the urine leakage for you?
1
2
3
A big problem
ÎGo to Question 44
A small problem
ÎGo to Question 44
Not a problem
ÎGo to Question 46
44. Have you talked with your current doctor or other health provider about your urine leakage
problem?
1
2
Yes
No
45. There are many ways to treat urinary incontinence including bladder training, exercises,
medication, and surgery. Have you received these or any other treatments for your current
urine leakage problem?
1
2
Yes
No
Probe: In your own words, please tell me what this question is asking.
Probe: What does the term “bladder training” mean to you in this question? Have you heard that
term before today?
46. In the past 12 months, did you talk with a doctor or other health provider about your level
of exercise or physical activity? For example, a doctor or other health provider may ask if
you exercise regularly or take part in physical exercise.
1
2
3
Yes
ÎGo to Question 47
No
ÎGo to Question 47
I had no visits in the past 12 months
ÎGo to Question 48
Probe: In your own words, please tell me what this question is asking.
Probe: What does “physical activity” mean to you in this question?
47. In the past 12 months, did a doctor or other health provider advise you to start, increase or
maintain your level of exercise or physical activity? For example, in order to improve your
health, your doctor or other health provider may advise you to start taking the stairs,
increase walking from 10 to 20 minutes every day or to maintain your current exercise
program.
1
2
Yes
No
B-13
Probe: In your own words, please tell me what this question is asking.
Probe: [IF YES] What did your doctor advise you to do?
48. A fall is when your body goes to the ground without being pushed. In the past 12 months,
did you talk with your doctor or other health provider about falling or problems with balance
or walking?
1
2
3
Yes
No
I had no visits in the past 12 months
49. Did you fall in the past 12 months?
1
2
Yes
No
INTERVIEWER: Note any problems with the order of Q48 and Q49 (do R’s that say no to Q48
think that Q49 is not relevant)?
50. In the past 12 months, have you had a problem with balance or walking?
1
2
Yes
No
Probe: In your own words, please tell me what this question is asking.
Probe: [IF YES] Please tell me about the problem you have had (was the problem with balance,
walking or both?).
51. Has your doctor or other health provider done anything to help prevent falls or treat
problems with balance or walking? Some things they might do include:
• Suggest that you use a cane or walker.
• Check your blood pressure lying or standing.
• Suggest that you do an exercise or physical therapy program.
• Suggest a vision or hearing testing.
1
2
3
Yes
No
I had no visits in the past 12 months
Probe: In your own words, please tell me what this question is asking.
B-14
Probe: Are there other things that a health provider might recommend to prevent falls or treat
problems with balance or walking?
52. Have you ever had a bone density test to check for osteoporosis, sometimes thought of
as “brittle bones”? This test may have been done to your back, hip, wrist, heel, or finger.
1
2
Yes
No
Probe: What does the term “bone density test” mean to you? Have you heard the term before
today?
Probe: What does the term “osteoporosis” mean to you? Have you heard the term before
today?
53. How much do you weigh in pounds (lbs.)?
01
90 lbs. or less
08
151–160 lbs.
15
221–230 lbs.
22
291–300 lbs.
02
91–100 lbs.
09
161–170 lbs.
16
231–240 lbs.
23
301–310 lbs.
03
101–110 lbs.
10
171–180 lbs.
17
241–250 lbs.
24
311–320 lbs.
04
111–120 lbs.
11
181–190 lbs.
18
251–260 lbs.
25
321 lbs. or more
05
121–130 lbs.
12
191–200 lbs.
19
261–270 lbs.
06
131–140 lbs.
13
201–210 lbs.
20
271–280 lbs.
07
141–150 lbs.
14
211–220 lbs.
21
281–290 lbs.
54. How tall are you without shoes on in feet (ft.) and inches (in.)? (If 1/2 in., please round up.)
01
5 ft. 00 in. or less
05
5 ft. 04 in.
09
5 ft. 08 in.
13
6 ft. 00 in.
02
5 ft. 01 in.
06
5 ft. 05 in.
10
5 ft. 09 in.
14
6 ft. 01 in.
03
5 ft. 02 in.
07
5 ft. 06 in.
11
5 ft. 10 in.
15
6 ft. 02 in.
04
5 ft. 03 in.
08
5 ft. 07 in.
12
5 ft. 11 in.
16
6 ft. 03 in. or more
55. In what year were you born? Please provide your year of birth only.
INTERVIEWER: note any problems with this question
B-15
56. Are you male or female?
1
2
Male
Female
57. Are you of Hispanic or Latino origin or descent?
1
2
Yes, Hispanic or Latino
No, not Hispanic or Latino
58. How would you describe your race? Please mark one or more.
a
b
c
d
e
f
American Indian or Alaska Native
Asian
Black or African American
Native Hawaiian or Other Pacific Islander
White
Another race
59. What is your current marital status?
1
2
3
4
5
Married
Divorced
Separated
Widowed
Never married
60. What is the highest grade or level of school that you have completed?
1
2
3
4
5
6
8th grade or less
Some high school, but did not graduate
High school graduate or GED
Some college or 2 year degree
4 year college graduate
More than a 4 year college degree
B-16
61. Is the house or apartment you currently live in:
1
2
3
4
5
Owned or being bought by you
Owned or being bought by someone in your family other than you
Rented for money
Not owned and one in which you live without payment of rent
None of the above
62. Who completed this survey form?
1
2
3
4
Person to whom survey was addressed
ÎGo to Question 64
Family member or relative of person to whom the survey was addressed
Friend of person to whom the survey was addressed
Professional caregiver of person to whom the survey was addressed
Probe: What does the first answer choice (Person to whom the survey was addressed) mean to
you? What about the term “Person selected for the survey.”? [INTERVIEWER: probe to
determine which term R’s prefer].
63. What is the name of the person who completed this survey form? Please print clearly.
First Name
Last Name
B-17
64. Which of the following categories best represents the combined income for all family
members in your household for the past 12 months?
01
02
03
04
05
06
07
08
09
10
Less than $5,000
$5,000–$9,999
$10,000–$19,999
$20,000–$29,999
$30,000–$39,999
$40,000–$49,999
$50,000–$79,999
$80,000–$99,999
$100,000 or more
Don’t know
You Have Completed the Survey. Thank You.
B-18
“According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of
information that does not display a valid OMB control number. The valid OMB control number for this
information collection is 0938-0701. The time required to complete this information collection is estimated
to average 20 minutes including the time to review instructions, search existing data resources, gather the
data needed, and complete and review the information collection. If you have comments concerning the
accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500
Security Boulevard, C3-16-27, Baltimore, Maryland 21244-1850.”
Insert Vendor Contact Information Here
B-19
APPENDIX C:
ROUND 2 RECRUITMENT SCREENER
Medicare Health Outcomes Survey
Materials Testing
Recruitment Screener: Round 2
Introduction
Ask to speak to an adult male or female age 18 or older living in the household:
Hello, my name is _______________ and I’m from (name of company). We are calling on
behalf of RTI International, a nonprofit research organization.
We are not selling or promoting any product. We are calling to recruit people to take part in a
research study about a health survey. The purpose of the research is to get feedback from people
about some materials that are used in the study. The study is sponsored by the Centers for
Medicare & Medicaid Services (CMS), the Federal government agency that operates the
Medicare and Medicaid programs.
We will be conducting interviews with several people and we would like to conduct one with
you. We will be conducting interviews on [DATES]. We have various time slots available and
will work with you to find a time that fits your schedule. The interview will last up to 1 ½ hours.
In appreciation for your participation, you will be reimbursed for your time, effort, and travel
expenses.
To see if you are eligible for this study, I need to ask you a few questions. All of your comments
will be kept private. My questions will only take a few minutes. May I proceed?
Yes – CONTINUE
No – Thank you and end call
1. RECORD GENDER (Ask if necessary)
1. Female
2. Male
[Attempt Mix]
2. How old were you on your last birthday? ______________
1. CODE AGE RANGE BELOW
2. 18–64 ÆTerminate
3. 65 or over
C-1
3. What is the highest level of education you have completed?
1. Less than high school
2. High school graduate
3. Some college or post-high school education
4. College (university) graduate or higher (See note below)
Note: No more than 3 Rs per round in category 4
4. How would you describe your race? (can choose more than 1)
1. American Indian or Alaska Native
2. Asian
3. Black or African American
4. Native Hawaiian or Other Pacific Islander
5. White
6. Another race
[Attempt mix]
5. How long have you lived in North Carolina?
_______________________________________
6.
[If Q5 – not whole life] Where did you live before moving to North Carolina?
_______________________________________
NOTE: You can probe about where R lived directly before NC, as well as other times in their
life.
[Attempt mix from different states and regions]
Invitation:
Thank you for answering all of my questions. As I mentioned earlier, we are conducting a
research study, sponsored by CMS to get feedback on a materials used as part of a health survey.
We would like to invite you to take part in an informal, personal interview. The interviews are
being scheduled on [DAYS/DATE] at a time that would be convenient for you [GIVE
AVAILABLE TIMES]. The discussion will last about 90 minutes. No one will attempt to sell
you anything and no one will call on you for other studies as a result of your participation in this
study. As part of the study, you may be asked if it is ok to audiotape your interview. However,
this is not required, and you may still participate if you prefer not to be recorded. To help repay
you for your time, effort, and travel expenses, you will receive $75 at the time of the interview.
C-2
This is an important research effort and we hope that you will be part of it. Can we schedule your
attendance?
Facility to provide:
•
Directions/map
•
Interviewer name and phone number
Closing for Ineligible Participants:
Thank you for answering my questions. At this time you are not eligible to be in this study
because [R is either under age 65 or is in a category that is full]. We value your interest in this
research study. Thank you for being willing to help us.
C-3
Information Sheet:
We are asking for your contact information only for the purpose of sending you a reminder letter
and giving you a call to remind you of your interview. We will destroy all contact information
upon conclusion of the interviews.
NAME: _________________________________________________________________
ADDRESS: _____________________________________________________________
CITY: __________________________________________________________________
ZIP CODE: ______________________________________________________________
EMAIL: ________________________________________________________________
What is the best time to reach you? What is the best telephone number to reach you at that time?
BEST TIME TO BE REACHED: __________________________________________________
BEST PHONE NUMBER: _______________________________________________________
Is there another time to call and number to reach you we can try if we miss you?
ALTERNATIVE TIME: _________________________________________________________
ALTERNATE PHONE NUMBER: ________________________________________________
Your participation in this study is very important. If for some reason you will not be able to
attend, please let us know right away. You can call us anytime at [INSERT PHONE NUMBER],
and if we are not here, please leave a message.
Interviewer: _____________________________________
Supervisor Confirm: _______________________________
C-4
APPENDIX D:
ROUND 2 DISCUSSION GUIDE
Round 2 Testing Protocol
Mailing Materials
We will be testing the following materials:
Baseline Material
1. Prenotification post card—This is sent to survey respondents 1 week before the initial survey
is mailed.
2. Cover letter #1—This letter is mailed to respondents with the questionnaire 1 week after the
prenotification post card.
3. Reminder/thank you post card—This post card is mailed to respondents 4–10 days after the
first questionnaire is sent. It is a reminder for those who have not yet returned their survey
and thanks those who have.
4. Cover letter #2—A second questionnaire and cover letter (different text) is sent to
nonrespondents approximately 28 days after sending the first questionnaire.
Follow-Up Materials
Medicare beneficiaries who complete the Medicare HOS are asked to complete a follow-up
survey 2 years later. They receive the materials outlined above (but a different version of the
prenotification post card and the cover letters for each of the questionnaire mailings; the
reminder/thank you post card is the same at baseline and follow-up).
D-1
Interviewer: Read/Paraphrase this Introduction
Hi, my name is ________. I want to thank you for coming today. Before we get started, I’m
going to give you some information about the study to read.
Give R the Informed Consent to read. Ask if R has any questions. Give R a copy of the
consent form and note phone numbers if they have follow-up questions.
Today we’re going to be talking about the Medicare Health Outcomes Survey (HOS). The HOS
is a large survey conducted by the Centers for Medicare & Medicaid Services (CMS), the
Federal agency that runs Medicare. The goal of the Medicare HOS program is to find out more
about the health care that is provided to people with Medicare. When someone is selected for the
survey, they receive notification by mail. Today I’m going to ask you to look at some of the
mailings someone would receive if selected for the survey.
We are interested in finding out what you think of them: are they easy to read, do you have any
questions after reading them, etc.
Do you have any questions before we begin?
Process
Respondents should receive materials in the order listed below (each interviewer will have a
folder for each piece that will include an original and a copy for making notations). For each
piece, direct respondents to read thoroughly and make notations next to anything that is
confusing (notations should be made on the copy).
After each piece, ask questions outlined in the guide.
Baseline Materials
1. Baseline prenotification post card
2. Baseline cover letter for first questionnaire
3. Baseline reminder/thank you post card
4. Baseline cover letter for second questionnaire
Follow-up Materials
5. Follow-up prenotification post card
6. Follow-up cover letter
7. Follow-up reminder/thank you post card (this is the same version used for baseline; it does
not need to be tested again)
8. Follow-up cover letter for second questionnaire
D-2
INTERVIEWER: This is the text for a post card that you might receive about the study. Please
take a few minutes to read it. If there are any words or concepts that are unclear or do not sound
right to you, please mark them with this pen so we can discuss them later.
[CMS Logo]
MEDICARE HEALTH OUTCOMES SURVEY
Dear Medicare Beneficiary:
The Centers for Medicare & Medicaid Services (CMS), the Federal agency that runs Medicare,
is conducting a survey to find out more about the care that is provided to people with Medicare.
This survey is called the “Medicare Health Outcomes Survey.” Your name was selected at
random from your health plan. In a few days, you will receive a questionnaire in the mail. We
would greatly appreciate your taking the time to complete the form.
Please look for the Medicare Health Outcomes Survey in the mail in a few days. Thank you for
helping with this important survey.
Sincerely,
Walter Stone
CMS Privacy Officer
En un esfuerzo por enterarnos más sobre la calidad de la asistencia que se brinda a los
beneficiarios del programa Medicare, la CMS está llevando a cabo una encuesta de los
beneficiarios. Si desea recibir la Encuesta de Medicare Sobre la Salud en español sírvase
llamar gratuitamente a [1-800 number].
Main Points
•
CMS, the Federal agency that runs Medicare, is conducting a survey with people who
have Medicare.
•
The survey is about the care provided to people with Medicare.
•
The person receiving the survey was selected at random.
•
The sample member will receive the survey in a few days.
Probe: In your own words, please tell me what the main points of this post card are.
D-3
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
Probe: Is there anything that you would want to know about the survey (at this point) that isn’t
included? If so, what else would you want to know?
D-4
Baseline Cover Letter 1
INTERVIEWER: This is a letter that you would receive in the mail with the questionnaire. Please
take a couple of minutes to read it. If there are any words or concepts that are unclear or do not
sound right to you, please mark them with this pen so we can discuss them later.
[CMS Letterhead]
Dear Medicare Beneficiary:
The Centers for Medicare & Medicaid Services (CMS) is the Federal agency that administers
the Medicare program. Our responsibility is to make sure that you get high quality care. One of
the ways we can fulfill that responsibility is to find out directly from you about how the care you
are currently receiving under the Medicare program affects your health.
CMS is conducting a survey of people with Medicare called the Medicare Health Outcomes
Survey. Your name was selected at random by CMS from among the people in your health plan
with Medicare. We would greatly appreciate your taking the time to fill out this questionnaire. A
postage-paid return envelope is enclosed.
Your answers to the survey will provide information about the state of your health. You will be
contacted two years from now and asked to complete the survey again. Your answers to the two
surveys will be compared to determine if the care you receive is keeping you as healthy as
possible. After the study is completed, your responses will be shared with your health plan. Your
plan will use this information to improve the quality of care.
Learning about the state of your health is very important to us. While your participation is
voluntary, we hope that you will take the time to answer the questionnaire. Your answers will
have no effect on your Medicare benefits.
[SURVEY VENDOR NAME] is a survey research organization working with us to carry out this
survey. If you have any problems completing the survey or have other questions about it, please
don’t hesitate to call [SURVEY VENDOR NAME toll free] at [1-800-Number] or e-mail [SURVEY
VENDOR NAME] at [e-mail address].
Thank you for your help with this important survey.
Sincerely,
Walter Stone
CMS Privacy Officer
Enclosures
ESPANOL AL OTRO LADO
D-5
Baseline Cover Letter 1
Main Points
•
CMS is the Federal agency that runs the Medicare program.
•
CMS is conducting a survey to find out how the care received under the Medicare
program affects people’s health.
•
The survey is called the Medicare Health Outcomes Survey.
•
The person receiving the survey was selected at random.
•
The sample member is being asked to complete a survey and return it in an enclosed
postage-paid envelope.
•
The sample member will be asked to complete the survey again in 2 years.
•
After both surveys have been completed, health plans will receive information about
responses so they can improve the quality of care that they provide.
•
Participation is voluntary.
•
Answers will not affect Medicare benefits.
•
Contact information is provided if the sample member has questions.
Probe: In your own words, please tell me what this letter says.
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
Probe: Do you think the letter provides enough information about the survey?
IF YES: Do you think it provides too much information (if yes, probe to see what information is
not needed)?
IF NO: What else would you like to know?
D-6
Baseline Reminder/Thank You Postcard
Interviewer: This is a post card that you might receive in the mail if you are a participant in the
study. Please take a couple of minutes to read it. If there are any words or concepts that are
unclear or do not sound right to you, please mark them with this pen so we can discuss them
later.
[CMS Logo]
MEDICARE HEALTH OUTCOMES SURVEY
Hello!
About a week ago you received the Medicare Health Outcomes Survey in the mail.
This is a reminder that we would like you to fill out the questionnaire and mail it back in the
postage-paid envelope that came with it. We need your answers. This is your chance to help
your health plan serve you better.
If you have returned the completed questionnaire, thank you!
If you did not get the questionnaire or have misplaced it, please call [SURVEY VENDOR NAME]
toll free at [1-800-Number] or e-mail [SURVEY VENDOR NAME ] at [e-mail address] and we will
mail a questionnaire to you.
THANK YOU!
[SURVEY VENDOR NAME]
Main Points
•
This post card is sent to remind sample members to mail back their completed
questionnaire (and thanks them if they have already done so).
•
This post card provides contact information if they did not receive the survey, or if it has
been misplaced.
Probe: In your own words, please tell me what the main points of this post card are.
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
D-7
Baseline Cover Letter 2
Interviewer: This is a letter that you might receive in the mail if you are a participant in the
study. Please take a couple of minutes to read it. If there are any words or concepts that are
unclear or do not sound right to you, please mark them with this pen so we can discuss them
later.
[CMS Letterhead]
Dear Medicare Beneficiary:
The Centers for Medicare & Medicaid Services (CMS) is conducting a survey of people with
Medicare in managed care plans to learn if the care you receive is keeping you as healthy as
possible. Your name was randomly selected from your health plan.
Recently, we mailed the Medicare Health Outcomes Survey to you but have not received your
response. We are enclosing another copy and would greatly appreciate your taking the time to
complete the survey. Please return it in the enclosed postage-paid envelope.
Learning about the state of your health is very important to us. While your participation is
voluntary, we hope you will help us by completing the questionnaire. Your answers will have no
effect on your Medicare benefits.
You will be contacted again two years from now and asked to complete the survey again. Your
answers to the two surveys will be compared to determine if the care you receive is keeping you
as healthy as possible. After the study is completed, your responses will be shared with your
health plan. Your plan will use this information to improve the quality of care.
[SURVEY VENDOR NAME] is a survey research organization working with us to carry out this
survey. If you have any problems completing the survey or have other questions about the
survey, please don’t hesitate to call [SURVEY VENDOR NAME] toll free at [1-800-Number] or
e-mail [SURVEY VENDOR NAME] at [e-mail address].
Thank you for your help with this important survey.
Sincerely,
Walter Stone
CMS Privacy Officer
Enclosures
D-8
Baseline Cover Letter 2
Main Points
•
CMS is conducting a survey of people with Medicare in managed care plans to learn
about the care that they receive.
•
The person receiving the survey was selected at random from their health plan.
•
The Medicare Health Outcomes Survey was recently sent to the sample member, but not
returned yet.
•
This mailing includes another copy of the survey.
•
Participation is voluntary.
•
Answers will not affect Medicare benefits.
•
The sample member will be asked to complete the survey again in 2 years.
•
After both surveys have been completed, health plans will receive information about
responses so they can improve the quality of care that they provide.
•
Contact information is provided if the sample member has questions.
Probe: In your own words, please tell me what this letter says.
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
Probe: What is the difference between this letter and the first letter you looked at? (Does R
notice different information—e.g., Medicare managed care plan)
D-9
Follow-Up Prenotification Postcard
Interviewer: This is the text for a post card you might receive if you are in the follow-up study.
Please take a couple of minutes to read it. If there are any words or concepts that are unclear or
do not sound right to you, please mark them with this pen so we can discuss them later.
[CMS Logo]
MEDICARE HEALTH OUTCOMES SURVEY
Dear Medicare Beneficiary:
Two years ago you completed the Medicare Health Outcomes Survey. At that time, we said that
we would like you to complete the survey again in the year 2010. In a few days, you will receive
a questionnaire in the mail. We would greatly appreciate your taking the time to complete the
form.
The Centers for Medicare & Medicaid Services (CMS) conducts this survey to find out more
about the care that is provided to people with Medicare. Your answers can help your health plan
to improve the quality of care.
Please look for the Medicare Health Outcomes Survey in the mail in a few days. Thank you for
your continued help in this important study.
Sincerely,
Walter Stone
CMS Privacy Officer
Main Points
•
The sample member completed the Medicare Health Outcomes Survey 2 years ago (and
was told that he/she would be asked to complete it again in 2 years).
•
The sample member will receive the survey in a few days.
•
CMS conducts the survey to find out more about the care that is provided to people with
Medicare. Answers to this survey can help health plans improve the quality of care that
they provide.
•
The sample member is thanked for their participation.
Probe: In your own words, please tell me what the main points of this post card are. (Does R
understand that this is for a follow-up survey 2 years after completing the first one?)
D-10
Follow-Up Prenotification Postcard
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
D-11
Follow-Up Cover Letter 1
Interviewer: This is a letter that you might receive in the mail if you are a participant in the
follow-up study. Please take a couple of minutes to read it. If there are any words or concepts
that are unclear or do not sound right to you, please mark them with this pen so we can discuss
them later.
[CMS Letterhead]
Dear Medicare Beneficiary:
Two years ago you completed the Medicare Health Outcomes Survey. At that time, we said that
we would like you to complete the survey again in the year 2010. Therefore, we are writing to
ask for your continued help. We would greatly appreciate your taking the time to fill out the
enclosed questionnaire. A postage-paid return envelope is enclosed.
After the study is completed, your responses will be shared with your health plan. Your plan will
use this information to improve the quality of care. Your responses to this follow-up survey will
also help the Centers for Medicare & Medicaid Services (CMS) to determine if the care you
receive is keeping you as healthy as possible.
[SURVEY VENDOR NAME] is a survey research organization working with us to carry out this
survey. If you have any problems completing the survey or have other questions about it, please
don’t hesitate to call [SURVEY VENDOR NAME] toll free at [1-800-Number] or e-mail [SURVEY
VENDOR NAME] at [e-mail address].
Thank you for your help with this important survey.
Sincerely,
Walter Stone
CMS Privacy Officer
Enclosures
Main Points
•
The letter reminds the sample member that they completed the Medicare HOS 2 years
ago and at that time, were asked to complete a follow-up survey in 2 years.
•
The sample member is asked to complete the enclosed questionnaire and return it in the
enclosed postage-paid envelope.
•
Responses to the survey will be shared with health plans so that they can use the
information to improve the quality of care that they provide. CMS will also use the
information to determine if the care people receive keeps them as healthy as possible.
•
Contact information is provided if the sample member has questions.
D-12
Follow-Up Cover Letter 1
Probe: In your own words, please tell me what this letter says.
INTERVIEWER: Ask—Did you mark anything on your copy as confusing or unclear?
[IF YES]—Review with R as needed
[IF NO]—Were there any words or ideas that were confusing or unclear?
D-13
Follow-Up Cover Letter 2
Interviewer: This is a letter that you might receive in the mail if you are a participant in the
follow-up study. Please take a couple of minutes to read it. If there are any words or concepts
that are unclear or do not sound right to you, please mark them with this pen so we can discuss
them later.
[CMS Letterhead]
Dear Medicare Beneficiary:
Two years ago you completed the Medicare Health Outcomes Survey. At that time, we said that
we would like you to complete the survey again in the year 2010. Recently we wrote to ask for
your continued help but have not received your response. We would greatly appreciate your
taking the time to fill out the enclosed questionnaire. A postage-paid return envelope is
enclosed.
After the study is completed, your responses will be shared with your health plan. Your plan will
use this information to improve the quality of care. Your responses to this follow-up survey will
also help the Centers for Medicare & Medicaid Services (CMS) to determine if the care you
receive is keeping you as healthy as possible.
[SURVEY VENDOR NAME] is a survey research organization working with us to carry out this
survey. If you have any problems completing the survey or have other questions about it, please
don’t hesitate to call [SURVEY VENDOR NAME] toll free at [1-800-Number] or e-mail [SURVEY
VENDOR NAME] at [e-mail address].
Thank you for your help with this important survey.
Sincerely,
Walter Stone
CMS Privacy Officer
Enclosures
Main Points
•
The letter reminds the sample member that they completed the Medicare HOS 2 years
ago and at that time, were asked to complete a follow-up survey in 2 years.
•
The Medicare HOS was recently sent to the sample member, but not returned yet.
•
Another questionnaire is enclosed with this mailing. The sample member is being asked
to return it in the enclosed postage-paid envelope.
•
Responses to the survey will be shared with health plans so that they can use the
information to improve the quality of care that they provide. CMS will also use the
information to determine if the care people receive keeps them as healthy as possible.
•
Contact information is provided if the sample member has questions.
D-14
Follow-Up Cover Letter 2
Probe: In your own words, please tell me what this letter says.
INTERVIEWER: Ask – Did you mark anything on your copy as confusing or unclear?
[IF YES] – Review with R as needed
[IF NO] – Were there any words or ideas that were confusing or unclear?
Summary
1. Now that you’ve read through all of the material, please tell me what you learned about
the Medicare HOS?
2. How easy or difficult were the materials to understand?
•
Was there enough information? If not, what else would you want to know?
•
Was there too much information?
3. Based on what you read, would you participate in the study? Why or why not?
4. Do you have any suggestions for how the materials that you looked at could be
improved?
D-15
| File Type | application/pdf |
| File Title | Microsoft Word - NCQA Round 2 draft_revise 7-21_changes accepted.doc |
| Author | smb |
| File Modified | 2010-07-28 |
| File Created | 2010-07-23 |