Download:
pdf |
pdfMEDICARE CURRENT BENEFICIARY SURVEY (MCBS)
2023 Content Management Cycle
Cognitive Testing Report
Contract 75FCMC19D0092; Task Order 75FCMC21F0001
MARCH 15, 2022
PRESENTED TO:
PRESENTED BY:
William Long, Contracting Officer’s
Representative
Centers for Medicare & Medicaid
Services
7111 Security Boulevard
Baltimore, MD 21244
Susan Schechter
Project Director
NORC at the University of Chicago
55 East Monroe Street
30th Floor
Chicago, IL 60603
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Table of Contents
Executive Summary .................................................................................................................................... 1
Introduction ................................................................................................................................................. 2
Original Items ......................................................................................................................................... 4
Adapted Items......................................................................................................................................... 5
Methods........................................................................................................................................................ 6
Sampling, Recruitment, and Screener .................................................................................................... 6
Data Collection ....................................................................................................................................... 7
Data Analysis Methods .......................................................................................................................... 7
Results and Recommendations .................................................................................................................. 7
Use of the Medicare Savings Program (MSP)........................................................................................ 8
Beneficiary Knowledge—Right to File a Complaint or Appeal .......................................................... 11
Use of CBD for Pain Management....................................................................................................... 13
Bowel Incontinence .............................................................................................................................. 14
Health Equity—Perceived Discrimination from Health Providers ...................................................... 17
Health Equity—Socio-Demographic Items about Religious Affiliation .............................................. 18
Discussion .................................................................................................................................................. 19
Appendix A: Cognitive Interview Screening Instrument ...................................................................... 21
Appendix B: Cognitive Interview Instrument ........................................................................................ 23
REPORT | i
NORC
| 2023 Content Management Cycle Cognitive Testing Report
List of Exhibits
Exhibit 1. 2023 Content Cycle Requests and Reasons for Cognitive Testing ........................................... 3
Exhibit 2. Distribution of Sex, Ethnicity, Race, and Educational Attainment among Cognitive Interview
Respondents .............................................................................................................................. 6
Exhibit 3. Medicare Savings Program, Item Q1-Q4 Question Text .......................................................... 8
Exhibit 4. Responses to Q1 Before and After Probing .............................................................................. 9
Exhibit 5. Proposed Revisions to Medicare Savings Program Items ....................................................... 11
Exhibit 6. Right to Appeal Series—Introductory Items ........................................................................... 12
Exhibit 7. Right to Appeal Series—Main Items ...................................................................................... 13
Exhibit 8. Use of CBD for Pain Management.......................................................................................... 14
Exhibit 9. Bowel Incontinence Items ....................................................................................................... 15
Exhibit 10. Proposed Revisions to Initial Bowel Incontinence Items ........................................................ 17
Exhibit 11. Health Equity Items—Perceived Discrimination from Health Providers................................ 17
Exhibit 12. Health Equity Items—Socio-Demographic Items about Religious Affiliation ....................... 18
Exhibit 13. Items Recommended for Implementation without Revision ................................................... 19
Exhibit 14. Items Recommended for Implementation with Minor Revision ............................................. 20
REPORT | ii
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Executive Summary
The Medicare Current Beneficiary Survey (MCBS) is a continuously fielded survey of a
nationally representative sample of the Medicare population conducted by the Centers for
Medicare & Medicaid Services (CMS) through a contract with NORC at the University of
Chicago (NORC). The Medicare population includes all Medicare eligible persons aged 65 and
over, and persons under age 65 with certain disabilities or with end-stage renal disease (ESRD).
The MCBS uses a rotating panel design and collects data from Medicare beneficiaries up to
eleven times over a span of four years. Incoming panels are sampled and recruited in the fall of
each year to replace the panel that rotates out in the winter. The survey covers topics including
health care utilization and expenditures, sources of health insurance coverage, and health status
and functioning. Data are collected for sampled beneficiaries living in noninstitutionalized (e.g.,
households) and institutionalized (e.g., nursing homes) settings.
Each year, CMS solicits content changes to the MCBS questionnaires from partners and
stakeholders. As a result of this call in 2021, CMS obtained requests to consider a number of
new items to be added to the MBCS Community questionnaire in 2023. These requests span a
series of health-related topics and stem from a variety of sources (Exhibit 1). Based on draft
wording of the items (either directly from other surveys or from other resources), a small-scale
cognitive testing round was designed to assess comprehension, sensitivity, and overall flow of
administration. Six interviews in English were conducted by phone in the Fall of 2021; two
interviews in Spanish were conducted by phone in Winter of 2022.
Based on the results of this small-scale testing effort, items measuring the use of cannabidiol
(CBD) for pain management, health equity items (related to perceived discrimination by health
care providers and socio-demographic items about religious affiliation intended to improve
analysis of health equity), use of the Medicare Savings Program (MSP), knowledge of the right
to file a complaint or appeal under the Medicare program, and the prevalence of bowel
incontinence are recommended for implementation. The testing found that a few questions
needed minor revisions to improve clarity but all performed well and administration was
straightforward in both English and Spanish. Adding these new items beginning with the 2023
MCBS will close important data gaps and improve analytic capability.
REPORT | 1
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Introduction
The Medicare Current Beneficiary Survey (MCBS) is a continuously fielded survey of a
nationally representative sample of the Medicare population conducted by the Centers for
Medicare & Medicaid Services (CMS) through a contract with NORC at the University of
Chicago (NORC). The Medicare population includes all Medicare eligible persons aged 65 and
over, and persons under age 65 with certain disabilities or with end-stage renal disease (ESRD).
The MCBS uses a rotating panel design and collects data from Medicare beneficiaries up to
eleven times over a span of four years. Incoming panels are sampled and recruited in the fall of
each year to replace the panel that rotates out in the winter. The survey covers topics including
health care utilization and expenditures, sources of health insurance coverage, and health status
and functioning. Data are collected for sampled beneficiaries living in noninstitutionalized (e.g.,
households) and institutionalized (e.g., nursing homes) settings.
Each year, CMS solicits content changes to the MCBS questionnaires from partners and
stakeholders. As a result of this call in 2021, CMS obtained requests for a number of new items
to be added to the MBCS Community questionnaire in 2023. These requests span a series of
health-related topics and stem from a variety of sources (Exhibit 1); some items required
development and others were derived from other surveys. One round of cognitive testing
involving eight interviews (six in English, two in Spanish) were conducted to assess
comprehension, sensitivity, and overall flow of administration.
REPORT | 2
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Exhibit 1. 2023 Content Cycle Requests and Reasons for Cognitive Testing
Item Source
Primary Reason for
Cognitive Testing
Use of the Medicare Savings
Program
Original Items
Assess comprehension
Beneficiary Knowledge of their Right
to File a Complaint or Appeal
Original Items
Assess comprehension
Content Request
REPORT | 3
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Item Source
Primary Reason for
Cognitive Testing
Original Item
Assess comprehension
Bowel Incontinence
Adapted from 2004 Mayo Clinic
Study
Assess comprehension
Health Equity—Perceived
Discrimination from Health Providers
Medicare Advantage (MA) and
Prescription Drug Plan (PDP)
Consumer Assessment of
Healthcare Providers and
Systems (CAHPS) Web Mode
Field Test
Assess sensitivity
General Social Survey (GSS)
Assess sensitivity
Content Request
Use of CBD for Pain Management
Health Equity—Expand SocioDemographic Items to Include
Religious Affiliation
Original Items
Several of the requests for new content are original items that require testing to assess
comprehension and clarity among Medicare beneficiaries. These include:
■
Items about use of the Medicare Savings Program (MSP) to capture information about
beneficiaries who are eligible but not enrolled in the MSP program. These items are
formatted in a way similar to existing items in the MCBS Drug Coverage Questionnaire
REPORT | 4
NORC
| 2023 Content Management Cycle Cognitive Testing Report
(RXQ) about the Low-Income Subsidy (LIS) program which asks for a similar measure
(awareness of a program that provides financial assistance to those in need).
■
■
Items about the beneficiary’s knowledge of their rights to file complaints or appeals under
the Medicare program).
An item on the use of cannabidiol (CBD) for pain management for inclusion to an existing
pain management grid item in the MCBS Chronic Pain Questionnaire (CPQ) that asks about
non-medication management methods used to manage pain. This item asks beneficiaries to
identify whether they use specific methods, such as physical therapy, chiropractic care, or
massage, to manage their chronic pain.
Adapted Items
Several of the requests for new content have been taken or adapted from other surveys. These
include:
■
■
■
An item on bowel incontinence, sourced from a 2004 Mayo Clinic Study1 and adapted to
model the urinary incontinence items in the MCBS Health Status and Functioning
Questionnaire (HFQ) to capture prevalence and management of stool leakage.
Eight items about health equity, including perceived discrimination by health care providers
based on aspects of the beneficiary’s identity. This series of items was developed and will be
undergoing testing for implementation on the Medicare Advantage (MA) and Prescription
Drug Plan (PDP) Consumer Assessment of Healthcare Providers and Systems (CAHPS) Web
Mode Field Test (pending OMB approval).
Two socio-demographic items about religious affiliation sourced from the General Social
Survey (GSS) 2. These items are intended to improve measures of health equity as they will
allow analysts to broaden socio-demographic analysis and assess whether key health
outcomes differ by factors such as religious affiliation. These items will also help to evaluate
whether perceived discrimination from health care providers varies based on religious
affiliation.
Bharucha AE, Locke GR 3rd, Seide BM, Zinsmeister AR. A new questionnaire for constipation and faecal
incontinence. Aliment Pharmacol Ther. 2004 Aug 1;20(3):355-64. doi: 10.1111/j.1365-2036.2004.02028.x. PMID:
15274673
2
The 2020 General Social Survey Questionnaire is available from:
http://gss.norc.org/Documents/quex/GSS2020panel_Ballot1_English_CAPI.pdf
1
REPORT | 5
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Methods
The purpose of this cognitive testing study is to understand the processes people use to answer
questions about several health-related topics and to identify potential problems in the questions
and response categories. A qualitative analysis of the interviews was conducted.
Sampling, Recruitment, and Screener
As is typical of cognitive interviewing, convenience sampling was used by reaching out to a pool
of potential participants or asking project staff to forward study information to their personal
networks. Respondents were screened prior to interview administration via telephone. The
screener consisted of items to measure eligibility, such as age, Medicare status, and basic
demographics (see English-language screening instrument in Appendix A). To be eligible to
participate, respondents had to self-report eligibility for Medicare and have at least one chronic
health condition.
A total of eight interviews were completed. A total of six interviews were conducted in English
in October 2021. Two additional interviews were conducted in Spanish in February 2022.
Respondents were provided a $40 incentive for participating. They ranged in age from 69 to 90
(mean age = 77.3. Exhibit 2 shows distributions for sex, ethnicity, race, and educational
attainment as reported during screening.
Exhibit 2. Distribution of Sex, Ethnicity, Race, and Educational Attainment among Cognitive
Interview Respondents
Characteristic
Count
Sex
Female
6
Male
2
Ethnicity
Non-Hispanic
5
Hispanic
3
Race
American Indian/Alaska Native and Black/African American
1
Black or African American alone
2
White
4
Don’t Know/Refused
1
REPORT | 6
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Characteristic
Count
Education
No schooling completed
1
Some college, but no degree
3
Bachelor’s degree
3
Master’s, Professional, or Doctorate degree
1
Data Collection
Interviewers administered the questionnaire as a paper-and-pencil interview (PAPI) remotely via
telephone or the Zoom video conferencing platform. Once potential respondents were screened
and determined to be eligible, interviewers reviewed and obtained informed consent and
administered the test questionnaire in full. Interviews were audio or video recorded, with
respondents’ permission, so that analysts could reference the recording during analysis as
needed. After each series of items was administered, interviewers debriefed respondents,
reviewing the questions and their answers and probing for information about how they
interpreted the questions and arrived at their answers. The English version of the test
questionnaire instrument is shown in Appendix B.
Data Analysis Methods
This cognitive testing effort primarily used ‘think aloud’ and retrospective probing techniques.
Participants were asked to think aloud as they offered answers. Additionally, respondents were
retrospectively asked additional questions about the items being tested. During and after each
interview, interviewers wrote notes summarizing the themes and responses they heard. Analysts
then reviewed the notes; listened to the interview recordings for additional clarity when needed;
identified themes in the responses; and created typologies of key themes observed. This analysis
provided insights into any confusion respondents may have had in responding to the draft items
and identified items that respondents did not answer as intended. It also identified possible
opportunities for improvement of some measures.
Results and Recommendations
The discussion below is structured around each set of items: (1) beneficiary use of the MSP
program, (2) beneficiary knowledge about their right to file a complaint or appeal under the
Medicare program, (3) use of CBD for pain management, (4) prevalence of bowel incontinence,
(5) health equity—perceived discrimination from health providers, and (6) health equity—
expanding socio-demographic items to include religious affiliation. For each group, this section
REPORT | 7
NORC
| 2023 Content Management Cycle Cognitive Testing Report
first describes the testing results and then, when appropriate, includes recommendations to
improve or revise the questions. For each group of test questions, the report includes descriptive
statistics and a discussion of item performance.
Use of the Medicare Savings Program (MSP)
Item Q1 (shown in Exhibit 3) asks respondents if they receive help in paying for their health care
costs through the MSP program. If the respondent reported they do not, items Q2-4 were
included in the cognitive testing questionnaire to further assess comprehension about the MSP
program. The intent of these follow-up items was to gather additional information about how
respondents are interacting with the MSP program by asking whether they have applied for help
to their state Medicare office, whether their application was accepted or denied, and their
awareness of the MSP program.
Exhibit 3. Medicare Savings Program, Item Q1-Q4 Question Text
Variable
Name
Q1
Question Text
Code List
We’re interested in learning more about how people
covered under Medicare navigate certain programs
available to beneficiaries—particularly those meant to
help people pay for their healthcare costs. As you may
know, the government has programs that help
beneficiaries pay for the costs associated Medicare such as
Part A or Part B premiums, deductibles, coinsurance, and
copayments. We’re going to ask you a few questions
about these programs, and what your experience, if any,
has been with them.
(A) YES Skip rest of series
(B) NO
(C) DON’T KNOW
(D) REFUSED
[Are you/Is (SP)] receiving this type of help to pay for
[your/his/her] (CURRENT YEAR) health care costs?
[EXPLAIN IF NECESSARY: These programs are called
Medicare Savings Programs. There are four different
Medicare Savings Programs: Qualified Medicare
Beneficiary Program, Specified Low-Income Medicare
Beneficiary Program, Qualifying Individual Program, and
Qualified Disabled and Working Individuals Program.]
Q2
Did [you/(SP)] apply to the [STATE] Medicare office for
help with (CURRENT YEAR) expenses?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
REPORT | 8
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Variable
Name
Question Text
Code List
Q3
Was [your/(SP's)] application for a Medicare Savings
Program accepted or denied?
(01) ACCEPTED
(02) DENIED
(03) STILL PENDING/NO
DECISION YET
(-8) Don't Know
(-9) Refused
Q4
Before today, were you aware of these programs?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Half of respondents (n=4) initially stated that they had help through the MSP program to pay for
health care costs. After probing, however, and further explanation of the MSP program, it
became evident that several participants thought the item was referring to secondary insurance
policies (Exhibit 4). For the remaining items (Q2-4), none of the respondents reported having
applied to their state Medicare office for help with expenses and none were aware of the MSP
program before today.
Exhibit 4. Responses to Q1 Before and After Probing
[Are you/Is (SP)] receiving this type of
help to pay for [your/his/her]
(CURRENT YEAR) health care costs?
Initial Response
Response After Probing and
Explanation of MSP Program
Yes
4
0
No
4
8
Analysis and Recommendations
The largest comprehension issue arising from this set of questions was that respondents thought
the questions asked about private insurance policies in addition to Medicare policies. Each
respondent had another insurance plan, largely through their employer-sponsored retirement
benefit plans, which helped them with costs and expenses not covered by Medicare. This resulted
in overreporting of MSP as respondents interpreted any coverage they have beyond Medicare to
fit the conceptual definition.
Additional questions and probing revealed that even those who reported living on small, fixed
incomes and needing help paying for their prescriptions and other medical needs had not heard
of support programs through Medicare and did not know there was an option to reach out to a
state-based Medicare office. Two of the participants were aware that they had materials (e.g.,
Medicare cards, beneficiary literature) where they might find information on such a program.
REPORT | 9
NORC
| 2023 Content Management Cycle Cognitive Testing Report
However, two other respondents said they viewed Medicare as a large government effort so were
more likely to reach out to a family member or online source to try to find information on
alternate support options.
The terms “Qualified” used in the MSP program names in the help text at Q1 also caused
confusion. Respondents were unsure how to interpret this term in the program name and thought
they meant someone from Medicare might reach out to notify them they were qualified. Similar
results were observed during testing of the items in Spanish.
A clearer and more direct definition of MSP at Q1 (proposed in Exhibit 5) would alleviate some
of the confusion in this series as would introductory text helping respondents to differentiate
MSP from other privately held insurances. As there was no variation in responses to Q2-4, it is
recommended to not implement all three items in the MCBS survey. Instead, Q1 followed by Q2
would be sufficient to obtain the initial measure of whether respondents are receiving MSP
assistance and whether they have applied for assistance.
REPORT | 10
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Exhibit 5. Proposed Revisions to Medicare Savings Program Items
Variable
Name
Q1
Question Text
We’re interested in learning about how Medicare
beneficiaries navigate certain programs available to help
them pay for their health care costs. As you may know,
the government has a set of programs, called Medicare
Savings Programs (MSP), that help beneficiaries pay for
the costs associated with Medicare, such as Part A
(Hospital Insurance) or Part B (Medical Insurance)
premiums, deductibles, coinsurance, and copayments.
Unlike additional insurance plans that require a monthly
premium, Medicare Savings Programs provide financial
help at no cost to eligible beneficiaries who have limited
income and resources. We’re going to ask you a few
questions about these programs, and what [your/(SP)’s]
experience, if any, has been with them.
Code List
(A) YESNext Series
(B) NOQ2
(C) DON’T KNOWQ2
(D) REFUSEDQ2
[Are you/Is (SP)] receiving any assistance from a
Medicare Savings Program (MSP) to help pay for
[your/his/her] (CURRENT YEAR) health care costs?
[EXPLAIN IF NECESSARY: Medicare Savings
Programs pay for remaining costs (premiums,
deductibles, coinsurance, and copayments) not covered by
Medicare. These programs are different from additional
insurance plans, such as Medicare Supplement Insurance
(Medigap) or private insurance plans, in that beneficiaries
will not pay for this extra financial help. Instead,
beneficiaries must be eligible (i.e., have limited resources
or income) and apply to receive this financial assistance
from an MSP.
Q2
Did [you/(SP)] apply to the [STATE] Medicare office for
help with (CURRENT YEAR) expenses?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Beneficiary Knowledge—Right to File a Complaint or Appeal
Testing items that measure knowledge of the right to file a complaint or appeal were focused on
whether respondents would know about these rights and be able to answer the questions. For the
cognitive test, introductory items not intended for survey implementation were used in the series
to help better understand challenges individuals may face in using their Medicare coverage; these
can be found in Exhibit 6, Q1—6. These introductory items provided context for the main items
REPORT | 11
NORC
| 2023 Content Management Cycle Cognitive Testing Report
being tested (Q7-9), to help determine whether the terms or approaches covered in items Q7-9
would be understood without additional explanations or needs for interviewer probing.
When asked what they would do if they had difficulty with obtaining medical services, all
respondents said they would take action. Contacting Medicare was not reported as a possible
outcome to addresses issues with access to quality healthcare. Instead, respondents said they
would speak to members of management at a doctors’ office or medical facility, contact agebased advocacy groups, use resources from private insurance plans/retirement plans, or consult
friends/family.
Exhibit 6. Right to Appeal Series—Introductory Items
Variable
Name
Question Text
Code List
Q1
Think about all of the times you’ve sought healthcare
under Medicare coverage. Have you ever had a
difficult time getting an appointment with a healthcare
provider you needed to see?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Q2
Think about all of the times you’ve sought healthcare
under Medicare coverage. Have you ever had a
difficult time getting the care you needed during your
appointment(s) with a healthcare provider you saw?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Q3
If you had concerns about the quality of care you were
receiving from a healthcare provider or facility, would
you know how to file a complaint with Medicare?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Q4
In your own words, can you tell me a bit about what
you would do if you had difficulty with getting an
appointment, getting the care you need, or getting the
care you received paid for properly?
(01) CONTINUOUS ANSWER
Q5
Have you ever heard the term ‘right to appeal’? What
do you think it means?
(01) CONTINUOUS ANSWER
Q6
What do you know about filing a complaint, or your
right to file a complaint, based on concerns you have
around the healthcare you’ve received?
(01) CONTINUOUS ANSWER
Following the introductory series, a statement (using Medicare text) on the definitions of the
right to appeal was read verbatim to the respondents. Respondents were then asked items Q7-9
(shown in Exhibit 7). Even with the information provided in the introductory series, none knew
how they would go about contacting Medicare to invoke their rights to appeal or file a complaint
and reported that they would either use the information sources reported earlier including
reaching out to advocacy groups or looking it up online or in the Medicare book.
REPORT | 12
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Exhibit 7. Right to Appeal Series—Main Items
Variable
Name
Question Text
Code List
Q7
If you felt that a healthcare provider was ending your
healthcare services too soon, would you know who to
contact in Medicare to protect your rights and appeal?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Q8
If you had concerns about the safety of care you were
receiving from a healthcare provider or facility, would
you know how to file a complaint with Medicare?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Q9
If you had concerns about the quality of care you were
receiving from a healthcare provider or facility, would
you know how to file a complaint with Medicare?
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
Analysis and Recommendations
While items Q7-9 were clearly understood, and participants all reported they would take action if
needed, they do not measure a priori knowledge about rights to file a complaint or appeal. Also,
it was unclear whether respondents would distinguish carefully between the three questions, each
of which seeks to measure a slightly different issue that would lead to filing a complaint or
appeal. Therefore, it is recommended that one question among this series is selected to add to the
2023 MCBS Questionnaire. It is also recommended to adapt Q3 to ask about knowledge of both
the right to appeal and file a complaint by modifying it to ask: “If you had concerns about the
quality of care you were receiving from a healthcare provider or facility, would you know how
to file a complaint or an appeal with Medicare?” Similar results were observed during Spanishlanguage testing.
Use of CBD for Pain Management
An item on the use of CBD to manage pain was tested as part of an existing series about the use
of non-medication management methods from the MCBS Chronic Pain Questionnaire (CPQ), as
shown in Exhibit 8. This item asks beneficiaries to identify whether they use specific methods,
such as physical therapy, chiropractic care, or massage, to manage their chronic pain. All but two
respondents reported pain on some or more days in the past three months and therefore qualified
for this series. Of the six respondents who were administered Q2, none reported use of CBD to
manage pain.
REPORT | 13
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Exhibit 8. Use of CBD for Pain Management
Variable
Name
Q1
Question Text
Code List
Now I would like to ask about your experiences with
pain. In the past three months, how often did you have
pain? Would you say:
(A) Never Skip rest of series
(B) some daysQ2
(C) most daysQ2
(D) every dayQ2
(E) Don’t Know Skip rest of
series
(F) RefusedSkip rest of series
Q2
Since (TODAY'S MONTH AND YEAR - 3
MONTH), did you use any of the following to manage
your pain? Please indicate yes or no to each one:
Yes
No
Physical therapy, rehabilitative therapy, or
occupational therapy
Spinal manipulation or other forms of chiropractic care
Yes
No
Yes
No
Massage
Yes
No
Meditation, guided imagery, or other relaxation
techniques
CBD (cannabidiol)
Yes
No
Yes
No
[IF NEEDED: CBD is different from Marijuana or
THC. CBD is one of the many compounds that can be
present in the cannabis plant and does not cause a
“high” by itself. CBD is available in different forms,
such as oils and tinctures (extracts), capsules
(pills/softgels), edibles (gummies/candies), and
topicals (creams/lotions/ointments).
Analysis and Recommendations
Questions on pain management were not challenging and were easily understood in both English
and Spanish. Respondents knew what CBD was and had no challenge reporting on it.
Bowel Incontinence
The next series of items assessed prevalence and frequency of bowel incontinence (Exhibit 9).
The follow-up items in this series were asked more for the interviewers to obtain additional
information on the degree to which this series might eventually be expanded to mirror the
REPORT | 14
NORC
| 2023 Content Management Cycle Cognitive Testing Report
urinary incontinence series currently on the MCBS. As only one participant reported yes to Q1,
only one person received follow-up items Q2-Q7.
Exhibit 9. Bowel Incontinence Items
Variable
Name
Question Text
Code List
Q1
We are now going to ask you some questions about (A) YESQ2
your ability to control your bowel movements.
(B) NO SKIP REST OF
SERIES
In the last 12 months, have you had any problems
(C) DON’T KNOWSKIP REST
with leakage of stool, for example leaking gas,
OF SERIES
accidents, or soiling?
(D) REFUSEDSKIP REST OF
[IF NEEDED: Was that because you were sick?]
SERIES
SELECT 'NO' IF THE RESPONDENT HAD ANY
PROBLEMS DUE TO A SHORT-TERM
DIARRHEAL ILLNESSES SUCH AS THE FLU
OR A VIRUS.
Q2
Did you have a problem with leaking gas only; a
small amount of stool; a moderate amount of stool,
often requiring a change of underwear; or a large
amount of liquid stool, often requiring a complete
change of clothes?
SELECT ALL THAT APPLY
Q3
In the last 12 months, did you have this problem
more than once a week, about once a week, 2-3
times a month, about once a month, every 2-3
months, or once or twice a year?
[IF NEEDED: Include all types of stool leakage,
including: gas only; a small amount of stool; a
moderate amount of stool, and/or a large amount of
liquid stool.]
(A) LEAKING GAS ONLY
(B) A SMALL AMOUNT OF
STOOL
(C) A MODERATE AMOUNT
OF STOOL, OFTEN
REQUIRING A CHANGE
OF UNDERWEAR; OR
(D) A LARGE AMOUNT OF
LIQUID STOOL, OFTEN
REQUIRING A COMPLETE
CHANGE OF CLOTHES?
(E) DON’T KNOW
(F) REFUSED
(A) MORE THAN ONCE A
WEEK
(B) ABOUT ONCE A WEEK
(C) 2-3 TIMES A MONTH
(D) ABOUT ONCE A MONTH
(E) EVERY 2-3 MONTHS
(F) ONCE OR TWICE A YEAR?
(G) DON’T KNOW
(H) REFUSED
REPORT | 15
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Variable
Name
Question Text
Code List
Q4
Have you talked about your problem with stool
leakage with [your/his/her] doctor or other health
professional?
[IF NECESSARY: This is also referred to as bowel
or fecal incontinence.]
(A)
(B)
(C)
(D)
YES
NO
DON’T KNOW
REFUSED
Q5
Has your doctor or other health professional asked
you about how you feel about this problem?
(A)
(B)
(C)
(D)
YES
NO
DON’T KNOW
REFUSED
Q6
Has your doctor or other health professional
examined you to figure out why you have a
problem with stool leakage?
[IF NECESSARY: This is also referred to as bowel
or fecal incontinence.]
(A)
(B)
(C)
(D)
YES
NO
DON’T KNOW
REFUSED
Q7
Has your doctor or other health professional talked
with you about taking medicine or having surgery
for this problem?
[IF NECESSARY: Please answer based on your
usual use of medication.]
(A)
(B)
(C)
(D)
YES
NO
DON’T KNOW
REFUSED
Analysis and Recommendations
Despite limited testing on follow-up items in this series, use of open-ended probes at Q1 revealed
the possibility of underreporting of bowel-related issues in this series using the current question
structure. When asked more generally about health-related issues, several respondents reported
severe constipation and stomach issues from medication. To avoid being too restrictive in this
series, it is recommended to combine Q1 and Q2 into a series where respondents are asked
whether or not they have experienced each type of bowel incontinence issues (Exhibit 10).
While combining Q1 and Q2 will more accurately measure prevalence of bowel incontinence
issues, the format recommended is a yes/no to each of the bowel-related problems. Beneficiaries
would first be asked if they have experienced several types of bowel incontinence including
leaking gas, leaking a small amount of stool, leaking a moderate amount of stool, and leaking a
large amount of liquid stool. Beneficiaries who respond affirmatively to any type of bowel
incontinence would next be asked if they have talked with their health care provider about this
issue. It is recommended to limit follow-up questions to one item (Q4) to reduce respondent
burden. This revised format will allow CMS to understand the prevalence of bowel incontinence
among beneficiaries and develop better awareness and outreach regarding this medical issue.
This revised format was tested with two Spanish-speaking respondents and performed well.
REPORT | 16
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Exhibit 10. Proposed Revisions to Initial Bowel Incontinence Items
Variable
Name
Combined
Q1/Q2
Question Text
Code List
We are now going to ask you some questions about your
ability to control your bowel movements. In the last 12
months, have you had any of the following problems?
Yes
No
Don’t Know
Leaking a small amount of stool?
Yes
No
Don’t Know
Leaking a moderate amount of stool, requiring a change of
underwear?
Yes
No
Don’t Know
Leaking a large amount of liquid stool, requiring a complete
change of clothes?
Yes
No
Don’t Know
Have you talked about your problem with stool leakage with
[your/his/her] doctor or other health professional?
[IF NECESSARY: This is also referred to as bowel or fecal
incontinence.]
(A) YES
(B) NO
(C) DON’T KNOW
(D) REFUSED
[IF NEEDED: Was that because [you/(SP)] [were/was]
sick?]
SELECT 'NO' IF THE RESPONDENT HAD ANY
PROBLEMS DUE TO SHORT-TERM DIARRHEAL
ILLNESSES SUCH AS THE FLU OR A VIRUS.
Leaking gas?
Q4
Health Equity—Perceived Discrimination from Health Providers
The first health equity series (shown in Exhibit 11) asked if respondents perceived that they were
treated in an unfair or sensitive way by health providers due to certain personal attributes. All
eight respondents reported “no” to each item in this series.
Exhibit 11. Health Equity Items—Perceived Discrimination from Health Providers
Variable
Name
Q1
Question Text
Code List
In the last 12 months, did anyone from a clinic,
emergency room, or doctor’s office where you got care
treat you in an unfair or insensitive way because of any of
the following things about you?
Yes
No
Don’t Know
Race or ethnicity
Yes
No
Don’t Know
Language or accent
Yes
No
Don’t Know
Gender or gender identity
Yes
No
Don’t Know
Sexual orientation
Yes
No
Don’t Know
REPORT | 17
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Variable
Name
Question Text
Code List
Age
Yes
No
Don’t Know
Culture or religion
Yes
No
Don’t Know
Disability
Yes
No
Don’t Know
Medical history
Yes
No
Don’t Know
Analysis and Recommendations
The items on health equity performed well as tested in both English and Spanish. Respondents
did not find these items to be sensitive nor did they require definitions to clarify terms. Including
standard definitions and help text for interviewers will be important for uniform interpretation of
terms.
Health Equity—Socio-Demographic Items about Religious Affiliation
The final item asked about the respondent’s religious preference (Exhibit 12). Three respondents
reported that they identified as Jehovah’s witness, two reported being affiliated with the Church
of Latter-Day Saints, one respondent reported identifying as Catholic, and two respondents
reported identifying as Christian.
Exhibit 12. Health Equity Items—Socio-Demographic Items about Religious Affiliation
Variable
Name
Q1
Question Text
What is your religious preference?
(A) PROTESTANT
(B) CATHOLIC
IF NEEDED: Please select the religion with which
you most closely identify.
Q2
Code List
What is your religious preference?
(C) EASTERN ORTHODOX,
SUCH AS GREEK OR
RUSSIAN ORTHODOX
(D) JEWISH
(E) BUDDHIST
(F) HINDU
(G) MUSLIM, ISLAM, SUFI,
SUNNI, OR SHIA
(H) SOME OTHER RELIGION
(I) NO RELIGION
(J) DON'T KNOW
(K) REFUSED
(L) NA
(A) Continuous Answer
REPORT | 18
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Analysis and Recommendations
The item on religious preference performed well as tested in both English and Spanish. The item
was easily understood, every participant was able to report on it, and all stated no hesitation in
doing so.
Discussion
This section includes recommendations for whether the draft items require additional revisions
before implementation.
■
Recommended for Implementation Without Revision. Items measuring the use of CBD
for pain management, health equity, and religious affiliation were easily understood in both
English and Spanish and were not considered sensitive by respondents (Exhibit 13). Based on
the results of this small testing effort, it is recommended that these three sets of items be
included in a full clearance revision to the main MCBS (0938-0568) for implementation in
2023.
Exhibit 13.
Items Recommended for Implementation without Revision
# of
Items
Questionnaire Section
Administration Schedule
Use of CBD
1
Chronic Pain Questionnaire (CPQ)
Annually, Summer Round
Health Equity
8
Satisfaction with Care (SCQ)
Annually, Fall Round
Religious Affiliation
2
Demographics and Income
Questionnaire (DIQ)
Baseline Interview Only,
Fall Round
Content Request
■
Recommended for Implementation with Minor Revisions. Three additional new measures
are recommended for implementation as part of the 2023 MCBS questionnaires with minor
revisions.
►
►
Items measuring use of the MSP program were somewhat confusing to respondents, who
interpreted them as asking about secondary sources of insurance. Clarifying the text and
terms in the initial question, and limiting the number of follow-up items, is likely to
reduce overreporting of MSP participation.
Items measuring knowledge of the right to file a complaint or appeal under the Medicare
program were somewhat challenging as it is difficult to measure knowledge about
something that is very program specific and usually requires education or broad
promotional materials. However, the question could be implemented if it were revised to
measure whether a respondent would know how to submit a complaint or an appeal.
REPORT | 19
NORC
►
| 2023 Content Management Cycle Cognitive Testing Report
Similarly, items measuring the prevalence and frequency of bowel incontinence would
benefit from minor question restructuring to avoid underreporting. Using a yes/no format
to ask beneficiaries about the prevalence of each bowel-related problem will improve
comprehension and allow respondents to answer each item separately. Limiting followup questions to one item will provide necessary information for analysts while
minimizing respondent burden. This revised format tested well with two Spanishspeaking respondents.
Exhibit 14.
Items Recommended for Implementation with Minor Revision
Content Request
# of
Items
Questionnaire Section
Administration
Schedule
Use of MSP Program
2
Beneficiary Knowledge and
Decision-Making Questionnaire
(KNQ)
Annually, Winter
Round
Right to File a
Complaint or Appeal
1
Beneficiary Knowledge and
Decision-Making Questionnaire
(KNQ)
Annually, Winter
Round
Bowel Incontinence
5
Health Status and Functioning
Questionnaire (HFQ)
Annually, Fall Round
Following OMB approval and CMS implementation, all questions will be closely monitored in
2023 to determine if any residual comprehension issues need to be addressed via interviewer
training or on-screen help text within the questionnaire.
REPORT | 20
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Appendix A: Cognitive Interview Screening Instrument
PARTICIPANT NUMERIC IDENTIFIER: ___________________________
1. What is your sex? Are you male, female, or do you identify yourself another way?
a) MALE
b) FEMALE
c) I IDENTIFY AS________
d) I don’t know
e) Refused
2. How old are you?
_______ years
3. Medicare is the federal health insurance program for people who are 65 or older, certain younger
people with disabilities, and people with End-Stage Renal Disease. Do you receive health insurance
through Medicare?
a) YES
b) NO I am sorry, but only people who receive insurance through Medicare are eligible
for this study.
4.
What is the highest degree or level of school you have completed?
a) NO SCHOOLING COMPLETED
b) NURSERY SCHOOL TO 8TH GRADE
c) 9TH-12TH GRADE, NO DIPLOMA
d) HIGH SCHOOL GRADUATE (HIGH SCHOOL DIPLOMA OR THE EQUIVALENT)
e) VOCATIONAL/TECHNICAL/BUSINESS/TRADE SCHOOL CERTIFICATE OR
DIPLOMA (BEYOND THE HIGH SCHOOL LEVEL)
f) SOME COLLEGE, BUT NO DEGREE
g) ASSOCIATE DEGREE
h) BACHELOR'S DEGREE
i) MASTER'S, PROFESSIONAL OR DOCTORATE DEGREE
5. Are you of Hispanic, Latino, or Spanish origin?
a) YES
b) NO
7. What is your race? [SELECT ONE OR MORE. READ RESPONSE OPTIONS IF NEEDED.]
a) American Indian or Alaska Native
b) Asian
c) Black or African American
d) Native Hawaiian or Other Pacific Islander
e) White
f) Something else:__________________
REPORT | 21
NORC
| 2023 Content Management Cycle Cognitive Testing Report
8. Would you prefer to conduct the interview over the phone or over video, using Zoom?
a) PHONE
b) ZOOM
9. CONFIRM CONTACT INFORMATION AND SCHEDULE APPOINTMENT:
Ok, let’s schedule an appointment to do the interview.
[TAKE INFORMATION]
IF PHONE:
10. Can you confirm that [PHONE NUMBER] is the best number to contact you on [DATE] for the
interview?
[TAKE INFORMATION]
IF ZOOM:
11. Could you please provide an email address to send you the Zoom link information to for the
interview?
[TAKE INFORMATION]
Thank you for volunteering to participate. We will speak with you on [REPEAT DATE AND TIME OF
APPOINTMENT] by [REPEAT MODE]. I will reach out the day before the interview to remind you of
the appointment and to confirm the name [and phone number] of the interviewer who will be contacting
you.
With your permission, your interview session will be recorded to allow us to ensure we capture all of the
feedback you provide us. Only staff directly involved in the project will have access to the recording. If
you do not want to be recorded, we can do the interview without recording. Your answers will always be
kept private, and none of the information that you provide will be used for any purpose other than
research.
AFTER HANGING UP, ENTER THE INFORMATION FROM THIS SCREENER INTO THE
RECRUITMENT SPREADSHEET ALONG WITH THE DATE, TIME, AND MODE OF THE
APPOINTMENT.
REPORT | 22
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Appendix B: Cognitive Interview Instrument
MATERIALS NEEDED FOR INTERVIEW
□
□
□
□
INTERVIEWER PROTOCOL BOOKLET (THIS BOOKLET)
INFORMED CONSENT
RECORDING DEVICE/VOICE RECORDING APP
PENS AND PENCILS; SOFTWARE FOR DIGITAL NOTETAKING
STEP 1: INFORMED CONSENT
INT1. The Medicare Current Beneficiary Survey (MCBS) asks Medicare beneficiaries about their health
status, sources of health care, satisfaction with care, and health care expenditures. In today’s interview I
will be asking you about your health conditions and experiences health care providers.
Periodically during the interview, I will ask you some questions about how you decided to answer some
of the survey questions. Getting your feedback on the questions can help make the questions better.
[CONTINUE]
INT2. All survey information will be kept private to the extent permitted by law, as prescribed by the
Privacy Act of 1974. Medicare benefits will not be affected in any way by survey responses or
participation.
Do you agree to participate in this interview?
a. YES -> GO TO NEXT QUESTION
b. NO -> STOP INTERVIEW AND THANK THEM FOR THEIR TIME.
INT3. In order to have a complete record of your comments, with your permission, your interview session
will be audio taped. The recording will be stored electronically on NORC’s secure servers. We plan to use
the recording to ensure that we capture all of the feedback you provide us. Only staff at directly involved
in this research project will have access to the recording. Any quotes used in presentations and
publications will not include any names or any information that could identify any participant.
Is it okay for me to proceed with the recording on?
a. YES -> TURN RECORDING ON
b. NO -> PROCEED WITHOUT RECORDING
REPORT | 23
NORC
| 2023 Content Management Cycle Cognitive Testing Report
STEP 2: COMPLETION OF THE QUESTIONNAIRE
The Medicare Current Beneficiary Survey (MCBS) asks Medicare beneficiaries about their health status,
sources of health care, satisfaction with care, and health care expenditures. In today’s interview I will be
asking you about your recent health care and some of your medical conditions.
After we have finished the survey, I would like to talk with you about some of the questions in the survey.
Getting your feedback on the questions can help make the questions better.
[INTERVIEWER: EACH TEST QUESTION IN THE PROTOCOL IS IMMEDIATELY FOLLOWED
BY A SET OF EXAMPLE PROBES AND INTERVIEWER NOTES DESIGNED TO AID THE
DEBRIEFING PROCESS; WHILE SPECIFIC PROBES WILL VARY, THE EXAMPLES PROVIDED
IN THE PROTOCOL ARE MEANT TO BE A STARTING POINT FOR DIALOG WITH THE
RESPONDENT. ALTHOUGH THESE PROBES APPEAR ADJACENT TO THE QUESTIONS
THEMSELVES, IT IS GENERAL PRACTICE TO ADMINISTER THE QUESTIONNAIRE IN FULL
BEFORE USING THE EXAMPLE PROBES AND INTERVIEWER NOTES TO DEBRIEF THE
RESPONDENT. REMIND THE RESPONDENT OF THE QUESTION AND HIS/HER RESPONSE IN
ADVANCE OF USING PROBES.]
REPORT | 24
NORC
| 2023 Content Management Cycle Cognitive Testing Report
MEDICARE Savings Program
[INTERVIEWER READ Instructions]: We’re interested in learning more about how people covered
under Medicare navigate certain programs available to beneficiaries—particularly those meant to help
people pay for their healthcare costs. As you may know, the government has programs that help
beneficiaries pay for the costs associated Medicare such as Part A or Part B premiums, deductibles,
coinsurance, and copayments. We’re going to ask you a few questions about these programs, and what
your experience, if any, has been with them.
NEWVAR1
[Are you/Is (SP)] receiving this type
of help to pay for [your/his/her]
(CURRENT YEAR) health care
costs?
(01) YES
(02) NO
(-8) Don't Know
(-9) Refused
(01) skip rest of MSP series
(02) NEWVAR2
(-8) NEWVAR2
(-9) NEWVAR2
[EXPLAIN IF NECESSARY: These
programs are called Medicare
Savings Programs. There are four
different Medicare Savings Programs:
Qualified Medicare Beneficiary
Program, Specified Low-Income
Medicare Beneficiary Program,
Qualifying Individual Program, and
Qualified Disabled and Working
Individuals Program.]
Required probes:
• Please tell me in your own words what you think this question is asking.
• If receiving: How did you first become aware of these programs?
• If not receiving: Have you heard of these programs before?
NEWVAR2
Did [you/(SP)] apply to the [STATE]
Medicare office for help with
(CURRENT YEAR) expenses?
(01) YES
(02) NO
(-8) Don't Know
(-9) Refused
Required probes:
• Please tell me in your own words what you think this question is asking.
• What did you think about when answering this question? What factors did you consider?
NEWVAR3
Was [your/(SP's)] application for a
Medicare Savings Program accepted
or denied?
(01) ACCEPTED
(02) DENIED
(03) STILL PENDING/NO DECISION YET
(-8) Don't Know
(-9) Refused
REPORT | 25
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Required probes:
• Please tell me in your own words what you think this question is asking.
• What did you think about when answering this question? What factors did you consider?
• IF DENIED: Do you know why it was denied? What else are you doing now that it’s been
denied?
NEWVAR4 Before today, were you aware of
(01) YES
these programs?
(02) NO
(-8) Don't Know
(-9) Refused
Required probes:
• What did you know about these programs? Where did you get that information?
• If you didn’t know much, or wanted to know more after this conversation, where would you go
to get information?
REPORT | 26
NORC
| 2023 Content Management Cycle Cognitive Testing Report
RIGHT TO APPEAL/COMPLAINTS
These next questions ask about your experiences getting access to quality healthcare, and what you might
do if you have any challenges to getting access to quality healthcare.
1. Think about all of the times you’ve sought healthcare under Medicare coverage. Have you ever had a
difficult time getting an appointment with a healthcare provider you needed to see?
a. Yes
b. No
c. I don’t know
2. Think about all of the times you’ve sought healthcare under Medicare coverage. Have you ever had a
difficult time getting the care you needed during your appointment(s) with a healthcare provider you
saw?
a. Yes
b. No
c. I don’t know
3. Think about all of the times you’ve sought healthcare under Medicare coverage. Have you ever had a
difficult time getting your care paid for through your Medicare insurance with a healthcare provider
you saw because they were not submitting the correct claim information?
a. Yes
b. No
c. I don’t know
4. In your own words, can you tell me a bit about what you would do if you had difficulty with getting
an appointment, getting the care you need, or getting the care you received paid for properly?
5. Have you ever heard the term ‘right to appeal’? What do you think it means?
6. What do you know about filing a complaint, or your right to file a complaint, based on concerns you
have around the healthcare you’ve received?
[INTERVIEWER READ-ALOUD]: Everyone covered by Medicare has certain rights and protections
under their Medicare coverage.
These rights include the right to file a complaint or appeal any decision or action made by a medical
provider if you think you are being unfairly denied coverage, or denied adequate and complete treatment
of your condition.
An appeal is the action you can take if you disagree with a coverage or payment decision by Medicare or
your Medicare plan. For example, you can appeal if Medicare or your plan denies:
• A request for a health care service, supply, item, or drug you think Medicare should cover.
REPORT | 27
NORC
•
•
| 2023 Content Management Cycle Cognitive Testing Report
A request for payment of a health care service, supply, item, or drug you already got.
A request to change the amount you must pay for a health care service, supply, item, or drug.
You can also appeal:
• If Medicare or your plan stops providing or paying for all or part of a health care service, supply,
item, or drug you think you still need.
• An at-risk determination made under a drug management program that limits access to coverage
for frequently abused drugs, like opioids and benzodiazepines.
A complaint is about the quality of care you got or are getting. For example, you can file a complaint if
you have a problem calling the plan, or you're unhappy with how a staff person at the plan treated you.
You file an appeal if you have an issue with a plan's refusal to cover a service, supply, or prescription.
Learn more about appeals.
Thinking about this definition of your rights to complaints and appeals, please think about the following
questions.
1. If you felt that a healthcare provider was ending your healthcare services too soon, would you know
who to contact in Medicare to protect your rights and appeal?
2. If you had concerns about the safety of care you were receiving from a healthcare provider or facility,
would you know how to file a complaint with Medicare?
3. If you had concerns about the quality of care you were receiving from a healthcare provider or
facility, would you know how to file a complaint with Medicare?
REPORT | 28
NORC
| 2023 Content Management Cycle Cognitive Testing Report
USE OF CBD FOR PAIN MANAGEMENT
Q1: Now I would like to ask about your experiences with pain. In the past three months, how often did
you have pain? Would you say:
a. Never
b. some days
c. most days
d. every day
e. Don’t Know
f.
Refused
If response is “Never, Don’t Know, or Refused” then skip to next series of items (Bowel
Incontinence).
Otherwise, continue to Q2.
Q2: Since (TODAY'S MONTH AND YEAR - 3 MONTH), did you use any of the following to manage
your pain? Please indicate yes or no to each one:
Yes No
Physical therapy, rehabilitative therapy, or occupational therapy
Spinal manipulation or other forms of chiropractic care
Massage
Meditation, guided imagery, or other relaxation techniques
CBD (cannabidiol)
[IF NEEDED: CBD is different from Marijuana or THC. CBD
is one of the many compounds that can be present in the
cannabis plant and does not cause a “high” by itself. CBD is
available in different forms, such as oils and tinctures (extracts),
capsules (pills/softgels), edibles (gummies/candies), and topicals
(creams/lotions/ointments).]
REPORT | 29
NORC
| 2023 Content Management Cycle Cognitive Testing Report
BOWEL INCONTINENCE
Q1: We are now going to ask you some questions about your ability to control your bowel movements.
In the last 12 months, have you had any problems with leakage of stool, for example leaking gas,
accidents, or soiling?
[IF NEEDED: Was that because you were sick?]
SELECT 'NO' IF THE RESPONDENT HAD ANY PROBLEMS DUE TO A SHORT-TERM
DIARRHEAL ILLNESSES SUCH AS THE FLU OR A VIRUS.
a. Yes
b. No
c. Don’t Know
d. Refused
If response is “No, Don’t Know, or Refused” then skip to next series of items (Health Equity).
Otherwise, continue to Q2.
Q2: Did you have a problem with leaking gas only; a small amount of stool; a moderate amount of stool,
often requiring a change of underwear; or a large amount of liquid stool, often requiring a complete
change of clothes?
SELECT ALL THAT APPLY
(A) leaking gas only
(B) a small amount of stool
(C) a moderate amount of stool, often requiring a change of underwear; or
(D) a large amount of liquid stool, often requiring a complete change of clothes?
(E) Don’t Know
(F) Refused
Q3: In the last 12 months, did you have this problem more than once a week, about once a week, 2-3
times a month, about once a month, every 2-3 months, or once or twice a year?
[IF NEEDED: Include all types of stool leakage, including: gas only; a small amount of stool; a
moderate amount of stool, and/or a large amount of liquid stool.]
a. more than once a week
b. about once a week
c. 2-3 times a month
d. about once a month
REPORT | 30
NORC
| 2023 Content Management Cycle Cognitive Testing Report
e. every 2-3 months
e. once or twice a year?
f.
Don’t Know
g. Refused
Q4: Have you talked about your problem with stool leakage with [your/his/her] doctor or other health
professional?
[IF NECESSARY: This is also referred to as bowel or fecal incontinence.]
a. Yes
b. No
c. Don’t Know
d. Refused
Q5: Has your doctor or other health professional asked you about how you feel about this problem?
a. Yes
b. No
c. Don’t Know
d. Refused
Q6: Has your doctor or other health professional examined you to figure out why you have a problem
with stool leakage?
[IF NECESSARY: This is also referred to as bowel or fecal incontinence.]
a. Yes
b. No
c. Don’t Know
d. Refused
Q7: Has your doctor or other health professional talked with you about taking medicine or having surgery
for this problem?
[IF NECESSARY: Please answer based on your usual use of medication.]
a. Yes
b. No
c. Don’t Know
d. Refused
REPORT | 31
NORC
| 2023 Content Management Cycle Cognitive Testing Report
HEALTH EQUITY—Perceived Discrimination
Q1: In the last 12 months, did anyone from a clinic, emergency room, or doctor’s office where you got
care treat you in an unfair or insensitive way because of any of the following things about you?
(Response Options: Yes, No, Don’t know)
Yes
No
Don’t know if this
happened to me
Don’t know what this
question is asking
Race or ethnicity
Language or accent
Gender or gender identity
Sexual orientation
Age
Culture or religion
Disability
Medical history
REQUIRED PROBES:
-
-
What do you think these questions are asking?
IF DON’T KNOW: Can you tell me more about what you’re thinking? If you were to answer
these questions on your own (without me here) how do you think you’d guess at the meaning or
answer? Do you not know if this happened to you or do you not know what this question is
asking?
When you were/if you were ever treated negatively, what would you do about it? Would you file
a complaint or appeal with Medicare?
Are there any reasons you think you’ve been treated negatively, that we didn’t list here?
DEFINITIONS:
[INTERVIEWER READ AS NECESSARY] Please think about the following terms and definitions when
you answer these next questions. We are interested in how you personally identify, and whether you
think that has influenced your care in the [REFERENCE PERIOD] regardless of whether you have
told your care provider about these identities directly.
Race or ethnicity: This is how you think about or what you call your racial or ethnic background. For
example, Black, Dominican, Hispanic, Mexican, Asian-American, Chinese.
Age: This is how old you are, in years. You might also think of it as being older, younger, or middle
aged.
Sex: This is what is on your original birth certificate, such as male or female.
Gender: This is often what you call yourself, such as man or woman.
REPORT | 32
NORC
| 2023 Content Management Cycle Cognitive Testing Report
Gender Identity: This is often what you call yourself, and how you present yourself—such as man,
woman, nonbinary, or genderqueer.
Sexual orientation: This is how you describe your sexual preferences and identities, such as straight,
gay, lesbian, or bisexual.
Religion; This is the religion/religious preference with which you identify, such as Christian, Jewish,
Muslim, or Agnostic.
Disability: This may mean a physical, mental, or other disability that you have currently or have
experienced in the past year.
Health Equity—Religious Affiliation
Q1. What is your religious
preference?
IF NEEDED: Please select the
religion with which you most
closely identify.
Q2. What is your religious
preference?
(01) PROTESTANT
(01) End
(02) CATHOLIC
(02) End
(03) EASTERN ORTHODOX, SUCH AS
(03) End
GREEK OR RUSSIAN ORTHODOX
(04) JEWISH
(05) BUDDHIST
(06) HINDU
(07) MUSLIM, ISLAM, SUFI, SUNNI,
OR SHIA
(08) SOME OTHER RELIGION
(09) NO RELIGION
(10) DON'T KNOW
(11) REFUSED
(12) NA
(01) CONTINUOUS ANSWER
(04) End
(05) End
(06) End
(07) End
(08) Q2
(09) End
(10) End
(11) End
(12) End
END
REQUIRED PROBES:
STEP 3: THANK YOU AND INCENTIVE
[INTERVIEWER] Thank you for participating in the interview today. To thank you for your time, we’d
like to send you a check for $40. Could you please provide me with an address to mail the check to?
NAME: __________________________________
STREET: _________________________________________________________________
CITY: ___________________________________________________________________
STATE: __________________________ ZIP: _________________________________
REPORT | 33
| File Type | application/pdf |
| File Title | Attachment 7. 2023 Testing Report |
| Subject | MCBS; questionnaire; options paper |
| Author | NORC at the University of Chicago |
| File Modified | 2022-04-07 |
| File Created | 2022-03-24 |