Crosswalk of Nursing Home Survey

Crosswalk for Changes to
Evaluation of the CMS Network of Quality Improvement and Innovation Contractors Program (NQIIC) (CMS-10769)
Appendix A Nursing Home Survey Questions
#
1

2

Question Used in Pretests
Since March 2020, has your facility
changed or developed new processes or
protocols to prevent and control the spread
of COVID-19 infection?
a. Yes [GO TO Q2]
b. No [SKIP TO Q3]
Off the top of your head, what would you
consider the most important change you
made to your processes or protocols to
respond to the COVID-19 pandemic?
a. Response category 1
b. Response category 2
c. Response category 3
d. Other [open ended]

Recommendation/Decision
• Recommend underlining “your
facility”

Reason
As recommended by
the pretest, interviewer
needs to refer to the
facility, not the
respondent personally

As recommended by
• Remove “Off the top of your
the pretest
head”
• Include timeframe - “during the
first 6 months”
• Change to, “What would you
consider the top 2 or 3
changes...” Include note to
interviewer that they do not need
to be in order of importance

Recommended Revised Question
Since March 2020, has your facility changed or
developed new processes or protocols to prevent and
control the spread of COVID-19 infection?
a. Yes [GO TO Q2]
b. No [SKIP TO Q3]
During the first 6 months of COVID-19, what would
you consider the top two or three changes you made to
your processes or protocols to respond to the
pandemic? [DO NOT NEED TO BE IN ORDER OF
IMPORTANCE]
a. Response category 1
b. Response category 2
c. Response category 3
d. Other [open ended]

1

#
4

Question Used in Pretests
I’m going to read a list of programs,
agencies, or resources that you may have
used over the past 18 months to improve
the quality of care to residents at your
facility. As I read each one, please tell me
if you recall working with or using this
resource over the last 18 months or since
the beginning of the COVID-19 pandemic.
a. Project ECHO or AHRQ’s
(pronounced “ARK”) National
Nursing Home COVID-19
Coordinating Center or AHRQ’s
National Nursing Home COVID-19
Action Network”)
b. The Quality Improvement
Organization (QIO) Program, or
QIOs, also known as [QIO Name] in
your area
c. Government agencies like the Centers
for Disease Control and Prevention,
or CDC, or the [INSERT NAME
FOR STATE DEPARTMENT OF
HEALTH]
d. CMS Scenario-based Training also
known as the CMS Targeted COVID19 Training for Nursing Homes
(online modules for frontline staff and
nursing home management)
e. AHCA (American Health Care
Association) or state chapter of
AHCA resources
f. Leading Age or [INSERT STATE]
Chapter of Leading Age resources
Are there any other sources of information
you used when working on preventing and
controlling the spread of COVID-19?
Record response #1
Record response #2
Record response #3
NA. No additional sources of
information

Recommendation/Decision
Reason
• A – Just include “Project Echo” As recommended by
BUT If the interviewee responds the pretest
‘no,’ the interviewer will read the
entire response item to clarify
and confirm the interviewee
understands what is meant by
Project Echo.
• C – Split into 2 response items.
o Just use “CDC”
o Type in state name for
“State Department of
Health”
• D- Change to “CMS targeted
COVID-19 training” BUT If the
interviewee responds ‘no,’ the
interviewer will read the entire
response item to clarify and
confirm the interviewee
understands the question refers
to CMS Scenario-based
Training.
• E – Remove “resources” and
type in name of state
• F – Remove question and record
response if it is mentioned in
“other sources of information”
• ADD response items
o “Peers and/or other
facilities”
o “Local (county or city)
departments of health”

Recommended Revised Question
I’m going to read a list of programs, agencies, or
resources that you may have used over the past 18
months to improve the quality of care to residents at
your facility. As I read each one, please tell me if you
recall working with or using this resource over the last
18 months or since the beginning of the COVID-19
pandemic.
a. Project ECHO [IF RESPONDENT ANSWERS
‘NO’ PROVIDE ADDITIONAL INFORMATION
TO CLARIFY: Project ECHO (Extension for
Community Healthcare Outcomes) or AHRQ’s
(pronounced “ARK”) National Nursing Home
COVID-19 Coordinating Center or AHRQ’s
National Nursing Home COVID-19 Action
Network”]
b. The Quality Improvement Organization (QIO)
Program, or QIOs, also known as [QIO Name] in
your area
c. CDC
d. [TYPE IN NAME FOR STATE DEPARTMENT
OF HEALTH]
e. CMS targeted COVID-19 training [IF
RESPONDENT ANSWERS ‘NO’ PROVIDE
ADDITIONAL INFORMATION TO CLARIFY:
Scenario-based Training also known as the CMS
Targeted COVID-19 Training for Nursing Homes
(online modules for frontline staff and nursing
home management)]
f. AHCA (American Health Care Association) or
[TYPE IN NAME OF STATE] chapter of AHCA
g. Peers and/or other facilities
h. Local departments of health (city/county)
Are there any other sources of information you used
when working on preventing and controlling the spread
of COVID-19?
Record response #1
Record response #2
Record response #3
NA. No additional sources of information

2

#
5

6

7

9

Question Used in Pretests
You mentioned that you use [INSERT
ANSWERS FROM Q4], Off the top of
your head, what would you consider the
sources of information, guidance, or
assistance that had the most impact on
your facility’s ability to prevent and
control the spread of COVID-19? [Open
ended]
Have you ever heard of Quality
Improvement Organizations, otherwise
referred to as QIOs?
a. Yes
b. No
Have you ever heard of [QIO Name or
QIO contact person]?
a. Yes
b. No
Which best describes the reason(s) why
this facility was not fully engaged with
[QIO Name or ‘the QIO that serves your
area’] while you were working to prevent
and control the spread of COVID-19?
Would you say this is because…
[Enter all that apply]
a. No such opportunity presented itself
b. The resources seemed redundant with
other efforts we are involved in
c. We had all the support needed within
this facility
d. The assistance available didn’t seem
to be helpful or worth the effort
e. We didn’t have enough time to
participate in another effort
f. The quality of resources or assistance
from [QIO Name or ‘the QIO that
serves your area’] was sub-optimal
g. The scheduling was inconvenient
h. Other [Open ended]

Recommendation/Decision
Reason
Remove “Off the top of your head” As recommended by
the pretest

Recommended Revised Question
You mentioned that you use [INSERT ANSWERS
FROM Q4], What would you consider the sources of
information, guidance, or assistance that had the most
impact on your facility’s ability to prevent and control
the spread of COVID-19? [Open ended]

As recommended by
the pretest and to be
consistent with
question wording in
Hospital Survey
As recommended by
the pretest

Before this survey, had you ever heard of Quality
Improvement Organizations, otherwise referred to as
QIOs?
a. Yes
b. No
The name of your QIO is [QIO Name], have you ever
heard of them? [CONFIRM NAME IS IN LIST]
a. Yes
b. No

• Add “Before this survey” to the
beginning of the sentence and
change “Have” to “had.”

• Change to “The name of your
QIO is [QIO Name], have you
ever heard of them?”
• Add note for the interviewer to
confirm that we have the contact
person in the list
“Redundant” sounds
• B - Remove “redundant” and
negative
replace with “similar” or
“overlapping” efforts we were
involved with
• G – Reword to say, “We could
not find a convenient time to
schedule meetings”

Which best describes the reason(s) why this facility
was not fully engaged with [QIO Name or ‘the QIO
that serves your area’] while you were working to
prevent and control the spread of COVID-19? Would
you say this is because…
[Enter all that apply]
b. The resources seemed similar to or overlapped
with other efforts we are involved in
g. We could not find a convenient time to schedule
meetings

3

#
Question Used in Pretests
Recommendation/Decision
10 To the best of your knowledge did your
• Replace “changes” with
interactions with [QIO Name or ‘the QIO
“improvements”
that serves your area’] result in any
changes to the processes or protocols
your facility uses to prevent COVID19 infection, or infection control in
general?
a. Yes [GO TO Q11]
b. No [IF Q8 = c OR d (minimally or
not engaged), GO TO Q13; ELSE GO
TO Q12]
11 Please describe what processes or
• Remove “processes or protocols”
protocols changed as a result of the
and revise to “Please describe
interactions with [QIO Name or ‘the QIO
what improved…”
that serves your area.’] (PROBE)
a. Record response #1
b. Record response #2
c. Record response #3
d. Other [GO TO Q13]

Reason
As recommended by
the pretest

Recommended Revised Question
To the best of your knowledge did your interactions
with [QIO Name or ‘the QIO that serves your
area’] result in any improvements to the processes or
protocols your facility uses to prevent COVID19 infection, or infection control in general?
a. Yes [GO TO Q11]
b. No [IF Q8 = c OR d (minimally or not
engaged), GO TO Q13; ELSE GO TO Q12]

As recommended by
the pretest

Please describe what improved as a result of the
interactions with [QIO Name or ‘the QIO that serves
your area.’] (PROBE)

4

#
Question Used in Pretests
12 You said earlier that your facility did not
change processes or procedures from
working with [QIO Name or ‘the QIO that
serves your area.’] Which response(s) best
describes why that was the case?
[Select all that apply]
a. Our facility already made changes
needed before we began working with
[QIO Name or ‘the QIO that serves
your area’]
b. The resources from [QIO Name or
‘the QIO that serves your area’]
seemed redundant with other efforts
we were using
c. We had all the support needed within
this facility
d. The assistance available didn’t seem
to be helpful or worth the effort
e. We didn’t have time to implement the
changes the [QIO Name or ‘the QIO
that serves your area’] recommended
f. The quality of resources or assistance
from [QIO Name or ‘the QIO that
serves your area’] was sub-optimal
g. The scheduling was inconvenient
h. Other [Open ended]
14 How satisfied are you with the amount of
contact between your facility and [QIO
Name or ‘the QIO that serves your area’]?
Would you say the amount of contact is…
a. Not enough
b. Just about right
c. Too much

Recommendation/Decision
• Change “procedures” to
“protocols”

Reason
As recommended by
the pretest

Recommended Revised Question
You said earlier your interactions with [QIO Name or
‘the QIO that serves your area’] did not result in any
improvements to the processes or
protocols your facility uses to prevent COVID19 infection, or infection control in general? Which
response(s) best describes why that was the case?
[Select all that apply]
b. The resources from [QIO Name or ‘the QIO that
serves your area’] seemed similar to or overlapped
with other efforts we are involved in
g. We could not find a convenient time to schedule
meetings

• Add “Overall” before “How” and As recommended by
change question to Likert scale: the pretest
“Overall, how satisfied are
you...”
• Add:
- Don’t know/not sure
- Declined to answer

Overall, how satisfied are you with the amount of
contact between your facility and [QIO Name or ‘the
QIO that serves your area’]? Would you say you are…
a. Very Satisfied
b. Somewhat Satisfied
c. Neither Satisfied or Dissatisfied
d. Somewhat Dissatisfied
e. Very Dissatisfied
f. Don’t know/not sure
g. Declined to answer

• B – Remove “redundant” and
replace with “similar” or
“overlapping” efforts we were
involved with
• G - Reword to say, “We could
not find a convenient time to
schedule meetings”

5

#
Question Used in Pretests
16 How satisfied are you with the timeliness
of the QIO’s response to your question
or request for assistance? Would you say
you are…
a. Very Satisfied
b. Somewhat Satisfied
c. Neither Satisfied nor Dissatisfied
d. Somewhat Dissatisfied
e. Very Dissatisfied

Recommendation/Decision
• Add “Overall” before “How”
• Replace “the QIO’s” with [QIO
Name or ‘the QIO that serves
your area,’]
• Add:
- Don’t know/not sure
- Declined to answer

Reason
As recommended by
the pretest

17 Thinking about all interactions with [QIO • Move “Overall” so it is before
Name or ‘the QIO that serves your area,’]
“How”
how satisfied are you with
• Replace “the QIO” with “[QIO
your relationship with the QIO
NAME]”
overall? Would you say you are…
• Add:
a. Very Satisfied
- Don’t know/not sure
b. Somewhat Satisfied
- Declined to answer
c. Neither Satisfied nor Dissatisfied
d. Somewhat Dissatisfied
e. Very Dissatisfied

As recommended by
the pretest

18 Please indicate your level of agreement
with the following statements about the
information and assistance provided by
the QIO.
The assistance we received from the QIO
was key to the efficient implementation
of our protocols to prevent and control
COVID-19 infection. Would you…
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree

As recommended by
the pretest

• Replace “the QIO” with “[QIO
NAME]”
• Underline the word “key”
• Revise “Would you…” to
“Would you say you…”
• Add:
- Don’t know/not sure
- Declined to answer

Recommended Revised Question
Overall, how satisfied are you with the timeliness of
[QIO Name or ‘the QIO that serves your area,’]
response to your question or request for assistance?
Would you say you are…
a. Very Satisfied
b. Somewhat Satisfied
c. Neither Satisfied or Dissatisfied
d. Somewhat Dissatisfied
e. Very Dissatisfied
f. Don’t know/not sure
g. Declined to answer
Thinking about all interactions with [QIO Name or ‘the
QIO that serves your area,’], overall, how satisfied are
you with your relationship with [QIO NAME]? Would
you say you are…
a. Very Satisfied
b. Somewhat Satisfied
c. Neither Satisfied or Dissatisfied
d. Somewhat Dissatisfied
e. Very Dissatisfied
f. Don’t know/not sure
g. Declined to answer
Please indicate your level of agreement with the
following statements about the information
and assistance provided by the QIO.
The assistance we received from [QIO NAME] was
key to the efficient implementation of our protocols to
prevent and control COVID-19 infection. Would you
say you…
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree
f. Don’t know/not sure
g. Declined to answer

6

#
Question Used in Pretests
19 The service we received from this QIO
was worth the time or effort required on
the part of our staff. Would you …
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree

Recommendation/Decision
• Replace “this QIO” with “[QIO
NAME]”
• Revise “Would you…” to
“Would you say you…”
• Add:
- Don’t know/not sure
- Declined to answer

Reason
As recommended by
the pretest

20 Our organization has benefited from
having received services from this
QIO. Would you…
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree

• Replace “this QIO” with “[QIO
NAME]”
• Revise “Would you…” to
“Would you say you…”
• Add:
- Don’t know/not sure
- Declined to answer

As recommended by
the pretest

22 What quality improvement areas are you
most in need of for additional assistance?
[Open ended]
NA. No quality improvement areas are in
need of additional assistance

• Add “at this point” to the
beginning of the sentence

As recommended by
the pretest

Recommended Revised Question
The service we received from [QIO NAME] was worth
the time or effort required on the part of our staff.
Would you say you …
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree
f. Don’t know/not sure
g. Declined to answer
Our organization has benefited from having received
services from [QIO NAME]. Would you say you…
a. Strongly Agree
b. Somewhat Agree
c. Neither Agree nor Disagree
d. Somewhat Disagree
e. Strongly Disagree
f. Don’t know/not sure
g. Declined to answer
At this point, what quality improvement areas are you
most in need of for additional assistance? [Open ended]
NA. No quality improvement areas are in need of
additional assistance

7