CMS-10294.Appendix E (8-23-10)

CMS-10294.Appendix E (8-23-10).pdf

Program Evaluation of the Ninth Scope of Work Quality Improvement Organization Program (CMS-10294)

CMS-10294.Appendix E (8-23-10)

OMB: 0938-1104

Document [pdf]

Download: pdf | pdf

APPENDIX E
PROVIDER SURVEYS AND LETTERS

Mathematica Reference No.: 06514.260

Ninth Scope of Work
QIO Program
Evaluation: Hospital
Survey
Final Draft Questionnaire
January 5, 2010

A. INTRODUCTION

INITIAL CONTACT WITH HOSPITAL: Hello, may I please speak with [NAME OF CEO/The
CEO/or Assistant to the CEO of your hospital]?
IF YOU DON’T HAVE (HIS/HER) NAME,
ASK FOR IT AND RECORD HERE:
IF NEEDED: My name is ________ and I’m calling on behalf of the Centers for Medicare &
Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF CEO/the CEO or
CEO’s assistant] a letter from CMS requesting the hospital’s participation in a survey for an
evaluation of the Ninth Scope of Work of the Quality Improvement Organization Program.
A1.

WHEN SPEAKING WITH CEO/Assistant to the CEO: Hello, [Dr./Mr./Ms.]
[LAST NAME], my name is __________, and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS. A short time ago, you should have received a
letter from CMS requesting your hospital’s participation in a survey for an evaluation of
the Ninth Scope of Work of the Quality Improvement Organization Program.
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made. FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning
how its efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
We would like to conduct a brief interview with the QI Director (or whoever is directly
involved with quality improvement at the hospital). Would you please give me the
name and contact information for this person?
YES .......................................................................... 1

GO TO A2

NO ............................................................................ 0
SET CALLBACK
DON’T KNOW .......................................................... d
A2.

RECORD NAME AND CONTACT INFORMATION FOR QI DIRECTOR.

Prepared by Mathematica Policy Research

E.8

|-|

A3.

FOLLOW-UP INFORMATION:
PURPOSE:
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how well the QIOs are working to help hospitals improve
quality and patient safety and what changes if any, need to be made.
FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning how its
efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
OR SAY: It is critical that our study understand quality improvement processes and
thinking in hospitals that have not been working with QIOs as well as those that have,
in order to understand the added value of the QIO program.

CONFIDENTIALITY: Please be assured that your responses to the survey will remain
confidential to the extent permitted by law. All data collected for the purposes of this
study will be combined and reported in aggregate form only. Neither you nor your
organization will be identified by name in any reports or documents produced from the
study findings. Only Mathematica staff that work directly on the evaluation will have
access to the name of your organization and your name.
SELECTION: Your hospital was randomly selected from U.S. hospitals so as to
support evaluation of CMS’ Quality Improvement Program. Some were selected to
represent participating hospitals that worked with QIOs and some were selected to
represent hospitals that did not work with QIOs.
A4.

INTRODUCTION WITH QI DIRECTOR:
Hello, may I please speak with [NAME OF QI DIRECTOR]?
IF NEEDED: My name is ___________ and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF CEO] a
letter from CMS requesting the hospital’s participation in a survey for an evaluation of
the Ninth Scope of Work of the Quality Improvement Organization Program.

Prepared by Mathematica Policy Research

E.9

A5.

WHEN SPEAKING WITH QI DIRECTOR:
Hello, [Dr. /Mr. /Ms.] [LAST NAME], my name is ________, and I’m calling on behalf of
the Centers for Medicare & Medicaid Services or CMS. We recently spoke with
[NAME OF CEO], who gave us your name as someone directly involved with quality
improvement at the hospital. We are conducting a survey for an evaluation of the Ninth
Scope of Work of the Quality Improvement Organization Program.
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made. FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning
how its efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
from outside organizations even if they are not currently working with CMS programs
on quality issues.
The survey interview takes roughly 30 minutes, depending upon your answers. I can
conduct it now, or at any time that's convenient for you.
START INTERVIEW NOW ....................................... 1

GO TO B1

NOT NOW, SET UP APPT/CALLBACK ................... 2

SET APPT.

NEEDS MORE INFORMATION ............................... 3

GO TO FU SCREEN

REFUSED ................................................................ r

REFUSAL

Prepared by Mathematica Policy Research

E.10

FOLLOW-UP INFORMATION SCREEN:
PURPOSE:
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made.
FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning how its
efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
OR SAY: It is critical that our study understand quality improvement processes and
thinking in hospitals that have not been working with QIOs as well as those that have,
in order to understand the added value of the QIO program.
CONFIDENTIALITY: Please be assured that your responses to the survey will remain
confidential to the extent permitted by law. All data collected for the purposes of this
study will be combined and reported in aggregate form only. Neither you nor your
organization will be identified by name in any reports or documents produced from the
study findings. Only Mathematica staff that work directly on the evaluation will have
access to the name of your organization and your name.
SELECTION: Your hospital was randomly selected from U.S. hospitals so as to
support evaluation of CMS’ Quality Improvement Program. Some were selected to
represent participating hospitals that worked with QIOs and some were selected to
represent hospitals that did not work with QIOs.
REFUSAL SCREEN: Thank you for your time. Have a nice day.

Prepared by Mathematica Policy Research

E.11

B. QIO INTERACTIONS

The first few questions are about staff interactions with [NAME OF QIO].
B1.

Is this hospital participating with [NAME OF QIO] on a quality improvement initiative
related to any of the following topics… [READ DOWN LIST]

YES

DON’T
KNOW

REFUSED

a. Surgical Care Infection Prevention?..................

b. Heart Failure? ...................................................

c. Pressure Ulcers Reduction? .............................

d. MRSA Infection Prevention and Transmission
Reduction? ........................................................

e. Care Transitions (Reducing Readmissions)?....

Any Other Topic? (SPECIFY) ..........................

B1aa.

[FOR EACH CATEGORY THAT RECEIVED A 'NO' RESPONSE, ASK B1aa AFTER
YOU HAVE READ THROUGH THE ENTIRE LIST IN B1.]
Were you invited to participate with [NAME OF QIO] on a quality improvement initiative
related to [TOPIC]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B2

REFUSED ................................................................ r
B1ab.

Why did you choose not to participate with [NAME OF QIO] on a quality improvement
initiative? RECORD VERBATIM

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.12

B2.

The next few questions ask about how often hospital staff may have met with [NAME
OF QIO], either in-person or by telephone. Since August 2008, how many times have
hospital personnel met with [NAME OF QIO] in-person at this hospital?
PROBE: Your best estimate is fine.
|

| TIMES

NONE ....................................................................... 0
DON’T KNOW .......................................................... d

GO TO B3

REFUSED ................................................................ r

B2a.

[IF B2 = 2 OR MORE, DISPLAY “How often did…”; IF B2 = 1, DISPLAY “Did…”]
(How often did/Did) the following people attend the meeting with [NAME OF QIO]
[READ ITEM]?
[IF B2 = 2 OR MORE, DISPLAY: Would you say always, usually, sometimes, or
never?] [IF B2 = 1, DISPLAY ONLY CATEGORIES “ALWAYS” AND “NEVER”]

ALWAYS

USUALLY

SOMETIMES

NEVER

DON’T
KNOW

REFUSED

a. Physician leaders for the
clinical areas being
discussed? .............................

b. One or more members of
senior hospital (“C-Suite”)
leadership?.............................

c. The quality improvement
director? .................................

d. Nursing leadership? ...............

e. Any other leadership staff?
(SPECIFY) .............................

Prepared by Mathematica Policy Research

E.13

B3.

Since August 2008, how many other in-person meetings have hospital personnel
attended where [NAME OF QIO] was an active participant? Please include in-person
meetings held inside and outside the hospital.
PROBE: Your best estimate is fine.
|

| TIMES

NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B4.

Since August 2008, approximately how many times have hospital personnel met
by phone with [NAME OF QIO]? Please do not include large conference calls.
PROBE: Your best estimate is fine.
|

| TIMES

Since August 2008, how many other telephone conference calls or web-ex meetings
have hospital personnel attended that [NAME OF QIO] convened?
PROBE: Your best estimate is fine.
|

| TIMES

Prepared by Mathematica Policy Research

E.14

B6.

I’m going to read a list of reasons why you might have met with [NAME OF QIO] since
August 2008. After each one, please tell me if this was a reason for (any of) the
in-person or phone meeting(s) with [NAME OF QIO]. [READ LIST]
[ROTATE ORDER OF LIST, BUT ALWAYS END WITH CATEGORY “n” LAST]

YES

DON’T
KNOW

REFUSED

a. Complaint or case review follow-up .........................

b. Understanding [NAME OF QIO]’s plans for
activities and opportunities to participate .................

c. Routine meetings as part of participating with
[NAME OF QIO] on a quality improvement effort .....

d. To discuss this hospital’s performance data ............

e. Applying [NAME OF QIO] staff’s expertise to
improve this hospital’s routine performance
measurement ...........................................................

Hearing about best practices of other hospitals .......

g. To learn about a new tool or recommended
process for quality improvement ..............................

h. Other staff development or training ..........................

Presentation(s) to help create buy-in to
quality improvement beyond the quality
improvement staff.....................................................

To discuss issues, methods, and/or timeframes for
quality reporting to CMS...........................................

k. To attend a broad-based regional or statewide
meeting on quality improvement where
[NAME OF QIO] was an active participant ...............

This hospital asked for and received some
assistance or information from the QIO ...................

m. To receive assistance regarding reporting of quality
data for the Reporting of Hospital Quality Data for
Annual Payment Update, or “RHQDAPU”................

n. Any other reason? (SPECIFY) ................................

Prepared by Mathematica Policy Research

E.15

B7.

Which of the following describe the aim(s) of the assistance you received?
[FOR THOSE WHO RESPOND ‘YES’ TO ANY ITEM IN B6]
IMPROVE OR ENSURE COMPLETENESS
OF THE REPORTED DATA ..................................... 1
ADVISE ON ISSUES RELATED TO
TRANSMISSION OF THE DATA ............................. 2
OTHER (SPECIFY) .................................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B7a.

How effective was this assistance in achieving its aim(s)?
VERY EFFECTIVE ................................................... 1
SOMEWHAT EFFECTIVE ....................................... 2
NOT EFFECTIVE ..................................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.16

B8.

How valuable to the hospital was meeting (about) [FILL EACH CATEGORY THAT
RECEIVED A YES RESPONSE AT B6]?
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES TO B6]
[READ ITEM] Would you say it was very valuable, somewhat valuable, or not
valuable?
[IF ‘VERY VALUABLE’, ‘SOMEWHAT VALUABLE’, OR ‘NOT VALUABLE’ IS
CHOSEN, ASK B9 OR B10 BEFORE MOVING ON TO NEXT ITEM]
VERY
VALUABLE

SOMEWHAT
VALUABLE

NOT
VALUABLE

a. Complaint or case review follow-up ...

b. Understanding [NAME OF QIO]’s
plans for activities and opportunities
to participate.......................................

c. Routine meetings as part of
participating with [NAME OF QIO] on
a quality improvement effort ...............

d. To discuss this hospital’s
performance data ...............................

e. Applying [NAME OF QIO] staff’s
expertise to improve this hospital’s
routine performance measurement ....

Hearing about best practices of other
hospitals .............................................

g. To learn about a new tool or
recommended process for quality
improvement ......................................

h. Other staff development or training ....

Presentation(s) to help create buy-in
to quality improvement beyond the
quality improvement staff ...................

To discuss issues, methods, and/or
timeframes for quality reporting to
CMS ...................................................

k. To attend a broad-based regional or
statewide meeting on quality
improvement where [NAME OF QIO]
was an active participant ....................

This hospital asked for and received
some assistance or information from
the QIO...............................................

m. Other reason (SPECIFY) ...................

Prepared by Mathematica Policy Research

E.17

DON’T
KNOW

REFUSED

B9.

[IF ‘NOT VALUABLE’ SELECTED AT B8, ASK B9 IMMEDIATELY AFTER B8:]
Why were these meetings not valuable?
CODE ALL THAT APPLY
THE MEETING MERELY FULFILLED AN
OBLIGATION (SUCH AS COOPERATION
WITH CASE REVIEWS) ........................................... 1
THIS HOSPITAL IS SO ADVANCED, THERE
IS NOTHING WE CAN LEARN FROM
[NAME OF QIO] ....................................................... 2
[NAME OF QIO] STAFF DID NOT HAVE
ENOUGH EXPERTISE OF THE RIGHT TYPE ........ 3
[NAME OF QIO] STAFF WAS NOT
WELL-PREPARED ................................................... 4
THE RIGHT PEOPLE WERE NOT
AT THE MEETING ................................................... 5
THE INFORMATION PROVIDED WAS
NOT APPLICABLE TO THIS HOSPITAL’S
SITUATION .............................................................. 6
THE MEETING WAS REDUNDANT WITH
INFORMATION WE ALREADY HAD ....................... 7
PROGRESS AT THIS HOSPITAL ON THE
MEETING TOPIC IS NOT FEASIBLE AT
THIS TIME ................................................................ 8
OTHER (SPECIFY) .................................................. 9

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.18

B10.

[IF ‘VERY VALUABLE’ OR ‘SOMEWHAT VALUABLE’ SELECTED AT B8, ASK B10
IMMEDIATELY AFTER B8:]
In what ways were these meetings valuable?
CODE ALL THAT APPLY
PROVIDED NEW, USEFUL INFORMATION ON
HOW TO IMPROVE PERFORMANCE .................... 1
PROVIDED NEW, USEFUL INFORMATION
ON ANOTHER TOPIC (SUCH AS QUALITY
REPORTING, USE OF EHR) ................................... 2
INCREASED MOTIVATION TO IMPROVE
FROM ONE OR MORE EXECUTIVES .................... 3
INCREASED MOTIVATION TO IMPROVE
FROM KEY PHYSICIANS ........................................ 4
INCREASED MOTIVATION TO IMPROVE
FROM OTHER KEY HOSPITAL STAFF .................. 5
OTHER (SPECIFY) .................................................. 6

DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B11.

Did any of the meetings lead to changes at the hospital that ultimately improved patient
care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B12

REFUSED ................................................................ r
B11a.

Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.19

GO TO B12

B11b.

Please tell me which measures were improved.
[READ LIST IF NEEDED]
SCIP (SURGICAL CARE INFECTION PREVENTION)
Surgery patients on a beta blocker prior
to arrival who received a beta blocker
during the perioperative period ................................. 1
Prophylactic antibiotic received on time—within
one hour prior to surgical incision ............................. 2
Prophylactic antibiotic selection for
surgical patients ....................................................... 3
Prophylactic antibiotics discontinued within
24 hours after surgery end time................................ 4
Cardiac surgery patients with controlled 6 am
postoperative serum glucose ................................... 5
Surgery patients with appropriate hair removal ........ 6
Surgery patients with recommended VTE
prophylaxis ordered .................................................. 7
Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ............................. 8
HEART FAILURE
Heart failure patients with left ventricular
systolic dysfunction without ACEI and
ARB contraindications who are prescribed
ACEI/ARB at discharge ............................................ 9
PRESSURE ULCERS
Incidence of pressure ulcers .................................... 10
MRSA
MRSA-1 Infection rate .............................................. 11
MRSA-2 Transmission rate ...................................... 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.20

B12.

(In addition to meetings), did you receive other educational materials, tools, or quality
improvement news from [NAME OF QIO]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B13

REFUSED ................................................................ r
B12a.

Did you receive these items from . . .
CODE ALL THAT APPLY
A newsletter, ............................................................. 1
An email or listserv, .................................................. 2
At in-person meeting, ............................................... 3
At a teleconference or web-ex, ................................. 4
Or some other way? (SPECIFY) ............................. 5
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B12b.

Overall, how valuable were these educational materials and/or tools? Would you
say . . .
Very valuable, ........................................................... 1
Somewhat valuable, ................................................. 2
Not very valuable, or ................................................ 3
Not at all valuable? ................................................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B12c.

Did any of these educational materials or tools from [NAME OF QIO] thus far lead to
changes at the hospital that ultimately improved patient care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.21

GO TO B13

B12d.

GO TO B13

REFUSED ................................................................ r
B12e.

Please tell me which measures were improved. [READ LIST IF NEEDED]
SCIP (SURGICAL CARE INFECTION PREVENTION)
Surgery patients on a beta blocker prior
to arrival who received a beta blocker
during the perioperative period ................................. 1
Prophylactic antibiotic received on time—within
one hour prior to surgical incision ............................. 2
Prophylactic antibiotic selection for
surgical patients ....................................................... 3
Prophylactic antibiotics discontinued within
24 hours after surgery end time................................ 4
Cardiac surgery patients with controlled 6 am
postoperative serum glucose ................................... 5
Surgery patients with appropriate hair removal ........ 6
Surgery patients with recommended VTE
prophylaxis ordered .................................................. 7
Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ............................. 8
HEART FAILURE
Heart failure patients with left ventricular
systolic dysfunction without ACEI and
ARB contraindications who are prescribed
ACEI/ARB at discharge ............................................ 9
PRESSURE ULCERS
Incidence of pressure ulcers .................................... 10
MRSA
MRSA-1 Infection rate .............................................. 11
MRSA-2 Transmission rate ...................................... 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.22

B13.

Do you routinely receive data feedback from [NAME OF QIO] on this hospital’s quality
performance?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B14

REFUSED ................................................................ r
B13a. How widely within the hospital do you typically share the feedback [NAME OF QIO]
provides, or highlights that you derive from it? Is it . . .
CODE ONE ONLY
Shared with a wide array of relevant
physicians and staff, ................................................. 1
Shared with a few key individuals, or ....................... 2
Rarely or never shared? ........................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B13b.

Has the feedback from [NAME OF QIO] on this hospital’s quality performance been
important to the hospital’s quality improvement efforts?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B14.

Did this hospital complete a Hospital Leadership Quality Assessment Tool© survey?
PROBE: It is also known as the HLQAT (“HELLCAT”) survey.
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.23

GO TO C1

B15.

Did the results help identify things that could be strengthened to better support quality
improvements at the hospital?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B16.

Have any changes been made as a result of the survey or related follow-up that
strengthened quality at the hospital?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO C1

REFUSED ................................................................ r
B16a.

Would you consider these changes to be important or not very important?
IMPORTANT ............................................................ 1
NOT VERY IMPORTANT ......................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.24

C. OTHER QI INITIATIVES

EXTERNAL INITIATIVES
The next questions are about quality improvement initiatives that involve external organizations.
C1.

Is this hospital part of or affiliated with a larger organization with central quality
expertise and an array of quality initiatives that extend to owned or affiliated
organizations?
YES, LARGER ORGANIZATION ............................. 1
YES, AFFILIATED ORGANIZATION........................ 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C1a.

To what extent are your hospital’s quality improvement actions influenced by this
(larger/affiliated organization)? Would you say to a large extent, a moderate extent,
or a small or no extent?
LARGE EXTENT ...................................................... 1
MODERATE EXTENT .............................................. 2
SMALL OR NO EXTENT .......................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C2.

Setting aside any interactions with [NAME OF QIO] (and the (larger/affiliated)
organization just discussed), is the hospital actively involved in any other quality
improvement efforts involving outside organizations?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.25

GO TO C3

C2a.

Please describe the nature of each of these initiatives and the sponsoring organization.
RECORD VERBATIM
PROBE FOR BOTH INITIATIVE AND SPONSOR.
PROBE: Please do not include efforts where the hospital’s participation is not active
or that focus on quality reporting but not improvement.

DON’T KNOW .......................................................... d
REFUSED ................................................................ r
INTERNAL INITIATIVES
Now I’d like to ask about internal quality improvement initiatives within the hospital.
C3.

How many full-time equivalent staff are currently devoted to quality improvement in the
hospital?
PROBE: For example, if the hospital has four staff who each devote a quarter time to
quality improvement, then the number of full-time equivalent staff at the
hospital is one.
PROBE: Your best estimate is fine.
|

| NUMBER OF FTE’s

LESS THAN ONE (OR A FRACTION) ..................... f
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.26

C4.

Since August 2008, please indicate if the hospital has had internal quality improvement
efforts that have improved hospital performance on any of the following measures . . .
[READ DOWN LIST]
YES

DON’T
KNOW

REFUSED

Surgery patients on a beta blocker prior to
arrival who received a beta blocker during
the perioperative period ................................

b. Prophylactic antibiotic received on time—
within one hour prior to surgical incision .......

c. Prophylactic antibiotic selection for surgical
patients..........................................................

d. Prophylactic antibiotics discontinued within
24 hours after surgery end time ....................

e. Cardiac surgery patients with controlled
6 AM postoperative serum glucose ...............

Surgery patients with appropriate hair
removal .........................................................

g. VTE prophylaxis measures? .........................

Surgery patients with recommended VTE
prophylaxis ordered.......................................

h. Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ..................

Heart failure patients prescribed ACEI/ARB
at discharge?.................................................

(heart failure patients with left ventricular
systolic dysfunction without ACEI and ARB
contraindications who are prescribed
ACEI/ARB at discharge)................................

Incidence of pressure ulcers .........................

k. MRSA-1 Infection rate ...................................

a. SCIP measures? ...........................................

NOT
APPLICABLE

IF YES, ASK: Which ones? [IF NO, GO TO g]

IF YES, ASK: Which ones? [IF NO, GO TO i]

MRSA-2 Transmission rate ...........................

Prepared by Mathematica Policy Research

E.27

C5.

Has the hospital undertaken an analysis on any of the measures to identify the reasons
why the relevant guideline sometimes is not followed, or why the undesirable outcome
sometimes occurs?
YES .......................................................................... 1
NO ............................................................................ 0
NOT APPLICABLE – PERFORMANCE
IS 100% RELIABLE .................................................. n

GO TO C7

DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C5a.

For which measures has the hospital undertaken an analysis? [READ LIST IF
NECESSARY]
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM C4]
PROBE: IF RESPONDENT SAYS “SCIP MEASURES,” ASK: “Which ones?”
ANALYSIS

A. SURGERY PATIENTS ON A BETA BLOCKER PRIOR TO ARRIVAL
WHO RECEIVED A BETA BLOCKER DURING THE
PERIOPERATIVE PERIOD ....................................................................

B. PROPHYLACTIC ANTIBIOTIC RECEIVED ON TIME—WITHIN ONE
HOUR PRIOR TO SURGICAL INCISION ...............................................

C. PROPHYLACTIC ANTIBIOTIC SELECTION FOR SURGICAL
PATIENTS...............................................................................................

D. PROPHYLACTIC ANTIBIOTICS DISCONTINUED WITHIN 24 HOURS
AFTER SURGERY END TIME ...............................................................

E. CARDIAC SURGERY PATIENTS WITH CONTROLLED 6 AM
POSTOPERATIVE SERUM GLUCOSE .................................................

F. SURGERY PATIENTS WITH APPROPRIATE HAIR REMOVAL ...........

G. SURGERY PATIENTS WITH RECOMMENDED VTE PROPHYLAXIS
ORDERED ..............................................................................................

H. SURGERY PATIENTS WHO RECEIVED APPROPRIATE VTE
PROPHYLAXIS WITHIN 24 HOURS PRIOR TO SURGERY TO
24 HOURS AFTER SURGERY...............................................................

HEART FAILURE PATIENTS WITH LEFT VENTRICULAR SYSTOLIC
DYSFUNCTION WITHOUT ACEI AND ARB CONTRAINDICATIONS
WHO ARE PRESCRIBED ACEI/ARB AT DISCHARGE .........................

J. INCIDENCE OF PRESSURE ULCERS ..................................................

K. MRSA-1 INFECTION RATE....................................................................

L. MRSA-2 TRANSMISSION RATE............................................................

Prepared by Mathematica Policy Research

E.28

C5b.

[FOR EACH ‘YES’ RESPONSE AT C4, ASK:]
What type(s) of effort(s) has the hospital undertaken to improve on [ITEM FROM C4]
[If respondents report the same efforts for multiple measures, use code 13]?
CODE ALL THAT APPLY
AN IMPROVEMENT TEAM USED A
PLAN-DO-STUDY-ACT (OR SIMILAR)
TECHNIQUE ............................................................ 1
USED SIX SIGMA .................................................... 2
USED LEAN PROCESS .......................................... 3
ADJUSTED ELECTRONIC HEALTH RECORD
OR OTHER ELECTRONIC SYSTEM TO
INCLUDE AND DISPLAY KEY INFORMATION
IN A SEARCHABLE FIELD ...................................... 4
IMPLEMENTED A CHECKLIST ............................... 5
ESTABLISHED A PROTOCOL THAT IS
USED ROUTINELY .................................................. 6
CHANGED WHO IS RESPONSIBLE FOR
KEY TASKS RELATED TO THE MEASURE ........... 7
CHANGED ANOTHER ASPECT OF THE
PROCESS (BUT NOT THROUGH A
FORMAL PROTOCOL) ............................................ 8
DEPARTMENT HEADS OR OTHER KEY
PHYSICIANS MET WITH OTHER
PHYSICIANS TO GAIN THEIR BUY-IN TO
IMPROVEMENT ....................................................... 9
PRODUCED AND SHARED
PHYSICIAN-SPECIFIC DATA
ON THE MEASURE ................................................. 10
PERFORMED CONCURRENT CHART
REVIEWS TO IDENTIFY FAILING
CASES FOR FOLLOW-UP ...................................... 11
OTHER (SPECIFY) .................................................. 12

SAME EFFORTS AS FOR PREVIOUS
MEASURE ................................................................ 13
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.29

C6.

To what extent have you had both the leadership support and resources you needed to
accomplish the improvements you sought on these measures?
On improvements for [READ ITEM], would you say you had all the leadership support
and resources you needed, somewhat less than you needed, or a lot less than you
needed?
[PROGRAM WILL DISPLAY FOR EACH ‘YES’ RESPONSE TO C4]

SOMEWHAT
LESS THAN
YOU NEEDED

A LOT
LESS
THAN
YOU
NEEDED

DON’T
KNOW

REFUSED

Prophylactic antibiotic selection for
surgical patients .................................

d. Prophylactic antibiotics discontinued
within 24 hours after surgery end
time ....................................................

e. Cardiac surgery patients with
controlled 6 AM postoperative serum
glucose ..............................................

g. Surgery patients with recommended
VTE prophylaxis ordered ...................

h. Surgery patients who received
appropriate VTE prophylaxis within
24 hours prior to surgery to 24 hours
after surgery.......................................

Heart failure patients with left
ventricular systolic dysfunction
without ACEI and ARB
contraindications who are prescribed
ACEI/ARB at discharge .....................

Incidence of pressure ulcers..............

MRSA-1 Infection rate .......................

MRSA-2 Transmission rate ...............

ALL THE
LEADERSHIP
SUPPORT AND
RESOURCES
YOU NEEDED

a. Surgery patients on a beta blocker
prior to arrival who received a beta
blocker during the perioperative
period .................................................
b. Prophylactic antibiotic received on
time—within one hour prior to
surgical incision .................................

SCIP MEASURES

Surgery patients with appropriate
hair removal .......................................

VTE PROPHYLAXIS MEASURES

HEART FAILURE MEASURE
i.

Prepared by Mathematica Policy Research

E.30

C7.

Does the hospital provide physicians with physician-level data for any of the SCIP,
heart failure or MRSA measures?
IF YES, ASK: Which ones? [PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM
C4] [READ LIST IF NECESSARY]
IF NO, GO TO C8

YES

DON’T
KNOW

REFUSED

a. Surgery patients on a beta blocker prior to arrival
who received a beta blocker during the perioperative
period .............................................................................

b. Prophylactic antibiotic received on time—within one
hour prior to surgical incision .........................................

Prophylactic antibiotic selection for surgical patients ....

d. Prophylactic antibiotics discontinued within 24 hours
after surgery end time ....................................................

e. Cardiac surgery patients with controlled 6 AM
postoperative serum glucose.........................................

g. Surgery patients with recommended VTE prophylaxis
ordered ..........................................................................

h. Surgery patients who received appropriate VTE
prophylaxis within 24 hours prior to surgery to
24 hours after surgery ...................................................

Heart failure patients with left ventricular systolic
dysfunction without ACEI and ARB contraindications
who are prescribed ACEI/ARB at discharge .................

Incidence of pressure ulcers..........................................

MRSA-1 Infection rate ...................................................

MRSA-2 Transmission rate ...........................................

SCIP MEASURES

Surgery patients with appropriate hair removal .............

VTE PROPHYLAXIS MEASURES

HEART FAILURE MEASURE
i.

Prepared by Mathematica Policy Research

E.31

C8.

In general, how well do the hospital’s electronic health record or other information
systems support measurement on the SCIP, heart failure, and MRSA measures?
Would you say very well, somewhat well, not very well, or not at all well?
VERY WELL ............................................................. 1
SOMEWHAT WELL ................................................. 2
NOT VERY WELL .................................................... 3
NOT AT ALL WELL .................................................. 4

C8a.

Are there any specific measures where the hospital’s information systems support the
measure well?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.32

GO TO C9

C8b.

Which ones?
[PROGRAM WILL LIST FOR EACH ‘YES’ RESPONSE TO C4]
HOSPITAL INFORMATION
SYSTEM SUPPORTS
MEASURES WELL
(CHECK ALL THAT APPLY)

SCIP MEASURES
a. Surgery patients on a beta blocker prior to arrival who
received a beta blocker during the perioperative period .....

□

b. Prophylactic antibiotic received on time—within one hour
prior to surgical incision.......................................................

□

c. Prophylactic antibiotic selection for surgical patients ..........

□

d. Prophylactic antibiotics discontinued within 24 hours after
surgery end time .................................................................

□

e. Cardiac surgery patients with controlled 6 AM
postoperative serum glucose ..............................................

□

g. Surgery patients with recommended VTE prophylaxis
ordered ................................................................................

□

h. Surgery patients who received appropriate VTE
prophylaxis within 24 hours prior to surgery to 24 hours
after surgery ........................................................................

□

Heart failure patients with left ventricular systolic
dysfunction without ACEI and ARB contraindications who
are prescribed ACEI/ARB at discharge ...............................

□

Incidence of pressure ulcers ...............................................

□

k. MRSA-1 Infection rate .........................................................

□

Surgery patients with appropriate hair removal ...................

VTE PROPHYLAXIS MEASURES

HEART FAILURE MEASURE
i.

MRSA-2 Transmission rate .................................................

Prepared by Mathematica Policy Research

E.33

BARRIERS TO IMPROVEMENT AND INTEREST IN EXTERNAL ASSISTANCE
C9.

I’m going to read a list of barriers that hospitals may face in improving their
performance on the SCIP, heart failure, and MRSA measures. As I read each one,
please tell me whether each is currently a major barrier, a minor barrier, or not a barrier
for this hospital.
[ROTATE ORDER OF LIST a – i, BUT ALWAYS END WITH j AND THEN k]
CODE ONE FOR EACH
MAJOR
BARRIER

MINOR
BARRIER

NOT A
BARRIER

DON’T
KNOW

REFUSED

a. The hospital lacks enough
staff trained in quality
improvement .........................

b. Resource constraints, other
than staffing, limit
improvement strategies. ........

c. Lack of physician interest or
involvement ...........................

d. Documentation of the care
that is actually given is a
major problem .......................

e. Physicians at the hospital
disagree with selection of the
measure or its definition ........

The hospital has other higher
priorities.................................

g. The hospital is unsure of how
to improve performance ........

h. The hospital has no incentive
to improve .............................

YES

DON’T
KNOW

REFUSED

Insufficient senior
management leadership and
support ..................................

Any other barrier I haven’t
mentioned? (SPECIFY)........

k. IF YES, ASK: Was this a
major or a minor barrier?.......

Prepared by Mathematica Policy Research

MAJOR

MINOR

DON’T
KNOW

REFUSED

E.34

C10.

Is this hospital potentially interested in future technical assistance sponsored by CMS
to help reduce any remaining barriers and thereby boost its performance?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO C11b

REFUSED ................................................................ r
C11a.

What would be the two most important topics on which this hospital would like
additional support?
1.
2.
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
GO TO C12

C11b.

Please tell us if any of the following are reasons why you would not be interested in
future technical assistance to boost the hospital’s performance.
Hospital already has plans to use supporting
initiatives or organizations that should be
sufficient ................................................................... 1
No assistance from external organizations
is needed—just internal work ................................... 2
Existing outside organizations lack the
necessary expertise ................................................. 3
Hospital has other more important priorities ............. 4
Hospital lacks staff resources to participate
in any more improvement initiatives ......................... 5
Other (SPECIFY) ...................................................... 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.35

IMPORTANT SOURCES OF QI INFORMATION
C12.

The last few questions are about sources of information that may have improved the
quality of care at this hospital.
Do you believe the quality of care at this hospital in one or more clinical areas is better
this year than last year?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO END

REFUSED ................................................................ r
C13.

In addition to your own experience and data, what were the three most important
sources of information that have helped your hospital improve its quality of care over
the past year?
CODE UP TO THREE
DIRECT SHARING OF EXPERIENCES AND
BEST PRACTICES AMONG HOSPITALS ............... 1
IHI WEBSITE ............................................................ 2
OTHER WEBSITES VISITED ROUTINELY ............. 3
USE OF SEARCH ENGINES TO IDENTIFY
RELEVANT MATERIAL ON THE WEB .................... 4
CONFERENCE OR MEETING MATERIALS ........... 5
WEBINARS OR TELECONFERENCE
PRESENTATIONS ................................................... 6
MEETING WITH CONSULTANTS ........................... 7
OTHER (SPECIFY) .................................................. 8

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.36

C13a.

[IF C13=1]
You mentioned direct sharing of experiences and best practices as one of three most
important sources of information that have helped your hospital. Who facilitated the
sharing? Was it . . .
The hospital association, .......................................... 1
[NAME OF QIO], or .................................................. 2
Another organization? .............................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C13b.

[IF C13=3]
You mentioned other websites that were visited routinely as one of three most
important sources of information that have helped your hospital. What are those
websites? RECORD VERBATIM

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C13c.

[IF C13=5]
You mentioned conferences or meeting materials as one of the three most important
sources of information that have helped your hospital. Who sponsored the conference
or provided the meeting materials? Was it . . .
IHI, ............................................................................ 1
The hospital association, .......................................... 2
[NAME OF QIO], or .................................................. 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.37

C13d.

[IF C13=6]
You mentioned webinars or teleconference presentations as one of the three most
important sources of information that have helped your hospital. Who sponsored the
webinar or teleconference? Was it . . .
IHI, ............................................................................ 1
The hospital association, .......................................... 2
[NAME OF QIO], or .................................................. 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

END.

Those are all the questions I have. Do you have any final comments you’d like to
share?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO THANK

REFUSED ................................................................ r
RECORD VERBATIM

THANK. Thank you very much for participating in this survey, and taking the time to speak with
me. Have a great (day/evening).

Prepared by Mathematica Policy Research

E.38

Mathematica Reference No.: 06514.260

Ninth Scope of Work
QIO Program
Evaluation: Nursing
Home Survey
Final Draft Questionnaire
January 5, 2010

A. INTRODUCTION

A1.

INITIAL CONTACT WITH NURSING HOME: Hello, may I please speak with [NAME OF
ADMINISTRATOR/The administrator of the nursing home]?
IF YOU DON’T HAVE (HIS/HER) NAME, ASK FOR IT.
RECORD NAME AND CONTACT INFORMATION FOR ADMINISTRATOR.

NAME OF NURSING HOME ADMINISTRATOR
IF NEEDED: My name is ________ and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF
ADMINISTRATOR/the administrator] a letter from CMS requesting the nursing home’s
participation in a survey for an evaluation of the Ninth Scope of Work of the Quality
Improvement Organization Program.
We would like to conduct a brief interview with the QI Director (or whoever is directly
involved with quality improvement at the nursing home). Would you please give me the
name and contact information for this person?
IF ADMINISTRATOR IS UNAVAILABLE, BUT YOU ARE SPEAKING WITH (HIS/HER)
SECRETARY OR ASSISTANT.
A1a.

My name is ________ and I’m calling on behalf of the Centers for Medicare & Medicaid
Services or CMS. A short time ago, we sent [NAME OF ADMINISTRATOR] a letter from
CMS requesting the nursing home’s participation in a survey for an evaluation of the
Ninth Scope of Work of the Quality Improvement Organization Program.
We would like to conduct a brief interview with the QI director, director of nursing, or
whoever is directly involved with quality improvement at the nursing home. I was hoping
to speak with the director to obtain the name and contact information for this person.
Since (he/she) is unavailable, would you be able to give me the name and contact
information of the QI director or director of nursing?
CAN TALK NOW ...................................................... 1

GO TO A2

SET CALLBACK ....................................................... 0
SET CALLBACK
DON’T KNOW .......................................................... d

Prepared by Mathematica Policy Research

E.40

A2.

RECORD NAME AND CONTACT INFORMATION FOR QI/NURSING DIRECTOR.

|-|

WHEN SPEAKING WITH QI/NURSING DIRECTOR: Hello, [Dr./Mr./Ms.] [LAST NAME],
my name is ________, and I’m calling on behalf of the Centers for Medicare & Medicaid
Services or CMS. A short time ago, you should have received a letter from CMS
requesting your nursing home’s participation in a survey for an evaluation of the Ninth
Scope of Work of the Quality Improvement Organization Program.
Your nursing home’s input is crucial to assure that CMS learns all it can about how the
QIOs are working and what changes if any, need to be made. [FOR NONPARTICIPATING NURSING HOMES: It is critical that our study understand quality
improvement processes and thinking in nursing homes that have not been working with
QIOs as well as those that have, in order to understand the added value of the QIO
program.]
We would like to conduct a brief interview with you (or whoever is directly involved with
quality improvement at your nursing home). The survey interview takes roughly
28 minutes, depending upon your answers. I can conduct it now, or at any time that’s
convenient for you.
START INTERVIEW NOW ....................................... 1

GO TO B1

NOT NOW, SET UP APPT/CALLBACK ................... 2

SET APPT.

NEEDS MORE INFORMATION ............................... 3

GO TO A2b

REFUSED ................................................................ r

REFUSAL

Prepared by Mathematica Policy Research

E.41

A2b.

FOLLOW-UP INFORMATION
PURPOSE: CMS is interested in learning about the experience of nursing homes
involved in the Ninth Scope of Work of the Quality Improvement Organization Program.
Your nursing home’s input is crucial to assure that CMS learns all it can about how the
QIOs are working and what changes if any, need to be made.
[FOR NON-PARTICIPATING NURSING HOMES: It is critical that our study understand
quality improvement processes and thinking in nursing homes that have not been
working with QIOs as well as those that have, in order to understand the added value of
the QIO program.]
CONFIDENTIALITY: Please be assured that your responses to the survey will remain
confidential to the extent permitted by law. All data collected for the purposes of this
study will be combined and reported in aggregate form only. Neither you nor your
organization will be identified by name in any reports or documents produced from the
study findings. Only Mathematica staff that work directly on the evaluation will have
access to the name of your organization and your name.
SELECTION: Your nursing home was randomly selected from U.S. nursing homes
eligible for the study. Some were selected to represent participating nursing homes that
worked with QIOs and some were selected to represent nursing homes that did not work
with QIOs.

REFUSAL SCREEN: Thank you for your time. Have a nice day. END CALL.

Prepared by Mathematica Policy Research

E.42

B. QIO INTERACTIONS

The first few questions are about staff interactions with [NAME OF QIO].
B1.

Is this nursing home participating with [NAME OF QIO] on a quality improvement
initiative related to any of the following topics . . . [READ DOWN LIST]

a. Physical restraint use ..............................................

YES
1

NO
0

DON’T
KNOW
d

REFUSED
r

b. Pressure ulcer reduction .........................................

c. Reducing re-hospitalizations ...................................

d. General assistance improving quality .....................

e. Any other topic? (SPECIFY) ..................................

[FOR EACH CATEGORY THAT RECEIVED A ‘NO’ RESPONSE, ASK B1aa AFTER
YOU HAVE READ THROUGH THE ENTIRE LIST IN B1.]
B1aa. Were you invited to participate with [NAME OF QIO] on a quality improvement initiative
related to [TOPIC]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B2

REFUSED ................................................................ r
B1ab. Why did you choose not to participate with [NAME OF QIO] on a quality improvement
initiative? RECORD VERBATIM

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.43

B2.

The next few questions ask about how often nursing home staff may have met with
[NAME OF QIO], either in-person or by telephone. Since August 2008, how many times
have nursing home personnel met with [NAME OF QIO] in-person at this nursing
home?
PROBE: Please do not include telephone conference calls or web-ex meetings.
|

| TIMES

NONE ....................................................................... 0
DON’T KNOW .......................................................... d

GO TO B3

REFUSED ................................................................ r
Bracket grouping No, Don't Know, and Refused responses and pointing to.

B2a.

[IF B2 = 2 OR MORE, DISPLAY “How often did…”; IF B2 = 1, DISPLAY “Did…”]
(How often did/Did) the following people attend the meeting with [NAME OF QIO]?
[READ ITEM]
[IF B2 = 2 OR MORE, DISPLAY: Would you say always, usually, sometimes, or
never?] [IF B2 = 1, DISPLAY ONLY CATEGORIES “ALWAYS” AND “NEVER”]

ALWAYS

USUALLY

SOMETIMES

NEVER

DON’T
KNOW

REFUSED

a. The administrator ......................

b. The director of nursing ..............

c. The medical director..................

d. The quality improvement
coordinator ................................

e. The staff developer ...................

B3.

Any other staff? (SPECIFY) .....

Since August 2008, how many other in-person meetings have nursing home personnel
attended where [NAME OF QIO] was an active participant? Please include in-person
meetings held inside and outside the nursing home.
|

| TIMES

Prepared by Mathematica Policy Research

E.44

B4.

Since August 2008, approximately how many times have nursing home personnel met
by phone with [NAME OF QIO]? Please do not include large conference calls.
|

| TIMES

Since August 2008, how many other telephone conference calls or web-ex meetings
have nursing home personnel attended that [NAME OF QIO] convened?
|

| TIMES

Prepared by Mathematica Policy Research

E.45

B6.

DON’T
KNOW

REFUSED

a. Self-referral to [NAME OF QIO] ..................................

b. Understanding [NAME OF QIO]’s plans for activities
and opportunities to participate ...................................

c. Routine meetings as part of participating with [NAME
OF QIO] on a quality improvement effort ....................

d. To discuss this nursing home’s performance data ......

e. Applying [NAME OF QIO] staff’s expertise to improve
this nursing home’s quality measures .........................

Hearing about best practices of other nursing homes.

g. To learn about a new tool or recommended process
for quality improvement ...............................................

h. Other staff development or training .............................

Presentation(s) to help create buy-in to quality
improvement beyond the quality improvement staff....

To discuss issues, methods, and/or timeframes for
quality reporting to CMS..............................................

k. To attend a broad-based regional or statewide
meeting on quality improvement where [NAME OF
QIO] was an active participant ....................................

i.
j.

Some other reason? (SPECIFY) ................................

Prepared by Mathematica Policy Research

E.46

B7.

Which of the following describe the aim(s) of the assistance you received?
[FOR THOSE WHO RESPOND ‘YES’ TO ANY ITEM IN B6]
IMPROVE OR ENSURE COMPLETENESS OF
THE REPORTED DATA ........................................... 1
ADVISE ON ISSUES RELATED TO
TRANSMISSION OF THE DATA ............................. 2
OTHER (SPECIFY) .................................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

B7a.

Prepared by Mathematica Policy Research

E.47

B8.

How valuable to the nursing home were each of these types of meetings?
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES TO B6.]
[READ ITEM] Would you say it was very valuable, somewhat valuable, or not valuable?
[IF ‘VERY VALUABLE’, ‘SOMEWHAT VALUABLE’, OR ‘NOT VALUABLE’ IS
CHOSEN, ASK B9 OR B10 BEFORE MOVING ON TO NEXT ITEM.]
VERY
VALUABLE

SOMEWHAT
VALUABLE

NOT
VALUABLE

a. Self-referral to [NAME OF QIO] .........

b. Understanding [NAME OF QIO]’s
plans for activities and opportunities
to participate.......................................

c. Routine meetings as part of
participating with [NAME OF QIO] on
a quality improvement effort ...............

d. To discuss this nursing home’s
performance data ...............................

e. Applying [NAME OF QIO] staff’s
expertise to improve this nursing
home’s quality measures ...................

Hearing about best practices of other
nursing homes....................................

g. To learn about a new tool or
recommended process for quality
improvement ......................................

h. Other staff development or training ....

Presentation(s) to help create buy-in
to quality improvement beyond the
quality improvement staff ...................

To discuss issues, methods, and/or
timeframes for quality reporting to
CMS ...................................................

k. To attend a broad-based regional or
statewide meeting on quality
improvement where [NAME OF QIO]
was an active participant ....................

Other reason (SPECIFY) ...................

Prepared by Mathematica Policy Research

E.48

DON’T
KNOW

REFUSED

B9.

[IF ‘NOT VALUABLE’ SELECTED AT B8, ASK B9 IMMEDIATELY AFTER B8:]
Why were these meetings not valuable?
CODE ALL THAT APPLY
THE MEETING MERELY FULFILLED AN
OBLIGATION (SUCH AS GUIDANCE OR
DIRECTION FROM THE STATE
SURVEY AGENCY) ................................................. 1
THIS NURSING HOME IS SO ADVANCED,
THERE IS NOTHING WE CAN LEARN
FROM [NAME OF QIO] ............................................ 2
[NAME OF QIO] STAFF DID NOT HAVE
ENOUGH EXPERTISE OF THE RIGHT TYPE ........ 3
[NAME OF QIO] STAFF WAS NOT
WELL-PREPARED ................................................... 4
THE RIGHT PEOPLE WERE NOT
AT THE MEETING ................................................... 5
THE INFORMATION PROVIDED WAS
NOT APPLICABLE TO THIS NURSING
HOME’S SITUATION ............................................... 6
THE MEETING WAS REDUNDANT WITH
INFORMATION WE ALREADY HAD ....................... 7
PROGRESS AT THIS NURSING HOME ON
THE MEETING TOPIC IS NOT FEASIBLE
AT THIS TIME .......................................................... 8
OTHER (SPECIFY) .................................................. 9

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.49

B10.

[IF ‘VERY VALUABLE’ OR ‘SOMEWHAT VALUABLE’ SELECTED AT B8, ASK B10
IMMEDIATELY AFTER B8:]
In what ways were these meetings valuable?
CODE ALL THAT APPLY
PROVIDED NEW, USEFUL INFORMATION ON
HOW TO IMPROVE PERFORMANCE .................... 1
PROVIDED NEW, USEFUL INFORMATION
ON ANOTHER TOPIC (SUCH AS QUALITY
REPORTING, USE OF EHR) ................................... 2
INCREASED MOTIVATION TO IMPROVE
FROM ONE OR MORE EXECUTIVE LEADERS ..... 3
INCREASED MOTIVATION TO IMPROVE
FROM MEDICAL DIRECTOR .................................. 4
INCREASED MOTIVATION TO IMPROVE
FROM KEY NURSING STAFF ................................. 5
OTHER (SPECIFY) .................................................. 6

DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B11.

Did any of the meetings lead to changes at the nursing home that ultimately improved
resident care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B12

REFUSED ................................................................ r
B11a. Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.50

GO TO B12

B11b. Please tell me which measures were improved at least partly due to help from
[NAME OF QIO].
[READ LIST IF NEEDED]
CODE ALL THAT APPLY
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
RATE OF RE-HOSPITALIZATIONS ........................ 3
OTHER (SPECIFY) .................................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12.

GO TO B13

REFUSED ................................................................ r
B12a. Did you receive these items from . . .
CODE ALL THAT APPLY
A newsletter, ............................................................. 1
An email or listserv, .................................................. 2
At an in-person meeting, .......................................... 3
At a teleconference or web-ex, ................................. 4
Or some other way? (SPECIFY) ............................. 5
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.51

B12b. Overall, how valuable were these educational materials and/or tools? Would you
say . . .
Very valuable, ........................................................... 1
Somewhat valuable, ................................................. 2
Not very valuable, or ................................................ 3
Not at all valuable? ................................................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12c. Did any of these educational materials or tools from [NAME OF QIO] thus far lead to
changes at the nursing home that ultimately improved resident care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B13

REFUSED ................................................................ r
B12d. Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO B13

REFUSED ................................................................ r
B12e. Please tell me which measures were improved at least in part due to changes that
stemmed from the educational materials or tools from [NAME OF QIO].
[READ LIST IF NEEDED]
CODE ALL THAT APPLY
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
RATE OF RE-HOSPITALIZATIONS ........................ 3
OTHER (SPECIFY) .................................................. 4

DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.52

B13.

Do you routinely receive data feedback from [NAME OF QIO] on this nursing home’s
quality performance?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO C1

REFUSED ................................................................ r
B13a. How widely within the nursing home do you typically share the feedback [NAME OF
QIO] provides, or highlights that you derive from it? Is it . . .
CODE ONE ONLY
Shared with a wide array of relevant
nursing and direct care staff, .................................... 1
Shared with a few key individuals, or ....................... 2
Rarely or never shared? ........................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B13b. Has the feedback from [NAME OF QIO] been important to the nursing home's quality
improvement efforts?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.53

C. OTHER QI INITIATIVES

EXTERNAL INITIATIVES
The next questions are about quality improvement initiatives that involve external organizations.
C1.

Is this nursing home part of a corporate chain or otherwise-affiliated group of nursing
home providers?
YES, CORPORATE CHAIN ..................................... 1
YES, AFFILIATED GROUP ...................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO C2

REFUSED ................................................................ r
C1a.

To what extent are your nursing home’s quality improvement actions influenced by this
(larger corporate chain/affiliated group) of nursing homes? Would you say to a large
extent, a moderate extent, or a small or no extent?
LARGE EXTENT ...................................................... 1
MODERATE EXTENT .............................................. 2
SMALL OR NO EXTENT .......................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C2.

Setting aside any interactions with [NAME OF QIO] (and the (larger corporate chain/
affiliation) just discussed), is your nursing home actively involved in any other quality
improvement efforts involving outside organizations?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.54

GO TO C3

C2a.

Is your nursing home involved with any of the following?
CODE ALL THAT APPLY
The Advancing Excellence in Nursing Homes
Campaign ................................................................. 1
The Pressure Ulcer Collaborative............................. 2
State-initiated quality improvement projects
on pressure ulcer reduction, restraint use,
or other issues .......................................................... 3
Any other quality improvement effort with
an outside organization? (SPECIFY)....................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

INTERNAL INITIATIVES
Now I’d like to ask about internal quality improvement initiatives within the nursing home.
C3.

How many full-time equivalent staff are currently devoted to quality improvement in the
nursing home?
PROBE:

For example, if the nursing home has four staff who each devote a quarter
time to quality improvement, then the number of full-time equivalent staff at
the nursing home is one.

PROBE:

Your best estimate is fine.

| NUMBER OF FTE’s

Prepared by Mathematica Policy Research

E.55

C4.

Since August 2008, please indicate if the nursing home has had internal quality
improvement efforts that have improved nursing home performance on any of the
following measures. [READ DOWN LIST]
YES

DON’T
KNOW

REFUSED

a. Physical restraint use ..............................................

b. Pressure ulcers .......................................................

c. Influenza (flu) vaccination .......................................

d. Pneumococcal vaccination......................................

e. Urinary tract infection ..............................................

Urinary catheter use ................................................

g. Depression or anxiety .............................................

h. Moderate to severe pain .........................................

Patient mobility ........................................................

Weight loss..............................................................

k. Help with daily activity .............................................

Any other measure? (SPECIFY) ............................

[IF C4a THROUGH C4l ALL EQUAL ‘NO,’ GO TO C7. ELSE GO TO C5]
C5.

Has the nursing home performed an analysis on any of the measures to identify the
reasons why the relevant guideline sometimes is not followed, or why the undesirable
outcome sometimes occurs?
YES .......................................................................... 1
NO ............................................................................ 0
NOT APPLICABLE – PERFORMANCE
IS 100% RELIABLE .................................................. 2
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.56

GO TO C7

C5a.

For which measures has the nursing home performed an analysis?
[READ LIST IF NECESSARY]
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM C3a]
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
INFLUENZA (FLU) VACCINATION RATES ............. 3
PNEUMOCOCCAL VACCINATION RATES ............ 4
URINARY TRACT INFECTION RATES ................... 5
URINARY CATHETER USE..................................... 6
DEPRESSION OR ANXIETY RATES ...................... 7
MODERATE TO SEVERE PAIN RATES ................. 8
PATIENT MOBILITY RATES ................................... 9
WEIGHT LOSS RATES............................................ 10
DAILY ACTIVITY RATES ......................................... 11
OTHER (SPECIFY) .................................................. 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.57

C5b.

[FOR EACH ‘YES’ RESPONSE AT C4, ASK:]
What type(s) of effort(s) has the nursing home undertaken to improve on [ITEM FROM
C4] [If respondents report the same efforts for multiple measures, use code 10]?
CODE ALL THAT APPLY
AN IMPROVEMENT TEAM USED A
PLAN-DO-STUDY-ACT (OR SIMILAR)
TECHNIQUE ............................................................ 1
ADJUSTED ELECTRONIC HEALTH RECORD
OR OTHER ELECTRONIC SYSTEM TO
INCLUDE AND DISPLAY KEY INFORMATION
IN A SEARCHABLE FIELD ...................................... 2
IMPLEMENTED A CHECKLIST ............................... 3
ESTABLISHED A PROTOCOL THAT IS
USED ROUTINELY .................................................. 4
[HIRED A CLINICAL CONSULTANT TO] TRAIN
NURSING AND DIRECT CARE STAFF ON
IMPROVING CARE IN THE MEASURE AREA........ 5
CHANGED ANOTHER ASPECT OF THE
PROCESS (BUT NOT THROUGH A
FORMAL PROTOCOL) ............................................ 6
PRODUCED AND SHARED UNIT LEVEL
DATA WITH NURSING STAFF ON THE
MEASURE ................................................................ 7
PERFORMED CHART REVIEWS TO IDENTIFY
ASSESSMENT ERRORS OR OTHER
POTENTIAL PROCESS IMPROVEMENTS ............. 8
OTHER (SPECIFY) .................................................. 9

SAME EFFORTS AS FOR PREVIOUS
MEASURE ................................................................ 10
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.58

C6.

SOMEWHAT
LESS THAN
YOU NEEDED

A LOT
LESS
THAN YOU
NEEDED

DON’T
KNOW

REFUSED

a. Physical restraint use ....................

b. Pressure ulcers .............................

c. Influenza (flu) vaccination .............

d. Pneumococcal vaccination ...........

e. Urinary tract infection ....................

f. Urinary catheter use .....................

g. Depression or anxiety ...................

h. Moderate to severe pain ...............

i. Patient mobility .............................

j. Weight loss ...................................

k. Help with daily activity ...................

l. Any other measure? (SPECIFY) ..

Prepared by Mathematica Policy Research

E.59

BARRIERS TO IMPROVEMENT AND INTEREST IN EXTERNAL ASSISTANCE
C7.

I’m going to read a list of barriers that nursing homes may face in improving their
performance on these measures. As I read each one, please tell me whether each
is currently a major barrier, a minor barrier, or not a barrier for this nursing home.
[ROTATE ORDER OF LIST, BUT ALWAYS END WITH CATEGORY “J” AND “K”]
CODE ONE FOR EACH
MAJOR
BARRIER

MINOR
BARRIER

NOT A
BARRIER

DON’T
KNOW

REFUSED

a. The nursing home lacks
enough staff trained in
quality improvement ..............

b. Resource constraints, other
than staffing, limit
improvement strategies. ........

c. Lack of nursing staff interest
or involvement .......................

d. Documentation of the care
that is given is a problem ......

e. Staff at the nursing home
disagree with selection of the
measure or its definition ........

The nursing home has other
higher priorities......................

g. The nursing home is unsure
of how to improve
performance ..........................

h. The nursing home has no
incentive to improve ..............

YES

DON’T
KNOW

REFUSED

Insufficient senior
management leadership and
support ..................................

Any other barrier I haven’t
mentioned? (SPECIFY)........

k. IF YES, ASK: Was this a
major or a minor barrier?.......

Prepared by Mathematica Policy Research

MAJOR

MINOR

DON’T
KNOW

REFUSED

E.60

C8a.

Is this nursing home potentially interested in future technical assistance sponsored by
CMS to help reduce any remaining barriers and thereby boost its performance?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0

GO TO C9

DON’T KNOW .......................................................... d
GO TO C10
REFUSED ................................................................ r
C8b.

What would be the two most important topics on which this nursing home would like
additional support?
1.
2.
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

C9.

Please tell me if any of the following are reasons why you would not be interested in
future technical assistance to boost the nursing home’s performance.
CODE ALL THAT APPLY
Our nursing home already has plans to use
other supporting initiatives or organizations ............. 1
No assistance from external organizations
is needed, just internal work ..................................... 2
Existing outside organizations lack the
necessary expertise ................................................. 3
Our nursing home has other more important
priorities .................................................................... 4
Our nursing home lacks staff resources to
participate in any more improvement initiatives........ 5
Any other reason? (SPECIFY) ................................ 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.61

IMPORTANT SOURCES OF QI INFORMATION
The last few questions are about sources of information that may have improved the quality of
care at this nursing home.
C10.

Do you believe the quality of care at this nursing home in one or more clinical areas is
better this year than last year?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d

GO TO END

REFUSED ................................................................ r
C11.

In addition to your own experience and data, what were the three most important
sources of information that have helped your nursing home improve its quality of
care over the past year?
CODE UP TO THREE
DIRECT SHARING OF EXPERIENCES AND
BEST PRACTICES AMONG NURSING HOMES .... 1
MedQIC WEBSITE ................................................... 2
OTHER WEBSITES VISITED ROUTINELY ............. 3
USE OF SEARCH ENGINES TO IDENTIFY
RELEVANT MATERIAL ON THE WEB .................... 4
CONFERENCES OR SEMINARS ............................ 5
WEBINARS OR TELECONFERENCE
PRESENTATIONS ................................................... 6
MEETING WITH CONSULTANTS ........................... 7
OTHER (SPECIFY) .................................................. 8
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.62

[IF C11=1]
C11a. You mentioned direct sharing of experiences and best practices as one of three most
important sources of information that have helped your nursing home. Who facilitated
the sharing? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF C11=3]
C11b. You mentioned other websites that were visited routinely as one of three most important
sources of information that have helped your nursing home. What are those websites?
RECORD VERBATIM

DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF C11=5]
C11c. You mentioned conferences or seminars as one of the three most important sources of
information that have helped your nursing home. Who sponsored the conference or
seminar? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.63

C11d.

[IF C11=6]
You mentioned webinars or teleconference presentations as one of the three most
important sources of information that have helped your nursing home. Who sponsored
the webinar or teleconference? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

END.

Those are all the questions I have. Do you have any final comments you’d like to
share? RECORD VERBATIM

THANK. Thank you very much for participating in this survey, and taking the time to speak with
me. Have a great (day/evening).
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r

Prepared by Mathematica Policy Research

E.64

GO TO THANK

File Type	application/pdf
File Title	Program Evaluation of the Eighth and Ninth Scope of Work Quality Improvement Program: Supporting Statement for Paperwork Reducti
Author	Martha Kovac, Sue Felt-Lisk, Arnold Chen, John Hall
File Modified	2010-08-27
File Created	2010-08-11