Download:
pdf |
pdfAPPENDIX C
QIO SURVEYS AND LETTERS
CMS LETTERHEAD
ADVANCE LETTER – QIO SURVEY FOR THE 9TH SOW QIO PROGRAM
EVALUATION
[Date], 2010
«QIO Director»
«Name_of_QIO»
«Mailing_Address»
«City», «State» «Zip_Code»
Dear [Mr./Ms./Dr.] «QIO Director»:
The Centers for Medicare & Medicaid Services (CMS) would very much appreciate your
thoughtful input on your QIO’s experience with the 9th SOW. In the near future, you will be
contacted by Mathematica Policy Research, Inc (Mathematica) to participate in the 9th SOW
Evaluation Survey. The survey is being conducted as part of Mathematica’s Evaluation of the 8th
and 9th SOW Quality Improvement Program, and will include a questionnaire for you (QIO
Director Survey) and one for the leader of each theme or Patient Safety sub-theme (QIO Theme
Leader Survey).
Mathematica will soon be asking for your help in identify the most appropriate respondents
for the two surveys and will request current contact information for each, including name, e-mail
address, and telephone number. Once this information is gathered, Mathematica will send
personalized emails to each individual inviting him/her to participate in the appropriate survey.
Please be assured that responses to the surveys will remain confidential. Mathematica will
report information to CMS in aggregate form only. Your input will be valuable in assisting CMS
in continuing to improve the QIO program with each new scope of work. I urge you to
participate.
Please note that Section C.4.14 of the 9th SOW contract requires each QIO to provide data
for evaluation, thus, your time in providing contact information and your time to participate in
the survey is an expense covered under the contract. If you have any questions, please feel free to
call me at [phone number]. If you have questions about the survey, please call Martha Kovac,
Mathematica’s survey director, at 609-275-2331.
Sincerely,
[Signature block]
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The time required to complete
this information collection is estimated to average 0.5 to 0.75 hours or 30 to 45 minutes per response, including the time to review
instructions, search existing data resources, and gather the data needed, and complete and review the information collection. If you have
comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security
Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.
C.3
REQUEST FOR CONTACT INFORMATION FROM MATHEMATICA
[Date], 2010
Dear [Mr./Ms./Dr.] «QIO Director»:
Recently, the Centers for Medicare & Medicaid Services (CMS) sent you a letter about the upcoming
9th SOW Evaluation Survey being conducted by Mathematica Policy Research, Inc. This is part of
Mathematica’s evaluation of the 8th and 9th SOW Quality Improvement Program. The 9th SOW Evaluation
Survey will include a questionnaire for you (QIO Director Survey) and one for the leader of each theme or
Patient Safety sub-theme (QIO Theme Leader Survey). We need your help in identify the most appropriate
respondents for these two surveys.
The QIO Director Survey is intended for the QIO executive with ongoing management responsibility
and knowledge of the QIO’s experience operating the program under the 9th SOW contract. If you are the
best person to respond, please record your name, email address, and telephone number on the Excel
spreadsheet attached. If there is someone else at your organization that is better able to response, please
record his/her information on the form instead.
The QIO Theme Leader Survey is intended for those individuals with responsibility and day-to-day
knowledge of the QIO’s experience with themes or Patient Safety sub-themes in which the QIO is engaged.
On the attached Excel spreadsheet, please record the name, email, and telephone number for each theme
and Patient Safety sub-theme leader, as applicable.
Please complete the attached Excel spreadsheet and email it back to Mathematica
([email protected]) or via fax (Attn: Sarah Croake) at 202-863-1763 by
xx/xx/xxxx.
After receiving your completed spreadsheet, email invitations to the applicable web surveys will be
sent to the individuals you have identified. All survey responses will remain confidential. Mathematica will
report information to CMS in aggregate form only. Your input will be valuable in assisting CMS in
continuing to improve the QIO program with each new scope of work.
Please note that Section C.4.14 of the 9th SOW contract requires each QIO to provide data for
evaluation, thus, your time in providing contact information and your time to participate in the survey is an
expense covered under the contract. If you have any questions, please feel free to call me at [phone
number]. If you have questions about the survey, please call Martha Kovac, Mathematica’s survey director,
at 609-275-2331.
Sincerely,
Sue Felt-Lisk
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid
OMB control number. The valid OMB control number for this information collection is xxx-XXX. The time required to complete this information
collection is estimated to average 0.5 to 0.75 hours or 30 to 45 minutes per response, including the time to review instructions, search existing data
resources, and gather the data needed, and complete and review the information collection. If you have comments concerning the accuracy of the
time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer,
Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.
C.5
QUALITY IMPROVEMENT ORGANIZATION NAME
Instructions: In the spaces provided, please first indicate the most knowledgeable respondent for the QIO Director Survey. Please
provide his/her first and last name, email, and telephone number. Then, please identify the most knowledgeable respondent for the
QIO Theme Leader Survey, for each theme at your QIO. Again, please provide the first and last name, email, and telephone number
for each person listed.
Recommended Respondent
First Name
Last Name
Email Address
Telephone
QIO Director Survey
X' next to each QIO theme
Patient Safety‐Pressure Ulcers
Patient Safety‐ Physical Restraints
Patient Safety‐Surgical Care Improvement Project
Patient Safety‐MRSA
Patient Safety‐Drug Safety
Patient Safety ‐ Nursing Homes in Need
Prevention
Prevention ‐ Disparities
Care Transitions
Chronic Kidney Disease
QIO Theme Leader Survey
C.7
QIO DIRECTOR SURVEY – INVITATION EMAIL
Dear ${name},
Recently, CMS sent a letter to your Quality Improvement Organization (QIO) about an
important web-based survey we are conducting. The QIO Director Survey, which you are being
asked to complete, will gather input for the program evaluation of the 9th SOW along with the
Theme Leader Survey, which was separately sent to theme leaders within the QIO. The surveys
collect information about QIO activities, experience, environment, and suggestions for program
improvement. Information you provide will support Mathematica’s evaluation team in
understanding the trends and patterns in outcomes we will be studying, and in developing
program improvement recommendations for CMS.
You were identified by [name of CEO or CMS principal contact] as the best person to
complete the QIO Director Survey. Please note that Section C.4.14 of the 9th SOW contract
requires each QIO to provide data for evaluation. Your time to participate in the survey is an
expense covered under the contract. The survey should take less than 15 minutes to complete.
Your survey responses will remain confidential. Mathematica will report information to CMS in
aggregate form only. Your input is critical in assisting CMS in its efforts to improve the QIO
program with each new scope of work.
To begin the QIO Director Survey, click the link below:
[link would be embedded in email]
We look forward to receiving your completed survey within the next few days. Thanks in
advance for your participation. If you have any questions, please do not hesitate to contact me at
[email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher,
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.9
QIO DIRECTOR SURVEY – REMINDER #1
Dear ${name},
Recently, CMS sent a letter to your Quality Improvement Organization (QIO) about an
important web-based survey we are conducting. About one week ago, we sent you an email
invitation to participate in the QIO Director Survey. This survey gathers input for the program
evaluation of the 9th SOW along with the Theme Leader Survey, which was separately sent to
theme leaders within the QIO. The surveys collect information about QIO activities, experience,
environment, and suggestions for program improvement. Information you provide will support
Mathematica’s evaluation team in understanding the trends and patterns in outcomes we will be
studying, and in developing program improvement recommendations for CMS.
We have not yet received your completed survey. Please note that Section C.4.14 of the 9th
SOW contract requires each QIO to provide data for evaluation. Your time to participate in the
survey is an expense covered under the contract. The survey should take less than 15 minutes to
complete. Your survey responses will remain confidential. Mathematica will report
information to CMS in aggregate form only. Your input is critical in assisting CMS in its efforts
to improve the QIO program with each new scope of work.
To begin the QIO Director Survey, click the link below:
[link would be embedded in email]
We look forward to receiving your completed survey within the next few days. Thanks in
advance for your participation. If you have any questions, please do not hesitate to contact me at
[email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.10
QIO DIRECTOR SURVEY – REMINDER #2
Dear ${name},
We recently we sent you an email invitation to participate in the QIO Director Survey. This
survey gathers input for the program evaluation of the 9th. The surveys collect information about
QIO activities, experience, environment, and suggestions for program improvement. Information
you provide will support Mathematica’s evaluation team in understanding the trends and patterns
in outcomes we will be studying, and in developing program improvement recommendations for
CMS.
Your input is very important. Unfortunately, we have not yet received your completed
survey. Section C.4.14 of the 9th SOW contract requires each QIO to provide data for
evaluation. Your time to participate in the survey is an expense covered under the contract. The
survey should take less than 15 minutes to complete. Your survey responses will remain
confidential. Mathematica will report information to CMS in aggregate form only. Your input is
critical in assisting CMS in its efforts to improve the QIO program with each new scope of work.
To begin the QIO Director Survey, click the link below:
[link would be embedded in email]
We look forward to receiving your completed survey within the next few days. Thanks in
advance for your participation. If you have any questions, please do not hesitate to contact me at
[email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.11
QIO DIRECTOR SURVEY – REMINDER #3
Dear ${name},
Time is running out to participate in the QIO Director Survey! This survey gathers input for
the program evaluation of the 9th. Information you provide will support Mathematica’s
evaluation team in understanding the trends and patterns in outcomes we will be studying, and in
developing program improvement recommendations for CMS.
Section C.4.14 of the 9th SOW contract requires each QIO to provide data for evaluation.
Your time to participate in the survey is an expense covered under the contract.
Please take 15 minutes to complete the survey today.
Your survey responses will remain confidential. Mathematica will report information to
CMS in aggregate form only. Your input is critical in assisting CMS in its efforts to improve the
QIO program with each new scope of work.
To begin the QIO Director Survey, click the link below:
[link would be embedded in email]
If you have any questions, please do not hesitate to contact me at [email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.12
QIO THEME LEADER SURVEY – INVITATION EMAIL
Dear ${name},
Recently, CMS sent a letter to your Quality Improvement Organization (QIO) about an
important web-based survey we are conducting. The QIO Theme Leader Survey will gather input
for the program evaluation of the 9th SOW. The survey collects information about QIO
activities, experience, environment, and suggestions for program improvement specific to the
${theme} theme. Information you provide will support Mathematica’s evaluation team in
understanding the trends and patterns in outcomes we will be studying, and in developing
program improvement recommendations for CMS.
You were identified by [name of CEO or CMS principal contact] as the best person to
complete the survey for the ${theme} theme. Please note that Section C.4.14 of the 9th SOW
contract requires each QIO to provide data for evaluation. Your time to participate in the survey
is an expense covered under the contract.
The survey should take about 45 minutes to complete. Your survey responses will remain
confidential. Mathematica will report information to CMS in aggregate form only. As a theme
leader, your input is critical in assisting CMS in its efforts to improve the QIO program with
each new scope of work.
To begin the QIO Theme Leader Survey for the ${theme} theme, click the link below:
[link would be embedded in email]
We look forward to receiving your completed survey within the next few days. Thanks in
advance for your participation. If you have any questions, please do not hesitate to contact me at
[email protected] or (202) 484-4519.
Please note, if you are the theme leader for more than one theme at your QIO, you will
receive an email invitation to complete a QIO Theme Leader Survey for each theme that you
lead. This survey is specific to the ${theme} theme.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.13
QIO THEME LEADER SURVEY – REMINDER #1
Dear ${name},
Recently, CMS sent a letter to your Quality Improvement Organization (QIO) about an
important web-based survey we are conducting. About a week ago, we sent you an email
invitation to participate in the QIO Theme Leader Survey. The survey collects information about
QIO activities, experience, environment, and suggestions for program improvement specific to
the ${theme} theme. Information you provide will support Mathematica’s evaluation team in
understanding the trends and patterns in outcomes we will be studying, and in developing
program improvement recommendations for CMS.
You were identified by [name of CEO or CMS principal contact] as the best person to
complete the survey for the ${theme} theme. Section C.4.14 of the 9th SOW contract requires
each QIO to provide data for evaluation. Your time to participate in the survey is an expense
covered under the contract.
We have not yet received your completed survey for the ${theme} theme. The survey should
take about 45 minutes to complete. Your survey responses will remain confidential.
Mathematica will report information to CMS in aggregate form only. As a theme leader, your
input is critical in assisting CMS in its efforts to improve the QIO program with each new scope
of work.
To begin the QIO Theme Leader Survey for the ${theme} theme, click the link below:
[link would be embedded in email]
We look forward to receiving your completed survey within the next few days. Thanks in
advance for your participation. If you have any questions, please do not hesitate to contact me at
[email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.14
QIO THEME LEADER SURVEY – REMINDER #2
Dear ${name},
Recently, we sent you an email invitation to participate in the QIO Theme Leader Survey.
The survey collects information about QIO activities, experience, environment, and suggestions
for program improvement specific to the ${theme} theme. Information you provide will support
Mathematica’s evaluation team in understanding the trends and patterns in outcomes we will be
studying, and in developing program improvement recommendations for CMS.
We have not yet received your completed survey for the ${theme} theme. Section C.4.14 of
the 9th SOW contract requires each QIO to provide data for evaluation. Your time to participate
in the survey is an expense covered under the contract.
Please take time to complete the survey today – it should take about 45 minutes. Your
survey responses will remain confidential. Mathematica will report information to CMS in
aggregate form only. As a theme leader, your input is critical in assisting CMS in its efforts to
improve the QIO program with each new scope of work.
To begin the QIO Theme Leader Survey for the ${theme} theme, click the link below:
[link would be embedded in email]
Thanks in advance for your participation. If you have any questions, please do not hesitate to
contact me at [email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.15
QIO THEME LEADER SURVEY – REMINDER #3
Dear ${name},
Time is running out to participate in the QIO Theme Leader Survey! Information you
provide will support Mathematica’s evaluation team in understanding the trends and patterns in
outcomes we will be studying, and in developing program improvement recommendations for
CMS.
Section C.4.14 of the 9th SOW contract requires each QIO to provide data for evaluation.
Your time to participate in the survey is an expense covered under the contract.
Please complete the survey today.
Your survey responses will remain confidential. Mathematica will report information to
CMS in aggregate form only. As a theme leader, your input is critical in assisting CMS in its
efforts to improve the QIO program with each new scope of work.
To begin the QIO Theme Leader Survey for the ${theme} theme, click the link below:
[link would be embedded in email]
Thanks in advance for your participation. If you have any questions, please do not hesitate to
contact me at [email protected] or (202) 484-4519.
Sincerely,
Sue Felt-Lisk
Sr. Health Researcher
Co-Principal Investigator
Mathematica Policy Research
600 Maryland Ave., SW, Suite 550
Washington, DC 20024
C.16
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Director Survey
January 8, 2010
NINTH SCOPE OF WORK QIO PROGRAM EVALUATION:
QIO Director Survey
The QIO Director Survey is the key mechanism for gathering QIO input for the program
evaluation of the 9th SOW being conducted by Mathematica Policy Research (Mathematica).
Your suggestions for program improvement will support Mathematica’s evaluation team in
developing program- and theme-level recommendations for Centers for Medicare & Medicaid
Services (CMS). The survey should take 10-20 minutes to complete, and your time is a covered
expense under your contract per Section C.4.B.13 of the 9th SOW contract, which requires
each QIO to provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Prepared by Mathematica Policy Research
C.18
1.
In Column A, for each of the areas listed, please indicate if you would recommend that CMS make a
change to improve the QIO program’s success in generating quality improvement. If you do recommend
a change, please briefly describe your recommendation in Column B.
Column A
Column B
Recommend
that CMS make
a change
In the space provided, briefly describe what
change you would recommend.
Yes
No
a. Focus of QIO contract? ..............................
1
□
0
□
b. How QIOs are evaluated? ..........................
1
□
0
□
c. Program emphasis on QIOSCs?................
1
□
0
□
d. Program emphasis on data support
functions? ...................................................
1
□
0
□
e. How QIOs are expected to work with
other providers? .........................................
1
□
0
□
How QIOs are expected to work with
other health care organizations (such as
provider associations or health plans)?......
1
□
0
□
g. Other needed change (Please specify) ........
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.19
Challenges
2.
What area of quality improvement is presenting the most important challenge for you in the 9th SOW?
Recommended Changes
3.
What improvements would you suggest making to the tools or resources made available by CMS? Please list up to
three improvements that should be a high priority.
1.
2.
3.
4.
5.
Is there anything specific you would like to be doing to improve the quality of care in this state that you feel you
cannot do under the current contract?
1
□ Yes
GO TO Q.5
0
□ No
Thank you for completing this survey. See return instructions below.
Please list up to three things you would like to do to improve the quality of care in this state that you feel you cannot
do under the current contract.
1.
2.
3.
Thank you for completing the QIO Director Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.20
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Care Transitions
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Care Transitions
Prepared by Mathematica Policy Research
C.22
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.23
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff? Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree,
disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.24
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.25
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is
addressing...............................................................
1
□
0
□
b. Enable design of intervention with high likelihood
of success ...............................................................
1
□
0
□
c. Identify disparities related to this theme ..................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere..
1
□
0
□
n
□
e. Adequately justify the intervention to providers and
others ......................................................................
1
□
0
□
n
□
Prepared by Mathematica Policy Research
C.26
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
Medium
Value
Low
Value
Did Not
Use
a. Reports from the QIOSC that contain data
analysis .........................................................
1
□
2
□
3
□
0
□
b. Conference calls convened by the QIOSC ...
1
□
2
□
3
□
0
□
c. Tools provided by the QIOSC (regardless of
source) ..........................................................
1
□
2
□
3
□
0
□
d. Data from IFMC that the QIO analyzes itself
1
□
2
□
3
□
0
□
e. QualityNet (including MedQIC) .....................
1
□
2
□
3
□
0
□
f.
QualityNet conferences .................................
1
□
2
□
3
□
0
□
g. Conferences sponsored by other
organizations .................................................
1
□
2
□
3
□
0
□
h. Webinars or teleconferences sponsored by
other organizations........................................
1
□
2
□
3
□
0
□
i.
Key websites (other than MedQIC) ...............
1
□
2
□
3
□
0
□
j.
Personal contacts with other QIOs................
1
□
2
□
3
□
0
□
k. Personal contacts with other health care
organizations .................................................
1
□
2
□
3
□
0
□
1
□
2
□
3
□
0
□
l.
Other key information source (Please specify
below) ............................................................
___________________________________
Prepared by Mathematica Policy Research
C.27
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
13.
□
□
Yes
No
GO TO Q.14
Please identify the tools or other resource(s) and describe how they were adapted.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.28
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.29
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.30
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.31
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Care Transitions
20.
In Column A, for each of the following care transitions activities, indicate if it is a major
or minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality with regard to this theme. If an activity is “N/A” in Column A, then skip
Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Encouraging and training
on the use of the CARE
instrument ...........................
1
□
2
□
n
□
1
□
2
□
3
□
b. Use of a Transitions
Coach ..................................
1
□
2
□
n
□
1
□
2
□
3
□
Prepared by Mathematica Policy Research
C.32
SECTION V: STAFFING
21.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
| YEARS
| YEARS
Professional level:
1
2
3
4
5
22.
Associate’s degree
b.
|
e.
Some college
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify) ___________________________________________
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| STAFF
Prepared by Mathematica Policy Research
C.33
23.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Care Transitions. For each, please indicate his/her highest
level of education attainment, field of study, and years of relevant QI experience, and
years of experience working with the types of providers or organizations relevant to this
theme.
PERSON #1
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc]
Doctoral degree [EdD, PhD]
Other (please specify): __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
|
| YEARS
| YEARS
PERSON #2
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
|
| YEARS
| YEARS
Prepared by Mathematica Policy Research
C.34
24.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
25.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
26.
Regulatory oversight
Actively engaged with others (such as the QIO and/or
other provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.________________________________________________________________
2.________________________________________________________________
Prepared by Mathematica Policy Research
C.35
27.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 26.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
The association tends to work with a different set of providers than
the QIO..............................................................................................
1
□
0
□
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
1
□
0
□
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.36
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 26. If only one organization was listed at
Question 26, go to Question 29.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g. quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
f.
The association tends to work with a different set of providers than
the QIO..............................................................................................
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
Role of Large Provider Organizations
The following questions apply to the state level.
29.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
30.
□
□
Yes
No
GO TO Q.33
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1. __________________________________________________________________
2. __________________________________________________________________
Prepared by Mathematica Policy Research
C.37
31.
To what extent does the headquarters of the first organization listed in Question 30 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
32.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 30
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 30, then go to Question 33.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
33.
Please list up to three other external organizations whose efforts are proving important
to achieving improvements on this theme. (Please spell out the full name of the
organization.)
1. __________________________________________________________________
2. __________________________________________________________________
3. __________________________________________________________________
Prepared by Mathematica Policy Research
C.38
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
34.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
agree
Agree
Disagree
Strongly
disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.39
If you responded “agree” or “strongly agree” to Question 34, item g, go to Question 35.
Otherwise, go to Question 36.
35.
What types of providers lack motivation to improve on this theme?
1
2
3
4
36.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
37.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.40
If you responded “agree” or “strongly agree” to Question 37, item c, go to Question 38.
Otherwise, go to Question 39.
38.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
39.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline. .............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines. .........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines. .........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.41
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
40.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
Standardized announcement to eligible providers from the QIO
3
□
□
□
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
__________________________________________________
1
2
QIO initiated personalized contact with providers
Provider associations or other organizations outside the QIO
delivered announcement
Prepared by Mathematica Policy Research
C.42
41.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 42.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.43
NA
NA
42.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those
who asked questions but ultimately decided not to participate.
1
2
3
43.
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
44.
□
□
□
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough providers
GO TO Q.45
Please tell us what types of providers were the most difficult to persuade to participate?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
45.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
46.
Yes
No
GO TO Q.47
How many providers lobbied the QIO for participation?
1
2
3
47.
□
□
□
□
□
1 to 2
3 to 4
5 or more
What is the total number of participating providers at present?
|
|
|
| NUMBER
Prepared by Mathematica Policy Research
C.44
48.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
49.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
50.
100%
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.51
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Prepared by Mathematica Policy Research
C.45
Thank you for
completing this
survey
51.
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.46
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Chronic Kidney Disease
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format - neither you nor your QIO will be able to be identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Chronic Kidney Disease
Prepared by Mathematica Policy Research
C.48
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme. For
each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.49
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff. Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree, disagree,
or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.50
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel .........................
(specify role below):
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.51
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is
addressing.....................................................
1
□
0
□
b. Enable design of intervention with high
likelihood of success .....................................
1
□
0
□
c. Identify disparities related to this theme ........
1
□
0
□
n
□
d. Identify what interventions are working
elsewhere ......................................................
1
□
0
□
n
□
e. Adequately justify the intervention to
providers and others .....................................
1
□
0
□
n
□
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
a. Reports from the QIOSC that contain data
analysis .........................................................
1
b. Conference calls convened by the QIOSC ...
1
c. Tools provided by the QIOSC (regardless of
source) ..........................................................
1
d. Data from IFMC that the QIO analyzes itself
1
e. QualityNet (including MedQIC) .....................
1
f.
QualityNet conferences .................................
1
□
□
□
□
g. Conferences sponsored by other
organizations .................................................
1
□
h. Webinars or teleconferences sponsored by
other organizations........................................
1
i.
Key websites (other than MedQIC) ...............
1
j.
Personal contacts with other QIOs................
1
□
□
□
k. Personal contacts with other health care
organizations .................................................
1
1
l.
Other key information source (Please specify
below) ............................................................
___________________________________
Prepared by Mathematica Policy Research
C.52
□
□
Medium
Value
2
2
□
□
2
□
□
□
□
2
□
2
2
2
2
□
□
□
□
2
□
2
2
2
Low
Value
3
3
□
□
3
□
□
□
□
3
□
3
3
3
3
□
□
□
□
3
□
3
3
3
Did Not
Use
0
0
□
□
0
□
□
□
□
0
□
0
0
0
0
□
□
□
□
0
□
□
0
□
0
0
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
13.
□
□
Yes
No
Go to Question 14
Please identify the tools or other resource(s) and describe how they were adapted.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
Go to Question 16
Please describe the tool(s) or other resource(s) created.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.53
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.54
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ..........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.55
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.56
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
Prepared by Mathematica Policy Research
1
□
2
□
C.57
n
□
1
□
2
□
3
□
SECTION V: STAFFING
20.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
e.
1
2
3
4
5
21.
Some college
|
|
| YEARS
| YEARS
Professional level:
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify) __________________________________________
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.58
22.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Chronic Kidney Disease. For each, please indicate his/her
highest level of education attainment, field of study, and years of relevant QI experience,
and years of experience working with the types of providers or organizations relevant to
this theme.
PERSON #1
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
|
| YEARS
| YEARS
PERSON #2
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
|
| YEARS
| YEARS
Prepared by Mathematica Policy Research
C.59
23.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have
the right substantive expertise and
experience.....................................................
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.60
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
24.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
25.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.
2.
26.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 25.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.61
27.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 25. If only one organization was listed at
Question 25, go to Question 28.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
28.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
29.
□
□
Yes
No
GO TO Q.324
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1.
2.
Prepared by Mathematica Policy Research
C.62
31.
To what extent does the headquarters of the first organization listed in Question 29 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
31.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 29
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 29, then go to Question 32.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
32.
Please list up to three other external organizations whose efforts are proving important
to achieving improvements on this theme. (Please spell out the full name of the
organization.)
1.
2.
3.
Prepared by Mathematica Policy Research
C.63
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
33.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
agree
Agree
Disagree
Strongly
disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.64
If you responded “agree” or “strongly agree” to Question 33, item g, go to Question 34.
Otherwise, go to Question 35.
34.
What types of providers lack motivation to improve on this theme?
1
2
3
4
35.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
36.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.65
If you responded “agree” or “strongly agree” to Question 36, item c, go to Question 37.
Otherwise, go to Question 38.
37.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
38.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.66
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
39.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
1
□
Standardized announcement to eligible providers from the QIO
2
□
QIO initiated personalized contact with providers
3
□
Provider associations or other organizations outside the
QIO delivered announcement
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
Prepared by Mathematica Policy Research
C.67
40.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 41.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.68
NA
NA
41.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those who
asked questions but ultimately decided not to participate.
1
2
3
42.
□
□
□
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough provider
GO TO Q.44
43.
Please tell us what types of providers were the most difficult to persuade to participate?
44.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
45.
□
□
Yes
No
GO TO Q.46
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.69
46.
What is the total number of participating providers at present?
________ NUMBER
47.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
48.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
49.
100%
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.50
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Prepared by Mathematica Policy Research
C.70
Thank you for
completing this survey
50.
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.71
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Drug Safety
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Drug Safety
Prepared by Mathematica Policy Research
C.74
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.75
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff? Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree,
disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.76
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.77
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Interactions with Individual Providers
9.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Important of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ...
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated .................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers .
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations .....................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ...........
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI ........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them ......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations .......
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.78
Patient Safety
10.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
n
11.
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this theme?
1
2
3
4
d
12.
□
□
□
□
□
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
About what percent of QIO staff time devoted to the patient safety drug safety sub-theme
has been spent on activities with (or targeting) the following organizations?
Please round to the nearest percent. Percents should total 100.
a. Medicare providers and practitioners ..................................... |
|
|
| %
b. Medicare Advantage plans .................................................... |
|
|
| %
c. Prescription Drug Sponsor plans (PDPs) under Part D ......... |
|
|
| %
d. Other (Please specify).............................................................. |
|
|
| %
TOTAL 100%
Prepared by Mathematica Policy Research
C.79
13.
How interested did you find each of these types of organizations to be in working on
projects to reduce drug-drug interactions?
Very
Somewhat
interested interested
Little or
no interest
Don’t
know
a. Medicare providers and practitioners ..........
1
□
2
□
3
□
d
□
b. Medicare Advantage plans..........................
1
□
2
□
3
□
d
□
c. Prescription Drug Sponsor plans (PDPs)
under Part D ................................................
1
□
2
□
3
□
d
□
d. Other (Specify) .............................................
1
□
2
□
3
□
d
□
__________________________________
14.
How interested did you find each of these types of organizations to be in working on
projects to reduce prescriptions of potentially inappropriate medications?
Very
interested
Somewhat
interested
Little or no
interest
Don’t know
a. Medicare providers and practitioners ..........
1
□
2
□
3
□
d
□
b. Medicare Advantage plans..........................
1
□
2
□
3
□
d
□
c. Prescription Drug Sponsor plans (PDPs)
under Part D ................................................
1
□
2
□
3
□
d
□
d. Other (Specify) .............................................
1
□
2
□
3
□
d
□
__________________________________
15.
What has been the QIO’s most important contribution thus far under the drug safety subtheme?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.80
16.
What lessons have you learned about partnering with other organizations to improve drug
safety?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
17.
Do you have any suggestions for how CMS could improve the drug safety sub-theme?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.81
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Methicillin Resistant Staphylococcus Aureus
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Methicillin Resistant Staphylococcus Aureus
Prepared by Mathematica Policy Research
C.84
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. contract language at the time of award .........
1
□
2
□
3
□
4
□
c. contract modification(s) since award .............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.85
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff. Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree, disagree,
or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear..............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals.................................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable .....................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care .................
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable .....................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear..........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ..............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ......
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.86
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not
Enough
Contact
to Tell
a. Government Task Leader..................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ..............
1
□
2
□
3
□
4
□
5
□
c. Contract Officer .................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer...........................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.87
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is addressing ................
1
□
0
□
b. Enable design of intervention with high likelihood of success.......
1
□
0
□
c. Identify disparities related to this theme ........................................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere........................
1
□
0
□
n
□
e. Adequately justify the intervention to providers and others ...........
1
□
0
□
n
□
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
a. Reports from the QIOSC that contain data
analysis ...........................................................
1
b. Conference calls convened by the QIOSC .....
1
c. Tools provided by the QIOSC (regardless of
source) ............................................................
1
d. Data from IFMC that the QIO analyzes itself ..
1
e. QualityNet (including MedQIC) .......................
1
f.
QualityNet conferences ...................................
1
□
□
□
□
g. Conferences sponsored by other
organizations ...................................................
1
□
h. Webinars or teleconferences sponsored by
other organizations..........................................
1
i.
Key websites (other than MedQIC) .................
1
j.
Personal contacts with other QIOs..................
1
□
□
□
k. Personal contacts with other health care
organizations ...................................................
1
1
l.
Other key information source (Please specify
below) ..............................................................
Prepared by Mathematica Policy Research
C.88
□
□
Medium
Value
2
2
□
□
2
□
□
□
□
2
□
2
2
2
2
□
□
□
□
2
□
2
2
2
Low
Value
3
3
□
□
3
□
□
□
□
3
□
3
3
3
3
□
□
□
□
3
□
3
3
3
Did Not
Use
0
0
□
□
0
□
□
□
□
0
□
0
0
0
0
□
□
□
□
0
□
□
0
□
0
0
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
□
□
Yes
No
GO TO Q.14
13.
Please identify the tools or other resource(s) and describe how they were adapted.
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
Prepared by Mathematica Policy Research
C.89
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.90
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.91
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.92
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Patient Safety
20.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
5
21.
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this theme?
1
2
3
4
d
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
Prepared by Mathematica Policy Research
C.93
SECTION V: STAFFING
22.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience:
|
d.
e.
|
| YEARS
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
| YEARS
Professional level:
1
2
3
4
5
23.
□
□
□
□
□
□
□
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify)
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.94
24.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Methicillin Resistant Staphylococcus Aureus. For each,
please indicate his/her highest level of education attainment, field of study, and years of
relevant QI experience, and years of experience working with the types of providers or
organizations relevant to this theme.
PERSON #1
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
|
| YEARS
PERSON #2
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
Prepared by Mathematica Policy Research
|
| YEARS
C.95
25.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.96
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
26.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
27.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.
2.
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 27.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.97
29.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 27. If only one organization was listed at
Question 27, go to Question 30.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
30.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
□
□
Yes
No
GO TO Q.34
Prepared by Mathematica Policy Research
C.98
31.
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1.
2.
32.
To what extent does the headquarters of the first organization listed in Question 31 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
33.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 31
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 31, then go to Question 34.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
34.
Please list up to three other external organizations whose efforts are proving important to
achieving improvements on this theme. (Please spell out the full name of the
organization.)
1.
2.
3.
Prepared by Mathematica Policy Research
C.99
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
35.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
Strongly
agree
Agree
Disagree disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.100
If you responded “agree” or “strongly agree” to Question 35, item g, go to Question 36.
Otherwise, go to Question 37.
36.
What types of providers lack motivation to improve on this theme?
1
2
3
4
37.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
38.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.101
If you responded “agree” or “strongly agree” to Question 38, item c, go to Question 39.
Otherwise, go to Question 40.
39.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
40.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.102
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
41.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
1
□
Standardized announcement to eligible providers from the QIO
2
□
QIO initiated personalized contact with providers
3
□
Provider associations or other organizations outside the
QIO delivered announcement
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
Prepared by Mathematica Policy Research
C.103
42.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 43.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.104
NA
NA
43.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those who
asked questions but ultimately decided not to participate.
1
2
3
44.
□
□
□
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough provider
GO TO Q.46
45.
Please tell us what types of providers were the most difficult to persuade to participate?
46.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
47.
□
□
Yes
No
GO TO Q.48
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.105
48.
What is the total number of participating providers at present?
________ NUMBER
49.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
50.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.106
100%
51.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.52
52.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.107
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Nursing Homes in Need
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Nursing Homes in Need
Prepared by Mathematica Policy Research
C.110
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.111
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff? Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree,
disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.112
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.113
SECTION II: PATIENT SAFETY – NURSING HOMES IN NEED
8.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
n
9.
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this sub-theme?
1
2
3
4
d
10.
□
□
□
□
□
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
Think about the nursing home you first worked with under the nursing homes in need subtheme. About what percent of QIO staff time was spent on the following activities related
to the sub-theme?
Please round to the nearest percent. Percents should total 100.
Percent of Time
a. Root cause analysis ...............................................................
|
|
|
| %
b. Developing an action plan ......................................................
|
|
|
| %
c. Training or other interventions ...............................................
|
|
|
| %
d. Finding or preparing educational materials for the NHIN .......
|
|
|
| %
e. Follow-up assessment ...........................................................
|
|
|
| %
f.
|
|
|
| %
Other (Please specify)............................................................
TOTAL
Prepared by Mathematica Policy Research
C.114
100%
11.
Thinking again about the nursing home you first worked with under the nursing homes in
need sub-theme, to what extent do you agree with the following statements?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The root cause analysis led to identification
of problems that QIO staff were wellqualified to assist with ...................................
1
□
2
□
3
□
4
□
b. With the QIO’s assistance, the nursing
home was able to make substantial
progress on the major problems affecting its
quality ............................................................
1
□
2
□
3
□
4
□
c. Nursing home executives and staff seemed
to appreciate QIO assistance ........................
1
□
2
□
3
□
4
□
d. The nursing home was able to carry out its
action plan successfully ................................
1
□
2
□
3
□
4
□
12.
What has been the QIO’s most important contribution thus far under the nursing homes in
need sub-theme?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
13.
What lessons have you learned from your experience to date on the nursing homes in
need sub-theme?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.115
14.
Do you have any suggestions for how CMS could improve the nursing homes in need
sub-theme?
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.116
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Physical Restraints
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Physical Restraints
Prepared by Mathematica Policy Research
C.118
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. contract language at the time of award .........
1
□
2
□
3
□
4
□
c. contract modification(s) since award .............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.119
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff. Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree, disagree,
or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear..............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals.................................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable .....................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care .................
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable .....................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear..........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ..............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ......
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.120
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not
Enough
Contact
to Tell
a. Government Task Leader..................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ..............
1
□
2
□
3
□
4
□
5
□
c. Contract Officer .................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer...........................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.121
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is addressing ................
1
□
0
□
b. Enable design of intervention with high likelihood of success.......
1
□
0
□
c. Identify disparities related to this theme ........................................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere........................
1
□
0
□
n
□
e. Adequately justify the intervention to providers and others ...........
1
□
0
□
n
□
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
a. Reports from the QIOSC that contain data
analysis ...........................................................
1
b. Conference calls convened by the QIOSC .....
1
c. Tools provided by the QIOSC (regardless of
source) ............................................................
1
d. Data from IFMC that the QIO analyzes itself ..
1
e. QualityNet (including MedQIC) .......................
1
f.
QualityNet conferences ...................................
1
□
□
□
□
g. Conferences sponsored by other
organizations ...................................................
1
□
h. Webinars or teleconferences sponsored by
other organizations..........................................
1
i.
Key websites (other than MedQIC) .................
1
j.
Personal contacts with other QIOs..................
1
□
□
□
k. Personal contacts with other health care
organizations ...................................................
1
1
l.
Other key information source (Please specify
below) ..............................................................
Prepared by Mathematica Policy Research
C.122
□
□
Medium
Value
2
2
□
□
2
□
□
□
□
2
□
2
2
2
2
□
□
□
□
2
□
2
2
2
Low
Value
3
3
□
□
3
□
□
□
□
3
□
3
3
3
3
□
□
□
□
3
□
3
3
3
Did Not
Use
0
0
□
□
0
□
□
□
□
0
□
0
0
0
0
□
□
□
□
0
□
□
0
□
0
0
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
□
□
Yes
No
GO TO Q.14
13.
Please identify the tools or other resource(s) and describe how they were adapted.
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
Prepared by Mathematica Policy Research
C.123
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.124
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.125
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.126
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Patient Safety
20.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
5
21.
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this theme?
1
2
3
4
d
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
Prepared by Mathematica Policy Research
C.127
SECTION V: STAFFING
22.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience:
|
d.
e.
|
| YEARS
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
| YEARS
Professional level:
1
2
3
4
5
23.
□
□
□
□
□
□
□
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify)
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.128
24.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Physical Restraints. For each, please indicate his/her
highest level of education attainment, field of study, and years of relevant QI experience,
and years of experience working with the types of providers or organizations relevant to
this theme.
PERSON #1
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
|
| YEARS
PERSON #2
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
Prepared by Mathematica Policy Research
|
| YEARS
C.129
25.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.130
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
26.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
27.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.
2.
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 27.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.131
29.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 27. If only one organization was listed at
Question 27, go to Question 30.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
30.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
□
□
Yes
No
GO TO Q.34
Prepared by Mathematica Policy Research
C.132
31.
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1.
2.
32.
To what extent does the headquarters of the first organization listed in Question 31 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
33.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 31
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 31, then go to Question 34.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
34.
Please list up to three other external organizations whose efforts are proving important to
achieving improvements on this theme. (Please spell out the full name of the
organization.)
1.
2.
3.
Prepared by Mathematica Policy Research
C.133
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
35.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
Strongly
agree
Agree
Disagree disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.134
If you responded “agree” or “strongly agree” to Question 35, item g, go to Question 36.
Otherwise, go to Question 37.
36.
What types of providers lack motivation to improve on this theme?
1
2
3
4
37.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
38.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.135
If you responded “agree” or “strongly agree” to Question 38, item c, go to Question 39.
Otherwise, go to Question 40.
39.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
40.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.136
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
41.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
1
□
Standardized announcement to eligible providers from the QIO
2
□
QIO initiated personalized contact with providers
3
□
Provider associations or other organizations outside the
QIO delivered announcement
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
Prepared by Mathematica Policy Research
C.137
42.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 43.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.138
NA
NA
43.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those who
asked questions but ultimately decided not to participate.
1
2
3
44.
□
□
□
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough provider
GO TO Q.46
45.
Please tell us what types of providers were the most difficult to persuade to participate?
46.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
47.
□
□
Yes
No
GO TO Q.48
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.139
48.
What is the total number of participating providers at present?
________ NUMBER
49.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
50.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.140
100%
51.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.52
52.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.141
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Pressure Ulcers
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Pressure Ulcers
Prepared by Mathematica Policy Research
C.144
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. contract language at the time of award .........
1
□
2
□
3
□
4
□
c. contract modification(s) since award .............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.145
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff. Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree, disagree,
or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear..............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals.................................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable .....................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care .................
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable .....................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear..........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ..............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ......
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.146
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not
Enough
Contact
to Tell
a. Government Task Leader..................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ..............
1
□
2
□
3
□
4
□
5
□
c. Contract Officer .................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer...........................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.147
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is addressing ................
1
□
0
□
b. Enable design of intervention with high likelihood of success.......
1
□
0
□
c. Identify disparities related to this theme ........................................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere........................
1
□
0
□
n
□
e. Adequately justify the intervention to providers and others ...........
1
□
0
□
n
□
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
a. Reports from the QIOSC that contain data
analysis ...........................................................
1
b. Conference calls convened by the QIOSC .....
1
c. Tools provided by the QIOSC (regardless of
source) ............................................................
1
d. Data from IFMC that the QIO analyzes itself ..
1
e. QualityNet (including MedQIC) .......................
1
f.
QualityNet conferences ...................................
1
□
□
□
□
g. Conferences sponsored by other
organizations ...................................................
1
□
h. Webinars or teleconferences sponsored by
other organizations..........................................
1
i.
Key websites (other than MedQIC) .................
1
j.
Personal contacts with other QIOs..................
1
□
□
□
k. Personal contacts with other health care
organizations ...................................................
1
1
l.
Other key information source (Please specify
below) ..............................................................
Prepared by Mathematica Policy Research
C.148
□
□
Medium
Value
2
2
□
□
2
□
□
□
□
2
□
2
2
2
2
□
□
□
□
2
□
2
2
2
Low
Value
3
3
□
□
3
□
□
□
□
3
□
3
3
3
3
□
□
□
□
3
□
3
3
3
Did Not
Use
0
0
□
□
0
□
□
□
□
0
□
0
0
0
0
□
□
□
□
0
□
□
0
□
0
0
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
□
□
Yes
No
GO TO Q.14
13.
Please identify the tools or other resource(s) and describe how they were adapted.
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
Prepared by Mathematica Policy Research
C.149
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.150
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.151
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.152
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Patient Safety
20.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
5
21.
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this theme?
1
2
3
4
d
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
Prepared by Mathematica Policy Research
C.153
SECTION V: STAFFING
22.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience:
|
d.
e.
|
| YEARS
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
| YEARS
Professional level:
1
2
3
4
5
23.
□
□
□
□
□
□
□
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify)
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.154
24.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Pressure Ulcers. For each, please indicate his/her highest
level of education attainment, field of study, and years of relevant QI experience, and
years of experience working with the types of providers or organizations relevant to this
theme.
PERSON #1
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
|
| YEARS
PERSON #2
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
Prepared by Mathematica Policy Research
|
| YEARS
C.155
25.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.156
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
26.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
27.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.
2.
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 27.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.157
29.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 27. If only one organization was listed at
Question 27, go to Question 30.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
30.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
□
□
Yes
No
GO TO Q.34
Prepared by Mathematica Policy Research
C.158
31.
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1.
2.
32.
To what extent does the headquarters of the first organization listed in Question 31 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
33.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 31
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 31, then go to Question 34.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
34.
Please list up to three other external organizations whose efforts are proving important to
achieving improvements on this theme. (Please spell out the full name of the
organization.)
1.
2.
3.
Prepared by Mathematica Policy Research
C.159
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
35.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
Strongly
agree
Agree
Disagree disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.160
If you responded “agree” or “strongly agree” to Question 35, item g, go to Question 36.
Otherwise, go to Question 37.
36.
What types of providers lack motivation to improve on this theme?
1
2
3
4
37.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
38.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.161
If you responded “agree” or “strongly agree” to Question 38, item c, go to Question 39.
Otherwise, go to Question 40.
39.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
40.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.162
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
41.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
1
□
Standardized announcement to eligible providers from the QIO
2
□
QIO initiated personalized contact with providers
3
□
Provider associations or other organizations outside the
QIO delivered announcement
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
Prepared by Mathematica Policy Research
C.163
42.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 43.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.164
NA
NA
43.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those who
asked questions but ultimately decided not to participate.
1
2
3
44.
□
□
□
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough provider
GO TO Q.46
45.
Please tell us what types of providers were the most difficult to persuade to participate?
46.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
47.
□
□
Yes
No
GO TO Q.48
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.165
48.
What is the total number of participating providers at present?
________ NUMBER
49.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
50.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.166
100%
51.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.52
52.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.167
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Prevention - Disparities
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Prevention – Disparities
Prepared by Mathematica Policy Research
C.170
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.171
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff? Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree,
disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.172
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.173
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is
addressing...............................................................
1
□
0
□
b. Enable design of intervention with high likelihood
of success ...............................................................
1
□
0
□
c. Identify disparities related to this theme ..................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere..
1
□
0
□
n
□
e. Adequately justify the intervention to providers and
others ......................................................................
1
□
0
□
n
□
Prepared by Mathematica Policy Research
C.174
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
Medium
Value
Low
Value
Did Not
Use
a. Reports from the QIOSC that contain data
analysis .........................................................
1
□
2
□
3
□
0
□
b. Conference calls convened by the QIOSC ...
1
□
2
□
3
□
0
□
c. Tools provided by the QIOSC (regardless of
source) ..........................................................
1
□
2
□
3
□
0
□
d. Data from IFMC that the QIO analyzes itself
1
□
2
□
3
□
0
□
e. QualityNet (including MedQIC) .....................
1
□
2
□
3
□
0
□
f.
QualityNet conferences .................................
1
□
2
□
3
□
0
□
g. Conferences sponsored by other
organizations .................................................
1
□
2
□
3
□
0
□
h. Webinars or teleconferences sponsored by
other organizations........................................
1
□
2
□
3
□
0
□
i.
Key websites (other than MedQIC) ...............
1
□
2
□
3
□
0
□
j.
Personal contacts with other QIOs................
1
□
2
□
3
□
0
□
k. Personal contacts with other health care
organizations .................................................
1
□
2
□
3
□
0
□
1
□
2
□
3
□
0
□
l.
Other key information source (Please specify
below) ............................................................
___________________________________
Prepared by Mathematica Policy Research
C.175
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
13.
□
□
Yes
No
GO TO Q.14
Please identify the tools or other resource(s) and describe how they were adapted.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.176
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as
a health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.177
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If
the activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.178
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.179
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Disparities
20.
In Column A, for each of the following Prevention - Disparities activities, indicate if it is a
major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality with regard to this theme. If an activity is “N/A” in Column A, then skip
Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Obtaining clinical EHR-
based data from
practices ...........................
1
□
2
□
n
□
1
□
2
□
3
□
community health
workers .............................
1
□
2
□
n
□
1
□
2
□
3
□
Implementing DSME for
beneficiaries with
diabetes ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Recruiting and training
c.
Prepared by Mathematica Policy Research
C.180
21.
To meet the goals of providing Diabetes Self-Management Education, what mechanisms
were used by the QIO to recruit beneficiaries?
MARK ALL THAT APPLY
1
2
3
□
□
□
Recruitment through local providers
Recruitment directly from the community
Other mechanism(s) (Please specify)
_________________________________________
22.
Which of the following best describes the geographic area targeted under this theme?
1
2
3
4
5
□
□
□
□
□
All urban or suburban
Mostly urban or suburban
About evenly urban/suburban and rural
Mostly rural
All rural
Prepared by Mathematica Policy Research
C.181
SECTION V: STAFFING
23.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
| YEARS
| YEARS
Professional level:
1
2
3
4
5
24.
Associate’s degree
b.
|
e.
Some college
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify) ___________________________________________
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.182
25.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Prevention Disparities. For each, please indicate his/her
highest level of education attainment, field of study, and years of relevant QI experience,
and years of experience working with the types of providers or organizations relevant to
this theme.
PERSON #1
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
|
| YEARS
| YEARS
PERSON #2
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
|
| YEARS
| YEARS
Prepared by Mathematica Policy Research
C.183
26.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.184
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
27.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
28.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.________________________________________________________________
2.________________________________________________________________
29.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 28.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g. quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
The association tends to work with a different set of providers than
the QIO..............................................................................................
1
□
0
□
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
1
□
0
□
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.185
30.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 26. If only one organization was listed at
Question 28, go to Question 31.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
The association tends to work with a different set of providers than
the QIO..............................................................................................
1
□
0
□
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
1
□
0
□
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
31.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
32.
□
□
Yes
No
GO TO Q.35
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this
theme and have the greatest potential to affect its success.
1. __________________________________________________________________
2. __________________________________________________________________
Prepared by Mathematica Policy Research
C.186
33.
To what extent does the headquarters of the first organization listed in Question 32 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
34.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 32
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 32, then go to Question 35.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
35.
Please list up to three other external organizations whose efforts are proving important to
achieving improvements on this theme. (Please spell out the full name of the
organization.)
1. __________________________________________________________________
2. __________________________________________________________________
3. __________________________________________________________________
Prepared by Mathematica Policy Research
C.187
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
36.
The following are statements which related to motivation and quality improvement.
For each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
agree
Agree
Disagree
Strongly
disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.188
If you responded “agree” or “strongly agree” to Question 36, item g, go to Question 37.
Otherwise, go to Question 38.
37.
What types of providers lack motivation to improve on this theme?
1
2
3
4
38.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
39.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree,
disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
Know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.189
If you responded “agree” or “strongly agree” to Question 39, item c, go to Question 40.
Otherwise, go to Question 41.
40.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
41.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ............................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline. ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines. ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines. ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given .....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.190
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
42.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
Standardized announcement to eligible providers from the QIO
3
□
□
□
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
____________________________________________________________
1
2
QIO initiated personalized contact with providers
Provider associations or other organizations outside the QIO
delivered announcement
Prepared by Mathematica Policy Research
C.191
43.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 44.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.192
NA
NA
44.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those
who asked questions but ultimately decided not to participate.
1
2
3
45.
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
46.
□
□
□
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough providers
GO TO Q.47
Please tell us what types of providers were the most difficult to persuade to participate.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
47.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
48.
□
□
Yes
No
GO TO Q.49
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.193
49.
What is the total number of participating providers at present?
|
50.
|
|
| NUMBER
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
51.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.194
100%
52.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.53
53.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.195
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Prevention
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Prevention
Prepared by Mathematica Policy Research
C.198
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. Contract language at the time of award ........
1
□
2
□
3
□
4
□
c. Contract modification(s) since award ............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.199
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff? Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree,
disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals...............................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable ...................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care ...............
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable ...................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ....
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.200
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not Enough
Contact
to Tell
a. Government Task Leader ...................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ................
1
□
2
□
3
□
4
□
5
□
c. Contract Officer ...................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer ............................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
______________________________
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.201
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is
addressing...............................................................
1
□
0
□
b. Enable design of intervention with high likelihood
of success ...............................................................
1
□
0
□
c. Identify disparities related to this theme ..................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere..
1
□
0
□
n
□
e. Adequately justify the intervention to providers and
others ......................................................................
1
□
0
□
n
□
Prepared by Mathematica Policy Research
C.202
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
Medium
Value
Low
Value
Did Not
Use
a. Reports from the QIOSC that contain data
analysis .........................................................
1
□
2
□
3
□
0
□
b. Conference calls convened by the QIOSC ...
1
□
2
□
3
□
0
□
c. Tools provided by the QIOSC (regardless of
source) ..........................................................
1
□
2
□
3
□
0
□
d. Data from IFMC that the QIO analyzes itself
1
□
2
□
3
□
0
□
e. QualityNet (including MedQIC) .....................
1
□
2
□
3
□
0
□
f.
QualityNet conferences .................................
1
□
2
□
3
□
0
□
g. Conferences sponsored by other
organizations .................................................
1
□
2
□
3
□
0
□
h. Webinars or teleconferences sponsored by
other organizations........................................
1
□
2
□
3
□
0
□
i.
Key websites (other than MedQIC) ...............
1
□
2
□
3
□
0
□
j.
Personal contacts with other QIOs................
1
□
2
□
3
□
0
□
k. Personal contacts with other health care
organizations .................................................
1
□
2
□
3
□
0
□
1
□
2
□
3
□
0
□
l.
Other key information source (Please specify
below) ............................................................
___________________________________
Prepared by Mathematica Policy Research
C.203
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
13.
□
□
Yes
No
GO TO Q.14
Please identify the tools or other resource(s) and describe how they were adapted.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
Prepared by Mathematica Policy Research
C.204
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.205
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.206
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.207
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Prevention
20.
Do you think participating providers found clinical quality reporting to be beneficial to their
ongoing quality improvement efforts?
1
0
d
□
□
□
Yes
No
Don’t know
Prepared by Mathematica Policy Research
C.208
SECTION V: STAFFING
21.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) ___________________________________________
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme
|
|
| YEARS
| YEARS
Professional level:
1
2
3
4
5
22.
Associate’s degree
b.
|
e.
Some college
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify) ___________________________________________
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.209
23.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Prevention. For each, please indicate his/her highest level
of education attainment, field of study, and years of relevant QI experience, and years of
experience working with the types of providers or organizations relevant to this theme.
PERSON #1
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
|
| YEARS
| YEARS
PERSON #2
a.
1
2
3
4
5
6
7
Highest level of educational attainment:
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify) __________________________________________
b.
Field of study, if applicable: _______________________________________
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
|
| YEARS
| YEARS
Prepared by Mathematica Policy Research
C.210
24.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.211
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
25.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
26.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.________________________________________________________________
2.________________________________________________________________
27.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 26.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g. quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
The association tends to work with a different set of providers than
the QIO..............................................................................................
1
□
0
□
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
1
□
0
□
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.212
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 26. If only one organization was listed at
Question 26, go to Question 29.
Yes
No
a. The association employs at least one staff member with major
responsibility and time devoted to quality improvement....................
1
□
0
□
b. The association and QIO talk periodically (e.g. quarterly) to avoid
overlap ..............................................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored meetings at
least once per year............................................................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as a
Quality Council or a Quality Institute .................................................
1
□
0
□
e. The association and QIO work jointly on one or more QI efforts
substantial in scope (such as co-sponsoring in-person meetings
focused on QI)...................................................................................
1
□
0
□
f.
The association tends to work with a different set of providers than
the QIO..............................................................................................
g. The association tends to work on QI projects that are entirely
different from the QIO .......................................................................
h. The association primarily focuses on quality reporting issues rather
than quality improvement ..................................................................
Role of Large Provider Organizations
The following questions apply to the state level.
29.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
30.
□
□
Yes
No
GO TO Q.33
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this
theme and have the greatest potential to affect its success.
1. __________________________________________________________________
2. __________________________________________________________________
Prepared by Mathematica Policy Research
C.213
31.
To what extent does the headquarters of the first organization listed in Question 30 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
32.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 30
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 30, then go to Question 33.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
33.
Please list up to three other external organizations whose efforts are proving important
to achieving improvements on this theme. (Please spell out the full name of the
organization.)
1. __________________________________________________________________
2. __________________________________________________________________
3. __________________________________________________________________
Prepared by Mathematica Policy Research
C.214
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
34.
The following are statements which related to motivation and quality improvement.
For each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
agree
Agree
Disagree
Strongly
disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.215
If you responded “agree” or “strongly agree” to Question 34, item g, go to Question 35.
Otherwise, go to Question 36.
35.
What types of providers lack motivation to improve on this theme?
1
2
3
4
36.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
37.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree,
disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.216
If you responded “agree” or “strongly agree” to Question 37, item c, go to Question 38.
Otherwise, go to Question 39.
38.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
39.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have
been working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ............................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline. ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines. ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines. ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given .....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.217
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
40.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
Standardized announcement to eligible providers from the QIO
3
□
□
□
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
____________________________________________________________
1
2
QIO initiated personalized contact with providers
Provider associations or other organizations outside the QIO
delivered announcement
Prepared by Mathematica Policy Research
C.218
41.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 42.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.219
NA
NA
42.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those
who asked questions but ultimately decided not to participate.
1
2
3
43.
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
44.
□
□
□
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough providers
GO TO Q.45
Please tell us what types of providers were the most difficult to persuade to participate.
____________________________________________________________________
____________________________________________________________________
____________________________________________________________________
45.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
46.
Yes
No
GO TO Q.47
How many providers lobbied the QIO for participation?
1
2
3
47.
□
□
□
□
□
1 to 2
3 to 4
5 or more
What is the total number of participating providers at present?
|
|
|
| NUMBER
Prepared by Mathematica Policy Research
C.220
48.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
49.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.221
100%
50.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.51
51.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.222
Mathematica Reference No.: 06514.180
Ninth Scope of Work QIO
Program Evaluation:
QIO Theme Leader
Survey
January 8, 2010
Ninth Scope of Work QIO Program Evaluation:
QIO Theme Leader Survey
Patient Safety – Surgical Care Improvement Project
This survey is the key mechanism for QIO input into the program evaluation of the 9th SOW.
The information collected about QIO activities, experience, environment, and suggestions for
program improvement will allow quantitative and qualitative analysis to support the Mathematica
Policy Research evaluation team in developing program- and theme-level recommendations to
CMS. Each survey should take 10-20 minutes to complete, and time is a covered expense
under your contract per Section C.4.B.13 of the 9th SOW contract, which requires each QIO to
provide data for evaluation.
All information you provide will be kept strictly confidential. Information generated from this
survey will be shared in aggregate format—neither you nor your QIO will be able to be
identified.
If you have any questions about this survey, please call Sarah Croake at 202-554-7555 or email
at: [email protected].
Please answer all questions on this survey in relation to the following
9th SOW theme:
Patient Safety Theme – Surgical Care Improvement Project
Prepared by Mathematica Policy Research
C.224
SECTION I: 9TH SOW CONTRACT AND COMMUNICATION WITH CMS
Clarity of the Contract for the 9th SOW
1.
Please indicate the level of clarity for each of the following aspects of the 9th SOW
contract for this theme. For each, was it very clear, clear, unclear, or very unclear?
Very
clear
Clarity of…
Clear
Unclear
Very
unclear
a. RFP aims and requirements .........................
1
□
2
□
3
□
4
□
b. contract language at the time of award .........
1
□
2
□
3
□
4
□
c. contract modification(s) since award .............
1
□
2
□
3
□
4
□
d. SDPS or TOPS memos since award ............
1
□
2
□
3
□
4
□
Contract Documentation and Reporting Requirements for the 9th SOW
2.
Please indicate if you agree with each of the following statements about the
documentation and reporting requirements of the 9th SOW contract for this theme.
For each, do you strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS documentation and reporting
requirements are clear ..................................
1
□
2
□
3
□
4
□
b. Required reports to CMS capture
meaningful information about the progress
of the intervention..........................................
1
□
2
□
3
□
4
□
c. The amount of CMS required
documentation and reporting is reasonable ..
1
□
2
□
3
□
4
□
d. The PATRIOT system worked well in the
first six months of the contract ......................
1
□
2
□
3
□
4
□
e. The PATRIOT system worked well after the
first six months of the contract ......................
1
□
2
□
3
□
4
□
3.
In an average month how many total hours are spent fulfilling CMS reporting requirements
for this theme?
If reporting does not occur on a monthly basis, please determine total hours spent over a
single reporting period and divide by the total months in that reporting period to determine
the average hours per month.
|
|
|
| HOURS
Prepared by Mathematica Policy Research
C.225
4.
Of the hours spent in an average month reported above, what percent is spent by senior
staff, mid-level staff, and junior staff. Percents should total 100.
For themes where QIO staff work with providers, mid-level staff would include those who
work directly with providers on theme-relevant improvements, while junior staff would take
supporting roles and senior staff would lead and manage the various efforts.
Percent
a. Senior level staff.....................................................................
|
|
|
|
b. Mid-level staff .........................................................................
|
|
|
|
c. Junior staff..............................................................................
|
|
|
|
Contract Goals and Objectives for the 9th SOW
5.
Please indicate if you agree with each of the following statements about the goals,
objectives and improvement targets of the 9th SOW contract related to this theme
(as modified at the time of this survey). For each, do you strongly agree, agree, disagree,
or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. CMS goals and objectives are clear..............
1
□
2
□
3
□
4
□
b. Resources for this theme are sufficient to
support goals.................................................
1
□
2
□
3
□
4
□
c. Improvement targets set by CMS for this
theme are attainable .....................................
1
□
2
□
3
□
4
□
d. Improvement targets set by CMS represent
meaningful improvements in care .................
1
□
2
□
3
□
4
□
e. The timeframe for meeting improvement
targets is reasonable .....................................
1
□
2
□
3
□
4
□
f.
Method for evaluating the QIO is clear..........
1
□
2
□
3
□
4
□
g. The QIO contract focuses effort on
important areas of quality ..............................
1
□
2
□
3
□
4
□
h. The QIO contract focuses effort on
providers whose improvements will have
substantial impact on quality in the state ......
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.226
Contract Support and Communication for the 9th SOW
6.
For each of the following, please indicate if their knowledge base relative to their
responsibilities was excellent, good, fair, or poor.
Excellent
Good
Fair
Poor
Not
Enough
Contact
to Tell
a. Government Task Leader..................
1
□
2
□
3
□
4
□
5
□
b. Government Theme Leader ..............
1
□
2
□
3
□
4
□
5
□
c. Contract Officer .................................
1
□
2
□
3
□
4
□
5
□
d. CMS Project Officer...........................
1
□
2
□
3
□
4
□
5
□
e. Other CMS personnel (Specify role
below) ..................................................
1
□
2
□
3
□
4
□
5
□
7.
Please indicate if you agree with each of the following statements about the support and
communication for the 9th SOW contract as it relates to this theme. For each, do you
strongly agree, agree, disagree, or strongly disagree?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The CMS Project Officer is supportive and
helpful............................................................
1
□
2
□
3
□
4
□
b. The CMS Project Officer understands the
QIO’s interventions........................................
1
□
2
□
3
□
4
□
c. Oral communication by CMS personnel is
clear ..............................................................
1
□
2
□
3
□
4
□
d. Contract modification(s) required little effort
to implement..................................................
1
□
2
□
3
□
4
□
e. Contract modifications improved the
contract .........................................................
1
□
2
□
3
□
4
□
8.
How often was the communication you had with CMS consistent across different CMS
personnel with whom you interfaced?
1
2
3
4
□
□
□
□
Always consistent
Usually consistent
Rarely consistent
Never consistent
Prepared by Mathematica Policy Research
C.227
SECTION II: SUFFICIENCY OF INFORMATION AND RESOURCES
9.
Thinking about designing effective interventions that are known to work, have you had
sufficient data and information to do the following during the 9th SOW?
Yes
No
N/A
a. Understand the problem the intervention is addressing ................
1
□
0
□
b. Enable design of intervention with high likelihood of success.......
1
□
0
□
c. Identify disparities related to this theme ........................................
1
□
0
□
n
□
d. Identify what interventions are working elsewhere........................
1
□
0
□
n
□
e. Adequately justify the intervention to providers and others ...........
1
□
0
□
n
□
10.
Think about the information you get that helps you shape and refine your intervention over
time so that it targets the right techniques to the right providers during the 9th SOW.
Please rate the value of the information you currently obtain from:
High
Value
a. Reports from the QIOSC that contain data
analysis ...........................................................
1
b. Conference calls convened by the QIOSC .....
1
c. Tools provided by the QIOSC (regardless of
source) ............................................................
1
d. Data from IFMC that the QIO analyzes itself ..
1
e. QualityNet (including MedQIC) .......................
1
f.
QualityNet conferences ...................................
1
□
□
□
□
g. Conferences sponsored by other
organizations ...................................................
1
□
h. Webinars or teleconferences sponsored by
other organizations..........................................
1
i.
Key websites (other than MedQIC) .................
1
j.
Personal contacts with other QIOs..................
1
□
□
□
k. Personal contacts with other health care
organizations ...................................................
1
1
l.
Other key information source (Please specify
below) ..............................................................
Prepared by Mathematica Policy Research
C.228
□
□
Medium
Value
2
2
□
□
2
□
□
□
□
2
□
2
2
2
2
□
□
□
□
2
□
2
2
2
Low
Value
3
3
□
□
3
□
□
□
□
3
□
3
3
3
3
□
□
□
□
3
□
3
3
3
Did Not
Use
0
0
□
□
0
□
□
□
□
0
□
0
0
0
0
□
□
□
□
0
□
□
0
□
0
0
SECTION III: TOOLS AND OTHER RESOURCES
11.
The following are statements about tools and other resources used to support
interventions related to this theme. To what extent do you agree with each of the
following?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. The tools and other resources that are
available to support interventions related to
this theme are of high quality ........................
1
□
2
□
3
□
4
□
b. The tools and other resources that are
available to support interventions related to
this theme were available when we needed
them ..............................................................
1
□
2
□
3
□
4
□
c. The tools and specifications that are
available to support measurement related to
this theme work well ......................................
1
□
2
□
3
□
4
□
12.
Has the QIO made substantial adaptations to available tools or other resources to support
interventions related to this theme?
1
0
□
□
Yes
No
GO TO Q.14
13.
Please identify the tools or other resource(s) and describe how they were adapted.
14.
Has the QIO had to create any tools or other resources to support interventions related to
this theme?
1
0
15.
□
□
Yes
No
GO TO Q.16
Please describe the tool(s) or other resource(s) created.
Prepared by Mathematica Policy Research
C.229
SECTION IV: ACTIVITIES
Collaborative Activities
16.
In Column A, for each of the following collaborative activities, indicate if it is a major or
minor component for this theme. If the activity does not occur, check the corresponding
“N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Forming new collaborations
among providers .................
1
□
2
□
n
□
1
□
2
□
3
□
b. Forming new collaborations
that include health
organizations other than
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
Contributing to existing
collaborations......................
1
□
2
□
n
□
1
□
2
□
3
□
d. Supporting a large
organization (such as a
health delivery
organization, or health
plan) in their efforts to
improve ...............................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.230
Interactions with Individual Providers
17.
In Column A, for each of the following activities that involved interactions with
individual providers, indicate if it is a major or minor component for this theme. If the
activity does not occur, check the corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Problem-solving or
strategizing with individual
providers at their request ....
1
□
2
□
n
□
1
□
2
□
3
□
b. Problem-solving or
strategizing with individual
providers during meetings
the QIO initiated ..................
1
□
2
□
n
□
1
□
2
□
3
□
Making presentations onsite at individual providers ..
1
□
2
□
n
□
1
□
2
□
3
□
d. Interacting with top
leadership of provider
organizations ......................
1
□
2
□
n
□
1
□
2
□
3
□
e. Helping integrate clinical
guidelines into health
information systems ............
1
□
2
□
n
□
1
□
2
□
3
□
Helping providers better
use their health information
systems to support QI .........
1
□
2
□
n
□
1
□
2
□
3
□
g. Discussing providers’ own
performance with them .......
1
□
2
□
n
□
1
□
2
□
3
□
h. Training staff within
provider organizations ........
1
□
2
□
n
□
1
□
2
□
3
□
c.
f.
Prepared by Mathematica Policy Research
C.231
One-to-Many Activities
18.
In Column A, for each of the following group education/meeting activities indicate if it
is a major or minor component for this theme. If the activity does not occur, check the
corresponding “N/A” box.
In Column B, for each activity that occurs, indicate the extent to which it is important to
improving quality or patient safety with regard to this theme. If an activity is “N/A” in
Column A, then skip Column B for that activity.
Column A
Column B
Importance of Activity
Major
Component
Minor
Component
N/A
Very
important
Somewhat
important
Not very
important
a. Providing one-to-many
educational or shared
learning sessions via
telephone ............................
1
□
2
□
n
□
1
□
2
□
3
□
b. Large regional or statewide
in-person meetings .............
1
□
2
□
n
□
1
□
2
□
3
□
Routinely providing
provider-specific data to
providers with benchmarks .
1
□
2
□
n
□
1
□
2
□
3
□
d. Notifying providers of
quality improvementrelated opportunities
sponsored by others ...........
1
□
2
□
n
□
1
□
2
□
3
□
e. Summarizing quality
improvement tips or
information in a QIO or
provider association
newsletter, in paper or
electronic format .................
1
□
2
□
n
□
1
□
2
□
3
□
c.
Prepared by Mathematica Policy Research
C.232
Business Case Focus
19.
In Column A, for the following activity, indicate if it is a major or minor component for this
theme. If the activity does not occur, check the corresponding “N/A” box.
In Column B, if this activity occurs, indicate the extent to which it is important to improving
quality with regard to this theme. If this activity is “N/A” in Column A, then skip Column B
for that activity.
Column A
Column B
Importance of Activity
Major
component
Minor
component
N/A
Very
important
Somewhat
important
Not very
important
a. Developing or
incorporating information
into materials, talks,
consultations, etc.
regarding the business
case for quality
improvement relevant to
this theme............................
1
□
2
□
n
□
1
□
2
□
3
□
Patient Safety
20.
How valuable were the annual in-person meetings sponsored by CMS specific to patient
safety?
1
2
3
4
5
21.
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Did not attend any annual in-person meetings sponsored by CMS
How valuable was the “change package” that CMS developed for this theme?
1
2
3
4
d
□
□
□
□
□
Very valuable
Valuable
Marginally valuable
Not valuable at all
Don’t know
Prepared by Mathematica Policy Research
C.233
SECTION V: STAFFING
22.
Think about the person most responsible for design of the intervention materials and
processes for this theme. For this person, please indicate his/her highest level of
education attainment, field of study, years of relevant QI experience, years of experience
working with the types of providers or organizations relevant to this theme, and
professional level (executive, senior, etc.).
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience:
|
d.
e.
|
| YEARS
Years of experience working with the types of providers or organizations relevant to
this theme:
|
|
| YEARS
Professional level:
1
2
3
4
5
23.
□
□
□
□
□
□
□
□
□
□
□
□
Executive
Senior
Mid-Level
Junior
Other (Please specify)
How many QIO staff have interacted directly, on a frequent basis, with providers and
collaborating organizations for this theme?
|
|
| ENTER NUMBER
Prepared by Mathematica Policy Research
C.234
24.
Think about the one or two people who have interacted most frequently with providers and
collaborating organizations for Surgical Care Improvement Project. For each, please
indicate his/her highest level of education attainment, field of study, and years of relevant
QI experience, and years of experience working with the types of providers or
organizations relevant to this theme.
PERSON #1
a.
Highest level of educational attainment:
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
|
| YEARS
PERSON #2
1
2
3
4
5
6
7
□
□
□
□
□
□
□
Some college
Associate’s degree
Bachelor’s degree
Master’s degree
Professional degree [MD, DPM, DO, PharmD, etc.]
Doctoral degree [EdD, PhD]
Other (Please specify)
b.
Field of study, if applicable:
c.
Years of relevant QI experience: |
d.
Years of experience working with the types of providers or organizations relevant to
this theme: | | | YEARS
Prepared by Mathematica Policy Research
|
| YEARS
C.235
25.
To what extent do you agree with the following statements about staffing for this theme?
Strongly
agree
Agree
Disagree
Strongly
disagree
a. QIO staff assigned to this theme have the
right substantive expertise and experience ...
1
□
2
□
3
□
4
□
b. An adequate number of QIO staff have
been available to perform work on this
theme ............................................................
1
□
2
□
3
□
4
□
c. The QIO has been able to retain key staff
working on this theme (that is, turnover has
not been a problem) ......................................
1
□
2
□
3
□
4
□
Prepared by Mathematica Policy Research
C.236
SECTION VI: IMPACT OF EXTERNAL FACTORS ON THIS THEME
Role of Provider, Professional Associations and/or State Agencies
26.
Which role(s) does the state agency most relevant to this theme play in assuring and
improving quality?
2
□
□
3
□
1
27.
Regulatory oversight
Actively engaged with others (such as the QIO and/or other
provider organizations) to foster quality improvements
Both
Please list up to two provider or professional associations that are stakeholders in this
theme and could potentially affect its success that have been most relevant to this theme.
1.
2.
28.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the first organization
listed at Question 27.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Prepared by Mathematica Policy Research
C.237
29.
The following are items that characterize the types of activities and/or relationships that
may exist between QIO’s and provider and/or professional associations.
Please answer each of the following questions with regard to the second
organization listed at Question 27. If only one organization was listed at
Question 27, go to Question 30.
Yes
No
a. The association employs at least one staff member with
major responsibility and time devoted to quality
improvement ........................................................................
1
□
0
□
b. The association and QIO talk periodically (e.g., quarterly)
to avoid overlap ....................................................................
1
□
0
□
c. QIO staff attend and speak at association-sponsored
meetings at least once per year ...........................................
1
□
0
□
d. The association sponsors a quality-focused entity, such as
a Quality Council or a Quality Institute .................................
1
□
0
□
e. The association and QIO work jointly on one or more
QI efforts substantial in scope (such as co-sponsoring
in-person meetings focused on QI) ......................................
1
□
0
□
The association tends to work with a different set of
providers than the QIO .........................................................
1
□
0
□
g. The association tends to work on QI projects that are
entirely different from the QIO ..............................................
1
□
0
□
h. The association primarily focuses on quality reporting
issues rather than quality improvement ...............................
1
□
0
□
f.
Role of Large Provider Organizations
The following questions apply to the state level.
30.
Do any large provider organizations that are relevant to this theme, such as health
systems, hospitals, or nursing home chains, exist in your state?
1
0
□
□
Yes
No
GO TO Q.34
Prepared by Mathematica Policy Research
C.238
31.
Please list up to two large health care delivery organizations in your state (such as an
integrated delivery system or dominant medical group) that are stakeholders in this theme
and have the greatest potential to affect its success.
1.
2.
32.
To what extent does the headquarters of the first organization listed in Question 31 drive
quality in owned or affiliated organizations in this state?
1
2
3
4
d
33.
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
To what extent does the headquarters of the second organization listed in Question 31
drive quality in owned or affiliated organizations in this state?
If only one provider was listed at Question 31, then go to Question 34.
1
2
3
4
d
□
□
□
□
□
A lot
A moderate amount
A little
Not at all
Don’t know
Other Important External Players
34.
Please list up to three other external organizations whose efforts are proving important to
achieving improvements on this theme. (Please spell out the full name of the
organization.)
1.
2.
3.
Prepared by Mathematica Policy Research
C.239
SECTION VI: QIO VIEWS ON QUALITY IMPROVEMENT
Motivation and Quality Improvement
35.
The following are statements which related to motivation and quality improvement. For
each statement, please indicate if you strongly agree, agree, disagree, or strongly
disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective.
Strongly
Strongly
agree
Agree
Disagree disagree
a. Senior leaders at providers care about their
quality performance related to this theme .....
1
□
2
□
3
□
4
□
b. Providers regularly review data (generated
internally or received from another
organization) on their performance related
to this theme..................................................
1
□
2
□
3
□
4
□
c. The “business case” for quality, when it is
clear, is a key motivator for improvement for
most providers...............................................
1
□
2
□
3
□
4
□
d. Providers perceive a strong business case
for quality improvement on the measures
important to this theme..................................
1
□
2
□
3
□
4
□
e. Ongoing pay-for-performance efforts are a
key motivation for quality improvements in
this state ........................................................
1
□
2
□
3
□
4
□
One or more large health plans “tiers”
providers in their network in ways that
consider their quality performance ................
1
□
2
□
3
□
4
□
g. Many providers lack motivation to improve ...
1
□
2
□
3
□
4
□
h. Motivational speakers (such as key IHI
personnel, a prominent physician, or
leading QI thinkers who are physicians) are
effective motivators for improvement ............
1
□
2
□
3
□
4
□
The number of physician champions is
adequate to help facilitate improvement on
key measures for this theme .........................
1
□
2
□
3
□
4
□
Public reporting is a key motivator for
improvement for most providers....................
1
□
2
□
3
□
4
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.240
If you responded “agree” or “strongly agree” to Question 35, item g, go to Question 36.
Otherwise, go to Question 37.
36.
What types of providers lack motivation to improve on this theme?
1
2
3
4
37.
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Does state-level public reporting relevant to this theme exist in this state?
1
0
□
□
Yes
No
Knowledge and Information and Quality Improvement
38.
The following are statements related to knowledge and information and quality
improvement. For each statement, please indicate if you strongly agree, agree, disagree,
or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Enough information exists for
providers suggesting how they should
improve, once they are motivated and
link (or are helped to link) to this
information .........................................
1
□
2
□
3
□
4
□
d
□
b. Once motivated, providers tend to
seek out and find good information on
how to improve ...................................
1
□
2
□
3
□
4
□
d
□
c. Providers have staff who are
educated or otherwise qualified to
support improvement efforts ..............
1
□
2
□
3
□
4
□
d
□
d. The limitations of provider information
systems remain a large barrier to
improvement ......................................
1
□
2
□
3
□
4
□
d
□
e. Workforce turnover is a large barrier
to improvement ..................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.241
If you responded “agree” or “strongly agree” to Question 38, item c, go to Question 39.
Otherwise, go to Question 40.
39.
What types of providers are not so well educated or qualified to support improvement
efforts?
1
2
3
4
□
□
□
□
Hospitals
Physician practices
Nursing homes
Home health agencies
Poor Performers and Quality Improvement
40.
The following are statements about poor performance and quality improvement. For each
statement, please indicate if you strongly agree, agree, disagree, or strongly disagree.
For each item, please attempt to answer thinking about your theme from a statewide
perspective (that is, thinking about all providers in the state, not just those you have been
working with).
Strongly
agree
Agree
Strongly
Disagree disagree
Don’t
know
a. Poorly performing providers often
have financial and management
problems ...........................................
1
□
2
□
3
□
4
□
d
□
b. Poor performance on a particular
measure is often associated with
physicians who disagree with the
relevant guideline ..............................
1
□
2
□
3
□
4
□
d
□
c. Poor performance on a particular
measure is often associated with
corporate chain managers who
disagree with or do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
d. Poor performance on a particular
measure is often associated with
physicians who do not believe in
establishing care routines based on
guidelines ..........................................
1
□
2
□
3
□
4
□
d
□
e. QIO-initiated help to the poorest
performers is/would be ineffective
because of their limited ability to use
the help given ....................................
1
□
2
□
3
□
4
□
d
□
Prepared by Mathematica Policy Research
C.242
SECTION VII: PARTICIPATING PROVIDERS’ SELECTION PROCESS
41.
Which of the following strategies did you use when you first began recruiting providers for
work under this theme?
MARK ALL THAT APPLY
1
□
Standardized announcement to eligible providers from the QIO
2
□
QIO initiated personalized contact with providers
3
□
Provider associations or other organizations outside the
QIO delivered announcement
4
□
Joint announcement made by QIO and provider associations
or other organizations outside the QIO
5
□
Other (Please specify)
Prepared by Mathematica Policy Research
C.243
42.
The following table contains characteristics of providers that you may have targeted when
you first began recruitments. For each one, first indicate if the QIO targeted it in
recruitment (Column A) and, if so, how successful the recruitment effort was (Column B).
□
If no specific characteristics were considered when targeting providers, check this box
and go to Question 43.
Column A
Provider Characteristics among
those Eligible to Participate
Yes,
targeted
during
recruitment
Column B
How successful were you in
recruiting providers with this
characteristic?
Very
Somewhat
Not
successful successful successful
a. No special characteristics sought –
just any provider who was eligible..
1
□
NA
b. Providers who had past
experience working with the QIO ...
1
□
1
□
2
□
0
□
c. Providers who did not have past
experience working with the QIO ...
1
□
1
□
2
□
0
□
d. Providers known to have
the organizational capacity to
improve ..........................................
1
□
1
□
2
□
0
□
e. Providers above average on the
targeted measures among eligible
providers ........................................
1
□
1
□
2
□
0
□
Providers below average among
eligible providers on the targeted
measures .......................................
1
□
1
□
2
□
0
□
g. Providers viewed as “early
adopters” ........................................
1
□
1
□
2
□
0
□
h. Providers viewed as leader
organizations by their peers ...........
1
□
1
□
2
□
0
□
Providers whose leadership had
higher-than-average commitment
to quality .........................................
1
□
1
□
2
□
0
□
Providers with other special
characteristics (Please specify) ........
1
□
1
□
2
□
0
□
f.
i.
j.
Prepared by Mathematica Policy Research
C.244
NA
NA
43.
Which of these statements best describes the response you received from your initial
recruitment efforts? Please consider both those who decided to participate and those who
asked questions but ultimately decided not to participate.
1
2
3
44.
□
□
□
More providers than needed expressed interest
About the right number of providers expressed interest
Too few providers expressed interest
Which of these statements best describes the level of effort needed to persuade enough
of the right types of providers to commit to participate?
1
2
3
□
□
□
It took a lot of effort to secure enough providers
It took a moderate amount of effort to secure enough providers
It took only a little bit of effort to secure enough provider
GO TO Q.46
45.
Please tell us what types of providers were the most difficult to persuade to participate?
46.
Were any providers so interested in participating that they lobbied the QIO to be sure they
would be included?
1
0
47.
□
□
Yes
No
GO TO Q.48
How many providers lobbied the QIO for participation?
1
2
3
□
□
□
1 to 2
3 to 4
5 or more
Prepared by Mathematica Policy Research
C.245
48.
What is the total number of participating providers at present?
________ NUMBER
49.
At the start of the effort, please estimate how many of the participating providers had:
Number
a. Worked with the QIO on 3 or more projects over the prior 5 years? ........
________
b. Worked with the QIO on at least 1 but not more than 3 projects in the
prior 5 years? ...........................................................................................
________
c. Not worked with the QIO before? .............................................................
________
50.
Of those who began participating with the QIO, please indicate the percent that
participated in the manner indicated below.
Manner of Participation
Percent of Participants
a. Participated actively throughout ..............................
|
|
|
|
b. Participated less over time ......................................
|
|
|
|
c. Intermittent participation ..........................................
|
|
|
|
d. Never participated very actively ..............................
|
|
|
|
TOTAL
Prepared by Mathematica Policy Research
C.246
100%
51.
Did the QIO exclude providers who expressed interest in participating, due to limited
resources?
1
□
Yes, many providers who expressed interest had to be excluded
GO TO Q.52
52.
2
□
Yes, a few providers who expressed interest had to be excluded
0
□
No, no providers who expressed interest had to be excluded
Thank you for
completing this
survey
Approximately how many interested providers were excluded due to limited resources
available through your 9th SOW contract?
____ NUMBER OF PROVIDERS EXCLUDED
Thank you for completing the QIO Theme Leader Survey
Please return your completed survey to the following:
Via mail:
Via fax:
Mathematica Policy Research
c/o Sarah Croake
Suite 550, 600 Maryland Ave., SW
Washington, DC 20024
Attn: Sarah Croake
202-863-1763
Via email attachment:
[email protected]
Prepared by Mathematica Policy Research
C.247
APPENDIX D
PROJECT DESCRIPTION FOR ADVANCE MAILINGS
PROGRAM EVALUATION OF THE EIGHTH AND NINTH SCOPE OF WORK
QUALITY IMPROVEMENT PROGRAM
Sponsoring Agency: U.S. Department of Health and Human Services, Centers for Medicare &
Medicaid Services
Prime Contractor: Mathematica Policy Research
Contract #: HHSM-500-2005-00025I (0010)
Other Organizations Participating: Abt Associates Inc., Social & Scientific Systems, Inc.
Project Synopsis:
In August of 2008, Mathematica was selected by CMS to evaluate the 8th and 9th Scopes of
Work of the Quality Improvement Program. As a part of this large evaluation effort,
Mathematica is working to estimate the impact of the QIO Program on beneficiaries’ quality of
care, to identify the most effective methods, tools, and interventions for quality improvement,
and to recommend program improvements.
The 9th SOW evaluation began in May 2009, and includes the collection of new data
through surveys to providers and QIOs, and discussions with QIOs, partner staff and
beneficiaries. More specifically, data collection includes:
• A national web-based survey of QIO Directors and Theme Leaders (summer/fall
2010)
• Telephone discussions with QIO Partners for the Care Transitions and Prevention –
Chronic Kidney Disease themes (late 2010/early 2011)
• A national survey of hospitals and nursing homes (late 2010/early 2011)
• Week-long site visits to 12 states’ QIOs, providers that worked with them, and
community health leaders representing hospital, nursing home, and physician
stakeholders (late 2010/early 2011)
• Four focus groups of beneficiaries who participated in the Prevention – Disparities
theme by attending diabetes self-management education classes sponsored by the
QIO (late 2010/early 2011)
The evaluation will include both descriptive and multivariate analyses, using both the new
data collected by Mathematica, and secondary data reported by the QIOs. The evaluators have
access to data already reported by QIOs to CMS, and will maximize use of this data to ensure
efficiency. The progress and findings of the evaluation will be documented in quarterly reports
beginning in July 2009, a mid-course report in December 2010 and will culminate in a final
report in October 2011.
D.3
Project Director and Key Staff:
• Myles Maxfield, Ph.D, Vice President and Director of Health Research, MPR’s
Washington DC Office, Project Director - [email protected] or
(202) 484-4682
• Arnold Chen, MD, Senior Clinician Researcher, Principal Investigator for
Quantitative Analysis
• Suzanne Felt-Lisk, MPA, Senior Health Researcher, Principal Investigator for
Qualitative Analysis
D.4
APPENDIX E
PROVIDER SURVEYS AND LETTERS
CMS LETTERHEAD
ADVANCE LETTER EVALUATION OF THE 9TH QIO SOW
SURVEY OF HOSPITAL QI DIRECTOR
[DATE]
[NAME AND ADDRESS]
Dear [Dr./Mr./Ms.] [FILL LAST NAME]:
The Centers for Medicare & Medicaid Services (CMS) is sponsoring a study about the
Quality Improvement Organization (QIO) Program, which is a key component in CMS’ efforts
to improve the quality and efficiency of care for Medicare beneficiaries. The purpose of the
study is to evaluate the program’s effectiveness in helping hospitals to improve the quality of
care for Medicare beneficiaries and to find ways to improve the program.
Mathematica Policy Research, Inc. (MPR), an independent research organization, is
conducting the study on behalf of CMS. As part of this study, MPR will survey approximately
1,250 hospitals across the United States about their experiences with their local QIO and about
other quality improvement efforts they may have undertaken.
Your participation in the survey is voluntary, but your participation is invaluable for the
success of this important study. Your responses will help Medicare to improve the design and
effectiveness of the QIO program and to ultimately improve care for Medicare beneficiaries. A
telephone interviewer will be calling you to administer the survey; we anticipate the survey will
take about 30 minutes to complete. Your answers will remain completely confidential at
Mathematica. Neither your name nor your hospital’s name will ever be included in any reports
prepared as part of this study.
If you have any questions, or if you would prefer to complete the survey by mail, please call
MPR toll-free at 1-XXX-XXX-XXXX and ask for Martha Kovac. If you would like to learn
more about the QIO Program, please visit the CMS website at http://www.cms.
hhs.gov/QualityImprovementOrgs/. If you would like more information about the study, please
see the attached project description.
We look forward to including your valuable input in this study.
Sincerely,
CMS Privacy Officer
Enclosure
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless
it displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The
time required to complete this information collection is estimated to average 0.5 hours or 30 minutes per response, including
the time to review instructions, search existing data resources, and gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving
this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05,
Baltimore, Maryland 21244-1850.
E.3
CMS LETTERHEAD
ADVANCE LETTER EVALUATION OF THE 9TH QIO SOW
SURVEY OF NURSING HOME QI DIRECTOR
[DATE]
[NAME AND ADDRESS]
Dear [Dr./Mr./Ms.] [FILL LAST NAME]:
The Centers for Medicare & Medicaid Services (CMS) is sponsoring a study about the
Quality Improvement Organization (QIO) Program, which is a key component in CMS’ efforts
to improve the quality and efficiency of care for Medicare beneficiaries. The purpose of the
study is to evaluate the program’s effectiveness in helping nursing homes to improve the quality
of care for Medicare beneficiaries and to find ways to improve the program.
Mathematica Policy Research, Inc. (MPR), an independent research organization, is
conducting the study on behalf of CMS. As part of this study, MPR will survey approximately
1,250 nursing homes across the United States about their experiences with their local QIO and
about other quality improvement efforts they may have undertaken.
Your participation in the survey is voluntary, but your participation is invaluable for the
success of this important study. Your responses will help Medicare to improve the design and
effectiveness of the QIO program and to ultimately improve care for Medicare beneficiaries. A
telephone interviewer will be calling you to administer the survey; we anticipate the survey will
take about 20 minutes to complete. Your answers will remain completely confidential at
Mathematica. Neither your name nor your nursing home’s name will ever be included in any
reports prepared as part of this study.
If you have any questions, or if you would prefer to complete the survey by mail, please call
MPR toll-free at 1-XXX-XXX-XXXX and ask for Martha Kovac. If you would like to learn
more about the QIO Program, please visit the CMS website at http://www.cms.
hhs.gov/QualityImprovementOrgs/. If you would like more information about the study, please
see the attached project description.
We look forward to including your valuable input in this study.
Sincerely,
CMS Privacy Officer
Enclosure
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless
it displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The
time required to complete this information collection is estimated to average 0.33 hours or 20 minutes per response, including
the time to review instructions, search existing data resources, and gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving
this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05,
Baltimore, Maryland 21244-1850.
E.5
Mathematica Reference No.: 06514.260
Ninth Scope of Work
QIO Program
Evaluation: Hospital
Survey
Final Draft Questionnaire
January 5, 2010
A. INTRODUCTION
INITIAL CONTACT WITH HOSPITAL: Hello, may I please speak with [NAME OF CEO/The
CEO/or Assistant to the CEO of your hospital]?
IF YOU DON’T HAVE (HIS/HER) NAME,
ASK FOR IT AND RECORD HERE:
IF NEEDED: My name is ________ and I’m calling on behalf of the Centers for Medicare &
Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF CEO/the CEO or
CEO’s assistant] a letter from CMS requesting the hospital’s participation in a survey for an
evaluation of the Ninth Scope of Work of the Quality Improvement Organization Program.
A1.
WHEN SPEAKING WITH CEO/Assistant to the CEO: Hello, [Dr./Mr./Ms.]
[LAST NAME], my name is __________, and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS. A short time ago, you should have received a
letter from CMS requesting your hospital’s participation in a survey for an evaluation of
the Ninth Scope of Work of the Quality Improvement Organization Program.
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made. FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning
how its efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
We would like to conduct a brief interview with the QI Director (or whoever is directly
involved with quality improvement at the hospital). Would you please give me the
name and contact information for this person?
YES .......................................................................... 1
GO TO A2
NO ............................................................................ 0
SET CALLBACK
DON’T KNOW .......................................................... d
A2.
RECORD NAME AND CONTACT INFORMATION FOR QI DIRECTOR.
ENTER NAME OF QI DIRECTOR
ENTER TELEPHONE NUMBER: | | | |-|
Area Code
Prepared by Mathematica Policy Research
E.8
|
|
|-|
|
|
|
|
A3.
FOLLOW-UP INFORMATION:
PURPOSE:
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how well the QIOs are working to help hospitals improve
quality and patient safety and what changes if any, need to be made.
FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning how its
efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
OR SAY: It is critical that our study understand quality improvement processes and
thinking in hospitals that have not been working with QIOs as well as those that have,
in order to understand the added value of the QIO program.
CONFIDENTIALITY: The information you provide will be kept strictly confidential, and
used only for the purposes of contacting hospital staff to complete the questionnaire.
All information collected during the interview will be kept strictly confidential. Only
statistical totals will be reported. Neither your name, your staff’s names, nor the
hospital will be identified or publicly reported.
SELECTION: Your hospital was randomly selected from U.S. hospitals so as to
support evaluation of CMS’ Quality Improvement Program. Some were selected to
represent participating hospitals that worked with QIOs and some were selected to
represent hospitals that did not work with QIOs.
A4.
INTRODUCTION WITH QI DIRECTOR:
Hello, may I please speak with [NAME OF QI DIRECTOR]?
IF NEEDED: My name is ___________ and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF CEO] a
letter from CMS requesting the hospital’s participation in a survey for an evaluation of
the Ninth Scope of Work of the Quality Improvement Organization Program.
Prepared by Mathematica Policy Research
E.9
A5.
WHEN SPEAKING WITH QI DIRECTOR:
Hello, [Dr. /Mr. /Ms.] [LAST NAME], my name is ________, and I’m calling on behalf of
the Centers for Medicare & Medicaid Services or CMS. We recently spoke with
[NAME OF CEO], who gave us your name as someone directly involved with quality
improvement at the hospital. We are conducting a survey for an evaluation of the Ninth
Scope of Work of the Quality Improvement Organization Program.
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made. FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning
how its efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
from outside organizations even if they are not currently working with CMS programs
on quality issues.
The survey interview takes roughly 30 minutes, depending upon your answers. I can
conduct it now, or at any time that's convenient for you.
START INTERVIEW NOW ....................................... 1
GO TO B1
NOT NOW, SET UP APPT/CALLBACK ................... 2
SET APPT.
NEEDS MORE INFORMATION ............................... 3
GO TO FU SCREEN
REFUSED ................................................................ r
REFUSAL
Prepared by Mathematica Policy Research
E.10
FOLLOW-UP INFORMATION SCREEN:
PURPOSE:
FOR PARTICIPATING HOSPITALS: CMS is interested in learning about the
experience of hospitals involved in the Ninth Scope of Work of the Quality
Improvement Organization Program. Your hospital’s input is crucial to assure that
CMS learns all it can about how the QIOs are working and what changes if any, need
to be made.
FOR NON-PARTICIPATING HOSPITALS: CMS is interested in learning how its
efforts to support quality improvement in hospitals can be made more effective.
This requires understanding hospitals’ quality activities and interest in future assistance
even if they are not currently working with CMS programs on quality issues.
OR SAY: It is critical that our study understand quality improvement processes and
thinking in hospitals that have not been working with QIOs as well as those that have,
in order to understand the added value of the QIO program.
CONFIDENTIALITY: The information you provide will be kept strictly confidential.
Only statistical totals will be reported. Neither your name nor the hospital name will be
publicly reported.
SELECTION: Your hospital was randomly selected from U.S. hospitals so as to
support evaluation of CMS’ Quality Improvement Program. Some were selected to
represent participating hospitals that worked with QIOs and some were selected to
represent hospitals that did not work with QIOs.
REFUSAL SCREEN: Thank you for your time. Have a nice day.
Prepared by Mathematica Policy Research
E.11
B. QIO INTERACTIONS
The first few questions are about staff interactions with [NAME OF QIO].
B1.
Is this hospital participating with [NAME OF QIO] on a quality improvement initiative
related to any of the following topics… [READ DOWN LIST]
YES
NO
DON’T
KNOW
REFUSED
a. Surgical Care Infection Prevention?..................
1
0
d
r
b. Heart Failure? ...................................................
1
0
d
r
c. Pressure Ulcers Reduction? .............................
1
0
d
r
d. MRSA Infection Prevention and Transmission
Reduction? ........................................................
1
0
d
r
e. Care Transitions (Reducing Readmissions)?....
1
0
d
r
f.
1
0
d
r
Any Other Topic? (SPECIFY) ..........................
B1aa.
[FOR EACH CATEGORY THAT RECEIVED A 'NO' RESPONSE, ASK B1aa AFTER
YOU HAVE READ THROUGH THE ENTIRE LIST IN B1.]
Were you invited to participate with [NAME OF QIO] on a quality improvement initiative
related to [TOPIC]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B2
REFUSED ................................................................ r
B1ab.
Why did you choose not to participate with [NAME OF QIO] on a quality improvement
initiative? RECORD VERBATIM
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.12
B2.
The next few questions ask about how often hospital staff may have met with [NAME
OF QIO], either in-person or by telephone. Since August 2008, how many times have
hospital personnel met with [NAME OF QIO] in-person at this hospital?
PROBE: Your best estimate is fine.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
GO TO B3
REFUSED ................................................................ r
B2a.
[IF B2 = 2 OR MORE, DISPLAY “How often did…”; IF B2 = 1, DISPLAY “Did…”]
(How often did/Did) the following people attend the meeting with [NAME OF QIO]
[READ ITEM]?
[IF B2 = 2 OR MORE, DISPLAY: Would you say always, usually, sometimes, or
never?] [IF B2 = 1, DISPLAY ONLY CATEGORIES “ALWAYS” AND “NEVER”]
ALWAYS
USUALLY
SOMETIMES
NEVER
DON’T
KNOW
REFUSED
a. Physician leaders for the
clinical areas being
discussed? .............................
1
2
3
4
d
r
b. One or more members of
senior hospital (“C-Suite”)
leadership?.............................
1
2
3
4
d
r
c. The quality improvement
director? .................................
1
2
3
4
d
r
d. Nursing leadership? ...............
1
2
3
4
d
r
e. Any other leadership staff?
(SPECIFY) .............................
1
2
3
4
d
r
Prepared by Mathematica Policy Research
E.13
B3.
Since August 2008, how many other in-person meetings have hospital personnel
attended where [NAME OF QIO] was an active participant? Please include in-person
meetings held inside and outside the hospital.
PROBE: Your best estimate is fine.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B4.
Since August 2008, approximately how many times have hospital personnel met
by phone with [NAME OF QIO]? Please do not include large conference calls.
PROBE: Your best estimate is fine.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B5.
Since August 2008, how many other telephone conference calls or web-ex meetings
have hospital personnel attended that [NAME OF QIO] convened?
PROBE: Your best estimate is fine.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF B2 OR B3 =/> 1, GO TO B6. IF B2 AND B3 = 0, GO TO B12]
Prepared by Mathematica Policy Research
E.14
B6.
I’m going to read a list of reasons why you might have met with [NAME OF QIO] since
August 2008. After each one, please tell me if this was a reason for (any of) the
in-person or phone meeting(s) with [NAME OF QIO]. [READ LIST]
[ROTATE ORDER OF LIST, BUT ALWAYS END WITH CATEGORY “n” LAST]
YES
NO
DON’T
KNOW
REFUSED
a. Complaint or case review follow-up .........................
1
0
d
r
b. Understanding [NAME OF QIO]’s plans for
activities and opportunities to participate .................
1
0
d
r
c. Routine meetings as part of participating with
[NAME OF QIO] on a quality improvement effort .....
1
0
d
r
d. To discuss this hospital’s performance data ............
1
0
d
r
e. Applying [NAME OF QIO] staff’s expertise to
improve this hospital’s routine performance
measurement ...........................................................
1
0
d
r
f.
Hearing about best practices of other hospitals .......
1
0
d
r
g. To learn about a new tool or recommended
process for quality improvement ..............................
1
0
d
r
h. Other staff development or training ..........................
1
0
d
r
Presentation(s) to help create buy-in to
quality improvement beyond the quality
improvement staff.....................................................
1
0
d
r
To discuss issues, methods, and/or timeframes for
quality reporting to CMS...........................................
1
0
d
r
k. To attend a broad-based regional or statewide
meeting on quality improvement where
[NAME OF QIO] was an active participant ...............
1
0
d
r
This hospital asked for and received some
assistance or information from the QIO ...................
1
0
d
r
m. To receive assistance regarding reporting of quality
data for the Reporting of Hospital Quality Data for
Annual Payment Update, or “RHQDAPU”................
1
0
d
r
n. Any other reason? (SPECIFY) ................................
1
0
d
r
i.
j.
l.
Prepared by Mathematica Policy Research
E.15
B7.
Which of the following describe the aim(s) of the assistance you received?
[FOR THOSE WHO RESPOND ‘YES’ TO ANY ITEM IN B6]
IMPROVE OR ENSURE COMPLETENESS
OF THE REPORTED DATA ..................................... 1
ADVISE ON ISSUES RELATED TO
TRANSMISSION OF THE DATA ............................. 2
OTHER (SPECIFY) .................................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B7a.
How effective was this assistance in achieving its aim(s)?
VERY EFFECTIVE ................................................... 1
SOMEWHAT EFFECTIVE ....................................... 2
NOT EFFECTIVE ..................................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.16
B8.
How valuable to the hospital was meeting (about) [FILL EACH CATEGORY THAT
RECEIVED A YES RESPONSE AT B6]?
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES TO B6]
[READ ITEM] Would you say it was very valuable, somewhat valuable, or not
valuable?
[IF ‘VERY VALUABLE’, ‘SOMEWHAT VALUABLE’, OR ‘NOT VALUABLE’ IS
CHOSEN, ASK B9 OR B10 BEFORE MOVING ON TO NEXT ITEM]
VERY
VALUABLE
SOMEWHAT
VALUABLE
NOT
VALUABLE
a. Complaint or case review follow-up ...
1
2
3
d
r
b. Understanding [NAME OF QIO]’s
plans for activities and opportunities
to participate.......................................
1
2
3
d
r
c. Routine meetings as part of
participating with [NAME OF QIO] on
a quality improvement effort ...............
1
2
3
d
r
d. To discuss this hospital’s
performance data ...............................
1
2
3
d
r
e. Applying [NAME OF QIO] staff’s
expertise to improve this hospital’s
routine performance measurement ....
1
2
3
d
r
Hearing about best practices of other
hospitals .............................................
1
2
3
d
r
g. To learn about a new tool or
recommended process for quality
improvement ......................................
1
2
3
d
r
h. Other staff development or training ....
1
2
3
d
r
Presentation(s) to help create buy-in
to quality improvement beyond the
quality improvement staff ...................
1
2
3
d
r
To discuss issues, methods, and/or
timeframes for quality reporting to
CMS ...................................................
1
2
3
d
r
k. To attend a broad-based regional or
statewide meeting on quality
improvement where [NAME OF QIO]
was an active participant ....................
1
2
3
d
r
This hospital asked for and received
some assistance or information from
the QIO...............................................
1
2
3
d
r
m. Other reason (SPECIFY) ...................
1
2
3
d
r
f.
i.
j.
l.
Prepared by Mathematica Policy Research
E.17
DON’T
KNOW
REFUSED
B9.
[IF ‘NOT VALUABLE’ SELECTED AT B8, ASK B9 IMMEDIATELY AFTER B8:]
Why were these meetings not valuable?
CODE ALL THAT APPLY
THE MEETING MERELY FULFILLED AN
OBLIGATION (SUCH AS COOPERATION
WITH CASE REVIEWS) ........................................... 1
THIS HOSPITAL IS SO ADVANCED, THERE
IS NOTHING WE CAN LEARN FROM
[NAME OF QIO] ....................................................... 2
[NAME OF QIO] STAFF DID NOT HAVE
ENOUGH EXPERTISE OF THE RIGHT TYPE ........ 3
[NAME OF QIO] STAFF WAS NOT
WELL-PREPARED ................................................... 4
THE RIGHT PEOPLE WERE NOT
AT THE MEETING ................................................... 5
THE INFORMATION PROVIDED WAS
NOT APPLICABLE TO THIS HOSPITAL’S
SITUATION .............................................................. 6
THE MEETING WAS REDUNDANT WITH
INFORMATION WE ALREADY HAD ....................... 7
PROGRESS AT THIS HOSPITAL ON THE
MEETING TOPIC IS NOT FEASIBLE AT
THIS TIME ................................................................ 8
OTHER (SPECIFY) .................................................. 9
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.18
B10.
[IF ‘VERY VALUABLE’ OR ‘SOMEWHAT VALUABLE’ SELECTED AT B8, ASK B10
IMMEDIATELY AFTER B8:]
In what ways were these meetings valuable?
CODE ALL THAT APPLY
PROVIDED NEW, USEFUL INFORMATION ON
HOW TO IMPROVE PERFORMANCE .................... 1
PROVIDED NEW, USEFUL INFORMATION
ON ANOTHER TOPIC (SUCH AS QUALITY
REPORTING, USE OF EHR) ................................... 2
INCREASED MOTIVATION TO IMPROVE
FROM ONE OR MORE EXECUTIVES .................... 3
INCREASED MOTIVATION TO IMPROVE
FROM KEY PHYSICIANS ........................................ 4
INCREASED MOTIVATION TO IMPROVE
FROM OTHER KEY HOSPITAL STAFF .................. 5
OTHER (SPECIFY) .................................................. 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B11.
Did any of the meetings lead to changes at the hospital that ultimately improved patient
care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B12
REFUSED ................................................................ r
B11a.
Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.19
GO TO B12
B11b.
Please tell me which measures were improved.
[READ LIST IF NEEDED]
SCIP (SURGICAL CARE INFECTION PREVENTION)
Surgery patients on a beta blocker prior
to arrival who received a beta blocker
during the perioperative period ................................. 1
Prophylactic antibiotic received on time—within
one hour prior to surgical incision ............................. 2
Prophylactic antibiotic selection for
surgical patients ....................................................... 3
Prophylactic antibiotics discontinued within
24 hours after surgery end time................................ 4
Cardiac surgery patients with controlled 6 am
postoperative serum glucose ................................... 5
Surgery patients with appropriate hair removal ........ 6
Surgery patients with recommended VTE
prophylaxis ordered .................................................. 7
Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ............................. 8
HEART FAILURE
Heart failure patients with left ventricular
systolic dysfunction without ACEI and
ARB contraindications who are prescribed
ACEI/ARB at discharge ............................................ 9
PRESSURE ULCERS
Incidence of pressure ulcers .................................... 10
MRSA
MRSA-1 Infection rate .............................................. 11
MRSA-2 Transmission rate ...................................... 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.20
B12.
(In addition to meetings), did you receive other educational materials, tools, or quality
improvement news from [NAME OF QIO]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B13
REFUSED ................................................................ r
B12a.
Did you receive these items from . . .
CODE ALL THAT APPLY
A newsletter, ............................................................. 1
An email or listserv, .................................................. 2
At in-person meeting, ............................................... 3
At a teleconference or web-ex, ................................. 4
Or some other way? (SPECIFY) ............................. 5
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12b.
Overall, how valuable were these educational materials and/or tools? Would you
say . . .
Very valuable, ........................................................... 1
Somewhat valuable, ................................................. 2
Not very valuable, or ................................................ 3
Not at all valuable? ................................................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12c.
Did any of these educational materials or tools from [NAME OF QIO] thus far lead to
changes at the hospital that ultimately improved patient care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.21
GO TO B13
B12d.
Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B13
REFUSED ................................................................ r
B12e.
Please tell me which measures were improved. [READ LIST IF NEEDED]
SCIP (SURGICAL CARE INFECTION PREVENTION)
Surgery patients on a beta blocker prior
to arrival who received a beta blocker
during the perioperative period ................................. 1
Prophylactic antibiotic received on time—within
one hour prior to surgical incision ............................. 2
Prophylactic antibiotic selection for
surgical patients ....................................................... 3
Prophylactic antibiotics discontinued within
24 hours after surgery end time................................ 4
Cardiac surgery patients with controlled 6 am
postoperative serum glucose ................................... 5
Surgery patients with appropriate hair removal ........ 6
Surgery patients with recommended VTE
prophylaxis ordered .................................................. 7
Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ............................. 8
HEART FAILURE
Heart failure patients with left ventricular
systolic dysfunction without ACEI and
ARB contraindications who are prescribed
ACEI/ARB at discharge ............................................ 9
PRESSURE ULCERS
Incidence of pressure ulcers .................................... 10
MRSA
MRSA-1 Infection rate .............................................. 11
MRSA-2 Transmission rate ...................................... 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.22
B13.
Do you routinely receive data feedback from [NAME OF QIO] on this hospital’s quality
performance?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B14
REFUSED ................................................................ r
B13a. How widely within the hospital do you typically share the feedback [NAME OF QIO]
provides, or highlights that you derive from it? Is it . . .
CODE ONE ONLY
Shared with a wide array of relevant
physicians and staff, ................................................. 1
Shared with a few key individuals, or ....................... 2
Rarely or never shared? ........................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B13b.
Has the feedback from [NAME OF QIO] on this hospital’s quality performance been
important to the hospital’s quality improvement efforts?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B14.
Did this hospital complete a Hospital Leadership Quality Assessment Tool© survey?
PROBE: It is also known as the HLQAT (“HELLCAT”) survey.
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.23
GO TO C1
B15.
Did the results help identify things that could be strengthened to better support quality
improvements at the hospital?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B16.
Have any changes been made as a result of the survey or related follow-up that
strengthened quality at the hospital?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO C1
REFUSED ................................................................ r
B16a.
Would you consider these changes to be important or not very important?
IMPORTANT ............................................................ 1
NOT VERY IMPORTANT ......................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.24
C. OTHER QI INITIATIVES
EXTERNAL INITIATIVES
The next questions are about quality improvement initiatives that involve external organizations.
C1.
Is this hospital part of or affiliated with a larger organization with central quality
expertise and an array of quality initiatives that extend to owned or affiliated
organizations?
YES, LARGER ORGANIZATION ............................. 1
YES, AFFILIATED ORGANIZATION........................ 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C1a.
To what extent are your hospital’s quality improvement actions influenced by this
(larger/affiliated organization)? Would you say to a large extent, a moderate extent,
or a small or no extent?
LARGE EXTENT ...................................................... 1
MODERATE EXTENT .............................................. 2
SMALL OR NO EXTENT .......................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C2.
Setting aside any interactions with [NAME OF QIO] (and the (larger/affiliated)
organization just discussed), is the hospital actively involved in any other quality
improvement efforts involving outside organizations?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.25
GO TO C3
C2a.
Please describe the nature of each of these initiatives and the sponsoring organization.
RECORD VERBATIM
PROBE FOR BOTH INITIATIVE AND SPONSOR.
PROBE: Please do not include efforts where the hospital’s participation is not active
or that focus on quality reporting but not improvement.
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
INTERNAL INITIATIVES
Now I’d like to ask about internal quality improvement initiatives within the hospital.
C3.
How many full-time equivalent staff are currently devoted to quality improvement in the
hospital?
PROBE: For example, if the hospital has four staff who each devote a quarter time to
quality improvement, then the number of full-time equivalent staff at the
hospital is one.
PROBE: Your best estimate is fine.
|
|
| NUMBER OF FTE’s
LESS THAN ONE (OR A FRACTION) ..................... f
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.26
C4.
Since August 2008, please indicate if the hospital has had internal quality improvement
efforts that have improved hospital performance on any of the following measures . . .
[READ DOWN LIST]
YES
NO
DON’T
KNOW
REFUSED
1
0
d
r
Surgery patients on a beta blocker prior to
arrival who received a beta blocker during
the perioperative period ................................
1
0
d
r
b. Prophylactic antibiotic received on time—
within one hour prior to surgical incision .......
1
0
d
r
c. Prophylactic antibiotic selection for surgical
patients..........................................................
1
0
d
r
d. Prophylactic antibiotics discontinued within
24 hours after surgery end time ....................
1
0
d
r
e. Cardiac surgery patients with controlled
6 AM postoperative serum glucose ...............
1
0
d
r
Surgery patients with appropriate hair
removal .........................................................
1
0
d
r
g. VTE prophylaxis measures? .........................
1
0
d
r
Surgery patients with recommended VTE
prophylaxis ordered.......................................
1
0
d
r
h. Surgery patients who received appropriate
VTE prophylaxis within 24 hours prior to
surgery to 24 hours after surgery ..................
1
0
d
r
Heart failure patients prescribed ACEI/ARB
at discharge?.................................................
1
0
d
r
(heart failure patients with left ventricular
systolic dysfunction without ACEI and ARB
contraindications who are prescribed
ACEI/ARB at discharge)................................
1
0
d
r
Incidence of pressure ulcers .........................
1
0
d
r
k. MRSA-1 Infection rate ...................................
1
0
d
r
l.
1
0
d
r
a. SCIP measures? ...........................................
NOT
APPLICABLE
IF YES, ASK: Which ones? [IF NO, GO TO g]
f.
IF YES, ASK: Which ones? [IF NO, GO TO i]
i.
j.
MRSA-2 Transmission rate ...........................
Prepared by Mathematica Policy Research
E.27
n
C5.
Has the hospital undertaken an analysis on any of the measures to identify the reasons
why the relevant guideline sometimes is not followed, or why the undesirable outcome
sometimes occurs?
YES .......................................................................... 1
NO ............................................................................ 0
NOT APPLICABLE – PERFORMANCE
IS 100% RELIABLE .................................................. n
GO TO C7
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C5a.
For which measures has the hospital undertaken an analysis? [READ LIST IF
NECESSARY]
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM C4]
PROBE: IF RESPONDENT SAYS “SCIP MEASURES,” ASK: “Which ones?”
ANALYSIS
A. SURGERY PATIENTS ON A BETA BLOCKER PRIOR TO ARRIVAL
WHO RECEIVED A BETA BLOCKER DURING THE
PERIOPERATIVE PERIOD ....................................................................
1
B. PROPHYLACTIC ANTIBIOTIC RECEIVED ON TIME—WITHIN ONE
HOUR PRIOR TO SURGICAL INCISION ...............................................
2
C. PROPHYLACTIC ANTIBIOTIC SELECTION FOR SURGICAL
PATIENTS...............................................................................................
3
D. PROPHYLACTIC ANTIBIOTICS DISCONTINUED WITHIN 24 HOURS
AFTER SURGERY END TIME ...............................................................
4
E. CARDIAC SURGERY PATIENTS WITH CONTROLLED 6 AM
POSTOPERATIVE SERUM GLUCOSE .................................................
5
F. SURGERY PATIENTS WITH APPROPRIATE HAIR REMOVAL ...........
6
G. SURGERY PATIENTS WITH RECOMMENDED VTE PROPHYLAXIS
ORDERED ..............................................................................................
7
H. SURGERY PATIENTS WHO RECEIVED APPROPRIATE VTE
PROPHYLAXIS WITHIN 24 HOURS PRIOR TO SURGERY TO
24 HOURS AFTER SURGERY...............................................................
8
I.
HEART FAILURE PATIENTS WITH LEFT VENTRICULAR SYSTOLIC
DYSFUNCTION WITHOUT ACEI AND ARB CONTRAINDICATIONS
WHO ARE PRESCRIBED ACEI/ARB AT DISCHARGE .........................
9
J. INCIDENCE OF PRESSURE ULCERS ..................................................
10
K. MRSA-1 INFECTION RATE....................................................................
11
L. MRSA-2 TRANSMISSION RATE............................................................
12
Prepared by Mathematica Policy Research
E.28
C5b.
[FOR EACH ‘YES’ RESPONSE AT C4, ASK:]
What type(s) of effort(s) has the hospital undertaken to improve on [ITEM FROM C4]
[If respondents report the same efforts for multiple measures, use code 13]?
CODE ALL THAT APPLY
AN IMPROVEMENT TEAM USED A
PLAN-DO-STUDY-ACT (OR SIMILAR)
TECHNIQUE ............................................................ 1
USED SIX SIGMA .................................................... 2
USED LEAN PROCESS .......................................... 3
ADJUSTED ELECTRONIC HEALTH RECORD
OR OTHER ELECTRONIC SYSTEM TO
INCLUDE AND DISPLAY KEY INFORMATION
IN A SEARCHABLE FIELD ...................................... 4
IMPLEMENTED A CHECKLIST ............................... 5
ESTABLISHED A PROTOCOL THAT IS
USED ROUTINELY .................................................. 6
CHANGED WHO IS RESPONSIBLE FOR
KEY TASKS RELATED TO THE MEASURE ........... 7
CHANGED ANOTHER ASPECT OF THE
PROCESS (BUT NOT THROUGH A
FORMAL PROTOCOL) ............................................ 8
DEPARTMENT HEADS OR OTHER KEY
PHYSICIANS MET WITH OTHER
PHYSICIANS TO GAIN THEIR BUY-IN TO
IMPROVEMENT ....................................................... 9
PRODUCED AND SHARED
PHYSICIAN-SPECIFIC DATA
ON THE MEASURE ................................................. 10
PERFORMED CONCURRENT CHART
REVIEWS TO IDENTIFY FAILING
CASES FOR FOLLOW-UP ...................................... 11
OTHER (SPECIFY) .................................................. 12
SAME EFFORTS AS FOR PREVIOUS
MEASURE ................................................................ 13
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.29
C6.
To what extent have you had both the leadership support and resources you needed to
accomplish the improvements you sought on these measures?
On improvements for [READ ITEM], would you say you had all the leadership support
and resources you needed, somewhat less than you needed, or a lot less than you
needed?
[PROGRAM WILL DISPLAY FOR EACH ‘YES’ RESPONSE TO C4]
SOMEWHAT
LESS THAN
YOU NEEDED
A LOT
LESS
THAN
YOU
NEEDED
DON’T
KNOW
REFUSED
1
2
3
d
r
1
2
3
d
r
Prophylactic antibiotic selection for
surgical patients .................................
1
2
3
d
r
d. Prophylactic antibiotics discontinued
within 24 hours after surgery end
time ....................................................
1
2
3
d
r
e. Cardiac surgery patients with
controlled 6 AM postoperative serum
glucose ..............................................
1
2
3
d
r
1
2
3
d
r
g. Surgery patients with recommended
VTE prophylaxis ordered ...................
1
2
3
d
r
h. Surgery patients who received
appropriate VTE prophylaxis within
24 hours prior to surgery to 24 hours
after surgery.......................................
1
2
3
d
r
Heart failure patients with left
ventricular systolic dysfunction
without ACEI and ARB
contraindications who are prescribed
ACEI/ARB at discharge .....................
1
2
3
d
r
j.
Incidence of pressure ulcers..............
1
2
3
d
r
k.
MRSA-1 Infection rate .......................
1
2
3
d
r
l.
MRSA-2 Transmission rate ...............
1
2
3
d
r
ALL THE
LEADERSHIP
SUPPORT AND
RESOURCES
YOU NEEDED
a. Surgery patients on a beta blocker
prior to arrival who received a beta
blocker during the perioperative
period .................................................
b. Prophylactic antibiotic received on
time—within one hour prior to
surgical incision .................................
SCIP MEASURES
c.
f.
Surgery patients with appropriate
hair removal .......................................
VTE PROPHYLAXIS MEASURES
HEART FAILURE MEASURE
i.
Prepared by Mathematica Policy Research
E.30
C7.
Does the hospital provide physicians with physician-level data for any of the SCIP,
heart failure or MRSA measures?
IF YES, ASK: Which ones? [PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM
C4] [READ LIST IF NECESSARY]
IF NO, GO TO C8
YES
NO
DON’T
KNOW
REFUSED
a. Surgery patients on a beta blocker prior to arrival
who received a beta blocker during the perioperative
period .............................................................................
1
0
d
r
b. Prophylactic antibiotic received on time—within one
hour prior to surgical incision .........................................
1
0
d
r
c.
Prophylactic antibiotic selection for surgical patients ....
1
0
d
r
d. Prophylactic antibiotics discontinued within 24 hours
after surgery end time ....................................................
1
0
d
r
e. Cardiac surgery patients with controlled 6 AM
postoperative serum glucose.........................................
1
0
d
r
f.
1
0
d
r
g. Surgery patients with recommended VTE prophylaxis
ordered ..........................................................................
1
0
d
r
h. Surgery patients who received appropriate VTE
prophylaxis within 24 hours prior to surgery to
24 hours after surgery ...................................................
1
0
d
r
Heart failure patients with left ventricular systolic
dysfunction without ACEI and ARB contraindications
who are prescribed ACEI/ARB at discharge .................
1
0
d
r
j.
Incidence of pressure ulcers..........................................
1
0
d
r
k.
MRSA-1 Infection rate ...................................................
1
0
d
r
l.
MRSA-2 Transmission rate ...........................................
1
0
d
r
SCIP MEASURES
Surgery patients with appropriate hair removal .............
VTE PROPHYLAXIS MEASURES
HEART FAILURE MEASURE
i.
Prepared by Mathematica Policy Research
E.31
C8.
In general, how well do the hospital’s electronic health record or other information
systems support measurement on the SCIP, heart failure, and MRSA measures?
Would you say very well, somewhat well, not very well, or not at all well?
VERY WELL ............................................................. 1
SOMEWHAT WELL ................................................. 2
NOT VERY WELL .................................................... 3
NOT AT ALL WELL .................................................. 4
C8a.
Are there any specific measures where the hospital’s information systems support the
measure well?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.32
GO TO C9
C8b.
Which ones?
[PROGRAM WILL LIST FOR EACH ‘YES’ RESPONSE TO C4]
HOSPITAL INFORMATION
SYSTEM SUPPORTS
MEASURES WELL
(CHECK ALL THAT APPLY)
SCIP MEASURES
a. Surgery patients on a beta blocker prior to arrival who
received a beta blocker during the perioperative period .....
1
□
b. Prophylactic antibiotic received on time—within one hour
prior to surgical incision.......................................................
1
□
c. Prophylactic antibiotic selection for surgical patients ..........
1
□
d. Prophylactic antibiotics discontinued within 24 hours after
surgery end time .................................................................
1
□
e. Cardiac surgery patients with controlled 6 AM
postoperative serum glucose ..............................................
1
□
f.
1
□
g. Surgery patients with recommended VTE prophylaxis
ordered ................................................................................
1
□
h. Surgery patients who received appropriate VTE
prophylaxis within 24 hours prior to surgery to 24 hours
after surgery ........................................................................
1
□
Heart failure patients with left ventricular systolic
dysfunction without ACEI and ARB contraindications who
are prescribed ACEI/ARB at discharge ...............................
1
□
Incidence of pressure ulcers ...............................................
1
□
k. MRSA-1 Infection rate .........................................................
1
□
l.
1
□
Surgery patients with appropriate hair removal ...................
VTE PROPHYLAXIS MEASURES
HEART FAILURE MEASURE
i.
j.
MRSA-2 Transmission rate .................................................
Prepared by Mathematica Policy Research
E.33
BARRIERS TO IMPROVEMENT AND INTEREST IN EXTERNAL ASSISTANCE
C9.
I’m going to read a list of barriers that hospitals may face in improving their
performance on the SCIP, heart failure, and MRSA measures. As I read each one,
please tell me whether each is currently a major barrier, a minor barrier, or not a barrier
for this hospital.
[ROTATE ORDER OF LIST a – i, BUT ALWAYS END WITH j AND THEN k]
CODE ONE FOR EACH
MAJOR
BARRIER
MINOR
BARRIER
NOT A
BARRIER
DON’T
KNOW
REFUSED
a. The hospital lacks enough
staff trained in quality
improvement .........................
2
1
0
d
r
b. Resource constraints, other
than staffing, limit
improvement strategies. ........
2
1
0
d
r
c. Lack of physician interest or
involvement ...........................
2
1
0
d
r
d. Documentation of the care
that is actually given is a
major problem .......................
2
1
0
d
r
e. Physicians at the hospital
disagree with selection of the
measure or its definition ........
2
1
0
d
r
The hospital has other higher
priorities.................................
2
1
0
d
r
g. The hospital is unsure of how
to improve performance ........
2
1
0
d
r
h. The hospital has no incentive
to improve .............................
2
1
0
d
r
2
1
0
d
r
YES
NO
DON’T
KNOW
REFUSED
1
0
d
r
f.
i.
j.
Insufficient senior
management leadership and
support ..................................
Any other barrier I haven’t
mentioned? (SPECIFY)........
k. IF YES, ASK: Was this a
major or a minor barrier?.......
Prepared by Mathematica Policy Research
MAJOR
MINOR
DON’T
KNOW
REFUSED
2
1
d
r
E.34
C10.
Is this hospital potentially interested in future technical assistance sponsored by CMS
to help reduce any remaining barriers and thereby boost its performance?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO C11b
REFUSED ................................................................ r
C11a.
What would be the two most important topics on which this hospital would like
additional support?
1.
2.
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
GO TO C12
C11b.
Please tell us if any of the following are reasons why you would not be interested in
future technical assistance to boost the hospital’s performance.
Hospital already has plans to use supporting
initiatives or organizations that should be
sufficient ................................................................... 1
No assistance from external organizations
is needed—just internal work ................................... 2
Existing outside organizations lack the
necessary expertise ................................................. 3
Hospital has other more important priorities ............. 4
Hospital lacks staff resources to participate
in any more improvement initiatives ......................... 5
Other (SPECIFY) ...................................................... 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.35
IMPORTANT SOURCES OF QI INFORMATION
C12.
The last few questions are about sources of information that may have improved the
quality of care at this hospital.
Do you believe the quality of care at this hospital in one or more clinical areas is better
this year than last year?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO END
REFUSED ................................................................ r
C13.
In addition to your own experience and data, what were the three most important
sources of information that have helped your hospital improve its quality of care over
the past year?
CODE UP TO THREE
DIRECT SHARING OF EXPERIENCES AND
BEST PRACTICES AMONG HOSPITALS ............... 1
IHI WEBSITE ............................................................ 2
OTHER WEBSITES VISITED ROUTINELY ............. 3
USE OF SEARCH ENGINES TO IDENTIFY
RELEVANT MATERIAL ON THE WEB .................... 4
CONFERENCE OR MEETING MATERIALS ........... 5
WEBINARS OR TELECONFERENCE
PRESENTATIONS ................................................... 6
MEETING WITH CONSULTANTS ........................... 7
OTHER (SPECIFY) .................................................. 8
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.36
C13a.
[IF C13=1]
You mentioned direct sharing of experiences and best practices as one of three most
important sources of information that have helped your hospital. Who facilitated the
sharing? Was it . . .
The hospital association, .......................................... 1
[NAME OF QIO], or .................................................. 2
Another organization? .............................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C13b.
[IF C13=3]
You mentioned other websites that were visited routinely as one of three most
important sources of information that have helped your hospital. What are those
websites? RECORD VERBATIM
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C13c.
[IF C13=5]
You mentioned conferences or meeting materials as one of the three most important
sources of information that have helped your hospital. Who sponsored the conference
or provided the meeting materials? Was it . . .
IHI, ............................................................................ 1
The hospital association, .......................................... 2
[NAME OF QIO], or .................................................. 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.37
C13d.
[IF C13=6]
You mentioned webinars or teleconference presentations as one of the three most
important sources of information that have helped your hospital. Who sponsored the
webinar or teleconference? Was it . . .
IHI, ............................................................................ 1
The hospital association, .......................................... 2
[NAME OF QIO], or .................................................. 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
END.
Those are all the questions I have. Do you have any final comments you’d like to
share?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO THANK
REFUSED ................................................................ r
RECORD VERBATIM
THANK. Thank you very much for participating in this survey, and taking the time to speak with
me. Have a great (day/evening).
Prepared by Mathematica Policy Research
E.38
Mathematica Reference No.: 06514.260
Ninth Scope of Work
QIO Program
Evaluation: Nursing
Home Survey
Final Draft Questionnaire
January 5, 2010
A. INTRODUCTION
A1.
INITIAL CONTACT WITH NURSING HOME: Hello, may I please speak with [NAME OF
ADMINISTRATOR/The administrator of the nursing home]?
IF YOU DON’T HAVE (HIS/HER) NAME, ASK FOR IT.
RECORD NAME AND CONTACT INFORMATION FOR ADMINISTRATOR.
NAME OF NURSING HOME ADMINISTRATOR
IF NEEDED: My name is ________ and I’m calling on behalf of the Centers for
Medicare & Medicaid Services or CMS.
IF ASKED WHY YOU ARE CALLING: A short time ago, we sent [NAME OF
ADMINISTRATOR/the administrator] a letter from CMS requesting the nursing home’s
participation in a survey for an evaluation of the Ninth Scope of Work of the Quality
Improvement Organization Program.
We would like to conduct a brief interview with the QI Director (or whoever is directly
involved with quality improvement at the nursing home). Would you please give me the
name and contact information for this person?
IF ADMINISTRATOR IS UNAVAILABLE, BUT YOU ARE SPEAKING WITH (HIS/HER)
SECRETARY OR ASSISTANT.
A1a.
My name is ________ and I’m calling on behalf of the Centers for Medicare & Medicaid
Services or CMS. A short time ago, we sent [NAME OF ADMINISTRATOR] a letter from
CMS requesting the nursing home’s participation in a survey for an evaluation of the
Ninth Scope of Work of the Quality Improvement Organization Program.
We would like to conduct a brief interview with the QI director, director of nursing, or
whoever is directly involved with quality improvement at the nursing home. I was hoping
to speak with the director to obtain the name and contact information for this person.
Since (he/she) is unavailable, would you be able to give me the name and contact
information of the QI director or director of nursing?
CAN TALK NOW ...................................................... 1
GO TO A2
SET CALLBACK ....................................................... 0
SET CALLBACK
DON’T KNOW .......................................................... d
Prepared by Mathematica Policy Research
E.40
A2.
RECORD NAME AND CONTACT INFORMATION FOR QI/NURSING DIRECTOR.
ENTER NAME OF QI OR NURSING DIRECTOR
ENTER TELEPHONE NUMBER: | | | |-|
Area Code
A2a.
|
|
|-|
|
|
|
|
WHEN SPEAKING WITH QI/NURSING DIRECTOR: Hello, [Dr./Mr./Ms.] [LAST NAME],
my name is ________, and I’m calling on behalf of the Centers for Medicare & Medicaid
Services or CMS. A short time ago, you should have received a letter from CMS
requesting your nursing home’s participation in a survey for an evaluation of the Ninth
Scope of Work of the Quality Improvement Organization Program.
Your nursing home’s input is crucial to assure that CMS learns all it can about how the
QIOs are working and what changes if any, need to be made. [FOR NONPARTICIPATING NURSING HOMES: It is critical that our study understand quality
improvement processes and thinking in nursing homes that have not been working with
QIOs as well as those that have, in order to understand the added value of the QIO
program.]
We would like to conduct a brief interview with you (or whoever is directly involved with
quality improvement at your nursing home). The survey interview takes roughly
28 minutes, depending upon your answers. I can conduct it now, or at any time that’s
convenient for you.
START INTERVIEW NOW ....................................... 1
GO TO B1
NOT NOW, SET UP APPT/CALLBACK ................... 2
SET APPT.
NEEDS MORE INFORMATION ............................... 3
GO TO A2b
REFUSED ................................................................ r
REFUSAL
Prepared by Mathematica Policy Research
E.41
A2b.
FOLLOW-UP INFORMATION
PURPOSE: CMS is interested in learning about the experience of nursing homes
involved in the Ninth Scope of Work of the Quality Improvement Organization Program.
Your nursing home’s input is crucial to assure that CMS learns all it can about how the
QIOs are working and what changes if any, need to be made.
[FOR NON-PARTICIPATING NURSING HOMES: It is critical that our study understand
quality improvement processes and thinking in nursing homes that have not been
working with QIOs as well as those that have, in order to understand the added value of
the QIO program.]
CONFIDENTIALITY: The information you provide will be kept strictly confidential. Only
statistical totals will be reported. Neither your name nor the nursing home name will be
publicly reported.
SELECTION: Your nursing home was randomly selected from U.S. nursing homes
eligible for the study. Some were selected to represent participating nursing homes that
worked with QIOs and some were selected to represent nursing homes that did not work
with QIOs.
REFUSAL SCREEN: Thank you for your time. Have a nice day. END CALL.
Prepared by Mathematica Policy Research
E.42
B. QIO INTERACTIONS
The first few questions are about staff interactions with [NAME OF QIO].
B1.
Is this nursing home participating with [NAME OF QIO] on a quality improvement
initiative related to any of the following topics . . . [READ DOWN LIST]
a. Physical restraint use ..............................................
YES
1
NO
0
DON’T
KNOW
d
REFUSED
r
b. Pressure ulcer reduction .........................................
1
0
d
r
c. Reducing re-hospitalizations ...................................
1
0
d
r
d. General assistance improving quality .....................
1
0
d
r
e. Any other topic? (SPECIFY) ..................................
1
0
d
r
[FOR EACH CATEGORY THAT RECEIVED A ‘NO’ RESPONSE, ASK B1aa AFTER
YOU HAVE READ THROUGH THE ENTIRE LIST IN B1.]
B1aa. Were you invited to participate with [NAME OF QIO] on a quality improvement initiative
related to [TOPIC]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B2
REFUSED ................................................................ r
B1ab. Why did you choose not to participate with [NAME OF QIO] on a quality improvement
initiative? RECORD VERBATIM
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.43
B2.
The next few questions ask about how often nursing home staff may have met with
[NAME OF QIO], either in-person or by telephone. Since August 2008, how many times
have nursing home personnel met with [NAME OF QIO] in-person at this nursing
home?
PROBE: Please do not include telephone conference calls or web-ex meetings.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
GO TO B3
REFUSED ................................................................ r
Bracket grouping No, Don't Know, and Refused responses and pointing to.
B2a.
[IF B2 = 2 OR MORE, DISPLAY “How often did…”; IF B2 = 1, DISPLAY “Did…”]
(How often did/Did) the following people attend the meeting with [NAME OF QIO]?
[READ ITEM]
[IF B2 = 2 OR MORE, DISPLAY: Would you say always, usually, sometimes, or
never?] [IF B2 = 1, DISPLAY ONLY CATEGORIES “ALWAYS” AND “NEVER”]
ALWAYS
USUALLY
SOMETIMES
NEVER
DON’T
KNOW
REFUSED
a. The administrator ......................
1
2
3
4
d
r
b. The director of nursing ..............
1
2
3
4
d
r
c. The medical director..................
1
2
3
4
d
r
d. The quality improvement
coordinator ................................
1
2
3
4
d
r
e. The staff developer ...................
1
2
3
4
d
r
f.
1
2
3
4
d
r
B3.
Any other staff? (SPECIFY) .....
Since August 2008, how many other in-person meetings have nursing home personnel
attended where [NAME OF QIO] was an active participant? Please include in-person
meetings held inside and outside the nursing home.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.44
B4.
Since August 2008, approximately how many times have nursing home personnel met
by phone with [NAME OF QIO]? Please do not include large conference calls.
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B5.
Since August 2008, how many other telephone conference calls or web-ex meetings
have nursing home personnel attended that [NAME OF QIO] convened?
|
|
| TIMES
NONE ....................................................................... 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF B2 OR B3 = OR >1, GO TO B6. IF B2 AND B3 = 0, GO TO B12]
Prepared by Mathematica Policy Research
E.45
B6.
I’m going to read a list of reasons why you might have met with [NAME OF QIO] since
August 2008. After each one, please tell me if this was a reason for (any of) the
in-person or phone meeting(s) with [NAME OF QIO]. [READ LIST]
[ROTATE ORDER OF LIST, BUT ALWAYS END WITH CATEGORY “L” LAST]
YES
NO
DON’T
KNOW
REFUSED
a. Self-referral to [NAME OF QIO] ..................................
1
0
d
r
b. Understanding [NAME OF QIO]’s plans for activities
and opportunities to participate ...................................
1
0
d
r
c. Routine meetings as part of participating with [NAME
OF QIO] on a quality improvement effort ....................
1
0
d
r
d. To discuss this nursing home’s performance data ......
1
0
d
r
e. Applying [NAME OF QIO] staff’s expertise to improve
this nursing home’s quality measures .........................
1
0
d
r
f.
Hearing about best practices of other nursing homes.
1
0
d
r
g. To learn about a new tool or recommended process
for quality improvement ...............................................
1
0
d
r
h. Other staff development or training .............................
1
0
d
r
Presentation(s) to help create buy-in to quality
improvement beyond the quality improvement staff....
1
0
d
r
To discuss issues, methods, and/or timeframes for
quality reporting to CMS..............................................
1
0
d
r
k. To attend a broad-based regional or statewide
meeting on quality improvement where [NAME OF
QIO] was an active participant ....................................
1
0
d
r
l.
1
0
d
r
i.
j.
Some other reason? (SPECIFY) ................................
Prepared by Mathematica Policy Research
E.46
B7.
Which of the following describe the aim(s) of the assistance you received?
[FOR THOSE WHO RESPOND ‘YES’ TO ANY ITEM IN B6]
IMPROVE OR ENSURE COMPLETENESS OF
THE REPORTED DATA ........................................... 1
ADVISE ON ISSUES RELATED TO
TRANSMISSION OF THE DATA ............................. 2
OTHER (SPECIFY) .................................................. 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B7a.
How effective was this assistance in achieving its aim(s)?
VERY EFFECTIVE ................................................... 1
SOMEWHAT EFFECTIVE ....................................... 2
NOT EFFECTIVE ..................................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.47
B8.
How valuable to the nursing home were each of these types of meetings?
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES TO B6.]
[READ ITEM] Would you say it was very valuable, somewhat valuable, or not valuable?
[IF ‘VERY VALUABLE’, ‘SOMEWHAT VALUABLE’, OR ‘NOT VALUABLE’ IS
CHOSEN, ASK B9 OR B10 BEFORE MOVING ON TO NEXT ITEM.]
VERY
VALUABLE
SOMEWHAT
VALUABLE
NOT
VALUABLE
a. Self-referral to [NAME OF QIO] .........
1
2
3
d
r
b. Understanding [NAME OF QIO]’s
plans for activities and opportunities
to participate.......................................
1
2
3
d
r
c. Routine meetings as part of
participating with [NAME OF QIO] on
a quality improvement effort ...............
1
2
3
d
r
d. To discuss this nursing home’s
performance data ...............................
1
2
3
d
r
e. Applying [NAME OF QIO] staff’s
expertise to improve this nursing
home’s quality measures ...................
1
2
3
d
r
Hearing about best practices of other
nursing homes....................................
1
2
3
d
r
g. To learn about a new tool or
recommended process for quality
improvement ......................................
1
2
3
d
r
h. Other staff development or training ....
1
2
3
d
r
Presentation(s) to help create buy-in
to quality improvement beyond the
quality improvement staff ...................
1
2
3
d
r
To discuss issues, methods, and/or
timeframes for quality reporting to
CMS ...................................................
1
2
3
d
r
k. To attend a broad-based regional or
statewide meeting on quality
improvement where [NAME OF QIO]
was an active participant ....................
1
2
3
d
r
l.
1
2
3
d
r
f.
i.
j.
Other reason (SPECIFY) ...................
Prepared by Mathematica Policy Research
E.48
DON’T
KNOW
REFUSED
B9.
[IF ‘NOT VALUABLE’ SELECTED AT B8, ASK B9 IMMEDIATELY AFTER B8:]
Why were these meetings not valuable?
CODE ALL THAT APPLY
THE MEETING MERELY FULFILLED AN
OBLIGATION (SUCH AS GUIDANCE OR
DIRECTION FROM THE STATE
SURVEY AGENCY) ................................................. 1
THIS NURSING HOME IS SO ADVANCED,
THERE IS NOTHING WE CAN LEARN
FROM [NAME OF QIO] ............................................ 2
[NAME OF QIO] STAFF DID NOT HAVE
ENOUGH EXPERTISE OF THE RIGHT TYPE ........ 3
[NAME OF QIO] STAFF WAS NOT
WELL-PREPARED ................................................... 4
THE RIGHT PEOPLE WERE NOT
AT THE MEETING ................................................... 5
THE INFORMATION PROVIDED WAS
NOT APPLICABLE TO THIS NURSING
HOME’S SITUATION ............................................... 6
THE MEETING WAS REDUNDANT WITH
INFORMATION WE ALREADY HAD ....................... 7
PROGRESS AT THIS NURSING HOME ON
THE MEETING TOPIC IS NOT FEASIBLE
AT THIS TIME .......................................................... 8
OTHER (SPECIFY) .................................................. 9
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.49
B10.
[IF ‘VERY VALUABLE’ OR ‘SOMEWHAT VALUABLE’ SELECTED AT B8, ASK B10
IMMEDIATELY AFTER B8:]
In what ways were these meetings valuable?
CODE ALL THAT APPLY
PROVIDED NEW, USEFUL INFORMATION ON
HOW TO IMPROVE PERFORMANCE .................... 1
PROVIDED NEW, USEFUL INFORMATION
ON ANOTHER TOPIC (SUCH AS QUALITY
REPORTING, USE OF EHR) ................................... 2
INCREASED MOTIVATION TO IMPROVE
FROM ONE OR MORE EXECUTIVE LEADERS ..... 3
INCREASED MOTIVATION TO IMPROVE
FROM MEDICAL DIRECTOR .................................. 4
INCREASED MOTIVATION TO IMPROVE
FROM KEY NURSING STAFF ................................. 5
OTHER (SPECIFY) .................................................. 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B11.
Did any of the meetings lead to changes at the nursing home that ultimately improved
resident care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B12
REFUSED ................................................................ r
B11a. Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.50
GO TO B12
B11b. Please tell me which measures were improved at least partly due to help from
[NAME OF QIO].
[READ LIST IF NEEDED]
CODE ALL THAT APPLY
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
RATE OF RE-HOSPITALIZATIONS ........................ 3
OTHER (SPECIFY) .................................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12.
(In addition to meetings), did you receive other educational materials, tools, or quality
improvement news from [NAME OF QIO]?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B13
REFUSED ................................................................ r
B12a. Did you receive these items from . . .
CODE ALL THAT APPLY
A newsletter, ............................................................. 1
An email or listserv, .................................................. 2
At an in-person meeting, .......................................... 3
At a teleconference or web-ex, ................................. 4
Or some other way? (SPECIFY) ............................. 5
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.51
B12b. Overall, how valuable were these educational materials and/or tools? Would you
say . . .
Very valuable, ........................................................... 1
Somewhat valuable, ................................................. 2
Not very valuable, or ................................................ 3
Not at all valuable? ................................................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B12c. Did any of these educational materials or tools from [NAME OF QIO] thus far lead to
changes at the nursing home that ultimately improved resident care?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B13
REFUSED ................................................................ r
B12d. Did the changes contribute to improvements in any particular quality measure?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO B13
REFUSED ................................................................ r
B12e. Please tell me which measures were improved at least in part due to changes that
stemmed from the educational materials or tools from [NAME OF QIO].
[READ LIST IF NEEDED]
CODE ALL THAT APPLY
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
RATE OF RE-HOSPITALIZATIONS ........................ 3
OTHER (SPECIFY) .................................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.52
B13.
Do you routinely receive data feedback from [NAME OF QIO] on this nursing home’s
quality performance?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO C1
REFUSED ................................................................ r
B13a. How widely within the nursing home do you typically share the feedback [NAME OF
QIO] provides, or highlights that you derive from it? Is it . . .
CODE ONE ONLY
Shared with a wide array of relevant
nursing and direct care staff, .................................... 1
Shared with a few key individuals, or ....................... 2
Rarely or never shared? ........................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
B13b. Has the feedback from [NAME OF QIO] been important to the nursing home's quality
improvement efforts?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.53
C. OTHER QI INITIATIVES
EXTERNAL INITIATIVES
The next questions are about quality improvement initiatives that involve external organizations.
C1.
Is this nursing home part of a corporate chain or otherwise-affiliated group of nursing
home providers?
YES, CORPORATE CHAIN ..................................... 1
YES, AFFILIATED GROUP ...................................... 2
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO C2
REFUSED ................................................................ r
C1a.
To what extent are your nursing home’s quality improvement actions influenced by this
(larger corporate chain/affiliated group) of nursing homes? Would you say to a large
extent, a moderate extent, or a small or no extent?
LARGE EXTENT ...................................................... 1
MODERATE EXTENT .............................................. 2
SMALL OR NO EXTENT .......................................... 3
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C2.
Setting aside any interactions with [NAME OF QIO] (and the (larger corporate chain/
affiliation) just discussed), is your nursing home actively involved in any other quality
improvement efforts involving outside organizations?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.54
GO TO C3
C2a.
Is your nursing home involved with any of the following?
CODE ALL THAT APPLY
The Advancing Excellence in Nursing Homes
Campaign ................................................................. 1
The Pressure Ulcer Collaborative............................. 2
State-initiated quality improvement projects
on pressure ulcer reduction, restraint use,
or other issues .......................................................... 3
Any other quality improvement effort with
an outside organization? (SPECIFY)....................... 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
INTERNAL INITIATIVES
Now I’d like to ask about internal quality improvement initiatives within the nursing home.
C3.
How many full-time equivalent staff are currently devoted to quality improvement in the
nursing home?
PROBE:
For example, if the nursing home has four staff who each devote a quarter
time to quality improvement, then the number of full-time equivalent staff at
the nursing home is one.
PROBE:
Your best estimate is fine.
|
|
| NUMBER OF FTE’s
LESS THAN ONE (OR A FRACTION) ..................... f
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.55
C4.
Since August 2008, please indicate if the nursing home has had internal quality
improvement efforts that have improved nursing home performance on any of the
following measures. [READ DOWN LIST]
YES
NO
DON’T
KNOW
REFUSED
a. Physical restraint use ..............................................
1
0
d
r
b. Pressure ulcers .......................................................
1
0
d
r
c. Influenza (flu) vaccination .......................................
1
0
d
r
d. Pneumococcal vaccination......................................
1
0
d
r
e. Urinary tract infection ..............................................
1
0
d
r
f.
Urinary catheter use ................................................
1
0
d
r
g. Depression or anxiety .............................................
1
0
d
r
h. Moderate to severe pain .........................................
1
0
d
r
i.
Patient mobility ........................................................
1
0
d
r
j.
Weight loss..............................................................
1
0
d
r
k. Help with daily activity .............................................
1
0
d
r
l.
1
0
d
r
Any other measure? (SPECIFY) ............................
[IF C4a THROUGH C4l ALL EQUAL ‘NO,’ GO TO C7. ELSE GO TO C5]
C5.
Has the nursing home performed an analysis on any of the measures to identify the
reasons why the relevant guideline sometimes is not followed, or why the undesirable
outcome sometimes occurs?
YES .......................................................................... 1
NO ............................................................................ 0
NOT APPLICABLE – PERFORMANCE
IS 100% RELIABLE .................................................. 2
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.56
GO TO C7
C5a.
For which measures has the nursing home performed an analysis?
[READ LIST IF NECESSARY]
[PROGRAM WILL LIST ALL ‘YES’ RESPONSES FROM C3a]
PHYSICAL RESTRAINT USE .................................. 1
PRESSURE ULCER RATES ................................... 2
INFLUENZA (FLU) VACCINATION RATES ............. 3
PNEUMOCOCCAL VACCINATION RATES ............ 4
URINARY TRACT INFECTION RATES ................... 5
URINARY CATHETER USE..................................... 6
DEPRESSION OR ANXIETY RATES ...................... 7
MODERATE TO SEVERE PAIN RATES ................. 8
PATIENT MOBILITY RATES ................................... 9
WEIGHT LOSS RATES............................................ 10
DAILY ACTIVITY RATES ......................................... 11
OTHER (SPECIFY) .................................................. 12
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.57
C5b.
[FOR EACH ‘YES’ RESPONSE AT C4, ASK:]
What type(s) of effort(s) has the nursing home undertaken to improve on [ITEM FROM
C4] [If respondents report the same efforts for multiple measures, use code 10]?
CODE ALL THAT APPLY
AN IMPROVEMENT TEAM USED A
PLAN-DO-STUDY-ACT (OR SIMILAR)
TECHNIQUE ............................................................ 1
ADJUSTED ELECTRONIC HEALTH RECORD
OR OTHER ELECTRONIC SYSTEM TO
INCLUDE AND DISPLAY KEY INFORMATION
IN A SEARCHABLE FIELD ...................................... 2
IMPLEMENTED A CHECKLIST ............................... 3
ESTABLISHED A PROTOCOL THAT IS
USED ROUTINELY .................................................. 4
[HIRED A CLINICAL CONSULTANT TO] TRAIN
NURSING AND DIRECT CARE STAFF ON
IMPROVING CARE IN THE MEASURE AREA........ 5
CHANGED ANOTHER ASPECT OF THE
PROCESS (BUT NOT THROUGH A
FORMAL PROTOCOL) ............................................ 6
PRODUCED AND SHARED UNIT LEVEL
DATA WITH NURSING STAFF ON THE
MEASURE ................................................................ 7
PERFORMED CHART REVIEWS TO IDENTIFY
ASSESSMENT ERRORS OR OTHER
POTENTIAL PROCESS IMPROVEMENTS ............. 8
OTHER (SPECIFY) .................................................. 9
SAME EFFORTS AS FOR PREVIOUS
MEASURE ................................................................ 10
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.58
C6.
To what extent have you had both the leadership support and resources you needed to
accomplish the improvements you sought on these measures?
On improvements for [READ ITEM], would you say you had all the leadership support
and resources you needed, somewhat less than you needed, or a lot less than you
needed?
[PROGRAM WILL DISPLAY FOR EACH ‘YES’ RESPONSE TO C4]
ALL THE
LEADERSHIP
SUPPORT AND
RESOURCES
YOU NEEDED
SOMEWHAT
LESS THAN
YOU NEEDED
A LOT
LESS
THAN YOU
NEEDED
DON’T
KNOW
REFUSED
a. Physical restraint use ....................
1
2
3
d
r
b. Pressure ulcers .............................
1
2
3
d
r
c. Influenza (flu) vaccination .............
1
2
3
d
r
d. Pneumococcal vaccination ...........
1
2
3
d
r
e. Urinary tract infection ....................
1
2
3
d
r
f. Urinary catheter use .....................
1
2
3
d
r
g. Depression or anxiety ...................
1
2
3
d
r
h. Moderate to severe pain ...............
1
2
3
d
r
i. Patient mobility .............................
1
2
3
d
r
j. Weight loss ...................................
1
2
3
d
r
k. Help with daily activity ...................
1
2
3
d
r
l. Any other measure? (SPECIFY) ..
1
2
3
d
r
Prepared by Mathematica Policy Research
E.59
BARRIERS TO IMPROVEMENT AND INTEREST IN EXTERNAL ASSISTANCE
C7.
I’m going to read a list of barriers that nursing homes may face in improving their
performance on these measures. As I read each one, please tell me whether each
is currently a major barrier, a minor barrier, or not a barrier for this nursing home.
[ROTATE ORDER OF LIST, BUT ALWAYS END WITH CATEGORY “J” AND “K”]
CODE ONE FOR EACH
MAJOR
BARRIER
MINOR
BARRIER
NOT A
BARRIER
DON’T
KNOW
REFUSED
a. The nursing home lacks
enough staff trained in
quality improvement ..............
2
1
0
d
r
b. Resource constraints, other
than staffing, limit
improvement strategies. ........
2
1
0
d
r
c. Lack of nursing staff interest
or involvement .......................
2
1
0
d
r
d. Documentation of the care
that is given is a problem ......
2
1
0
d
r
e. Staff at the nursing home
disagree with selection of the
measure or its definition ........
2
1
0
d
r
The nursing home has other
higher priorities......................
2
1
0
d
r
g. The nursing home is unsure
of how to improve
performance ..........................
2
1
0
d
r
h. The nursing home has no
incentive to improve ..............
2
1
0
d
r
2
1
0
d
r
YES
NO
DON’T
KNOW
REFUSED
1
0
d
r
f.
i.
j.
Insufficient senior
management leadership and
support ..................................
Any other barrier I haven’t
mentioned? (SPECIFY)........
k. IF YES, ASK: Was this a
major or a minor barrier?.......
Prepared by Mathematica Policy Research
MAJOR
MINOR
DON’T
KNOW
REFUSED
2
1
d
r
E.60
C8a.
Is this nursing home potentially interested in future technical assistance sponsored by
CMS to help reduce any remaining barriers and thereby boost its performance?
YES .......................................................................... 1
PERHAPS/MAYBE ................................................... 2
NO ............................................................................ 0
GO TO C9
DON’T KNOW .......................................................... d
GO TO C10
REFUSED ................................................................ r
C8b.
What would be the two most important topics on which this nursing home would like
additional support?
1.
2.
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
C9.
Please tell me if any of the following are reasons why you would not be interested in
future technical assistance to boost the nursing home’s performance.
CODE ALL THAT APPLY
Our nursing home already has plans to use
other supporting initiatives or organizations ............. 1
No assistance from external organizations
is needed, just internal work ..................................... 2
Existing outside organizations lack the
necessary expertise ................................................. 3
Our nursing home has other more important
priorities .................................................................... 4
Our nursing home lacks staff resources to
participate in any more improvement initiatives........ 5
Any other reason? (SPECIFY) ................................ 6
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.61
IMPORTANT SOURCES OF QI INFORMATION
The last few questions are about sources of information that may have improved the quality of
care at this nursing home.
C10.
Do you believe the quality of care at this nursing home in one or more clinical areas is
better this year than last year?
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
GO TO END
REFUSED ................................................................ r
C11.
In addition to your own experience and data, what were the three most important
sources of information that have helped your nursing home improve its quality of
care over the past year?
CODE UP TO THREE
DIRECT SHARING OF EXPERIENCES AND
BEST PRACTICES AMONG NURSING HOMES .... 1
MedQIC WEBSITE ................................................... 2
OTHER WEBSITES VISITED ROUTINELY ............. 3
USE OF SEARCH ENGINES TO IDENTIFY
RELEVANT MATERIAL ON THE WEB .................... 4
CONFERENCES OR SEMINARS ............................ 5
WEBINARS OR TELECONFERENCE
PRESENTATIONS ................................................... 6
MEETING WITH CONSULTANTS ........................... 7
OTHER (SPECIFY) .................................................. 8
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.62
[IF C11=1]
C11a. You mentioned direct sharing of experiences and best practices as one of three most
important sources of information that have helped your nursing home. Who facilitated
the sharing? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF C11=3]
C11b. You mentioned other websites that were visited routinely as one of three most important
sources of information that have helped your nursing home. What are those websites?
RECORD VERBATIM
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
[IF C11=5]
C11c. You mentioned conferences or seminars as one of the three most important sources of
information that have helped your nursing home. Who sponsored the conference or
seminar? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.63
C11d.
[IF C11=6]
You mentioned webinars or teleconference presentations as one of the three most
important sources of information that have helped your nursing home. Who sponsored
the webinar or teleconference? Was it . . .
The nursing home trade association, ....................... 1
[NAME OF QIO], ...................................................... 2
AANAC, or ................................................................ 3
Another organization? .............................................. 4
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
END.
Those are all the questions I have. Do you have any final comments you’d like to
share? RECORD VERBATIM
THANK. Thank you very much for participating in this survey, and taking the time to speak with
me. Have a great (day/evening).
YES .......................................................................... 1
NO ............................................................................ 0
DON’T KNOW .......................................................... d
REFUSED ................................................................ r
Prepared by Mathematica Policy Research
E.64
GO TO THANK
APPENDIX F
CASE STUDY DISCUSSION GUIDES AND LETTERS
CMS LETTERHEAD
LETTER OF ENCOURAGEMENT TO PARTICIPATE - EVALUATION OF THE
9TH QIO SOW
PROVIDER ORGANIZATIONS SELECTED FOR SITE VISITS
[DATE]
[NAME AND ADDRESS]
Dear [Dr./Mr./Ms.] [FILL LAST NAME]:
The Centers for Medicare & Medicaid Services (CMS) is sponsoring a study about the
Quality Improvement Organization (QIO) Program, which is a key component in CMS’ efforts
to improve the quality and efficiency of care for Medicare beneficiaries. The purpose of the
study is to evaluate the program’s effectiveness in helping health care providers to improve the
quality of care for Medicare beneficiaries and to find ways to improve the program.
Mathematica Policy Research, Inc. (MPR), an independent research organization, is
conducting the study on behalf of CMS, with partnered organizations Social & Scientific
Systems and Abt Associates. As part of this study, MPR and its partners will visit providers in
twelve states who worked with their local QIO.
Your participation in the site visits is voluntary, but your participation is invaluable for the
success of this important study. Your input will help Medicare to improve the design and
effectiveness of the QIO program and to ultimately improve care for Medicare beneficiaries. I
urge you to participate as described in the invitation letter included with this one. Neither your
name nor your organization’s name will ever be included in any reports prepared for CMS or
others as part of this study.
We look forward to including your valuable input in this study.
Sincerely,
CMS Project Officer
Enclosure
F.3
QIO DIRECTOR INVITATION LETTER
600 Maryland Ave. S.W.
Suite 550
Washington D.C. 20024-2512
Telephone (202) 484-9220
Fax (202) 863-1763
www.mathematica-mpr.com
[Date], 2010/2011
QIO Director
QIO name
Street address
City, ST zip
Dear [QIO Director name]:
The Centers for Medicare & Medicaid Services (CMS) would very much appreciate your
thoughtful input on your QIO’s experience with the 9th SOW for [STATE]. This is an invitation to
participate in a site visit being conducted as part of the evaluation of the 9th SOW Quality
Improvement Program by Mathematica Policy Research and its subcontractors Social & Scientific
Systems and Abt Associates, sponsored by CMS. Participation will involve discussions with you
and the key QIO staff who lead themes or patient safety sub-themes for a total of [TOTAL
LENGTH OF INTERVIEWS] hours during the week of [TARGET WEEK]; this total includes
one hour for discussion with you, approximately 30 minutes per theme or patient safety sub-theme
with the leader of each of those efforts, and an additional 15 minutes for the overall patient safety
theme leader.
Your feedback is confidential. Your input will be valuable in assisting CMS in continuing to
improve the QIO program with each new scope of work. The site visits are part of a larger study to
evaluate the 8th and 9th Scopes of Work Quality Improvement Program, described in the
attachment.
Please note that Section C.4.B.13 of the 9th SOW contract requires each QIO to provide data
for evaluation, thus, your time to participate in the site visit is an expense covered under the
contract. If you have any questions, please feel free to call [CMS OFFICIAL] at [PHONE
NUMBER].
On-site, we would like to discuss the following key topics:
• The impact you have had on quality improvement and patient safety related to each
theme (note we do not expect you to provide data, the purpose is to obtain your
thinking in summary form about your accomplishments)
• Any difficulties you have faced with the program contract, infrastructure, and supports
• The value of QIOSCs in facilitating your work
• Which types of activities you have undertaken since August 2008 were more and less
valuable for quality improvement
F.5
An Affirmative Action/Equal Opportunity Employer
LETTER TO:
FROM:
DATE:
PAGE:
QIO Director
Myles Maxfeild
[Date], 2010/2011
2
• How you recruited providers to work with you during the 9th SOW
• Types of actions taken by providers to improve care on the program-targeted measures,
and what prompted the actions
• Lessons learned from the 9th SOW experience
• What types of key barriers remain to quality improvement in this state
• The state quality environment, such as the roles of other organizations influencing
quality, and the interest among providers in collaborating and working for quality
improvement
With your help, the evaluation will provide critical information to CMS to help refine the
Quality Improvement Program. An evaluation staff member will call you to follow up in the next
couple of days. In the meantime, please feel free to call or e-mail [CONTACT INFORMATION
OF EVALUATION STAFF MEMBER] with any questions or to initiate scheduling. Thank you
very much in advance for your assistance—your input into the evaluation is highly valued by
CMS.
Sincerely,
Myles Maxfield, Ph.D.
Project Director
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless
it displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The
time required to complete this information collection is estimated to average 0.8 hours or 48 minutes per response, including
the time to review instructions, search existing data resources, and gather the data needed, and complete and review the
information collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving
this form, please write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05,
Baltimore, Maryland 21244-1850.
F.6
PROVIDER INVITATION LETTER
600 Maryland Ave. S.W.,
Suite 550
Washington, DC 20024-2512
Telephone (202) 484-9220
Fax (202) 863-1763
www.mathematica-mpr.com
[Date], 2010/2011
Contact
Organization name
Street address
City, ST zip
Dear [contact name]:
This is an invitation to provide important feedback to the Centers for Medicare & Medicaid Services
(CMS) through agreeing to an on-site interview, roughly 45 to 50 minutes in length, to assist CMS in
evaluating and improving its Quality Improvement Program. The CMS Quality Improvement Program in this
state is operated through [NAME QIO]. If you have worked with [NAME QIO] on one or more quality
improvement efforts since August 2008, the evaluation research team would appreciate your feedback through
an on-site interview during [TARGET WEEK], to be scheduled at your convenience. You will not need to
make any special preparations for the visit, and your input will remain confidential. In particular, we would
like to discuss:
•
•
•
•
•
The overall value and effect of [NAME QIO] activities related to [NAME THEME(S) THE
PROVIDER WORKED WITH THEM ON]
Why you decided to work with [NAME QIO] (if applicable) and which types of interactions, tools,
resources, etc. that were provided by [NAME QIO] were more and less helpful to you
The story of this [TYPE OF PROVIDER]’s quality improvement related to [NAME THEME(S)]
since Summer 2008, and the major reasons for any changes in measured performance.
Lessons you have learned about how to improve quality since Summer 2008, and remaining barriers
you face to further improvement
What you know and think about the quality-related environment in your area, for example, how
interested [TYPE OF PROVIDER]s are in general in quality improvement, and whether they are
willing to share information
CMS has contracted with Mathematica Policy Research, Inc. and its partners Social & Scientific Systems
and Abt Associates, to conduct the site visits as part of a larger study to evaluate the 8th and 9th Scopes of
Work Quality Improvement Program. A description of the larger study is attached.
With your help, the evaluation will provide critical information to CMS to help refine its work to
improve the quality of care. Please see the attached letter of encouragement to participate from CMS. An
evaluation staff member will call you to follow up in the next couple of days. In the meantime, please feel free
to call or e-mail [CONTACT INFORMATION OF EVALUATION STAFF MEMBER] with any questions or
to initiate scheduling. Thank you very much in advance for your assistance—your input into the evaluation is
highly valued by CMS.
Sincerely,
Myles Maxfield, Ph.D., Project Director
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it
displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The time
required to complete this information collection is estimated to average 0.8 hours or 48 minutes per response, including the time
to review instructions, search existing data resources, and gather the data needed, and complete and review the information
collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please
write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland
21244-1850.
F.7
An Affirmative Action/Equal Opportunity Employer
COMMUNITY HEALTH LEADER
INVITATION LETTER
600 Maryland Ave. S.W.,
Suite 550
Washington, DC 20024-2512
Telephone (202) 484-9220
Fax (202) 863-1763
www.mathematica-mpr.com
[Date], 2010/2011
Contact
Organization name
Street address
City, ST zip
Dear [contact name]:
As a key health leader representing [HOSPITALS/NURSING HOMES/PHYSICIANS], you are no
doubt aware of the investment the Centers for Medicare & Medicaid Services (CMS) makes in improving
quality of care through its Quality Improvement Program, which in this state is operated through [NAME
QIO]. This is an invitation to provide important feedback to CMS through agreeing to an in-person interview,
roughly 45 to 50 minutes in length, to assist CMS in evaluating and improving its Quality Improvement
Program. Your input will remain confidential.
CMS has contracted with Mathematica Policy Research, Inc. and its partners Social & Scientific Systems
and Abt Associates, to conduct the site visits as part of a larger study to evaluate the 8th and 9th Scopes of
Work Quality Improvement Program. A description of the larger study is attached. The evaluation research
team would appreciate your feedback through an in-person interview during [TARGET WEEK], to be
scheduled at your convenience. You will not need to make any special preparations for the meeting. In
particular, we would like to discuss the following key topics:
•
•
•
•
The overall value and effect of [NAME QIO] activities related to quality improvement and patient
safety
Which types of activities by [NAME QIO] since August 2008 that were more and less valuable for
quality improvement
What types of key barriers remain to quality improvement in this state, and how the CMS Quality
Improvement Program could best help in the future
What the quality-related environment is like in your area, for example, how interested [TYPE OF
PROVIDER]s are in general in quality improvement, and whether they are willing to share
information
With your help, the evaluation will provide critical information to CMS to help refine its work to
improve the quality of care. Please see the attached letter of encouragement to participate from CMS. An
evaluation staff member will call you to follow up in the next couple of days. In the meantime, please feel free
to call or e-mail [CONTACT INFORMATION OF EVALUATION STAFF MEMBER] with any questions or
to initiate scheduling. Thank you very much in advance for your assistance—your input into the evaluation is
highly valued by CMS.
Sincerely,
Myles Maxfield, Ph.D., Project Director
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it
displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The time
required to complete this information collection is estimated to average 0.8 hours or 48 minutes per response, including the time
to review instructions, search existing data resources, and gather the data needed, and complete and review the information
collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please
write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland
21244-1850.
F.9
An Affirmative Action/Equal Opportunity Employer
DRUG SAFETY ORGANIZATION
INVITATION LETTER
600 Maryland Ave. S.W.,
Suite 550
Washington, DC 20024-2512
Telephone (202) 484-9220
Fax (202) 863-1763
www.mathematica-mpr.com
[Date], 2010/2011
Contact
Organization name
Street address
City, ST zip
Dear [contact name]:
As a [DESCRIBE ORGANIZATION TYPE IN APPROPRIATE, FLATTERING TERMS—such as
major health plan in the state], you are probably aware of the investment that the Centers for Medicare &
Medicaid Services (CMS) makes in improving quality of care through its Quality Improvement Program,
which in this state is operated through [NAME QIO]. This is an invitation to provide important feedback to
CMS through agreeing to an in-person interview, roughly 45 to 50 minutes in length, to assist CMS in
evaluating and improving its Quality Improvement Program. Your input will remain confidential.
CMS has contracted with Mathematica Policy Research, Inc. and its partners Social & Scientific Systems
and Abt Associates, to conduct site visits as part of a larger study to evaluate the 8th and 9th Scopes of Work
Quality Improvement Program. A description of the larger study is attached. The evaluation research team
would appreciate your feedback through an in-person interview during [TARGET WEEK], to be scheduled at
your convenience. You will not need to make any special preparations for the meeting. In particular, we
would like to discuss the following key topics:
•
•
•
•
The overall value and effect of [NAME QIO] activities related to improving drug safety by reducing
inappropriate medications and drug-on-drug interactions
Which types of activities by [NAME QIO] since August 2008 that were more and less valuable for
improving drug safety
What types of key barriers remain to further improving drug safety this state, and how the CMS
Quality Improvement Program could best help in the future
What the quality-related environment is like in your area, for example, how interested providers are
in general in improving drug safety, and whether they are willing to share information on how best
to do so
With your help, the evaluation will provide critical information to CMS to help refine its work to
improve patient safety. Please see the attached letter of encouragement to participate from CMS. An
evaluation staff member will call you to follow up in the next couple of days. In the meantime, please feel free
to call or e-mail [CONTACT INFORMATION OF EVALUATION STAFF MEMBER] with any questions or
to initiate scheduling. Thank you very much in advance for your assistance—your input into the evaluation is
highly valued by CMS.
Sincerely,
Myles Maxfield, Ph.D., Project Director
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless
it displays a valid OMB control number. The valid OMB control number for this information collection is xxx-XXX. The time
required to complete this information collection is estimated to average 0.8 hours or 48 minutes per response, including the time
to review instructions, search existing data resources, and gather the data needed, and complete and review the information
collection. If you have comments concerning the accuracy of the time estimate(s) or suggestions for improving this form, please
write to: CMS, 7500 Security Boulevard, Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland
21244-1850.
F.11
An Affirmative Action/Equal Opportunity Employer
CASE STUDY DISCUSSION GUIDES
F6A. QIO DIRECTOR AND THEME LEADERS
In the matrix that follows, due to limited space we have used abbreviated indicators for the QIO
staff who will be asked each question during the case study discussions.
Prior to the visits, we will produce a guide tailored to the staffing situation of the QIO, based on
information about its staffing that we learn as we schedule the visit. For example, if a single
individual is responsible for leading the Pressure Ulcer and Physical Restraints sub-themes, we
will concatenate the questions for each of those topics into a single guide. The likely wide
variation in who is responsible for which themes and sub-themes has caused us to prefer to
provide the questions in this matrix format until just prior to the visit.
Key to Abbreviations for QIO Director and Theme Leaders Participating in Case Study
Discussions:
QIO Dir: QIO Director
Pt Sfty: Patient Safety Theme Leader
PrU: Patient Safety – Pressure Ulcers leader
PR: Patient Safety – Physical Restraints leader
SCIP: Patient Safety - Leader for improving surgical safety and heart failure
MRSA: Patient Safety - Leader for reducing Methicillin-resistant Staphylococcus aureus in the
acute care setting
Rx Safety: Patient Safety - Leader for improving drug safety
Pre: Prevention Theme Leader
Pre Disp: Prevention – Disparities Theme Leader
Care Tr: Care Transitions Theme Leader
CKD: Prevention – Chronic Kidney Disease Theme Leader
F.13
QIO DIRECTOR AND THEME LEADERS
DISCUSSION GUIDE TOPICS
Topic for Discussion
F.14
1. Program Impact in the State
Has [QIO contract theme] had an impact on
quality of care or patient safety? [For QIO Dir,
repeat for each theme]
[If yes to pt. safety] Which components of
the patient safety theme have had the
largest impact on quality or safety during
the 9th SOW? Why?
[If yes] What do you view as the most
important evidence of impact?
Did all the contract evaluation measures
work reasonably well to assess progress?
If not, which ones were problematic?
Why?
Are there certain types of providers that
seemed to benefit more than others from
working with the QIO? If yes, which
types?
Are there any specific “success stories”
that you are aware of that you could share
with us?
Are you aware of the trend in quality among
providers in the state that you did not
specifically work with during the 9th SOW? [If
yes:] Did they improve?
QIO Pt
PrU PR SCIP MRSA NHIN Rx Pre Pre Care CKD
Dir Sfty
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.15
QIO Pt
PrU PR SCIP
Dir Sfty
[If providers not worked with also
X
X
X
improved, on average:] What do you think
caused the providers you didn’t work with
to improve? Would this also have affected
the providers you did work with?
In your survey response, you indicated [name X
area of quality improvement posing greatest
challenge] was the biggest/most important
challenge for you in the 9th SOW. Could you
elaborate a little on that and whether and how
it may have affected improvements in provider
quality measures?
2. Program Contract, Infrastructure, and
Supports
[Request QIO Dir & TLs to review their survey
responses and provide any updates prior to the
visit. If the respondent has changed positions,
the new QIO Dir or TL will be asked to
respond to the survey in advance of the visit.
The following questions are designed to follow
up on survey responses.]
X
X
X
[For any negative response on the contractrelated matters section of the survey—repeat
as necessary:] Could you tell me a little more
about the specifics of the problem you
indicated in your response to [name question]?
Do you believe this [name problem indicated]
X
X
X
significantly lessened the results you were able
to achieve? [If yes] Why?
Are there [QIO Dir: any; Theme Leads: any X
X
X
X
other] barriers to this QIO’s effectiveness that
stem from the contract or CMS procedures?
MRSA NHIN Rx Pre Pre Care CKD
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.16
QIO Pt
PrU PR SCIP
Dir Sfty
[If responded insufficient data and
X
X
X
information on any item, ask:] You indicated
you had insufficient data and information to
[name task]: could you tell me more about
what data and information you felt you needed
but did not have?
Do you believe this lack of information
X
X
X
significantly lessened the results you were able
to achieve? [If yes] Why?
You rated the following information types as
X
X
X
having high value in helping you shape and
refine interventions over time: [name them].
What was it about these information sources
that made them high value?
You rated the following information types as
X
X
X
having low value: [name them]. What was it
about these information sources that made
them low value?
[Follow-up on any negative responses to
X
X
X
knowledge base/tools items:] Could you
elaborate more on your response that [tailor to
the item]?
[If not already covered:] Could you talk about
X
the role of the QIOSC with respect to your
work? Specifically what did they contribute to
your ability to work effectively on your theme?
X
X
X
[Follow up on survey response:] I see from
your survey response that you found the
centrally developed change package
[useful/not useful] to your work. Could
you tell us more about that?
MRSA NHIN Rx Pre Pre Care CKD
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.17
QIO Pt
PrU PR SCIP MRSA NHIN Rx Pre
Dir Sfty
Sfty
[If not already covered:] Could you talk about X
X
X
X
X
X
X
X
your experience in using the PATRIOT
system—that is, was it reasonably easy to enter
the required information into the system, and
retrieve anything you needed to retrieve from
it? [If problematic:] To what extent did this
impact your ability to get your work done most
effectively and efficiently? Could you provide
examples?
Could you talk about the experience with
X
physician practices reporting data from their
EHRs to a new CMS management information
system?
X
X
X
X
X
X
I see from your survey response that you found
the annual in-person meetings sponsored by
CMS specific to the patient safety theme to be
[useful/not useful]. Could you tell me more
about how they [were/were not] useful?
[If survey suggested 1 or more improvements] X
In your survey, you suggested [non-zero
number up to 3] potential high-priority
improvements to CMS-funded tools or
resources. Could you elaborate a little on your
ideas on this and how they could help QIOs
achieve
better
quality
and
safety
improvements?
Do you have any additional suggested X
improvements at this time?
Pre Care CKD
Disp Tr
X
X
X
X
Topic for Discussion
F.18
QIO Pt
PrU PR SCIP
Dir Sfty
[For any negatives in the contract support
X
X
X
and communication section:] In the survey
you indicated that [certain CMS staff had fair
to poor knowledge relative to their
responsibilities/communications were not
usually consistent among different CMS
personnel]. Could you elaborate on the
problem?
[For any “excellent” responses:] You indicated
X
X
X
the knowledge base was strong among [name
type of CMS staff]. Please tell us more about
how this may have assisted you in your work.
3. Types of High- and Lower-Value
Activities
X
X
X
Collaborative Activities – Relative to other
types of activities, you rated collaborative
activities [summarize value rating relative to
other activities].
X
X
X
[For low-value items:] Could you tell us more
about why you rated this type of activity
relatively low in value?
[For high-value items:] Could you tell us more
X
X
X
about why you rated this type of activity highvalue?
[Repeat for: Interactions with Individual
Providers, One-to-Many Activities, Business
Case Focus, Theme-Specific Items]
Are there any other types of activities we
X
X
X
haven’t discussed that you have found to be
high-value? [If yes] What are they?
MRSA NHIN Rx Pre Pre Care CKD
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.19
4. Common Types of Actions Taken by
Providers
What were common types of improvement
actions taken by providers [for Rx Sfty: or
other organizations] you worked with?
About what proportion of providers you
worked with took meaningful actions?
[If not 100%] What do you think might explain
why some took meaningful actions and some
did not?
5. Lessons Learned, and QI Within the QIO
What lessons have you learned during the 9th
SOW about what works to improve quality or
safety?
In hindsight, is there anything you wish you
had done differently in the area of recruiting
providers [Rx Sfty: or other organizations] to
work with?
Is there anything you wish you had done
differently in the area of working with
providers [Rx Sfty: or other organizations]?
During the 9th SOW, all QIOs were required to
track their own approaches to QI and assess
their effectiveness, then use action plans and
PDSA to improve outcomes. Based on the
quarterly reports on this topic that we
reviewed, it looks like [summarize findings
from quarterly reports review]. Could you
elaborate on [name anything puzzling or
particularly interesting from the review]?
QIO Pt
PrU PR SCIP MRSA NHIN Rx Pre Pre Care CKD
Dir Sfty
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.20
QIO Pt
PrU PR SCIP MRSA NHIN Rx Pre Pre Care CKD
Dir Sfty
Sfty
Disp Tr
Has any part of your work included thinking
X
X
X
about the sustainability of any gains that are
achieved? What are the key issues in
sustainability?
[If some changes were recommended in X
survey:] In your survey response, you
suggested that CMS should make the following
change [name change recommended]. Could
you elaborate a bit on what led you to that
recommendation and any more specific
thoughts about what shape the change should
take? [repeat as necessary to cover all
recommended changes]
6. State Environment
Health Care Organizations Involved in Quality
X
X
X
X
X
X
Please tell us about any quality-related
activities undertaken by the relevant
professional or provider [Rx Sfty: or health
plan] association(s)? What if any role did the
QIO play in any of these?
X
X
X
X
X
X
Please tell us about any relevant quality or
safety-related activities undertaken by large
provider organizations [Rx Sfty: or health
plans] that operate in this state? Please be clear
if the QIO had a role in prompting or
facilitating any of these.
Topic for Discussion
F.21
QIO Pt
PrU PR SCIP MRSA NHIN Rx
Dir Sfty
Sfty
In your survey response, you listed the
X
X
X
following other organizations whose efforts
were proving important to achieving
improvements: [list]. Could you tell us a little
more about what they are doing, and why you
think their activities have been effective?
Please be clear if the QIO had a role in any of
these.
Provider Environment
During the 9th SOW, to what extent have
X
X
providers in this state been interested in
exchanging information with one another to
improve quality?
X
X
Has this [willingness/unwillingness] to share
been a significant [help/hindrance] to
improving quality during this period?
X
X
What do you think underlies the general
[willingness/unwillingness] to share in this
state?
X
X
X
X
X
Your survey response indicated that providers
in this state often [have enough/lack]
motivation to
improve
their
quality
performance. Could you say a little more about
why they have this level/lack of motivation?
X
X
X
X
X
Has this level of motivation been a significant
[help/hindrance] to improving quality during
this period?
X
X
Are there any other characteristics of the
culture of the providers in this state that make
assisting them with quality improvement
particularly challenging?
Pre Pre Care CKD
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.22
QIO Pt
PrU PR SCIP
Dir Sfty
What are the key barriers to further
X
X
X
improvements among providers [Rx Sfty: and
health plans] in this state?
Focusing on the poorest performers, let’s say
X
X
X
the bottom 10 percent, could you describe why
you think they are performing so poorly? What
if anything do you believe would help them
improve?
Are there characteristics of the provider [Rx
Sfty: or health plan] environment in this state
that make providers particularly receptive to
attempts to assist them with quality
improvement?
Are there characteristics of the provider [Rx
Sfty: or health plan] environment in this state
that make it particularly challenging for an
organization like the QIO to assist providers
with quality improvement?
7. Participating Providers Selection Process
X
X
X
How much discretion did the QIO have in
selecting practices to be participating
providers?
X
X
X
Please describe the state’s strategy in recruiting
providers [CKD and CT: and organizational
partners] under this theme—types of providers
[CKD and CT: and organizations] targeted,
type of approach, and selling points.
Could you tell us how this state ended up
working with the number of providers [CKD
and CT: and other partners] that it did?
MRSA NHIN Rx Pre Pre Care CKD
Sfty
Disp Tr
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
Topic for Discussion
F.23
QIO Pt
PrU PR SCIP MRSA NHIN Rx Pre Pre Care CKD
Dir Sfty
Sfty
Disp Tr
Did the geography of the areas you targeted
X
X
X
present
any
special
challenges
to
accomplishing the goals of the theme?
How receptive were providers [CKD and CT:
X
X
X
X
X
X
X
X
X
and other key organizations] to working with
you? Why?
We understand from your survey response that
X
X
X
X
X
X
[summarize selection process]. Could you tell
us about the advantages and disadvantages of
the way providers were selected to work with,
in terms of how well it worked to give you a
group that you could assist to improve quality
to meet your goals?
X
X
X
X
X
X
X
If you had instead worked with every provider
in the state who could have well-used your
help to improve performance, how much
bigger would the group have been?
8. Recruitment of Beneficiaries
X
Based on your survey response, we understand
this state recruited beneficiaries for DSME
[through
providers/directly
from
the
community]. Did you consider the other type
of strategy, and if so, why did you choose the
one you did?
How well did your recruiting strategy
X
work?
What lessons have you learned about
X
recruiting beneficiaries for DSME?
F6B. HOSPITAL QUALITY IMPROVEMENT DIRECTORS
All hospitals selected for site visits will have worked with the QIO on one or more themes/subthemes. In the matrix that follows, some questions include theme/sub-theme names in brackets.
This means the question would be asked in terms of one theme that is applicable to the selected
hospital. If there is time, the interviewer would cycle back and discuss a second topic if the
hospital worked with the QIO on more than one theme/sub-theme.
F.24
1. Background Information
[collect description of hospital organization prior to visit, including whether part of larger
system, level of services (e.g., tertiary, secondary), size of hospital, teaching status (major
teaching, minor teaching, non-teaching), tax status (FP/NFP), payor mix (% Medicaid; %
Medicare)]
[We will know from administrative data which themes and sub-themes the visited hospital
worked on with the QIO. For questions that are specific to a theme or sub-theme—where a
theme or sub-theme is indicated in brackets—we will decide in advance which theme/sub-theme
to cover first. If time, we will cycle back and continue the discussion regarding a second
applicable theme/sub-theme.]
2. Overall Quality and Quality Improvement
Where does quality fit into the hospital’s overall business strategy?
When it comes to how well this hospital is performing overall in terms of quality and safety
overall, on a scale of 1-10, where 10 is the best care possible, where would you place it
currently?
Where would you have placed it three years ago?
[If improved:] What have been the main factors that led to the improvement?
[If lower than 7:] What are the main reasons you wouldn’t give the hospital a higher score?
Setting aside this 1-10 scale, could you give us an overview of how the hospital measures its
quality?
In general, what types of activities has the hospital found to be most effective at improving
quality?
3. QIO Interactions
How did this hospital get involved in quality improvement activities with the QIO that have been
undertaken since August 2008?
Please tell us about this hospital’s interactions with [name QIO] over the past three years.
[capture frequency and types of interactions, positions/backgrounds of those who attended on
both sides]
[Make sure they are thinking about emails they received as well as any in-person or phone
conferences they attended]
[Make sure they cover whether they took a HLQAT and/or AHRQ Pt. Safety survey provided by
the QIO and whether there was any follow up interaction.]
Let me summarize what I think I heard regarding all the hospital’s interactions with the QIO
around [PrU, SCIP/HF, MRSA as applicable]: [summarize]. Is that correct?
4. QIO Impact on Hospital Operations
How valuable to the hospital was the interaction with [name QIO] around [PrU, SCIP/HF,
MRSA as applicable]?
[If valuable:] In what way was it valuable?
[If not valuable:] Why was it not valuable?
Did any of the interactions with [name QIO] around [PrU, SCIP/HF, MRSA, as applicable] lead
to changes in the hospital that ultimately improved patient care?
[If hospital took the HLQAT or AHRQ Pt. Safety survey:] What did you learn from the results?
Have any changes been made as a result of the survey(s) or related follow-up that strengthen
quality or safety at the hospital?
Did any other interactions with the QIO lead to changes in the hospital that ultimately improved
patient care?
F.25
If you were advising [name QIO] about improving the impacts of its interactions with hospitals
like yours, what would you say?
5. Story of Hospital’s Performance Trend on [PrU, SCIP/HF, MRSA]
Could you take us through the story of this hospital’s performance trend on [PrU, SCIP/HF,
MRSA] from the time you first started tracking it? By “story,” we mean what the trends were
and what lay behind them.
If [name measure(s)] improved over time, what did you do?
What motivated you to take these actions that led to improvement?
What knowledge and staff resources made the changes possible?
Do you believe this hospital has now achieved the best possible performance?
[If no] What barriers remain to achieving optimal performance on the [PrU, SCIP/HF, MRSA as
appropriate] measures?
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for this hospital to achieve optimal performance on these
measures? [If yes] What types of help do you need? [If SCIP/HF:] On which measures?
6. Hospital’s Focus on QI for Measures of Interest vs. Other Measures
How much of the hospital’s quality improvement efforts have been focused on the [PrU,
SCIP/HF, MRSA] measures vs. others? [We will provide each visited hospital in advance and
on-site with a list of the specific measures we are referencing.]
[SCIP/HF only:] Within the SCIP/HF measures, have some measures received more focus
on improvement than others? Why?
Are there other measures or quality improvement areas that received more focus than the [PrU,
SCIP/HF, MRSA] measures? Why?
What influences the priority that is given to improving on various measures?
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for this hospital to achieve optimal performance on other
measures you perceive as high priority? [If yes] What types of help do you need? On which
types of measures?
7. Interactions with Outside Organizations/People Regarding QI
What other organizations or people has this hospital interacted with over the past three years
regarding quality or patient safety improvement?
Have any of these had an important influence on the hospitals quality improvement or safetyrelated efforts? [If yes] Which one(s)?
[If yes] Please tell us about what the most important interactions were and what changed as a
result within the hospital.
Do you talk much with other hospitals about their perspectives on quality improvement?
How do these conversations tend to occur, for example, by phoning a friend, chatting at inperson meetings, via one-to-one emails, via group emails/listservs?
[If yes] How motivated to improve quality and safety are hospitals in this state?
[If yes] Do hospitals in this state generally perceive there to be a business case for quality
and safety improvement? [Please elaborate.]
[If yes] What other factors motivate hospitals in this state? Public reporting? Pay-forperformance?
[If no] Are hospitals in this state generally reluctant to share much information about what
they are doing?
F.26
8. Closing
In conclusion, if you were going to advise CMS about how to make the QIO program more
effective in assisting hospitals to improve quality and safety, what would your advice be?
What are the most important targets for improvement?
What are the most useful methods of assistance to focus on?
Turning to efficiency of the program, are there some things that [name QIO] has been doing that
could be done just as well for a larger region than just the state? In other words, could you tell us
whether the state-specific knowledge of [name QIO] was key to its effectiveness in assisting
hospitals over the past few years?
Are there some things that have been done have little value that CMS should ensure QIOs do not
do in the future?
Do you have any other comments on the QIO program or your experiences with [name QIO] that
you would like to share with us?
F.27
F6C. NURSING HOME QUALITY DIRECTORS
The questions that follow apply the same to all nursing homes selected for site visits—those that
worked with the QIO on the Patient Safety Theme, sub-themes Pressure Ulcers, Physical
Restraints, and/or Nursing Homes in Need.
F.28
1.Background Information
Collect descriptive information about the facility prior to visit, including:
Ownership
Profit/non-profit
Size
5-star rating on staffing, deficiencies, and QMs
Pressure ulcer prevalence rates over last three years
Restraint prevalence rates over last three years
Payor mix (% Medicaid; % Medicare)]
2. Overall Quality and Quality Improvement
Where does quality fit into the nursing home’s overall business strategy?
When it comes to how well this nursing home is performing in terms of quality and safety
overall, on a scale of 1-10, where 10 is the best care possible, where would you place it
currently?
Where would you have placed it three years ago?
[If improved:] What have been the main factors that led to the improvement?
[If lower than 7:] What are the main reasons you wouldn’t give your facility a higher score?
3. QIO Interactions
How did this nursing home get involved in quality improvement activities with the QIO that
have been undertaken since August 2008?
Please tell us about your interactions with [name QIO] over the past three years. [capture
frequency and types of interactions, positions/backgrounds of those who attended on both sides]
[Make sure they are thinking about emails they received as well as any in-person or phone
conferences they attended]
Let me summarize what I think I heard regarding your nursing home’s interactions with the QIO
around [PrU and Restraints]: [summarize]. Is that correct?
4. QIO Impact on Nursing Home Operations
How valuable to the facility was the interaction with [name QIO] around [PrU or Restraints, as
applicable]?
[If valuable:] In what way was it valuable?
[If not valuable:] Why was it not valuable?
Did any of the interactions with [name QIO] around [PrU or Restraints, as applicable] lead to
changes in the facility that ultimately improved resident care? If yes, please describe.
If you were advising [name QIO] about improving the impacts of its interactions with facilities
like yours, what would you say?
5. Nursing Home’s Performance Trend on [PrU or Restraints]
Could you provide some history on your facility’s performance on the [PrU or Restraint]
measure(s)? Specifically, how have your rates changed (or not) from the time you first started
tracking it/them?
If [name measure(s)] improved over time, what did you do?
What motivated you to take these actions that led to improvement?
What knowledge and staff resources made the changes possible?
Do you believe this facility has now achieved the best possible performance?
[If no] What barriers remain to achieving optimal performance on the [PrU or Restraint as
appropriate] measures?
F.29
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for your facility to achieve optimal performance on these
measures? [If yes] What types of help do you need?
6. Nursing Home’s Focus on QI for Measures of Interest vs. Other Measures
How much of your quality improvement efforts have been focused on the [PrU or Restraint]
measures vs. others?
Are there other quality measures or quality improvement areas that received more focus than the
[PrU and Restraint] measures? Why?
How do you prioritize your quality improvement activities? What measure or issue is most
important to your facility? Least important? Why?
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for your facility to achieve optimal performance on other
measures you perceive as high priority? [If yes] What types of help do you need? On which
types of measures or quality improvement areas?
7. Interactions with Outside Organizations/People Regarding QI
What other organizations or people has this facility interacted with over the past three years
regarding quality improvement?
Have any of these had an important influence on the facility’s quality improvement efforts? [If
yes] Which one(s)?
[If yes] Please tell us about what the most important interactions were and what changed as a
result within the nursing home.
Do you talk much with other nursing homes about their perspectives on quality improvement?
How do these conversations tend to occur, for example, by phoning a friend, chatting at inperson meetings, via one-to-one emails, via group emails/listservs?
[If yes] How motivated to improve quality are nursing homes in this State?
[If yes] Do nursing homes in this State generally perceive there to be a business case for
quality? [Please elaborate.]
[If yes] What other factors motivate nursing homes in this State? Public reporting? Pay-forperformance? Medicaid payment rates? Survey and certification?
8. Closing
In conclusion, if you were going to advise CMS about how to make the QIO program more
effective in assisting nursing homes to improve quality, what would your advice be?
What are the most important targets for improvement?
What are the most useful methods of assistance to focus on?
Turning to efficiency of the program, are there some things that [name QIO] has been doing that
could be done just as well for a larger region than just the state? In other words, could you tell us
whether the state-specific knowledge of [name QIO] was key to its effectiveness in assisting
nursing homes over the past few years?
Are there some things that have been done that lack value and should be discontinued?
Do you have any other comments on the QIO program or your experiences with [name QIO] that
you would like to share with us?
F.30
F6D. PHYSICIAN PRACTICE – LEAD PHYSICIAN CONTACT
The questions that follow apply to lead physician contacts from practices selected for visit
because they worked with the QIO on the Prevention theme, and/or the Prevention – Disparities
theme. Where indicated in brackets within questions, question wording is slightly tailored to
apply to the Prevention – Disparities theme.
F.31
1. Background Information
[collect description of practice prior to visit, including whether an FQHC or Rural Health Clinic,
whether part of larger medical group or health system, size of practice (number of physicians and
mid-level practitioners), specialties of clinicians, number of Medicare beneficiaries and % of
practice this represents][for disparities, also get % of practice that is underserved, and % with
diabetes]
Does the practice receive any quality report cards from outside organizations?
Does the practice participate in any quality improvement programs or activities, in addition to
the [preventive/diabetes] work you do with [name QIO]?
2. Interactions with the QIO and other CMS Contractors
How did the practice get involved in working with [name QIO] on [preventive/diabetes] care
activities?
What did the practice hope to gain?
[Pre – Disp only:] Are you aware of the diabetes self-management education that the QIO has
been sponsoring?
[Pre – Disp only:] In your view, how well do such programs, and this one in particular, work?
Please tell us about the practice’s [Pre – Disp: other] interactions with [name QIO] over the past
three years. [capture frequency and types of interactions, positions/backgrounds of those who
attended on both sides]
[Make sure they are thinking about emails they received as well as any in-person or phone
conferences they attended]
Please describe your experience with reporting data from your EHR to the CMS contractor on
the [preventive/diabetes] care measures—did all go smoothly?
[Pre – Disp only:] Did you submit blood pressure control data to PQRI? Why or why not? [If
yes] Was it easy to do?
3. QIO Impact on Practice Operations and Patients’ Health
How valuable to the practice was the interaction with [name QIO] around [preventive/diabetes]
care?
[If valuable:] In what way was it valuable?
[If not valuable:] Why was it not valuable?
[Pre – Disp only:] To your knowledge, did any of your patients receive diabetes selfmanagement education because of the QIO’s initiative to increase this? [If yes] About how
many?
[Pre – Disp only:] [If yes] Did you see any improvement in the condition of patients who
attended the training that you believe was attributable to the class?
Did any of the interactions between the practice and [name QIO] around [preventive/diabetes]
care lead to changes in practice operations that improved patient care?
If you were advising [name QIO] about improving the impacts of its interactions with practices
like yours, what would you say?
4. Practice’s Knowledge of its Performance and Trend on [Preventive/Diabetes] Care
Measures
What information does this practice generate or receive about the percentage of practice patients
that have received [measures of interest]?
When and how did the practice first become aware of how its patients were doing on these
measures?
F.32
Since that time, have you taken any particular actions to try to increase these rates? [If yes,]
What were they?
[If yes] What motivated you to take these actions?
How consistently are these actions applied across the practice?
What knowledge and staff (or consultant) resources made the changes possible?
Have you been able to see any trend yet in these measures that would indicate if your actions
were working?
Do you believe this practice has now achieved the best possible performance?
What is the role of your EHR in supporting good performance on [preventive/diabetes] care
measures?
[If no] What barriers remain to achieving optimal performance on the [preventive/diabetes care]
measures?
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for this practice to achieve optimal performance on these
measures? [If yes] What types of help do you need? On which measures?
5. Practice’s Focus on QI for Measures of Interest vs. Other Measures
How much of the practice’s quality improvement efforts have been focused on the
[preventive/diabetes care] measures vs. others? Why? [we will share a list with them of the
specific measures we are referencing]
Within the [preventive/diabetes care] measures, have some measures received more focus on
improvement than others? Why?
Are there other measures or clinical areas that received more focus on improving measured
performance than the [preventive/diabetes] care measures? [If so] Why?
What influences the priority that is given to improving on various measures?
Are there any needs you have from an outside organization such as a Medicare Quality
Improvement Organization in order for this hospital to achieve optimal performance on other
clinical performance measures you perceive as high priority? [If yes] What types of help do you
need? On which types of measures?
6. Interactions with Outside Organizations/People Regarding Performance Measurement
and Improvement
What if any other organizations or people has this practice interacted with over the past three
years regarding performance improvement?
Have any of these had an influence on the practice’s operations? [If yes] Which one(s)?
[If yes] Please tell us about what the most important interactions were and what changed as a
result within the practice.
Do you talk much with other practices about their perspectives on performance measurement and
improvement?
How do these conversations tend to occur, for example, by phoning a friend, chatting at inperson meetings, via one-to-one emails, via group emails/listservs?
Are most practices in this state aware of how they are doing on these [preventive/diabetes]
care measures?
[If yes] How motivated to improve performance on quality measures are practices in this
state at this time? Why?
Do practices in this state generally perceive there to be a business case for them to improve
on these quality measures? [Please elaborate.]
F.33
Aside from any inherent business case and professional integrity, do any other factors
motivate practices in this state? Future or current public reporting? Future or current payfor-performance?
What do you think it would take to get practices to change operations enough to improve the
rates of [preventive/diabetes] care measures in the state to their optimal level?
7. Closing
In conclusion, if you were going to advise CMS about how to make the QIO program more
effective in assisting practices to improve quality of care and safety for their patients, what
would your advice be?
What are the most important targets for improvement?
What are the most useful methods of assistance to focus on?
Turning to efficiency of the program, are there some things that [name QIO] has been doing that
could be done just as well for a larger region than just the state? In other words, could you tell us
whether the state-specific knowledge of [name QIO] or their ability to come on-site was
important to their ability to effectively assist physician practices over the past few years?
Are there some things that have been done have no apparent value that CMS should ensure QIOs
do not do in the future?
Do you have any other comments on the QIO program or your experiences with [name QIO] that
you would like to share with us?
F.34
F6E. COMMUNITY HEALTH LEADERS
The set of questions that follows applies to three “community health leaders” identified for each
case study state. Usually these would be provider or professional association representatives
knowledgeable about quality, with one representing the physician sector, one the hospital sector,
and one the nursing home sector. On occasion, another type of respondent rather than a provider
or professional association representative may be selected to provide an external vantage point,
such as the leader of a large quality alliance in the state.
F.35
1.Background Information
[collect description of the leader’s position and role in health care quality improvement]
2. QIO Program Impact in the State
Could you tell us how and to what extent you know about and/or involved in the work of [name
QIO] as a Medicare QIO over the past few years?
Has the work of [name QIO] over the past few years as a Medicare Quality Improvement
Organization influenced the quality of care in this state? How?
3. Most Effective QIO Activities
Among the different kinds of activities you may be aware of that [name QIO] undertakes in its
role as a Medicare QIO, are there any that you could identify as particularly high-value? Low
value?
Is there anything you think [name QIO] could have done differently to be more effective in its
work that you are aware of? What would it have taken for this to have happened?
4. Quality Improvement in the State
What are the key organizations that influence quality or patient safety improvement activities in
this state? What major contributions has each made in the past three years?
What are the key barriers to further improvements among quality and patient safety in this state?
Are there characteristics of the provider environment in this state that make providers
particularly receptive to attempts to assist them with quality improvement?
Are there characteristics of the provider environment in this state that make it particularly
challenging for an organization like the QIO to assist providers with quality improvement?
5. Closing
If you were going to advise CMS about how to make the QIO program more effective as a
catalyst or technical assistance resource to improve quality and patient safety, what would your
advice be?
What are the most important targets for improvement?
What are the most useful methods of assistance to focus on?
Turning to efficiency of the program, are there some things that [name QIO] has been doing that
could be done just as well for a larger region than just the state? In other words, could you tell us
whether the state-specific knowledge of [name QIO] was key to its effectiveness over the past
few years?
Are there some things that have been done have no apparent value that CMS should ensure QIOs
do not do in the future?
Do you have any other comments on the QIO program or your experiences with [name QIO] that
you would like to share with us?
F.36
F6F. DRUG SAFETY PARTNER ORGANIZATION
The drug safety partner organization may be a health plan, a provider organization, or another
type of organization who has partnered with the QIO to improve drug safety under the QIO’s
Patient Safety theme. The QIO will provide us with contact information to interview the lead
contact from this organization.
F.37
1.Background Information
[collect description of drug safety partner organization and the effort that they worked on with
the QIO in advance of the visit]
2. The Drug Safety Initiative
Where does drug safety as a topic area for improvement fit into this organization’s overall
business strategy?
What influences the priority that is given to improving on various measures of patient safety and
quality?
Please tell us about the drug safety initiative that the QIO has been involved in with you.
[motivators, goals and measures, other partnered organizations, scope of the effort, timeframe,
level of effort over time from the various partners]
3. QIO Interactions
How, why, and when did [name organization] get involved in [name drug safety-related
activities] with the QIO?
Please tell us about [name organization]’s interactions with [name QIO] around drug safety over
the past three years. [capture frequency and types of interactions, positions/backgrounds of those
who attended on both sides]
[Make sure they are thinking about emails and materials they may have received as well as any
in-person or phone conferences they attended]
4. QIO Impact on Drug Safety Initiative and Drug Safety
How valuable to the drug safety initiative was the interaction with [name QIO]?
[If valuable:] In what way was it valuable?
[If not valuable:] Why was it not valuable?
Did any of the interactions with [name QIO] around drug safety lead to changes that have
improved drug safety?
If you were advising [name QIO] about improving the impacts of its interactions with other
organizations in order to improve drug safety, what would you say?
5. Story of the Organization or Initiative’s Performance Trend on Drug Safety Measures
Could you take us through the story of this [organization’s or initiative’s] performance trend on
the drug safety measures you have been tracking, from the time you first started tracking it? By
“story,” we mean what the trends were and what lay behind them.
If [name measure(s)] improved over time, what did you do?
What motivated you to take these actions that led to improvement?
What knowledge and staff resources made the changes possible?
Do you believe this [organization/initiative] has now achieved the best possible drug safety
performance?
[If no] What barriers remain to achieving optimal performance on the drug safety measures?
Are there needs from an outside organization such as a Medicare Quality Improvement
Organization in order for providers in the state to achieve optimal performance on drug safety
measures? [If yes] What types of help are needed? To influence what measures?
6. Interactions with Outside Organizations/People Regarding Drug Safety
What if any other organizations or people has this organization interacted with over the past
three years regarding drug safety, that haven’t yet been mentioned?
Have any of these had an important influence on the organization’s drug safety-related efforts?
[If yes] Which one(s)?
F.38
If yes] Please tell us about what the most important interactions were and what changed as a
result.
7. Closing
In conclusion, if you were going to advise CMS about how to make the QIO program more
effective in assisting other organizations with improving drug safety, what would your advice
be?
What are the most important targets for improvement?
What are the most useful methods of assistance to focus on?
Turning to efficiency of the program, are there some things that [name QIO] has been doing that
could be done just as well for a larger region than just the state? In other words, could you tell us
whether the state-specific knowledge of [name QIO] was key to its effectiveness in assisting with
drug safety over the past few years?
Are there some things that have been done have no apparent value that CMS should ensure QIOs
do not do in the future?
Do you have any other comments on the QIO program or your experiences with [name QIO] that
you would like to share with us?
F.39
APPENDIX G
PARTNERS DISCUSSION GUIDE, SCREENER, AND LETTER
Chronic Kidney Disease (CKD)
QIO Partner Organization Screener Questions/Scheduling Call
I am a researcher with Social & Scientific Systems, a health care research company in Silver
Spring, Maryland. We were given your name by [QIO NAME] as a partner organization in the
effort to improve care for people with Chronic Kidney Disease, which I’ll call CKD, or to
prevent CKD. The Centers for Medicare & Medicaid Services has contracted with Mathematica
Policy Research, Inc. and with Social & Scientific Systems to evaluate the impact of the QIO
program on healthcare processes and outcomes in CKD. With your help, the evaluation will
provide critical information to CMS to help refine its work to improve the quality of care. You
should have received a letter from someone on the research team saying that we would be calling
you—did you receive that letter? I’d like to ask you a few questions about your work with [QIO
NAME].
1. Has your organization worked with [QIO NAME] on issues related to CKD?
[If yes, go to Q2; if no, go to Q4]
2. What is the level of [QIO NAME]’s involvement with the activities of
[ORGANIZATION BEING INTERVIEWED] related to CKD? Would you say it is very
involved, somewhat involved, or only minimally involved?
[IF RESPONSE TO Q2 IS ‘SOMEWHAT’ OR ‘MINIMALLY’ INVOLVED, THEN ASK Q2A
AND Q2B]
Q2A. Please provide a brief overview of how [QIO NAME] has been involved with this
organization’s activities.
2B. Has your level of involvement changed since you first started working with [QIO NAME]?
[If no, go to Q3] If yes, why is that?
a.
b.
c.
d.
Was the work not important or relevant to your organization?
Does your organization not have the resources to collaborate?
Were you already doing that sort of work?
Did you work with [QIO NAME] and not find it useful?
[IF RESPONSE TO Q2 IS ‘VERY INVOLVED’ ASK Q3 AND THEN SCHEDULE FULL
INTERVIEW]
3. How would you describe the value of the [QIO NAME] to this organization’s efforts
related to CKD? Would you say that their involvement has been very valuable, somewhat
valuable, not very valuable, or they’ve added no value at all?
[THANK RESPONDENT FOR THEIR TIME AND END INTERVIEW]
4. Did [QIO NAME] contact your organization or have you interacted at all with [QIO
NAME] about collaborating on issues related to CKD?
G.3
[IF NO, THANK RESPONDENT FOR THEIR TIME AND END INTERVIEW]]
5.
[if yes] Did your organization make a decision not to work with [QIO name}? Why was
that—
a. Was the work not important or relevant to your organization?
b. Does your organization not have the resources to collaborate?
c. Were you already doing that sort of work?
d. Did you work with [QIO NAME] and not find it useful?
[THANK RESPONDENT FOR THEIR TIME AND END INTERVIEW]
G.4
CARE TRANSITIONS THEME
PARTNER ORGANIZATION
INVITATION LETTER
600 Maryland Ave. S.W., Suite 550
Washington, DC 20024-2512
Telephone (202) 484-922
Fax (202) 863-1763
www.mathematica-mpr.com
[Date], 2010/2011
Contact
Organization name
Street address
City, ST zip
Dear [contact name]:
This is an invitation to provide important feedback to the Centers for Medicare & Medicaid Services
(CMS) through agreeing to a telephone meeting, 45 to 50 minutes in length, to assist CMS in evaluating and
improving its Quality Improvement Program. The CMS Quality Improvement Program in this state is
operated through [NAME QIO]. As a partner organization in the effort involving [NAME QIO] improving
care transitions and preventing re-hospitalizations, the evaluation research team would appreciate your
feedback through a telephone meeting during [TARGET WEEK], to be scheduled at your convenience. You
will not need to make any special preparations for the meeting. CMS has contracted with Mathematica Policy
Research, Inc. and with Social & Scientific Systems to conduct the telephone meetings as part of a larger
study to evaluate the 8th and 9th Scopes of Work Quality Improvement Program. A description of the larger
study is attached.
In particular, we would like to discuss:
• Activities of this organizations in the collaborative effort to improve care transitions and prevent rehospitalizations
• Role played by [NAME QIO] in the collaborative effort
• Any changes in care resulting from the work of the collaborative effort
• Most successful strategies or interventions by the collaborative
• Challenges faced and sustainability of the efforts and resulting changes
The information you share with us will be kept strictly confidential and will not affect your current or
future participation in the QIO program. The information gathered in aggregate will provide critical feedback
to CMS to help refine its work to improve the quality of care. Please see the attached letter of encouragement
to participate from CMS. An evaluation staff member from Social & Scientific Systems will call you to
follow up in the next couple of days. In the meantime, please feel free to call or e-mail [CONTACT
INFORMATION OF EVALUATION STAFF MEMBER] with any questions or to initiate scheduling. Thank
you very much in advance for your assistance—your input into the evaluation is highly valued by CMS.
Sincerely,
Myles Maxfield, Ph.D., Project Director
According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a
valid OMB control number. The valid OMB control number for this collection of information is XXX-XXXX. The time required to
complete this information collection is estimated to average 0.8 hours or 48 minutes per response, including the time to review instructions,
search existing data resources, and gather the data needed, and complete and review the information collection. If you have comments
concerning the accuracy of the time estimate(s) or suggestions for improving this form, please write to: CMS, 7500 Security Boulevard,
Attn: PRA Reports Clearance Officer, Mail Stop C4-26-05, Baltimore, Maryland 21244-1850.
An Affirmative Action/Equal Opportunity Employer
Chronic Kidney Disease (CKD) and Care Transitions
QIO Partners Discussion Guide
INTRODUCTION
Thank you for agreeing to participate in this interview to better understand the role of the (QIO
name) in working with the community partners on [CKD/CT]. My name is ______________. I
am a researcher with Social & Scientific Systems, a health care research company in Silver
Spring, Maryland. Our organization has received funding from the Centers for Medicare and
Medicaid Services to evaluate the impact of the QIO program on healthcare processes and
outcomes in [CKD/CT]. Our discussion will provide insights on issues related to the work of the
(QIO name) within your community (state).
Before we begin, are there any questions?
INTERVIEW GUIDE:
QUESTIONS
• TOPIC #1: Activities of the Partner in the Collaborative
CKD Partners: Let’s begin by talking about the role of your organization in the effort to improve the
detection and treatment of CKD.
CT Partners: Let’s begin by talking about the role of your organization in the effort to reduce hospital
readmissions.
1.1
[CKD] Tell me about what your organization is doing as part of the collaborative to
improve the care delivered to people with chronic kidney disease? We’re specifically
interested in activities conducted since summer 2008, both alone and in collaboration
with other organizations.
1.2
[CT] Tell me about what your organization is doing as part of the collaborative to
improve the transition of Medicare patients between care settings in your community, to
help reduce re-hospitalizations? We’re specifically interested in activities conducted
since summer 2008, both alone and in collaboration with other organizations.
1.3
[CKD] Did your work on this collaborative include direct interaction with Medicare
patients with diabetes and their caregivers? [If yes,] Please describe the extent of any
direct interactions.
1.4
[CT] Did your work on this initiative include direct interaction with Medicare patients
and their caregivers? [If yes,] Please describe the extent of any direct interactions.
G.7
• TOPIC #2: Activities of the QIO in the Collaborative
Next, I’d like to talk about the QIO’s role in the collaborative.
2.1
What were the most important things the QIO did to support the work of the
collaborative?
2.2
To what extent was there significant decision-making by the collaborative as a group,
where participating organizations then carried out actions according to the collaborative’s
decisions?
2.3
[If some decisions were made by the collaborative,] Please describe how decisions were
made among partners in this collaboration? For example, who decided on what the
priorities were?
2.4
What role did the QIO play in decision making?
2.5
Would the activities undertaken by the collaborative likely have occurred if the QIO had
not existed?
2.6
If you were starting over as a group, is there anything you would recommend be done
differently regarding who is in the collaborative, or how it functions?
• TOPIC #3: Changes in Care Resulting from the Work of the Collaborative
3.1
[For provider organizations:] Has your organization made any operational changes that
may affect care, as a result of participating in this initiative? [If yes,] When were these
changes made? [If yes,]Would your organization have made these changes without the
support of the QIO?
3.2
[For organizations other than providers:]Has your organization done anything as a result
of participating in the collaborative that you see as critical to care improvement? [If yes,]
Please describe what you did, and when. [If yes,] Why do you view it as critical to care
improvement? [If yes,] Would your organization have made these changes without the
support of the QIO?
3.3
Thinking about the collaborative as a whole, is there any evidence or anecdotes that
would show actual improvements to patient care as a result of the work of the
collaborative?
3.4
[If yes,] Was the QIO’s role in the collaborative important to producing these changes?
[If yes,] How?
3.5
What if any other efforts have been going on at the same time that could also lead to
[improved care for patients with CKD/fewer re-hospitalizations]?
G.8
• TOPIC #4: Strategies or Interventions that Improve Care
Next, let’s talk about interventions and strategies that have been put in place and how data may have
facilitated your work as part of this initiative thus far.
4.1
What strategies or interventions by the collaborative have been most successful thus
far—either leading to changes in care or most promising for care improvement? Why?
4.2
What strategies or interventions by the collaborative have not worked out to be as
successful as hoped, thus far? Why?
4.3
Tell me about the role the data may have played in identifying specific opportunities for
improvement and the selection of interventions?
4.4
Has your organization, and the collaborative as a whole, had timely access to data to
monitor how effective the efforts have been?
4.5
CKD: annual urinary microalbumin testing, use of ACE inhibitors and ARB drugs,
availability of arteriovenous fistula (AV fistula or AVF) at 1st dialysis
4.6
CT: rehospitalization rates, HCAHPS survey results
4.7
What was the QIO’s role in making the data available to you and the collaborative?
4.8
What have you learned from the data you have reviewed?
4.9
Were modifications made to the strategies/interventions as a result of feedback received
from the data? If so, tell me about these changes and when they occurred.
• TOPIC #5: Challenges and Sustainability
Now, I’d like to talk about the remaining challenges the collaborative faces and sustainability of the
progress that has been made.
5.1
Please describe any important remaining challenges to achieving the goals the
collaborative was established to address.
5.2
What if any plans are underway to address these challenges?
5.3
What will be key factors in whether these challenges can be overcome?
5.4
[If changes in care were reported in Topic #3,] How likely do you think it is that the
changes that have been made thus far will be lasting once the QIO is no longer available
as a resource? Why or why not?
5.5
Are there other group(s) that could assume the role(s) of the QIO?
G.9
5.6
Are there any plans to encourage adoption of any successful strategies tried under this
collaborative elsewhere within the state? Tell me about it.
• Closing
In closing,
I’m hearing that [summarize the respondent’s main point about how successful the collaborative
has been to date]. Do I have that right?
Could you summarize what advice you would have for another group that was starting up
with the goals of the collaborative, about what works well and not so well?
Do you have any advice for CMS as it looks to improve the QIO program’s effectiveness
and efficiency going forward?
G.10
APPENDIX H
FOCUS GROUP DISCUSSION GUIDE AND LETTER
SIGN UP FOR A DISCUSSION GROUP ABOUT THE DIABETES CLASSES
Medicare would like to talk to people who took diabetes classes in different places around the
country to find out how helpful they were and ways to improve them.
May we contact you after the diabetes classes are over, to talk in a small discussion group about
how well the classes went and how they might be improved?
If yes, please sign up below and tell us how best to reach you. Those who get invited and come
to the group will receive $50 as a thank you for their help. Not everyone who signs up will be
invited.
If you are selected to be invited, a staff person from the research companies paid by Medicare—
Mathematica Policy Research or Social & Scientific Systems—will contact you.
Name
Phone:
OMB Data Collection No.: xxxx-xxxx
Expiration Date: xx/xx/xxxx
H.3
Focus Groups of Beneficiaries Who Received Diabetes Self‐Management
Education as a Result of QIO Prevention – Disparities Theme Effort
Telephone Recruitment Script
Hello, this is [CALLER NAME]. You signed up to take part in a small discussion group about the
diabetes class that you were part of. Do you remember signing up? [If no, read the following: I
understand that you took a diabetes class. The Medicare program would like to talk to people who
took the diabetes classes to find out how helpful the classes were and ways to improve them. There
was a sign-up sheet for people who would be willing to take part in the discussion group and I have
your name on the list.] We will be having one of the discussion groups in your area on [DATE]. The
discussion will take place from [BEGIN TIME TO END TIME] and it will be at [LOCATION]. If
you take part, we will give you $50 as a thank you for your help. Will you be able to join us at that
time?
[IF ANSWER IS NO, SAY] Thank you for your time and have a nice day. END CALL.
[IF ANSWER IS YES, SAY] We would like to call you the day before the discussion group to make
sure that you will be coming. Is this the best phone number to reach you at? Thank you for your
time and we look forward to seeing you on [DATE]. Have a nice day! END CALL.
H.5
Focus Groups of Beneficiaries Who Received Diabetes Self‐Management
Education as a Result of QIO Prevention – Disparities Theme Effort
Telephone Recruitment Script
SPANISH VERSION
Hola, me llamo [CALLER NAME]. Usted se registró para participar en un pequeño grupo de
discusión sobre la clase de diabetes en la cual tomó parte. ¿Se recuerda que se inscribió? [If no, read
the following: Entiendo que usted tomó una clase sobre la diabetes. El programa de Medicare
quiere hablar con personas que tomaron la clase sobre la diabetes para averiguar si las clases
ayudaron y maneras de mejorarlas. Había una lista para firmar para personas que estaban dispuestas
a tomar parte en el grupo de discusión y su nombre está en la lista.] Tendremos un grupo de
discusión en su área en la fecha [DATE]. La discusión se llevará a cabo desde las [BEGIN TIME]
hasta las [END TIME] y será en [LOCATION]. Si usted toma parte, le daremos $50 (cincuenta
dólares) para agradecerle por su ayuda. ¿Podrá usted reunirse con nosotros a esa hora?
[IF ANSWER IS NO, SAY] Muchas gracias por su tiempo y le deseo un buen día. END CALL.
[IF ANSWER IS YES, SAY] Quisiéramos llamarle el día antes del grupo de discusión para
asegurarnos que usted vendrá. ¿Este es el mejor número d teléfono para alcanzarle? Muchas gracias
por su tiempo y anticipamos verlo(a) el [DATE]. ¡Y que tenga un muy buen día! END CALL.
H.7
Focus Group Guide: Beneficiaries Who Received Diabetes Self‐
Management Education as a Result of QIO Prevention –
Disparities Theme Effort
1. How the Beneficiaries Were Recruited
Goal: Explore how the participants found out about the DSME that they participated in. This is to
help us understand whether participants’ level of interest and engagement varies by recruitment
setting and background of or relationship with person who recruited them.
•
•
•
•
How did you first learn about the diabetes management class?
– Who told you about it—was it your regular doctor, a nurse at the doctor’s office
or clinic, or some other sort of person? If “some other sort of person”, do you
know what that person’s position or job is?
How did you feel about that person suggesting you participate in the class? Did you have
a relationship with that person? Did you trust that person to have your interests in mind
when suggesting you go to the class? Do you think they understood you and your health
issues?
Why do you think they suggested that you participate in the class?
– Did you know at the time that you had diabetes?
– Did you know it was a serious health problem?
– Did you know that there were things you could do to better manage your
diabetes or to prevent health problems from the diabetes?
Is this the first time you had any diabetes teaching?
2. Class Structure, Perceptions about Leader and Other Participants
Goal: Learn about the general set‐up of the class, perceptions of the leader and qualifications,
comfort levels with leader and other participants.
• Can you tell me about the class and how it was set up?
– Was there one teacher or leader? What kind of person was that—was it a nurse,
a social worker, or what kind or training did the person have?
– Did you think that the leader knew a lot about diabetes and how to treat it? Did
they know a lot about how a person with diabetes should take care of
themselves?
– Were they able to explain the things to you in ways that you could understand?
– Do you think they understood you? Did they understand your health issues,
your cultural background, and the problems you might face getting health care
or taking better care of yourself?
• How many participants were in the class? Did you feel comfortable with the other
participants? Do you think they were people like you? How were they alike? How were
they different?
H.9
3. Content of Class
Goal: Focus on what was learned and how it was taught. Purpose is to understand what kind of
format participants prefer.
•
•
•
•
During the class, did the teacher spend most of time talking to you and telling you about
what you should do?
– Did the teacher ask you questions and try to learn about you and your health?
Did they ask you about your worries and problems in getting health care or
taking care of yourself?
– Were there different kinds of activities—did you practice doing things or take
turns talking?
Was it hard for you to understand the things the teacher was telling you or did most
things make sense to you?
Were the classes interesting to you? Do you think that the kinds of things that were
being taught were the right things for a person like you?
Are there ways that you think the classes could have been better for you?
4. Impact of Class
Goal: To understand what the participants got out of the DSME class, whether they are more
knowledgeable, and whether they understand how to put the knowledge into practice.
•
•
•
Do you feel like you learned new things during the classes?
– Do you think you know more about diabetes than you did before the classes?
• Do you understand the risks or what could happen if the diabetes is not
treated?
– Do you think you understand what you can do to get the diabetes under
control?
• Do you understand what tests you need to have done by your doctor or
nurse? Do you understand how often they need to be done?
• How do you know if your diabetes is under control? Do you
check your blood sugar? Do you have a hemoglobin A1C test?
How often do you do these things? What are good test results?
• What other types of things did you learn in the class to prevent
the complications of diabetes? Control your blood pressure?
Have your cholesterol checked? Have your eyes checked?
• Do you understand the kinds of changes you should make in your eating
and in exercise?
– Was most or some of the information new to you about caring for your diabetes
and preventing health problems from diabetes?
Are you doing anything differently now to take care of your health, than you were
before the classes?
• Are you eating less? Are you eating different foods?
• Are you exercising more than you did before?
Are you visiting the doctor more often or less, or getting any different type of medical
care than you were before the classes?
H.10
–
–
For things you are doing differently,
• Have you noticed any effects on your health? In terms of test results?
Weight loss? Feeling better?
• Do you think you would have made these changes without the class?
• Could you have gotten this information and support from somewhere
else?
• Do you think you will be able to continue these changes? Why or why
not? What kinds of help or support do you need to keep these changes
going?
If you are not doing anything differently, why not?
• What kinds of difficulties do you face in terms of getting the medical
care you need, getting exercise, or eating in a more healthy way?
[Discuss to understand extent of any financial, knowledge, and
time barriers]
5. Closing
•
•
Thinking about the classes you attended, is there anything you would say to the people
that organized the classes about how to make them more helpful to people with
diabetes?
Is there anything you would say to the people that organized the classes about what
they should not change?
H.11
Focus Group Guide: Beneficiaries Who Received Diabetes Self‐
Management Education as a Result of QIO Prevention –
Disparities Theme Effort
Spanish Version
1. How the Beneficiaries Were Recruited – Como los Beneficiarios Fueron Reclutados
Goal: Explore how the participants found out about the DSME that they participated in. This is to
help us understand whether participants’ level of interest and engagement varies by recruitment
setting and background of or relationship with person who recruited them.
• ¿Cómo aprendió o escucho por primera vez sobre la clase de cómo manejar/controlar el
diabetes?
– ¿Quién le dijo—fue su doctor o médico regular, una enfermera en la oficina del
doctor o en su clínica o consultorio o algún otro tipo de persona? Si fue “algún
otro tipo de persona”, ¿sabe usted cuál es la posición o el trabajo de esta
persona?
• ¿Cómo se siente usted acerca de que esa persona le hizo la sugerencia que usted
participara en la clase? ¿Tiene usted una relación con esa persona? ¿Confía usted en
que esa persona tiene los intereses de usted en mente cuando le sugiere que usted vaya
a la clase? ¿Usted cree que él o ella le entendía a usted y a su situación de salud?
• ¿Por qué piensa usted que le sugirieron que usted participe en esa clase?
– ¿Sabía usted en ese tiempo que usted tenía diabetes?
– ¿Sabía usted que era un serio problema de salud?
– ¿Sabía usted que habían cosas que usted podía hacer para manejar/controlar
mejor a su diabetes o para prevenir problemas de salud causados por diabetes?
• ¿Es esta la primera vez que ha tenido alguna enseñanza sobre diabetes?
2. Class Structure, Perceptions about Leader and Other Participants – La Estructura de la Clase,
Percepciones sobre (el/la) Líder y los Otros Participantes
Goal: Learn about the general set‐up of the class, perceptions of the leader and qualifications,
comfort levels with leader and other participants.
• ¿Me puede contar sobre la clase y cómo estaba organizada?
– ¿Había un maestro o líder? ¿Qué tipo de persona era – era una enfermera,
trabajadora o asistente social (“social worker”) o qué tipo de entrenamiento
tenía la persona?
– ¿Cree usted que (el/la) líder sabía mucho sobre diabetes y de cómo tratar a esta
enfermedad? ¿Sabía bastante sobre cómo una persona con diabetes debería
cuidarse?
– ¿Pudieron explicar las cosas en forma que usted podía entender?
– ¿Cree que (lo/la) entendían a usted? ¿Entendían a su situación de salud, sus
antecedentes culturales y los problemas que usted podía confrontar en obtener
servicios de salud (‘healthcare’) o para cuidarse mejor a sí mismo(a)?
H.13
•
¿Cuántos participantes habían en la clase? ¿Usted se sentía cómodo(a) con los otros
participantes? ¿Usted cree que eran personas como usted? ¿Cómo eran parecidos a
usted? ¿Cómo eran diferentes?
3. Content of Class – Contenido de la Clase
Goal: Focus on what was learned and how it was taught. Purpose is to understand what kind of
format participants prefer.
• Durante la clase, ¿el maestro (o la maestra) pasaba la mayor parte del tiempo hablando
con usted y diciéndole lo que usted debería hacer?
– ¿El maestro (o la maestra) le hizo preguntas y trató de aprender sobre usted y
de su salud? ¿Le preguntaron sobre sus preocupaciones y problemas en obtener
servicios de salud (‘healthcare’) o en cuidarse a sí mismo(a)?
– ¿Habían diferentes tipos de actividades – ustedes practicaron cómo hacer cosas
o tomaron turnos para hablar?
• ¿Era difícil para usted entender las cosas que el maestro (o la maestra) le estaba
diciendo o la mayor parte de las cosas tenían sentido para usted?
• ¿Las clases eran interesantes para usted? ¿Usted cree que el tipo de cosas que estaban
enseñando eran las cosas correctas para una persona como usted?
• ¿Usted cree que hay maneras en las que las clases pudieran haber sido mejores para
usted?
4. Impact of Class – Impacto de la Clase
Goal: To understand what the participants got out of the DSME class, whether they are more
knowledgeable, and whether they understand how to put the knowledge into practice.
• ¿Se siente usted como si aprendió cosas nuevas durante las clases?
– ¿Usted piensa que usted sabe más sobre diabetes de lo que sabía antes de las
clases?
• ¿Usted entiende los riesgos o lo que puede pasar si la diabetes no es
tratada?
– ¿Usted piensa que entiende lo que puede hacer para poner a la diabetes bajo
control?
• ¿Usted entiende que pruebas o exámenes usted necesita que le hage su
doctor o enfermera? ¿Usted entiende con qué frecuencia necesiatan
hacerlas?
• ¿Cómo sabe si su diabetes está bajo control? ¿Usted chequea el
azúcar de su sangre? ¿Tiene usted una prueba de hemoglobina
A1C? ¿Con qué frecuencia hace estas cosas? ¿Cuáles son
resultados buenos de las pruebas?
• ¿Qué otros tipos de cosas usted aprendió en la clase para
prevenir las complicaciones de la diabetes? ¿Control de su
presión arterial? ¿Chequear su colesterol? ¿Chequear sus ojos?
• ¿Usted entiende los tipos de cambios que usted debería de hacer en sus
comidas y en ejercicio?
H.14
–
•
•
¿La mayoría o parte de la información sobre el cuidado de su diabetes y la
prevención de problemas de salud causadas por diabetes era nueva para
usted?
¿Usted está haciendo algo diferente ahora para cuidar a su salud, de lo que hacía antes
de las clases?
• ¿Usted está comiendo menos? ¿Está comiendo comidas
diferentes?
• ¿Usted está haciendo más ejercicio de lo que hacía antes?
¿Usted está visitando al doctor con más frecuencia o menos, o está obteniendo algún
tipo diferente de atención médica de lo que tenía antes de las clases?
– Para las cosas que está haciendo de forma diferente:
• ¿Ha notado algún efecto sobre su salud? ¿En término a los resultados
de pruebas o exámenes? ¿Ha bajado en peso? ¿Se siente mejor?
• ¿Usted piensa que pudiera haber hecho estos cambios sin la clase?
• ¿Usted pudiera haber conseguido esta información y este apoyo de
algún otro sitio?
• ¿Usted piensa que va a poder continuar con estos cambios? ¿Por qué sí
o por qué no? ¿Qué tipo de ayuda o apoyo necesita usted para
mantener estos cambios en marcha?
– Si usted no está haciendo nada diferente, ¿por qué no?
• ¿Qué tipo de dificultades enfrenta en términos de conseguir la atención
médica que usted necesita, hacer ejercicio o comer en forma más
saludable?
[Discutir para entender el grado de barreras económicas, de
conocimiento y de tiempo]
5. Closing‐ Final
• Pensando de las clases a las que usted asistió, ¿hay algo que usted le diría a la gente que
organizó las clases sobre cómo hacerlas de más ayuda para personas con diabetes?
• ¿Hay algo que usted le diría a la gente que organizó las clases sobre lo que no deben
cambiar?
H.15
| File Type | application/pdf |
| File Title | Program Evaluation of the Eighth and Ninth Scope of Work Quality Improvement Organization Program: Supporting Statement for Pape |
| Subject | QIO |
| Author | Martha Kovac, Sue Felt-Lisk, Arnold Chen, John Hall |
| File Modified | 2010-03-18 |
| File Created | 2009-05-12 |