CMS-10360.REVISED Supporting Statement Part A_080511

SUPPORTING STATEMENT
Part A

Consumer Research on Public Reporting of Hospital Quality Measures

Version 4
August 5, 2011

Division of Consumer Assessment & Plan Performance (DCAPP)
Centers for Medicare & Medicaid Services (CMS)

Table of contents
A. Background ................................................................................................................1
B. Justification ................................................................................................................5
1. Need and Legal Basis.............................................................................................5
2. Information Users ..................................................................................................5
3. Use of Information Technology .............................................................................5
4. Duplication of Efforts ............................................................................................6
5. Small Businesses ....................................................................................................6
6. Less Frequent Collection .......................................................................................6
7. Special Circumstances ...........................................................................................6
8. Federal Register/Outside Consultation ..................................................................6
9. Payments/Gifts to Respondents .............................................................................7
10. Confidentiality .....................................................................................................7
11. Sensitive Questions ..............................................................................................7
12. Burden Estimates (Hours & Wages) ....................................................................7
13. Capital Costs ........................................................................................................8
14. Cost to Federal Government ................................................................................8
15. Changes to Burden ...............................................................................................9
16. Publication/Tabulation Dates ...............................................................................9
17. Expiration Date ....................................................................................................9
18. Certification Statement………… …………..……………………………… ….9
Attachments

A. Background
One of the primary missions of the Centers for Medicare & Medicaid Services (CMS) is to
improve the quality and efficiency of care in the Fee-for-Service (FFS) program. One of the
several vehicles used for this mission is the public reporting of quality, efficiency and cost
information about hospital care on the Hospital Compare website. This vehicle also serves
to provide Medicare beneficiaries and other consumers with the type of data needed to make
informed decisions about which providers (in this case, hospitals) to use for their care.
Addition of New Quality Information
Organization and Display of New Quality Measures
CMS often adds important new information about quality of care provided in hospitals on the
Hospital Compare website. As new quality measures are added, the website programmers are
faced with deciding how to best integrate the new information into the existing website so
that it can be easily accessed and understood by consumers. In 2011, CMS will be adding
several new measures to Hospital Compare. These new measures will introduce new topics
and concepts for consumers and will also use new types of data, than what currently exists on
the website. New topics will include: individual measures and composite measures around
patient safety, hospital acquired conditions and healthcare associated infections; individual
measures around timelines of care provided in the emergency department; and individual and
composite measures around the effectiveness of care for heart attack, heart failure, and
surgical care. The addition of these new measures raises concern for CMS in how consumers
will interpret and be able to use the information. To address these concerns CMS would like
to conduct formative research with consumers around the most effective way to incorporate
this new information into the website.
Healthcare Associated Infections and Hospital Acquired Conditions
As mentioned above, CMS plans to add new quality information to the Hospital Compare in
2011. Some of that information will include patient safety measures in the area of hospitalacquired conditions (HACs). The introduction of this information comes on the heels of
recent legislation around Medicare reimbursement for hospital care. Specifically, the Deficit
Reduction Act (DRA), which was signed in February of 2006, contains language creating a
system for quality adjustment of Medicare payments for inpatient hospital services. The law
required the Secretary of Health and Human Services (HHS) to identify at least two hospitalacquired conditions (HAC) which could have reasonably been avoided through the
application of evidence based guidelines and would be subject to the adjustment in payment.
On July 31, 2008, in the Inpatient Prospective Payment System (IPPS) Fiscal Year (FY) 2009
Final Rule, CMS included 10 categories of conditions that were selected for HAC payment
provision. Similarly, based on HHS’ priority on reducing Healthcare Associated Infections
(HAIs), CMS published its final rule in July 2010 for Medicare reimbursement for hospitals,
which included the provision that hospitals would start reporting certain HAIs. This reporting
will be a part of the CMS Hospital Inpatient Quality Reporting Program, an initiative
1

intended to equip consumers with quality of care information to make more informed
decisions about their health care, while encouraging hospitals and clinicians to improve the
quality of inpatient care provided to all patients.
Consumer Research
Prior to publicly reporting new information in 2011, CMS proposes to conduct consumer
research to ensure that the information is presented in a way that is consumer friendly and
understandable for consumers to use when making health care decisions. CMS has contracted
to conduct exploratory or formative research around how to best organize and display the
new quality information, to assist CMS in developing consumer friendly displays of the
information and to gather individual feedback about some of the specific measures to be
added to Hospital Compare, using intensive individual interviews.
Formative/Exploratory Research
Given that the information presents new concepts around healthcare associated infections
(HAIs) and hospital acquired conditions (HACs), information which does not currently exist
on Hospital Compare, CMS proposes to conduct a round of formative research to explore
consumers’ perceptions, knowledge and understanding of this new topic and to also explore
options for effectively integrating the new information into the website, through the existing
organizational scheme or potentially through a revised schema. The research team will
conduct the exploratory research using focus groups. Focus groups are an effective tool for
eliciting individuals’ perspectives and examining the differences in their opinions and
reactions. Focus groups will allow the research team to gather baseline information from a
cross section of potential users, get multiple points of view at one time, and stimulate
discussion regarding perceptions and attitudes around the topics, and explore options for
language, display, and navigation through group discussion and brainstorming.
Specifically, the research team conducted a series of focus group discussions with consumers
and informal caregivers (in particular, family members or friends who help care for Medicare
beneficiaries). The consumer participants will range in age from 40 to 70 years of age and
will include persons with a mix of hospital experience (persons who have had a recent
hospital stay and persons who anticipate a hospital stay in the near future); the informal
caregiver participants will range in age from 18 to 70 years of age and help care for persons
who have had a mix of hospital experience.
Intensive Individual Interviews
Subsequent to the exploratory or formative research, the research team proposes to design
and gather feedback about mock data displays and explanatory language for the new quality
and safety measures using intensive individual interviews. The interviewer, for example, will
show subjects one of the new HAC and HAI measures, such as Central Line-Associated
Bloodstream Infections (CLABSI). The interviewer will explore how the research subjects
cognitively approach the topic of HAIs/HACs, how salient the information is to their
healthcare decisions, and how they react to the new information. In particular, the
2

interviewer will look for misinterpretations, unintended reactions or confusion, and changes
to the display or language that might improve them.
The goal of gathering such individual feedback about the mock-ups will be to determine how
to best integrate the new information into the existing website using the display devices
similar to those used for extant measures, while presenting the information clearly and in
such a way that consumers can draw accurate and useful inferences from the data.
The team proposes using one-on-one intensive interviews for this research, in order to gain
more in-depth feedback from individuals. Intensive individual interviews can yield detailed,
more nuanced information because of the focused time and rapport established with an
individual. Our team often uses intensive individual interviews to understand how individuals
understand, process, and respond to specific materials. We have found that issues related to
comprehension or interpretation of complex technical information is better probed in one-onone settings, as the dynamics of focus groups can make it difficult to tease out variations
among individuals. The team will conduct two rounds of intensive individual interviews on
the new information, utilizing a second round to fine tune the mock displays and to make
useful recommendations to the website programmers.
Specifically, the research team will conduct intensive individual interviews with hospital
patient consumers, informal family caregiver consumers and physicians. The hospital patient
consumer participants will range in age from 40 to 70 years of age and will include persons
with a mix of hospital experience (persons who have had a recent hospital stay and persons
who anticipate a hospital stay in the near future); the informal family caregiver consumer
participants will range in age from 18 to 70 years of age and help care for persons who have
had a mix of hospital experience; and the physician participants will include surgeons,
hospital internists, and primary care physicians who commonly refer patients to hospitals.
The consumer and physician interviews are considered to be separate information collections,
as there will be two separate interview guides.
The purpose of the consumer research will be to gain feedback from subjects who may
reflect the population of users of the website, without attempting to definitively determine
how any particular population of users would react to and use the site. That is, the team is
not attempting to achieve an understanding that can be generalized, but to identify particular
issues or concerns that should be avoided in the presentation of the data. Because this study
is qualitative, the data collection will employ convenience samples of individuals from
different segments of the population. An estimated breakdown of research participants is
displayed in exhibit 1.
Exhibit 1.
Research Round/Participant Type

Formative/Exploratory
Focus Groups

Intensive
Individual
Interviews

20

6-8

Patient consumers who have

3

Follow-up
Intensive
Individual
Interviews
6-8

Research Round/Participant Type

Formative/Exploratory
Focus Groups

Intensive
Individual
Interviews

Follow-up
Intensive
Individual
Interviews

20

6-8

6-8

0

6

6

40

20

20

had/anticipate having a hospital stay
within the next six months
Informal family caregivers of patients
who have had/anticipate having a
hospital stay within the next six
months
Physicians with some knowledge
and/or experience with these types of
infections
Total Research Participants

We will work with established, reliable market research firms to enlist the necessary
participants, providing them tailored screeners for the recruitment process. We have learned
from previous experience that engaging the assistance of research firms is advantageous not
only for ensuring access to a reliable pool of potential research participants but also for the
added benefit of access to facilities designed for conducting qualitative research. We have
developed ongoing relationships with several market firms that staff skilled recruiters and
project managers, experienced in identifying appropriate research participants, coordinating
the research schedule, and providing overall management of the research logistics. These
firms maintain large databases of various populations, including healthcare consumers,
patients, informal caregivers, and medical professionals. The firms also provide access to indepth interview and focus group rooms, including one-way mirrors for observing the
research. We have developed recruitment screeners that include a detailed script along with
specific criteria to use when contacting potential participants. The scripts include screening
questions designed to target the specific audience required for this research, including such
criteria as gender, age, race and ethnicity, and level of education. Our screening tools also
take into account certain exclusion criteria, screening out, for example, individuals who work
in the health care industry, to avoid response bias.
For each information collection request, the following data collection tools will be submitted:
1) Recruitment screening forms to be used by the research facilities to match the
participant pool to the recruitment goals of the research team (Attachment A);
2) Semi-structured interview protocols for patient consumers, informal family
caregivers, and physicians (Attachments B);
3) Mock ups of data displays and narrative content for public reporting of quality and
safety measures (Attachment C); and
4) Consent form (Attachment D
4

This project will support CMS’s efforts to understand how diverse populations react to and
utilize consumer information. The research shall be conducted in three different sites, to
include locations with diverse population characteristics and varying levels of hospital
choice. The first information collection will occur in June/July 2011, in order to meet Agency
deadlines for adding the new information to Hospital Compare by December 2011.

B. Justification
1.

Need and Legal Basis

As noted above, CMS is committed to being transparent and to making publicly reported
information useful to consumers. The work under this collection of information is needed to
insure the new rates of hospital acquired conditions and healthcare-associated infections and
other information meet those goals.
2.

Information Users

The information that is gathered during the formative research will be used to develop draft
displays of the new quality information and accompanying explanatory information for
consumer use. The research team will create paper-based mockups of the information to be
tested with research participants, during the second and third rounds of research. The
information that is collected from those interviews will be used to revise mockups of displays
and explanatory language for a final round of consumer testing. Areas of concern identified
during consumer testing will be presented with options and pros and cons for leadership
decision-making in a research report. For example, if two different display options have both
advantages and disadvantages, the report describing results of this research shall attempt to
provide leadership with a basis for making a decision. The format of the report will include
screenshots and detailed instructions to help website programmers accurately implement the
recommendations.
Each research participant will participate in one focus group or one intensive individual
interview session. Once the interview data are collected, they will be transcribed. A process
of qualitative analysis will be used to identify common themes across multiple audiences.
Transcripts, video and audio recordings will be reviewed to insure that no biases were
introduced (e.g., from leading questions) and that alternate interpretations of the data are
considered.
3. Use of Information Technology
This data collection will utilize digital recording technology to collect, store, and manage the
interview data. The research team has established procedures for ensuring the protection of
confidentiality for participants who have contributed personal information to a study,
including removing all names from the interview notes and assigning each participant a code
or pseudonym and removing major identifying details, such as participants’ surnames and
5

addresses, from any notes or records obtained from the screening process. Also, all of the
data will be destroyed within a period of one year after the research team has completed the
final report.
4. Duplication of Efforts
An extensive review of the literature will be conducted prior to commencing the consumer
research. Most of the information is new to public reporting and/or has not been reported as
quality information. Hence, while prior research provides some guidance on how to present
these data, additional research is needed to insure the data are accurately interpreted and
effectively used.
5. Small Businesses
There may be a limited number of physician participants who may own or work for a small
physician practice. Overall, however, this information collection does not involve or impact
small businesses or other small entities.
6. Less Frequent Collection
This is a one-time data collection.
7. Special Circumstances
This request is consistent with the general information collection guidelines of 5 CFR
1320.5(d)(2). No special circumstances apply.
8. Federal Register Notice and Outside Consultations
A Federal Register Notice was posted in November, 2010. CMS has committed itself to
publicly reporting new hospital quality information on Hospital Compare when it updates the
website in January 2012. Workflows for measure developers and website programmers
require that consumer testing recommendations be available in July 2011 to meet this
deadline.
Therefore, CMS published the Federal Register notice in November 2010, for a 60-day
comment period, prior to submission to OMB. The team did not receive public comments in
response to that notice. The team then completed a revised signature package, based on
updates from internal measurement and policy experts. The team published the revised
package in the Federal Register in February 2011, for a 30-day comment period. The team
did not receive any additional comments received during the 30-day comment period.
A stakeholder workgroup was convened by the contractor research team in March 2011 to
obtain feedback on the research approach from a diverse set of stakeholders. This workgroup
consisted of representatives of consumer advocacy organizations, hospital associations,
physicians, foundations that support public reporting of healthcare quality and quality
improvement, business associations, insurers and other purchasers. The contractor research
6

team will consider feedback received from the workgroup as they provide CMS with
feedback.
9. Payments/Gifts to Respondents
Intensive individual interview subjects will receive a stipend, based on the rates set by OMB.
These rates are currently set at 40 dollars for an hour long interview with consumers and 75
dollars for professionals. As the professionals being sought are physicians representative of
clinicians in a community, CMS may request a higher stipend rate from OMB to avoid
recruiting a biased sample.
10. Confidentiality
Individuals and organizations will be assured of the confidentiality of their replies under
Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the
purposes for which the information is collected and that, in accordance with this statute, any
identifiable information about them will not be used or disclosed for any other purpose.
Information that can directly identify the respondent, such as name and age, will be collected
by the research facilities. However, only the age and first name of each participant shall be
passed on to the CMS research contractor. Participant confidentiality will be protected by
de-identifying audio data upon transcription. All data will be stored on encrypted and
password protected files. Data will only be presented in aggregate and de-identified format.
Only the principal investigator and research staff necessary to conduct research will have
access to the data. All research data is maintained in locked cabinets within a locked data
storage area. All electronic files are password and encryption protected. All activities stated
in this project will be performed in concordance with the Health Insurance Portability and
Accountability Act (HIPPA) Privacy Rule, 45 CFR Parts 160 and 164. Informed consent
will be obtained from all who participate in the semi-structured interviews. (See Attachment
D).
11. Sensitive Questions
Interviews will be conducted in private with an individual focus group moderator or
interviewer and will focus on soliciting feedback on the quality information and measure
displays, rather than on any personal or sensitive information. However, interviewees
appearing distressed or confused by the interview shall be provided any additional
information or a referral to medical or counseling help as needed. Interviewees who wish to
investigate CMS consumer information further will be given links and/or phone numbers as
necessary.
12. Burden Estimates (Hours & Wages)
Over the course of the project, CMS anticipates screening 120 individual for 5 minutes with
phone calls to schedule 60 individuals for interviews and focus groups. Forty (40) consumer
individuals will participate in 1 ½ hour focus groups, 14 consumers and 6 physicians will
participate in 1 ½ hour intensive individual interviews. The burden estimates in terms of
7

hours and costs that are anticipated over the course of the project are listed below. Detailed
estimates for each information collection will be provided at the time each is submitted.
Exhibit on estimated annualized burden hours
Data Collection Mode
Screening form
*Focus group discussions
*Intensive Individual Interviews

Number of
respondents
80
40
20

Number of
responses per
respondent
1
1
1

Hours per
response
0.083
3
3

Total
burden
hours
6.64
120
60

Total
140
186.64
* Includes 1.5 hours for research and 1.5 hours assumed time required to travel to and from focus group
facility.

Exhibit on estimated annualized cost burden
Total cost
burden
Screening form
$20.90*
$139
Focus Group Discussions
$20.90*
$2508
$878 + 1902=
Intensive Individual Interviews
20
60
$20.90/$105.66*
2780
Total
60
186.64
$5427
* Hourly wage based on the mean hourly wage estimates from the May 2009 National Occupational
Employment and Wage Estimates, U. S. Bureau of Labor Statistics. Provider hourly wage based on mean wage
estimates for surgeons from the May 2009 Occupational Employment and Wages, Occupational Employment
Statistics, U.S. Department of Labor, Bureau of Labor Statistics.
Data Collection Mode

Number of
respondents
80
40

Total burden
hours
6.64
120

Hourly Stipend

13. Capital Costs
There are no direct costs to respondents other than their time to participate in the study.
14. Cost to Federal Government
Exhibit 4 shows the estimated cost to the Federal government for this six month project. The
total cost is $204,000. This amount includes all direct and indirect costs of the design, data
collection, analysis, and reporting phase of the study.
Exhibit 4. Estimated Cost

Cost Component
Project Development
Data Collection Activities
Data Processing and Analysis
Reporting of Results
Project Management

Estimated
Total Cost
15,000
75,000
24,000
15,000
15,000
8

Overhead
Total

60,000
204,000

15. Changes in Hour Burden
This is a new collection of information.
16. Publication/Tabulation Dates
Timeline

2010

2011

Q4
Nov

CMS
submits
OMB
"Clearance
Package" to
OMB

X

Federal
Register
Notice (60
day
comment
period)

X

CMS
revises
OMB
“Clearance
Package” t
o OMB
Federal
Register
Notice (30
day
comment
period)

Q1
Dec

Jan

Feb

2012

Q2
Mar

Apr

Q3

May

Jun

Jul

Aug

Q4
Sep

Oct

Nov

Q1
Dec

X

X

Consumers
Research

X

X

X

Publication
of data on
Hospital
Compare

X

17. Expiration Date
CMS would like to display the expiration date. It does not seek an exemption.
18. Certification Statement
9

Not applicable.
ATTACHMENTS that will be provided for each IC
Attachment A – Screening forms
Attachment B – Intensive Individual interview guides
Attachment C – Testing Materials/Mockups
Attachment D – Consent form

10