May 2015
National Implementation of the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) Survey
CMS-10102 (OMB 0938-0981)
OMB Supporting Statement - Part A
Prepared by
Division of Consumer Assessment & Plan Performance
Centers for Medicare & Medicaid Services
7500 Security Boulevard
Baltimore, MD 21244
Section Page
Terms of Clearance…………………………………………………………………… 3
Background…………………………………………………………………… 4
Justification…………………………………………………………………… 5
Need and Legal Basis…………………………………………….……… 5
Information Users………………………………………………………... 6
Use of Information Technology………………………………………….. 6
Duplication of Effort……………………………………………………… 7
Small Businesses…………………………………………………………. 8
Less Frequent Collection…………………………………………………. 8
Special Circumstances……………………………………………………. 9
Federal Register/Outside Consultation……………………………………. 9
Payments/Gifts to Respondents…………………………………………… 9
Confidentiality…………………………………………………………….. 9
Sensitive Questions……………………………………………………….. 9
Burden Estimates (Hours & Wages)……………………………………... 10
Capital Costs……………………………………………………………… 10
Cost to the Federal Government………………………………………….. 11
Changes to Burden………………………………………………………… 11
Publication/Tabulation Dates……………………………………………… 11
Expiration Date……………………………………………………………. 12
Certification Statement…………………………………………………….. 12
EXHIBITS
Exhibit A-12: Annual Hours/Cost Burden of the HCAHPS Survey….…………… 10
LIST OF ATTACHMENTS
Attachment A - HCAHPS Survey Instrument (Mail) and Supporting Material
Attachment B - Crosswalk: Non-substantive Changes Made to the HCAHPS 32 Question
Survey Instrument
Attachment C - HCAHPS Mode Experiment III
OMB SUPPORTING STATEMENT – Part A:
National Implementation of the Hospital CAHPS Survey
CMS-10102 (OMB 0938-0981)
The following bulleted items present the major results of the mode experiment. Please see Attachment C – HCAHPS Mode Experiment III for detailed analysis and results of the mode experiment III. There are 9 sections to the mode experiment report numbered Appendices A through I. Each appendix provides methods, results and conclusions.
The distributions of HCAHPS items are as expected, with the exception that self-reported emergency room admission (ERA) rates notably exceeded the rates seen for the administrative measures of emergency room admission.
The Coordination of Care (CoC) measure shows no evidence of a ceiling effect. The high Cronbach’s alpha suggests that the CoC measure has very good internal consistency reliability. The moderately high association with hospital rating and recommendation are evidence of validity and suggest that patients value good coordination of care. The moderate correlations with other HCAHPS measures indicate that the CoC composite is not redundant with other HCAHPS measures. The strong association of CoC with Communication with Nurses and Communication about Medicine suggests the importance of coordination of care to those domains.
The associations of Mental Health Perception (MHP) and ERA with HCAHPS items are in the direction expected. Future research will inform the advisability of these items as patient-mix adjustors.
The mode effect results for CoC are similar to what has been observed previously in Mode Experiment 1.
The new CoC measure has psychometric properties as good or better than current HCAHPS measures, having both interclass correlation coefficient (ICC) and reliability (at n=130) above the median with respect to the ten current measures.
Overall, ERA and MHP have very little impact on informativeness. Thus, we recommend against addition of either item as a patient-mix adjuster (PMA). However, CMS will continue collection of these variables for analysis and oversight purposes.
Overall, ERA and PMA have little effect on hospital scores. Furthermore, there are concerns about the validity of ERA because it is believed to be over-reported by patients, and MHP is moderately correlated with General Health Perception (GHP) and varies little from hospital to hospital. For these reasons, we recommend against addition of ERA or MHP as PMA.
Findings are similar to Mode Experiment I, with CATI (phone) having a larger adjustment for the new measure Coordination of Care.
Since the interactions of CATI by hospital and Mixed (mail with phone follow-up) mode by hospital are not significant there is no evidence that modes CATI and Mixed vary much from hospital to hospital with respect to CoC. However, the Interactive Voice Response (IVR) variance component is significant indicating some variability in IVR mode from hospital to hospital.
The HCAHPS (Hospital Consumer Assessment of Healthcare Providers and Systems) Survey is the first national, standardized, publicly reported survey of patients’ perspectives of their hospital care. HCAHPS is a 32-item survey instrument and data collection methodology for measuring patients’ perceptions of their hospital experience. Since 2008, HCAHPS has allowed valid comparisons to be made across hospitals locally, regionally and nationally.
Three broad goals have shaped HCAHPS. First, the standardized survey and implementation protocol produce data that allow objective and meaningful comparisons of hospitals on topics that are important to consumers. Second, public reporting of HCAHPS results creates new incentives for hospitals to improve quality of care. Third, public reporting enhances accountability in health care by increasing transparency of the quality of hospital care
provided in return for the public investment.
The HCAHPS Survey asks recently discharged patients about aspects of their hospital experience that they are uniquely suited to address. The core of the survey contains 21 items that ask “how often” or whether patients experienced a critical aspect of hospital care, rather than whether they were “satisfied” with their care. Also included in the survey are four screener items that direct patients to relevant questions, five items to adjust for the mix of patients across hospitals, and two items that support Congressionally-mandated reports. (See Attachment A: HCAHPS Survey Instrument (Mail) and Supporting Materials.)
Since March of 2008, results from the HCAHPS survey have been publicly reported on the Hospital Compare website which can be found at www.hospitalcompare.hhs.gov or through a link on www.medicare.gov. The HCAHPS Survey and its implementation protocols can be found in the current version of the HCAHPS Quality Assurance Guidelines (Version 10.0, March 2015), located at: http://www.hcahpsonline.org/qaguidelines.aspx.
B. Justification
1. Need and Legal Basis
Beginning in 2002, CMS partnered with the Agency for Healthcare Research and Quality (AHRQ), another agency in the federal Department of Health and Human Services, to develop and test the HCAHPS Survey. AHRQ and its CAHPS Consortium carried out a rigorous and multi-faceted scientific process, including a public call for measures; literature review; cognitive interviews; consumer focus groups; stakeholder input; a three-state pilot test; extensive psychometric analyses; consumer testing; and numerous small-scale field tests. CMS provided three separate opportunities for the public to comment on HCAHPS and responded to over a thousand comments. The survey, its methodology and the results it produces are in the public domain.
In May 2005, the HCAHPS Survey was endorsed by the National Quality Forum, a national organization that represents the consensus of many healthcare providers, consumer groups, professional associations, purchasers, federal agencies, and research organizations. In December 2005, the federal Office of Management and Budget gave its final approval for the national implementation of HCAHPS for public reporting purposes. CMS implemented the HCAHPS Survey in October 2006 and the first public reporting of HCAHPS results occurred in March 2008.
Enactment of the Deficit Reduction Act of 2005 created an additional incentive for acute care hospitals to participate in HCAHPS. Since July 2007, hospitals subject to the Inpatient Prospective Payment System (IPPS) annual payment update provisions ("subsection (d) hospitals") must collect and submit HCAHPS data in order to receive their full annual payment update.
The incentive for IPPS hospitals to improve patient experience was further strengthened by the Patient Protection and Affordable Care Act of 2010 (P.L. 111-148), which specifically included HCAHPS performance in the calculation of the value-based incentive payment in the Hospital Value-Based Purchasing program beginning with October 2012 discharges.
2. Information Users
As noted above, there are three broad goals of the HCAHPS Survey. These goals are of value to consumers and providers of health care services as well as to CMS. First, the standardized survey and implementation protocol produce data that allow objective and meaningful comparisons of hospitals on topics that are important to consumers. Second, public reporting of HCAHPS results creates new incentives for hospitals to improve quality of care. Third, public reporting enhances accountability in health care by increasing transparency of the quality of hospital care provided in return for the public investment. HCAHPS scores have been publicly reported on the Hospital Compare Web site since 2008 and since 2012 have been used in the payment determination for Inpatient Prospective Payment System (IPPS) hospitals that participate in the Hospital Value-Based Purchasing (Hospital VBP) program.
3. Use of Information Technology
The national implementation of HCAHPS is designed to allow third-party CMS-approved survey vendors to administer HCAHPS using mail-only, telephone-only, mixed-mode (mail with telephone follow-up), or active IVR (interactive voice response).
With respect to a telephone-only or mixed-mode survey, the CMS-approved survey vendors use electronic data collection or CATI systems. CATI is also used for telephone follow-up with mail survey non-respondents. With respect to IVR survey administration, the IVR technology gathers information from respondents by prompting respondents to answer questions by pushing the numbers on a touch-tone telephone. Patients selected for IVR mode are able to opt out of the interactive voice response system and return to a “live” interviewer if they wish to do so. There are numerous advantages to administering a telephone interview using a CATI system or IVR technology, including the following:
costs less than in-person data collection;
allows for a shorter data collection period;
allows for less item nonresponse because the system controls the flow of the interview;
increases data quality by allowing consistency and data range checks on respondent answers;
creates a centralization of process/quality control; and
reduces post-interview processing time and costs.
CMS has tested new modes for the HCAPHS Survey, specifically a Speech Enabled-Interactive Voice Response mode and a Web-based mode, but concluded that issues stemming from differences in response rate and mode effects across implementations make such models unsuitable for the HCAHPS Survey at this time. Results of the investigation for using these modes of survey implementation are presented in Elliott, Brown, et al. (2013), “A Randomized Experiment Investigating the Suitability of Speech-Enabled IVR and Web Modes for Publicly Reported Surveys of Patients’ Experience of Hospital Care”, Medical Care Research and Review, 70 (2): 165-184.
The HCAHPS Survey does not require a signature from respondents. In fact, all information obtained through the survey is reported in the aggregate and no individual respondent’s information is ever reported independently or with any identifying information.
4. Duplication of Efforts
HCAPHS collects information that is fundamentally different from other CAHPS or patient experience of care surveys. CMS is not aware of any existing validated survey instrument where the unit of analysis is the acute care hospital and the focus of the survey is patient-reported experience of care. The information collected through this survey will therefore not duplicate any other effort and is not obtainable from any other source.
Many hospitals carry out their own patient experience of care surveys. These diverse, proprietary surveys do not allow for comparisons across hospitals. Making comparative performance information available to the public assists consumers in making informed choices when selecting an acute care hospital and creates incentives for facilities to improve the care they provide.
5. Small Businesses
Hospitals are not generally considered to be small businesses. All hospitals have the option to conduct HCAHPS as a stand-alone survey or to integrate it with their existing survey activities. They can choose to administer HCAHPS by mail, phone, mail with telephone follow-up, or active IVR. Costs associated with collecting HCAHPS will vary depending on:
The method hospitals currently use to collect patient survey data,
The number of patients surveyed (target is 300 completed surveys per year), and
Whether it is possible to incorporate HCAHPS into their existing survey.
Some smaller hospitals that participate in HCAHPS might be unable to reach the target of 300 completed surveys in a 12-month period. In such cases, the hospital should sample all discharges (census) and attempt to obtain as many completes as possible. HCAHPS scores based on fewer than 100 or 50 completed surveys are publicly reported but the lower reliability of these scores is noted by an appropriate footnote.
Great effort was expended considering how often HCAHPS data should be collected. We solicited and received much comment on this issue. Two options for the frequency of data collection were suggested: once during the year or continuous sampling. The majority of hospitals/vendors suggested continuous sampling would be easier to integrate into their current data collection processes. Thus we decided to require sampling of discharges on a continuous basis (i.e., a monthly basis) and cumulate these samples to create rolling estimates based on 12-months of data. We chose to pursue the continuous sampling approach for the following reasons:
It is more easily integrated with many existing survey processes used for internal improvement,
Improvements in hospital care can be more quickly reflected in hospital scores (e.g., 12-month estimates could be updated on a quarterly or semi-annual basis),
Hospital scores are less susceptible to unique events that could affect hospital performance at a specific point in time,
It is less susceptible to gaming (e.g., hospitals being on their best behavior at the time of an annual survey), and
There is less variation in time between discharge and data collection.
7. Special Circumstances
There are no special circumstances associated with this information collection request.
8. Federal Register/Outside Consultation
The 60 day Federal Register notice published on March 13, 2015 (80 FR 13391). One comment letter was received. The comment letter along with our response have been attached to this package. No program changes or burden adjustments were made in response to those comments.
9. Payments/Gifts to Respondents
There are no provisions for payments or gifts to survey respondents.
10. Confidentiality
All information obtained through the HCAHPS Survey is reported in the aggregate. No individual respondent’s information is reported independently or with identifying information. We have designed the data files so that the hospital/vendor submits a de-identified dataset to CMS through a QIO according to 45 CFR Section § 164.514. No protected health information is submitted to CMS. In all the modes of survey administration, guidelines are included on issues related to confidentiality:
Cover letters are not to be attached to the survey
Respondents’ names are not to appear on the survey
Interviewers are not to leave messages on answering machines or with household members since this could violate a respondent’s privacy
Please see HCAHPS Quality Assurance Guidelines, V10.0, pp. 43-48, for detailed information on patient confidentiality, http://www.hcahpsonline.org/qaguidelines.aspx.
11. Sensitive Questions
There are no questions of a sensitive nature on the HCAHPS Survey.
12. Burden Estimates (Hours & Wages)
To calculate the cost per response, we employ the Average Hourly Wage Rate of $22.77/hr, based upon mean hourly wages, “National Compensation Survey: All United States December 2009-January 2011,” U.S. Department of Labor, Bureau of Labor Statistics. On average, it takes respondents 8 minutes (0.1333 hours) to complete the survey, for a total 413,230 hours annual burden. The annual cost burden of the HCAHPS Survey is thus $9,409,247.
EXHIBIT 1: Annual Hours/Cost Burden of the HCAHPS Survey
HCAHPS Survey |
Number of Respondents |
Total Burden Hours |
Average Hourly Wage* |
Estimated Data Collection Cost to Respondents |
Total |
3,100,000
|
413,230 |
$22.77 |
$9,409,247 |
*Average Hourly Wage Rate of $22.77, based upon mean hourly wages, “National Compensation Survey: All United States December 2009-January 2011,” U.S. Department of Labor, Bureau of Labor Statistics.
Over the next three years, we anticipate that about 4,200 hospitals will participate in HCAHPS. Using the estimate of $4,000 per hospital for HCAHPS data collection, the annual cost burden is $16,800,000. Assuming 1 hour per hospital, the annual burden for hospitals is 4,200 hours. The increase is due entirely to more hospitals participating in the HCAHPS Survey.
In total, the annual cost burden of the survey is (Patients $9,409,247) + (Hospitals $16,800,000) = $26,209,247, and the annual hour burden is (Patients 413,230) + (Hospitals 4,200 hours) = 417,430 hours.
13. Capital Costs
Hospitals have the option to conduct HCAHPS as a stand-alone survey or to integrate it with an existing survey. Hospitals can choose to administer HCAHPS by mail, phone, mail with telephone follow-up, or active IVR. Costs associated with collecting HCAHPS will vary depending on:
The method hospitals currently use to collect patient survey data,
The number of patients surveyed (target is 300 completed surveys per year), and
Whether it is possible to incorporate HCAHPS into their existing survey.
Over the next three years, we anticipate that about 4,200 hospitals will participate in HCAHPS. Using the estimate of $4,000 per hospital for HCAHPS data collection, the annual cost burden is $16,800,000.
14. Cost to the Federal Government
Costs to the government include: hospital/vendor training and technical assistance; approving hospitals/vendors for conducting surveys; ensuring the integrity of the data; accumulating the data; analyzing the data; making adjustments for patient-mix and mode of administration; and public reporting. The annual cost to the Federal Government is estimated to be $3,230,000.
15. Changes to Burden
The 2012 OMB packaged inadvertently included a 27-question survey (Appendix A). In the same package, the 32-question survey (Appendix C) was identical to the 27-question survey except that it added 5 questions. Importantly, the 2012 burden was for the 32-question survey.
This 2015 package does not include the 27-question survey since it was mistakenly added to the 2012 package and since it is no longer applicable. The 2015 package also makes several non-substantive changes to the 2012 32-question survey which are set out in Attachment B – Crosswalk. The changes have no impact on our burden estimates.
All changes in burden to our currently approved PRA package are due to an increase in the number of hospitals participating in the HCAHPS Survey. In 2012, 3,812 hospitals participated in HCAHPS. Over the next three years, we estimate that approximately 4,200 hospitals will participate in HCAHPS, an increase of about 388 hospitals (10.2%).
Similarly, the number of patient respondents is adjusted by 390,000 (2,710,000 in 2012 to 3,100,000).
16. Publication/Tabulation Dates
Since October 2006, the HCAHPS Survey has been administered on a continuous basis, and since March 2008, HCAHPS results have been publicly reported on the Hospital Compare website four times per year. This pattern will continue into the foreseeable future.
17. Expiration Date
CMS would like an exemption from displaying the expiration date on this collection as the survey will be used on a continuing basis. To include an expiration date would result in having to discard a potentially large number of surveys.
18. Certification Statement
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Christine Payne |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |