HHCAHPSSupportingStatementA

Supporting Statement Part A

The Home Health Care CAHPS® Survey (HHCAHPS) Mode Experiment (CMS-10784; OMB-0938-New)

Background

The Centers for Medicare & Medicaid Services (CMS) implements the Home Health Care Consumer Assessment of Healthcare Providers and Systems (HHCAHPS^®) Survey to measure and publicly report patients’ experiences with the health care they receive from Medicare-certified home health agencies (HHAs). The HHCAHPS Survey was launched in 2009 with the following goals:

• to produce comparable data on patients’ perspectives on care that allow objective and meaningful comparisons between HHAs on domains that are important to consumers;

• to create incentives for agencies to improve their quality of care through public reporting of survey results; and

• to enhance public accountability in health care by increasing the transparency of the quality of the care provided in return for the public investment.

National implementation of the survey launched in October 2009. Since October 1, 2010, HHAs have been required to participate on a monthly basis to receive the full Annual Payment Update. Public reporting of the results on a CMS website started in April 2012.

Implementation is ongoing with monthly data collection.

Current OMB/PRA Request. In recent years, CMS has received feedback from HHA stakeholders requesting (1) the option to administer the survey using a web mode as an alternative to the existing approved modes of mail only, telephone only, and mixed mode (mail with telephone follow-up); and (2) to shorten the survey instrument. Both suggestions, the focus of this OMB/PRA submission, are aimed in part at improving the survey response rate and reducing respondent burden.

The addition of a web-based mode has the potential to improve response rates, particularly among younger beneficiaries, who have historically responded to this survey at a lower rate. Younger patients are far less likely to respond to the HHCAHPS Survey (using calendar year 2019 survey data, when compared to patients age 75-84, patients age 18-49 are 3.0 times less likely to respond and patients age 50-64 are 1.8 times less likely to respond).

Estimates of middle-aged and elderly Americans who have access to the Internet are changing rapidly. Hall and colleagues (2015) found 76% of their aged 50+ survey respondents had Internet access. Among National Health and Aging Trends Study respondents aged 65+, 80% had a cell phone, 69% had a home computer, and 2% used a computer outside the home. About 51% reported Internet use at least once in the past month. Younger age groups were more likely to use the Internet (i.e., the 65 to 69 group were more likely to use the Internet than the 70 to 74 group; Choi & Dinotto, 2013). As younger, more Internet-savvy beneficiaries age into the use of home health services, a web-based mode for the HHCAHPS Survey is likely to be more widely used.

CMS proposes to conduct a mode experiment with the main goal of testing the effects of a web-based mode on response rates and scores as an addition to the three currently approved

modes. The addition of a web mode will give HHAs an alternative or an addition to the use of mail and telephone modes. CMS is also interested in testing a revised, shorter version of the HHCAHPS survey, based on feedback from patients and stakeholders. To that end, we have removed approximately 10 questions that psychometric analyses show are contributing less to publicly reported items over time and have added 3 questions focused on whether staff cared about the patient “as a person,” provided enough information to the patient’s family or friends, and improved the patient’s ability to take care of his or her health. All three of these are considered important indicators of patient experience, based on stakeholder input over time.

These changes are presented in a crosswalk document in Attachment A2 All four proposed study arms will use this revised version of the HHCAHPS Survey. The arms are:

1. Mail only

2. Telephone only

3. Mixed mode (mail with telephone follow-up)

4. (New) Web with mail follow-up mode

We propose the Web mode with a mail follow-up because mail data collection is the most common data collection mode for the national implementation of the HHCAHPS Survey. This collection request has been extracted from the currently approved OMB Umbrella Generic Clearance (OMB Control Number: 0938-1370).

Our mode experiment design will allow for (1) testing the web mode with the HHCAHPS Survey; (2) examination of the effects of a shortened survey on response rate and scores; (3) an assessment of the measure properties of a limited number of new survey items suggested by stakeholders; and (4) calculation of item-level mode adjustments for the shortened survey in the currently approved modes of HHCAHPS Survey. These analyses are discussed in more detail in Supporting Statement B.

1. Justification

1. Circumstances Making the Collection of Information Necessary

The reporting of quality data by HHAs is mandated by Section 1895(b)(3)(B)(v)(II) of the Social Security Act (“the Act”). This statute requires that ‘‘each home health agency shall submit to the Secretary such data that the Secretary determines are appropriate for the measurement of health care quality. Such data shall be submitted in a form and manner, and at a time, specified by the Secretary for purposes of this clause.’’ HHCAHPS data are mandated in the Medicare regulations at 42 C.F.R.§484.250(a), which requires HHAs to submit HHCAHPS data to meet the quality reporting requirements of section 1895(b)(3)(B)(v) of the Act. This collection of information is necessary to be able to test updates to the HHCAHPS survey and administration protocols.

2. Purpose and Use of Information

The data collected from the HHCAHPS Survey mode experiment will be used for the following purposes:

• test the shortened survey instrument, including several new items;

• compare survey responses across the four proposed modes to determine if adjustments are needed to ensure that data collection mode does not influence results; and

• determine if and by how much patient characteristics affect the patients’ rating of the care they receive and adjust results based on those factors.

The mode experiment is designed to examine the effects of the shortened survey on response rates and scores and to provide precise adjustment estimates for survey items and composites on the shortened survey instrument. Information from this mode experiment will help CMS determine whether an additional mode of administration (i.e. Web data collection) should be included and a shortened survey instrument should be used in the current national implementation of the HHCAHPS Survey.

The national HHCAHPS Survey is used by Medicare-certified HHAs to improve their internal quality assurance of the home health care they provide. Additionally, CMS uses the HHCAHPS Survey in the Home Health Quality Reporting Program, and consumers use the survey results to evaluate options for home health care. Given the importance and widespread use of this survey, CMS wants to improve current response rates and ensure that the survey remains relevant to all stakeholders.

3. Technological Collection Techniques

The HHCAHPS Survey (OMB-0938-1370) is currently approved for data collection using mail, telephone, and mail with telephone follow-up (also called mixed mode). Through this submission, CMS wishes to investigate the feasibility of implementing the HHCAHPS Survey using a Web-based mode. If CMS determines that response rates to a web survey are comparable to the traditional modes currently in use in the national implementation, an additional mode option for web with mail follow-up may be incorporated into HHCAHPS through the rulemaking process.

CMS proposes to implement the experimental mode (web with mail follow-up) in this mode experiment using the Voxco Command Center, an integrated interviewing and case management system that provides tools for conducting web and telephone survey research. The web survey will be formatted such that sample members can easily access the survey on a variety of devices (laptops, desktops, smart phones, tablets, etc.) and web browsers. All systems reside within CMS contractor’s NIST-Moderate network, providing the security controls needed to protect personally identifiable information/protected Health information data. This network is FIPS 140-2 Compliant. All sampled patients assigned to receive the experimental mode of data collection will receive an initial invitation (sent via email, if an email address is available, and mail) to complete the Web survey. A combination of email reminders and, for all non-respondents, a mailed hard copy survey, will be used to encourage survey completion for this mode of administration.

As with a computer-assisted telephone interview (CATI) system, Web administration offers numerous advantages, including the following:

• costs less than in-person data collection;

• reduces item nonresponse because the system controls the flow of the interview and complex routing;

• increases data quality by allowing consistency and data range checks on respondent answers;

• centralizes process/quality control; and

• reduces post-interview processing time and costs.

Since response rates are typically higher when multiple modes are used and not all home health patients have access to the Internet, CMS is electing to combine the Web mode with a mail survey follow-up option for non-respondents.

4. Efforts to Identify Duplication

Work carried out under this clearance will not duplicate any other survey being done by CMS or other federal agencies.

5. Impact on Small Businesses

Survey respondents are patients who currently receive home health care, or who received home health care over the last 2 months, from Medicare-certified home health care agencies. The proposed revised HHCAHPS Survey mode experiment does not impact small businesses or other small entities.

6. Consequences if Information is Collected Less Frequently

CMS is proposing a one-time mode experiment under this request. Agencies sampled for the mode experiment will provide three consecutive months of sample to CMS’s contractor (via the HHA’s regular HHCAHPS Survey vendor). The sample will be provided after the HHCAHPS Survey vendor has already selected the agency’s monthly sample for national implementation. Therefore, no impact is expected on the agency’s ability to provide data for CMS’s ongoing national implementation of the HHCAHPS Survey. This mode experiment will run for three sample months to ensure sufficient sample for the proposed analyses.

If this mode experiment is not conducted, CMS will not have the evidence it seeks to determine whether a shortened survey (with items expected to resonate with home health recipients) and additional Web mode option would be both feasible and preferred by the target population and result in increases in the response rate.

7. Special Circumstances

There are no special circumstances with the HHCAHPS Survey that would require an information collection to be conducted in a manner that requires respondents to:

• report information to the agency more often than once;

• prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• submit more than an original and two copies of any document;

• retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than 3 years;

• collect data in connection with a statistical survey that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• use a statistical data classification that has not been reviewed and approved by OMB;

• include a pledge of confidentiality that is not supported by authority established in statute or regulation that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• submit proprietary trade secrets or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information’s confidentiality to the extent permitted by law.

1. Federal Register Notice/Outside Consultations

Federal Register

The 60-day notice was published in the Federal Register on 8/5/2021 (86 FR 42841).

No comments were received during the public comment period.

The 30-day notice was published in the Federal Register on 10/14/2021 (86 FR 57151).

Outside Consultations

CMS’s contractor for the national implementation of the HHCAHPS Survey and for the proposed mode experiment, RTI International, convened a technical expert panel (TEP) composed of representatives from the home health industry, consumer advocacy organizations, the government, and research organizations. Members of the committee provided guidance to RTI on the development of the design for the mode experiment and the survey content. RTI, CMS, and members of the technical expert panel met on July 20, 2018, and April 17, 2019. TEP members were from the following organizations:

• AARP (American Association of Retired Persons)

• Duke Raleigh Hospital

• Duke Regional Hospital

• Encompass Health –Home Health and Hospice

• Kaiser Permanente

• Kindred at Home

• LeadingAge

• LHC Group

• National Association for Home Health Care and Hospice

• Visiting Nurse Service of New York

• WakeMed Health and Hospitals

9. Payments/Gifts to Respondents

No payments or gifts will be provided to respondents.

10. Assurance of Confidentiality

Individuals who are contacted as part of this data collection are assured of the confidentiality of their replies under 42 U.S.C. 1306, 20 CFR 401 and 422, 5 U.S.C. 552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974), and OMB Circular A-130.

During the mode experiment, participating HHAs will give permission for their regular HHCAHPS Survey vendor to provide RTI with an electronic file containing information about patients who received home health care in a designated month. The information will include personally identifiable information (PII; name, address, gender, date of birth, email address, and telephone number). The file will also include protected health information used for patient-mix adjustment analysis, including ICD-10 codes (indicator of primary diagnoses), and agency name. Social Security numbers will not be provided.

RTI will include an assurance of confidentiality of the data to all participants. For those participating in the mail or Web survey, the cover letter will include an assurance of confidentiality. The introductory script for those interviewed by telephone will contain an assurance of confidentiality. Those patients included in the Web with mail follow-up data collection could receive the assurance of confidentiality twice—in the Web survey invitation and in the mail cover letter, should the mail survey be necessary. (Attachments A, B, and C show the mail, telephone, and Web versions of the survey. Attachment D shows the survey cover letters and email communications.) The three survey versions (mail, telephone, and Web) are the same except that the CATI version includes an introductory script that covers the information provided in the lead letter and email, and the CATI version does not include Q26 and Q27, which collects information on whether the patient needed help responding to questions.

RTI understands the privacy and confidentiality concerns regarding access to HHCAHPS data. All RTI staff members who will have access to patient information will be required to sign and abide by the terms of a nondisclosure agreement, where they agree to protect the identity of patients included in the mode experiment and the data they provide. Following the mode experiment, RTI will destroy the Personal Health Information and PII and retain deidentified data for the analyses. RTI has redundant security protocols to protect data and computer systems. Servers are maintained in climate-controlled environments, with restricted access. A firewall stands between the internal systems and the Internet, requiring authentication of all users requesting access. User identification and passwords are unique and changed on a regular basis. Full backups are conducted on a weekly basis, with incremental backups performed nightly.

Copies of backup materials are stored offsite in a secure location in case of system failure.

As data are collected and assembled into databases for analysis and interpretation, RTI incorporates database security safeguards to protect data from accidental or intentional access and disclosure threats. RTI’s data collection and storage security measures include the following:

• Maintenance of all servers in RTI’s environmentally controlled Computer Center, where computers are in a center constructed of masonry with an automatically locking steel door that is locked at all times; fire protection is provided by a halon system with all servers having an Uninterruptible Power Supply.

• User ID and password authentication to access all systems. Where appropriate, systems are configured to support the use of Digital Security Certificates for additional user authentication.

• Encrypted transmission of data.

• Use of Transport Layer Security, the successor technology to Secure Socket Layer for encryption of data across the Internet.

• Connection to the Internet by an Internet firewall via a high-speed T2 (6.2 MBs) line. In the event of a failure, a T1 (1.544 MBs) backup will automatically provide uninterrupted Internet connectivity. Subscription to virus-protection services from McAfee VirusScan with automated update of virus signature files on all computers.

• Redundant servers with automatic switchover to ensure 24/7 availability.

• Daily incremental backups of all data files, with full backups created weekly.

• Offsite storage of data backups.

11. Questions of a Sensitive Nature

Information collected in this survey is not considered to be of a sensitive nature.

Questions in the survey are confined to respondent interactions and experiences with their HHA.

12. Estimates of Burden Hours and Costs

Burden Estimates

The mode experiment will be conducted with an estimated 23,150 home health patients sampled across the four study arms described in the Background section. Patients will be administered the shortened survey, which is 27 items long and is estimated to require an average administration time of approximately 10 minutes. We estimate that we will have a total of 6,280 respondents across all four of the survey arms. We estimate it will take 0.167 hours (10 minutes) at $27.07/hr for a home health patient to complete the proposed mode experiment HHCAHPS Survey. Our estimate is based on the written length of the survey and CMS’s experience with the national implementation of HHCAHPS, including average times observed by survey vendors currently conducting the longer, national HHCAHPS Survey.

Wage Estimates

Individuals. To derive the average costs for individuals, we used data from the U.S. Bureau of Labor Statistics’ May 2020 National Occupational Employment and Wage Estimates for our salary estimate (https://www.bls.gov/oes/current/oes_nat.htm#00-0000). We believe that the burden will be addressed under All Occupations (occupation code 00-0000) at $27.07/hr since the group of individual respondents varies widely from working and nonworking individuals and by respondent age, location, years of employment, and educational attainment, etc. We are not adjusting this figure for fringe benefits and overhead since the individuals’ activities would occur outside the scope of their employment.

In aggregate, we estimate a burden of 1,048.76 hours (6,280 patients x 0.167 hr) at a cost of $28,390 (1,048.76 hr x $27.07/hr) or $4.52 per survey ($28,390 /6,280 patients).

The survey instrument and procedures for completing the instrument are designed to minimize burden on all respondents.

Burden Summary for Patients

Respondent Type	Number of Respondents	Time per Response (hr)	Total Time (hr)	Labor Rate ($/hr)	Total Cost ($)
Patients	6,280	0.167	1,048.76	$27.07	$28,390

Patient Sample. Section 484.250 requires that an HHA submit HHCAHPS data to CMS, for CMS to administer the payment rate methodologies described in §§ 484.215, 484.230, and

484.235. The burden associated with this effort is the time and effort put forth by the HHA to submit the HHCAHPS patient files to its approved HHCAHPS Survey vendor. As part of this requirement, each HHA sponsoring an HHCAHPS Survey must prepare and submit to its survey vendor a file containing data on patients served during the preceding month, that is used by the survey vendor to select the sample and field the survey.

For this mode experiment, CMS is proposing to use leftover sample from the agencies’ regular monthly file submissions to their survey vendors, therefore limiting the burden to HHAs. Rather, participating agencies’ HHCAHPS Survey vendors will submit leftover sample to CMS’s mode experiment contractor, RTI, for each of the three successive months of the proposed mode experiment. The burden on vendors to submit leftover sample is expected to be minimal.

Information Collection Instruments and Instruction/Guidance Documents

Attachment A – HHCAHPS (Mail Survey)

Attachment A2 – Crosswalk of proposed new instrument versus current HHCAHPS Survey

Attachment B – HHCAHPS (Telephone Script) Attachment C – HHCAHPS (Web Survey Screenshots) Attachment D1 – Initial Cover Letter Mail Survey Attachment D2 – Letter Invitation for Web Survey

13. Capital Costs

HHCAHPS Survey respondents do not incur any capital costs. Since the sample will be selected from HHAs that are already participating in the HHCAHPS Survey, no capital costs are incurred by participating HHAs.

14. Estimates of Annualized Cost to the Government

The cost to the federal government for the mode experiment is $617,501. This is CMS’s cost for the mode experiment implementation task in the contract with the federal contractor managing the HHCAHPS mode experiment. RTI International is the federal contractor for the proposed HHCAHPS mode experiment and CMS’s national implementation contractor for the HHCAHPS Survey.

15. Program Changes or Adjustments to Annual Burden This is a new collection of information.

16. Tabulation and Publication of Results

The mode experiment will be a one-time (cross-sectional) survey of a sample of patients 18 years old and older who received home health care from an HHA during the sample month. Data collection for the mode experiment will take place in the spring of 2022 or as soon as possible after OMB clearance is received. Data collection for mail-only mode, telephone-only mode, mail with telephone follow-up mode, and Web with mail follow-up mode will be implemented concurrently for each of the three sample months. Each monthly sample will be fielded over a 6-week data collection period. The proposed schedule is shown in Table A-1. The schedule will be shifted (if needed) depending on when OMB approval is received.

Table A-1. Proposed schedule for Mode Experiment

Activity	Dates
Agency recruitment and onboarding	January – March 2022
Sampling and data collection, month 1 (April 2022)	May – July 2022
Sampling and data collection, month 2 (May 2022)	June – August 2022
Sampling and data collection, month 3 (June 2022)	July – September 2022
Analysis	September – November 2022
Technical Report to CMS	December 2022

Data from this mode experiment will help CMS determine whether adding a Web data collection mode will impact patients’ responses to the survey based on data collection mode. In addition, data from the mode experiment will be used to determine which, if any, patient characteristics affect CAHPS ratings and reported experience of the health care encounter. It will also be used to test the use of a revised, shortened survey. The following analyses will be conducted using the mode experiment data: psychometric analysis, mode and patient-mix analyses, and nonresponse analysis. These analyses are described in detail in Supporting Statement B, Section B.4.

Comparative results from the HHCAHPS mode experiment will not be publicly reported.

CMS’s HHCAHPS contractor will prepare a technical report describing the mode experiment results and recommendations for implementing the Web mode, adjusting survey results for nonresponse, mode, and patient-mix, should CMS elect to implement the shortened survey on a national scale. At this time, CMS does not anticipate making the report public, as it is expected to be used to inform internal discussions. Should CMS elect to move forward with a shorter survey or the Web-based mode, CMS will make the results of the mode experiment public via the HHCAHPS Survey website.

Assessment of Mode Effects and Patient-Mix

17. Display of OMB Expiration Date

The HHCAHPS mode experiment survey instrument will display the OMB Expiration Date and the PRA Disclosure Statement on the cover; this information will also be displayed on the Web instrument.

18. Exceptions to the Certification Statement None.

References

Choi, N. G., & Dinitto, D. M. (2013). Internet use among older adults: Association with health needs, psychological capital, and social capital. Journal of Medical Internet Research, 15(5), e97. https://doi.org/10.2196/jmir.2333

Hall, A. K., Bernhardt, J. M., Dodd, V., & Vollrath, M. W. (2015). The digital health divide: Evaluating online health information access and use among older adults. Health Education & Behavior, 42(2), 202–209. https://doi.org/10.1177/1090198114547815

1