Generic Supporting Statement

Application to Use Burden/Hours from Generic PRA Clearance:

Testing of Web Survey Design and Administration for CMS Experience of Care Surveys

(CMS-10694, OMB 0938-1370)

Generic Information Collection (GenIC) #1: Consumer Assessment of Healthcare Providers and Systems (CAHPS®) Hospice Survey Mode Experiment

Center for Medicare (CM)

Centers for Medicare & Medicaid Services (CMS)

1. Background

The CAHPS^® Hospice Survey assesses experiences of hospice care. It is administered to the primary caregivers (i.e., bereaved family members or close friends) of patients who died while receiving hospice care (“decedents”). The Centers for Medicare & Medicaid Services (CMS) launched the development of the survey in 2012 to:

• Provide a source of information from which selected measures could be publicly reported to beneficiaries and their family members as a decision aid for selection of a hospice program;

• Aid hospices with their internal quality improvement efforts and external benchmarking with other facilities; and

• Provide CMS with information for monitoring the care provided.

CMS announced its intention to implement the CAHPS^® Hospice Survey in the FY 2014 Hospice Wage Index and Payment Rate Update; Hospice Quality Reporting Requirements; and Updates on Payment Reform. National implementation of the survey launched on January 1, 2015 with hospices administering the survey for a “dry run” for at least one month in the first quarter of 2015. Since April 1, 2015, hospices have been required to participate on a monthly basis in order to receive the full Annual Payment Update (APU). Public reporting of the results on a CMS website started in 2018.

Implementation is ongoing. To date, there have been no changes to the questionnaire and only minor changes to survey administration procedures. However, in recent years, CMS has received feedback from hospice stakeholders requesting (1) the option to administer the survey using a web mode as an alternative to the existing approved modes of mail only, telephone only, and mixed mode (mail with telephone follow-up); and (2) that CMS shorten the survey instrument. Both suggestions are aimed at improving the survey’s response rate.

The addition of a web-based mode has the potential to improve response rates, particularly among the caregivers who have the lowest response rates to the CAHPS Hospice Survey: the response rate among children of hospice decedents is only 29 percent, compared to a response rate of 44 percent among spouses of decedents, who are likely to be older.¹ In the emergency department setting, research has shown that a higher survey response rate could be achieved by using a web survey as the initial invitation to the survey, followed by mail and telephone, as compared to mail and telephone without web invitation,² and that characteristics (e.g., age) of telephone and web respondents were more representative of the sampled population than mail respondents.³

CMS proposes to conduct a mode experiment with the goal of testing the effects of adding a web-based mode to the CAHPS Hospice Survey. We will examine the impact of a web-based mode on survey response rates and scores. The survey currently has three approved modes without any web component (mail, telephone and mail with telephone follow-up.). In addition, the test will allow for examination of the effects of a shortened survey (i.e., removing existing survey items) on response rate and scores; assessment of the measure properties of a limited number of supplemental survey items suggested by stakeholders; and calculation of item-level mode adjustments for the shortened survey in the currently-approved modes of CAHPS Hospice Survey administration, as well as the proposed new web-based mode.

B. Description of Information Collection

We are requesting clearance under the Testing of Web Survey Design and Administration for CMS Experience of Care Surveys generic clearance package which supports an array of research activities to support the decisions about whether and how to add the web administration to CMS patient experience of care surveys. This request focuses on the CAHPS Hospice Survey.

The CAHPS Hospice Survey is administered to the primary informal caregiver of a hospice patient who died under hospice care. Data collection begins between 2 and 4.5 months following the death of the hospice patient. Hospice patients and the primary informal caregivers noted in their hospice’s administrative records are eligible for inclusion in the sampling universe if:

• Patients were over the age of 18

• Patients died at least 48 hours following last admission to hospice care

• Patients had a caregiver listed or available and caregiver contact information is known

• Patients had a primary caregiver who is someone other than a non-familial legal guardian

• Patients had a primary caregiver who has a U.S. or U.S. Territory home address

Patients or caregivers of patients who voluntarily request that they not be contacted (those who sign “no publicity” requests while under the care of hospice or otherwise directly request not to be contacted) are excluded.

The proposed mode experiment design will apply all of these eligibility criteria, and will sample patients/caregivers across five arms. The first proposed arm will test a new web-mail mode, in which invitations to the web survey will be sent by email to those with email addresses. The email will be personalized to the respondent and will include a link to the web version of the survey, which can be completed on either a computer or a mobile device such as a smartphone or tablet. If the respondent does not complete the web survey after one week, or does not have a valid email address in which to send an email, up to two surveys will be sent by mail. This arm will use a shortened version of the CAHPS Hospice Survey.

In the next three arms, the shortened version of the CAHPS Hospice Survey instrument will be administered in the three currently-approved modes: mail only; telephone-only; and mixed mode (mail with telephone follow up).

The fifth arm, in which the current survey instrument will be administered via mail only survey administration, will serve as a comparison for all other arms.

Across all arms, half of sampled caregivers will receive a prenotification letter to examine the effects of such a letter on response rates.

Overall (across the five arms), CMS plans to sample 15,000 eligible caregivers from up to 50 hospices over a six- to seven-month period. Caregivers will be randomized within each hospice to one of the five arms in equal proportion.

The proposed design is of sufficient scale to examine response rates and the effect of the shortened survey on scores, and to provide precise adjustment estimates for survey items and composites on the shortened survey instrument. Specifically, this design provides 80% power to detect (1) differences in response rates as small as 3.6%, (2) small differences (Cohen’s d = 0.12-0.13) in response patterns by mode protocol, and (3) small differences (Cohen’s d = 0.13) in response patterns by instrument type (within mail only mode).

C. Deviations from Generic Request

No deviations are requested.

D. Burden Hour Deduction

The mode experiment will be conducted among a maximum of 15,000 caregivers. The 3,000 caregivers in the comparison arm will be administered the current CAHPS Hospice Survey as part of its national implementation (OMB control number: 0938-1257); therefore, we calculate the burden hours solely among the 12,000 caregivers in the other four study arms. The caregivers in these four arms will be administered the shortened survey, which is 44 items long and is estimated to require an average administration time of 9.8 minutes (at a pace of 4.5 items per minute).

These burden estimates are based on CMS’ experience with the CAHPS Hospice Survey and surveys of similar length that were fielded with Medicare beneficiaries. As shown in Table 1, based on these assumptions, the one-time total burden hours for survey participants are estimated to be 1,956 for participation in the mode experiment.

Table 1. Estimated Burden Hours for Respondents: CAHPS^® Hospice Survey Mode Experiment

Survey Version	Number of Respondents	Number of Responses per Respondent	Hours per Response	Total Burden Hours
Shortened CAHPS® Hospice Survey	12,000	1	0.163	1,956.0
Total	12,000	1	0.163	1,956.0

As not all sampled caregivers will complete the survey, this estimate reflects the maximum burden possible.

The total approved burden ceiling of the generic ICR is 17,000 hours. We are requesting a total deduction of 1,956 hours from the approved burden ceiling.

E. Timeline

The following table provides an overview of the project schedule, including data collection and analysis.

Project Month

Tasks

1

2

3

4

5

6

7

8

9

10

11

OMB Approval

X

Conduct survey

X

X

X

X

X

X

X

X

Data Analysis

X

X

X

X

X

Final Report

X

X

The following attachment is provided for this information collection:

Shortened CAHPS Hospice Survey

1 Parast, L., Haas A., et al. (2018) Effects of Caregiver and Decedent Characteristics on CAHPS Hospice Survey Scores. J Pain Symptom Manage. 56(4):519-529.

2 Mathews M, Parast L, Tolpadi A, Marc MN, Flow-Delwiche E, Becker K (2019). Methods for Improving Response Rates in an Emergency Department Setting - A Randomized Feasibility Study. Survey Practice, 12(1): 10.29115/SP-2019-0007.

3 Parast L., M. Mathews, et al. (2019). Effects of Push-to-Web Mixed Mode Approaches on Survey Response Rates: Evidence from a Randomized Experiment in Emergency Departments. Survey Practice, 12(2). https://doi.org/10.29115/SP-2019-0008.