Hospice Survey OMB_Statement_B

SUPPORTING STATEMENT CAHPS^® HOSPICE SURVEY

B. Collection of Information Employing Statistical Methods Introduction

The Consumer Assessment of Healthcare Providers and Systems (CAHPS) Hospice

Survey is administered to primary caregivers (i.e., bereaved family members or close friends) of adult patients who died while receiving hospice care (“decedents”). Survey administration begins two to three months following the death of the patient. The CAHPS Hospice Survey is not restricted to caregivers of Medicare decedents. Hospices must use a CMS-approved survey vendor to collect CAHPS Hospice Survey data.

In the FY 2025 Hospice Wage Index and Payment Rate Update [PLACEHOLDER FOR OFFICIAL RULE NAME] proposed rule, CMS is proposing an updated version of the CAHPS Hospice Survey, starting with January 2025 hospice decedents. The proposed updated CAHPS Hospice Survey is 37 items; survey items can be used to derive 9 quality measures, including seven composite multi-item measures and two single-item measures for the Hospice Quality Reporting Program (HQRP). One of the composite measures (Care preferences) is new, two other composite measures are substantially revised since the prior version of the survey (Communication with family and Training family to care for patient), and the remaining six measures are substantively unchanged (with some minor edits to simplify item language).

CMS made the proposed updates to CAHPS Hospice Survey in response to feedback from hospice stakeholders. The proposed revised CAHPS Hospice Survey, web-mail mode, and updates to survey administration procedures were empirically tested in a large- scale mode experiment in 2021. The measure developer has determined that the proposed updates to the CAHPS Hospice Survey will likely reduce burden, as the revised survey instrument is eight items shorter than the current instrument; consequently, the estimated average amount of time for respondents to complete the survey is estimated to decline from 10.42 minutes to 8.68 minutes.

The CAHPS Hospice Survey can be currently implemented in three survey modes: mail, telephone, mail with telephone follow-up. In the FY 2025 Hospice Wage Index and Payment Rate Update [PLACEHOLDER FOR OFFICIAL RULE NAME] proposed rule, CMS is proposing that starting with January 2025 hospice decedents, a survey mode of web with mail follow-up be added as an approved mode of survey administration.

National implementation of the CAHPS^® Hospice Survey started in 2015. Under the HQRP, hospices are required to participate in the survey in order to receive their full Annual Payment Update (APU). There are two exceptions to this requirement. The first exception is for size. Hospices with fewer than 50 survey-eligible decedents/caregivers during the prior calendar year are exempt from the CAHPS Hospice Survey data collection and reporting requirements for payment determination. The second exemption

is for newness. A hospice that received its Medicare provider number in a given calendar year is exempt from conducting the CAHPS Hospice Survey for that year. For example, any hospice that received its provider number during calendar year 2022 would be exempt from conducting the survey for 2022 decedents. That exemption would impact the 2024 APU.

Among hospices that are not exempt, those with 700 or more survey-eligible decedents/caregivers in the prior year can elect to conduct a simple random sample of at least 700, or they may survey all cases (i.e., conduct a census). Those with 50 to 699 survey-eligible decedents/caregivers in the prior year are required to survey all cases (i.e., conduct a census). Hospices with fewer than 50 survey-eligible decedents/caregivers may elect to participate in the survey voluntarily.

Hospices must survey caregivers on a monthly basis. The CAHPS Hospice Survey is currently available in eight languages including English, Spanish, Chinese (traditional and simplified), Russian, Portuguese, Vietnamese, Polish, and Korean. CMS will add additional translations over time as needed. The CAHPS Hospice Survey promotes standardized sampling and administration of the CAHPS Hospice Survey by survey vendors and comparability of resulting data across all participating hospitals. The FY 2025 Hospice proposed rule describes proposed updates to the CAHPS Hospice Survey instrument and administration modes and protocols. For more details regarding eligibility, sampling, survey administration, and analysis of the CAHPS Hospice Survey, please refer to the current CAHPS Hospice Survey Quality Assurance Guidelines, which can be found on the official CAHPS Hospice Survey Web site at: https://hospicecahpssurvey.org/en/quality-assurance-guidelines/.

B1. Respondent Universe and Respondent Selection

Eligibility criteria for hospice decedents and their caregivers were determined in consultation with a Technical Expert Panel. The following groups of hospice decedents are eligible for inclusion in the sampling universe:

• Decedents age of 18 and over at the time of death

• Decedents with death at least 48 hours following last admission to hospice care

• Decedents for whom there is a caregiver of record

• Decedents whose primary caregiver is someone other than a non-familial legal guardian

• Decedents for whom the caregiver has a U.S. or U.S. Territory home address

Decedents or caregivers who voluntarily request that they not be contacted (those with “no publicity” requests while under the care of hospice or otherwise directly request not

to be contacted) are excluded. Decedents for whom any part of their date of death is missing must not be included in the sampling universe.

Survey vendors participating in the CAHPS Hospice Survey are responsible for generating complete, accurate, and valid sample frame data files each month that contain all administrative information on all decedents/caregivers who meet the eligible population criteria. Prior to generating the CAHPS Hospice Survey sample frame, survey vendors must apply the eligibility criteria and remove ineligible decedents/caregivers from the decedents/caregivers list received from the hospice. The steps below must be followed when creating the sample frame:

• Decedents/Caregivers whose eligibility status is uncertain must be included in the sample frame

• The sample frame for a particular month must include all survey-eligible decedents/caregivers from the first through the last day of the month (e.g., for January, any qualifying patient deaths from the 1st through 31st)

• Records with missing or incomplete decedent or caregiver names, addresses and/or telephone numbers must not be removed from the sample frame

• Patients whose eligibility status is uncertain must be included in the sample frame

The survey vendor must retain the original decedents/caregivers list, the sample frames (the entire list of eligible CAHPS Hospice Survey decedents/caregivers from which each hospice’s sample is drawn), the sample, and ineligibility counts in each category for a minimum of three years.

There are no requirements for the number of completed surveys that hospices must submit per year; however, non-exempt hospices with 700 or more survey-eligible decedents/caregivers in the prior year must sample a minimum of 700 cases, those with 50 to 699 survey-eligible decedents/caregivers in the prior year are required to survey all cases (i.e., conduct a census).

1. Statistical Methodology for Stratification and Sample Selection

The basic sampling procedure for the CAHPS Hospice Survey entails drawing a random sample of all eligible decedents/caregivers on a monthly basis. Survey vendors must sample from every month in the reporting period even if they have already achieved the required number of sampled decedents/caregivers.

Sampling for the CAHPS Hospice Survey is based on the survey-eligible decedents/caregivers (CAHPS Hospice Survey sample frame) for a calendar month. An equiprobable approach is being used, as every eligible decedent/caregiver for a given month has the same probability of being sampled.

There are two options for sampling patients for the CAHPS Hospice Survey: Simple Random Sampling (SRS), or Census Sampling. SRS is the most basic sampling type. A

group of decedents/caregivers (a sample) is randomly selected from a larger group of survey-eligible decedents/caregivers. Each decedent/caregiver is chosen entirely by chance, and each survey-eligible decedent/caregiver has an equal chance of being included in the sample. With Census Sampling, all eligible decedents/caregivers from the survey sample frame are selected.

Only one sample type may be used by a hospice within a quarter; “Sample Type” can only be changed at the beginning of a quarter. For more information about CAHPS Hospice sampling, please see the CAHPS Hospice Survey Quality Assurance Guidelines, V10.0, pp. 37-46, at https://www.hospicecahpssurvey.org/en/quality- assurance-guidelines/.

2. Estimation Procedures.

Not applicable to the CAHPS Hospice Survey.

3. Degree of Accuracy Needed for the Purpose Described in the Justification.

CAHPS Hospice Survey measure scores are reported for hospices with at least 30 completed surveys over eight calendar quarters. Additional information about the validity and reliability of the CAHPS Hospice Survey is provided in [https://p4qm.org/measures/2651].

4. Unusual Problems Requiring Specialized Sampling Procedures.

Not applicable to the CAHPS Hospice Survey.

B2. Data Collection Procedures

The CAHPS Hospice Survey currently has three approved modes of administration: (1) mail-only, including a mailed survey followed by a second survey mailed approximately 21 days later; (2) telephone-only, including up to 5 telephone attempts; and (3) mixed mode, including a mailed survey followed by up to 5 telephone attempts beginning approximately 21 days later.

The survey is administered between approximately 2 and 4.5 months following the death of the hospice patient. The survey is available in English, Spanish, and Russian for the telephone-only mode and mixed mode of administration. For the mail-only mode of administration, the survey is available in English, Spanish, Traditional Chinese, Simplified Chinese, Russian, Portuguese, Vietnamese, Polish, and Korean.

Beginning with survey administration in April 2025 (caregivers of January 2025 decedents), CMS will:

• Add web-mail mixed mode as an approved mode. In the web-mail mode, sampled caregivers with an available email address will receive up to two email invitations to complete the survey. Sampled caregivers without an available email

address, and those with an email address but who do not respond by web, will receive up to two mailings of a paper survey.

• Extend the survey field period by one week. The current CAHPS Hospice Survey administration field period is no longer than 42 days (six weeks), regardless of survey mode. Beginning with survey administration in April 2025, CMS will extend the survey field period by one week. For example, mailed surveys will be accepted for one additional week.

• Add a prenotification letter to all approved modes of CAHPS Hospice Survey administration. This letter would be sent one week prior to the start of data collection in all modes.

B3. Maximizing Response Rates and Ensuring Reliability and Validity

Implementation of the CAHPS Hospice Survey according to the protocols contained in the current CAHPS Hospice Survey Quality Assurance Guidelines, V10.0, helps hospice to attain the highest response rate. CMS examines response rates every quarter and works with hospices and survey vendors whose response rates are significantly below the national average. CMS strongly encourages hospices to offer the CAHPS Hospice Survey in the language spoken at home by their patients and their caregivers of record; in addition, each of the existing and proposed modes of administration employs multiple contacts to improve response rates. Proposed updates to survey administration procedures, including a prenotification letter and extended field period, are expected to improve response rates while imposing minimal burden on respondents. Analysis of similar CAHPS data indicates that the patient-mix adjustment applied to survey results adequately addresses the non-response bias that would exist without patient-mix adjustment; see “The Effects of Survey Mode, Patient Mix, and Nonresponse on CAHPS Hospital Survey Scores.” Elliott, Zaslavsky et al. (2009) Health Services Research, 44

(2): 501-518.

From Q1 2022 through Q4 2022, the overall CAHPS Hospice Survey response rate was 28.4%. As described above, beginning with January 2025 decedents (survey administration beginning in April 2025), CMS proposes to begin using an updated version of the CAHPS Hospice Survey instrument, allow for an additional survey mode of administration (Web with mail follow-up), add a prenotification letter prior to survey administration, and extend the field period by one week to increase response rates. Based on the 2021 mode experiment, CMS conservatively estimates that together, these changes will result in at least a 5-percentage point increase in overall response rates across hospices (i.e., a minimum response rate of approximately 33%).

Data from the revised CAHPS Hospice Survey will be used to calculate nine quality measures, including all existing measures and a new Care Preferences measure. In the 2021 mode experiment, the intraclass correlation coefficients (ICCs) of these nine Partnership for Quality Measurement-endorsed CAHPS Hospice Survey measures ranged from 0.012 to 0.030, indicating moderate variability in performance between hospices; see: Lyratzopoulos G, Elliott MN, Barbiere J, Staetsky L , Paddison C, Campbell J,

Roland M. (2011) “How can health care organizations be reliably compared?: Lessons from a national survey of patient experience.” Medical Care 49(8):724-733. At the average number of completes within each hospice for hospices projected to be included in public reporting, these would allow us to achieve reliability of 0.70 to 0.87 across measures (Table 1). Measure reliability greater than 0.70 is commonly considered adequate when entities such as hospices are being compared. ICC and reliability estimates are based on the 2021 mode experiment; these estimates may be conservative compared to the measure reliability in national implementation of the survey, since it is likely that there is more variation among hospices nationally than among the 56 hospices that participated in the mode experiment, which would result in higher ICCs and greater reliability than presented here.

Table 1. Hospice-Level Reliability of Revised CAHPS Hospice Survey Measures, 2021 Mode Experiment

Measure	Reliability at Average Number of Measure Respondents
Composite measures
Communication with family	0.84
Getting timely help	0.83
Treating patient with respect	0.77
Help for pain and other symptoms	0.70
Emotional and spiritual support	0.79
Care preferences1	0.78
Single-item measures
Training family to care for patient at home 2,3	0.70
Overall rating of this hospice	0.86
Willing to recommend this hospice	0.87

¹ Estimated ICCs based on the 2021 CAHPS Hospice Survey mode experiment. There were 5,731 respondents to this experiment. Average number of measure respondents is based on national CAHPS Hospice Survey data from Q3 2019 to Q4 2019 and Q3 2020 to Q4 2021 unless otherwise noted. Average number of measure respondents per hospice is similar across quarters.

² For the Care preferences and single-item Training measures, which are not on the current survey instrument, the average number of completes per hospice was extrapolated based on the percent of respondents completing those measures in the 2021 mode experiment.

³ ICCs and reliability for the Training measure are calculated only for those decedents who received hospice care at home or in an assisted living facility.

Table 2 displays the Cronbach’s Alpha statistics for the seven CAHPS Hospice Survey measures containing two or more items (i.e., composite measures), describing the internal consistency of these measures.

Table 2. Internal Consistency of Revised CAHPS Hospice Survey Composite Measures, 2021 Mode Experiment ¹

Measure	Cronbach’s Alpha
Communication with family	0.84
Getting timely help	0.62
Treating patient with respect	0.70
Help for pain and other symptoms	0.73
Emotional and spiritual support	0.71
Care preferences	0.75

NOTES: Cronbach’s alpha can only be calculated for composite measures; therefore, single-item measures not shown. There were 5,731 respondents to the 2021 mode experiment.

Construct validity indicates the extent to which measures assess the concept they were intended to evaluate. In the case of CAHPS Hospice Survey measures, construct validity can be established by examining whether measures of individual aspects of care, such as communication and timeliness, are related to caregivers’ assessments of the overall quality of hospice care. Applying Cohen’s rule of thumb that a correlation of 0.10-0.29 is small, 0.30-0.49 is medium, and 0.50 or above is large, the results in Table 3 demonstrate that CAHPS Hospice Survey measures exhibit medium to large associations with respondents’ Rating of this hospice at the individual level (range: r=0.43 to r=0.61).

Associations of Willingness to recommend this hospice and other measures are similar to those for Rating of this hospice; as is usually observed, the associations between Willingness to recommend and other measures are somewhat smaller than those between Rating of hospice and other measures, since Willingness to recommend may be affected by factors such as geographic location of the respondent. These results demonstrate the construct validity of the proposed measures.

Table 3. Individual-level Pearson Correlations Between Other CAHPS Hospice Survey Measures and Global Ratings, 2021 Mode Experiment

	Correlation with Overall Rating of Hospice	Correlation with Willingness to Recommend Hospice
Communication with family	0.61*	0.60*
Getting timely help	0.49*	0.47*
Treating patient with respect	0.53*	0.54*
Help for pain and other symptoms	0.48*	0.44*
Emotional and spiritual support	0.46*	0.43*
Care preferences	0.59*	0.61*
Training family to care for patient at home 1	0.43*	0.40*

NOTES: * p < 0.001. There were 5,731 respondents to the 2021 mode experiment.

¹ Correlations with the Training measure are calculated only for those decedents who received hospice care at home or in an assisted living facility.

As shown in Table 4, the correlations between the seven other measures and the two global measures are higher at the hospice level than at the individual level, ranging from 0.49 to 0.90. This is to be expected, as measurement is more stable at the entity level.

Table 4. Hospice-level Pearson Correlations Between Other CAHPS Hospice Survey Measures and Global Ratings, 2021 Mode Experiment

	Correlation with Overall Rating of Hospice	Correlation with Willingness to Recommend Hospice
Communication with family	0.84*	0.83*
Getting timely help	0.75*	0.71*
Treating patient with respect	0.85*	0.83*
Help for pain and other symptoms	0.67*	0.65*
Emotional and spiritual support	0.61*	0.73*
Care preferences	0.84*	0.90*
Training family to care for patient at home 1	0.56*	0.49*

NOTES: * p < 0.001. There were 5,731 respondents to the 2021 mode experiment.

¹ Correlations with the Training measure are calculated only for those decedents who received hospice care at home or in an assisted living facility.

To ensure that CAHPS Hospice Survey measure scores are comparable across hospices, CMS adjusts scores for differences in case mix across hospices. Case mix refers to characteristics of decedents and caregivers (e.g., education and age) that are not under the control of hospices, but may affect how caregivers respond to survey items. By adjusting for case mix, CMS is able to estimate hospices’ CAHPS Hospice Survey measure scores as if they all provided care to decedents and caregivers with similar characteristics. To be selected for inclusion as a case-mix adjustor, a characteristic must both be systematically associated with how caregivers respond to the CAHPS Hospice Survey and be differentially distributed across hospices.

Characteristics used in case mix adjustment are derived from hospices’ administrative records and from survey responses. Based on analyses of the 2021 mode experiment data, CMS proposes to continue to use the following variables in CAHPS Hospice Survey case-mix adjustment:

• Decedent age, payer for hospice care, primary diagnosis, length of final episode of hospice care

• Caregiver (respondent) education, language, and relationship to decedent

• Response percentile (calculated by ranking lag time)

On the survey there are two items that capture the race and ethnicity of the respondent. These items are not included in the case-mix adjustment model but are used as analytic variables to support the congressionally-mandated report, “The National Healthcare Quality and Disparities Report.” This report provides annual, national level breakdowns of CAHPS Hospice Survey scores by race and ethnicity. Many hospices collect information on race and ethnicity through their administrative systems, but coding is not standard. Thus, it was determined that administrative data are not adequate to support the analyses needed for the reports and the items should be included in the survey.

In addition to adjusting for case mix, CMS also makes a fixed adjustment to account for the effects of mode of survey administration, which can affect scores but is not related to the quality of hospice care. CMS currently applies survey mode adjustments derived from a 2015 CAHPS Hospice Survey mode experiment. With the introduction of a new mode of survey administration and new survey items, CMS is proposing to update the mode adjustments to those derived from the 2021 mode experiment beginning with January 2025 decedents (see the FY 2025 Hospice Rule [PLACEHOLDER FOR RULE NAME]).

B4. Tests of Procedures or Methods

In 2021, CMS conducted a large-scale OMB-approved mode experiment (OMB 0938- 1370) to test the web-mail mode, modification to survey administration protocols, and updates to the CAHPS Hospice Survey.

All nine quality measures derived from the revised CAHPS Hospice Survey received endorsement through the Partnership for Quality Measurement Consensus Standards Approval Committee (CSAC) Fall 2022 endorsement and maintenance cycle. Measures were also submitted through the Pre-Rulemaking Measure Review Measures Under Consideration process.

B5. Statistical and Data Collection Consultants

The revised survey, sampling approach, and updated data collection procedures were designed by the RAND Corporation under the leadership of:

Rebecca Anhang Price, PhD (Principal Investigator) Senior Policy Researcher

RAND Corporation 1200 South Hayes Street Arlington, VA 22202

(703) 413-1100 x5227 [email protected]

Melissa Bradley, BA (Co-Principal Investigator)

Associate Director, RAND Survey Research Group RAND Corporation

1200 South Hayes Street Arlington, VA 22202

(703) 413-1100 x5614 [email protected]

Marc N. Elliott, PhD

Distinguished Chair in Statistics; Senior Principal Researcher RAND Corporation

(310) 393-0411, x7931 [email protected]

Maria DeYoreo, PhD Senior Statistician RAND Corporation

(310) 393-0411, x6653 [email protected]

ATTACHMENTS

Attachment A: Revised CAHPS Hospice Survey

Attachment B: Crosswalk of Survey Changes

1