CMS-10537.FINAL - Supporting Statment Part B (10-29-14)

SUPPORTING STATEMENT CAHPS^® HOSPICE SURVEY

B. Collection of Information Employing Statistical Methods

B1. Respondent Universe and Respondent Selection

National implementation of the survey will occur in 2015. Hospices with fewer than 50 decedents during the prior calendar year are exempt from the CAHPS^® Hospice Survey data collection and reporting requirements for payment determination. Hospices with 50 to 699 decedents in the prior year will be required to survey all cases. For large hospices with 700 or more decedents in the prior year, a sample of 700 will be drawn under an equal-probability design.

For national implementation, we have assumed an eligibility rate of 85% and a response rate of 50%, based on experience in the 2013 field test of the CAHPS^® Hospice Survey instrument. (OMB Control number 0958-1208) A copy of the report can be found in Appendix A. These rates will result in an estimated 300 completed questionnaires for each large hospice and between 21 and 300 completed questionnaires for hospices with at least 50 decedents during the calendar year. Assuming a total of 300 completes within each hospice and an intraclass correlation coefficient (ICC) of 0.01, which measures the amount of variability between hospices, we would achieve an interunit reliability of 0.75. Note that in Medicare CAHPS^® a reliability of 0.75 is regarded as a minimal acceptable standard.

Interunit reliability is influenced by three main factors: agreement among respondents in a hospice (i.e., consistency of responses among respondents within the hospice), the magnitude of true differences in experience between hospices, as measured by the intraclass correlation coefficient (ICC), and the total number of respondents (i.e., sample size). Interunit reliability decreases as the sample size decreases. For example, assuming an ICC of 0.01, we would achieve an interunit reliability of 0.67 if the total number of completes within each hospice is 200 and an interunit reliability of 0.60 if the total number of completes within each hospice is 150.

Eligibility criteria for hospice patients and their primary caregivers were determined in consultation with a Technical Expert Panel, which was convened to provide guidance to our contractor, RAND Corp. The national implementation eligibility criteria parallel the criteria used in the field test of the survey. (The Technical Expert Panel did not issue a formal report or formal recommendations. Consultation occurred during a meeting with the panel on December 11. 2012 in Arlington, VA and in follow up telephone calls to panel members.) The following groups of hospice patients and the primary caregivers noted in their hospice’s administrative records are eligible for inclusion in the sampling universe:

• Patients over the age of 18

• Patients with death at least 48 hours following admission to hospice care

• Patients for whom a caregiver is listed or available and for whom caregiver contact information is known

• Patients whose primary caregiver is someone other than a non-familial legal guardian

• Patients for whom the primary caregiver has a U.S. or U.S. Territory home address

Patients or caregivers of patients who request that they not be contacted (those who sign “no publicity” requests while under the care of hospice or otherwise directly request not to be contacted) will be excluded. In the Hospice CAHPS field test this exclusion was available, but none of the participating hospices used it. “No publicity” is built into the eligibility estimate. The “no publicity” exclusion requires that the request for no contact originate with patients or their caregivers and cannot be suggested by the hospice.

In addition to national implementation of the CAHPS^® Hospice Survey, we will conduct a mode experiment to assess the differential impact of mail-only, telephone-only and mixed (mail with telephone follow-up) modes of survey administration on representativeness of respondents, survey response rates and response patterns. For the mode experiment, we will sample 60 hospices with sufficient decedent volume to achieve the targeted number of completes. Hospices will be sampled from among the largest

10% of hospices in the United States, measured as decedent volume in CMS claims data. To achieve precision similar to what was achieved in the HCAHPS mode experiments, we will sample an average of 300 deaths in each hospice. This sample is above and beyond the sample drawn for the purposes of national implementation of the survey. In keeping with the experience during the CAHPS^® Hospice Survey field test, we assume that approximately 15% of these deaths will be ineligible. Decedent and caregiver eligibility criteria will be aligned with the survey administration procedures outlined above for national implementation. Each hospice’s sample will be divided into three equal, randomly selected groups receiving mail only, telephone only, or mixed mode mail/telephone survey administration. Assuming a 50 percent response rate from caregivers, this will result in approximately 128 completes per hospice across the three modes, and an overall total across hospices of approximately 2,550 completes in each of the three modes.

The design will provide 80% power to detect differences in response rate as small as

2.8% (e.g. 48.6% vs. 51.4%) when comparing two survey modes in 2-sided tests, alpha=0.05. Similarly, when comparing responses to survey items, we will have 80% power to detect small-to-very-small differences (Cohen’s d=0.078) in 2-sided tests, alpha

= 0.05 and the adjustments we derive from the mode analysis will have a standard error of 0.055 SD. The design also provides excellent power to detect differences in distribution of respondent characteristics.

B2. Data Collection Procedures

We propose to use three survey modes: (1) mail-only, including a mailed survey followed

by a second survey mailed 21 days later; (2) telephone-only, including up to 5 telephone attempts; and (3) mixed mode, including a mailed survey followed by up to 5 telephone attempts beginning 21 days later. In keeping with HCAHPS guidelines, the entirety of the field period from will be no longer than 42 days (six weeks), regardless of survey mode.

The survey will be administered after a two-month lag between the death of the patient and the beginning of survey contact with the caregiver. The survey will be available in English and Spanish. We assume that 5 percent of the sample will need a Spanish survey. The low estimated proportion of Spanish surveys reflects the low representation of Hispanics among hospice patients.

B3. Response Rates and Non-Response

We anticipate a response rate of 50 percent, based on experience in the CAHPS^® Hospice

Survey field test. To minimize non-response, we will employ multiple mail contacts in the mail-only mode, multiple telephone contacts in the telephone-only mode, and both mail and telephone contacts in the mixed mode. CMS has issued highly detailed survey administration protocols, by mode, which can be found in the Quality Assurance Guidelines manual published on the survey web site, www.hospicechapssurvey.org. Copies of the questionnaires, scripts, suggested letters, etc., are in the appendices of the Manual.

B4. Tests of Procedures or Methods

This data collection effort includes a test to assess the differential impact of three survey administration modes: mail-only, telephone-only and mixed (mail with telephone follow- up) modes of survey administration on representativeness of respondents, survey

response rates and response patterns.

B5. Statistical and Data Collection Consultants

The survey, sampling approach, and data collection procedures were designed by the

RAND Corporation under the leadership of:

Rebecca Anhang Price, PhD RAND Corporation

1776 Main Street

Santa Monica, CA 90407

Melissa Bradley, BA RAND Corporation

1200 South Hayes Street

Arlington, VA 22202

Data for national implementation will be collected by survey vendors, to be determined. Data for the mode experiment will be collected by RAND Corporation.

ATTACHMENT

Attachment A: Hospice Survey Development Report

Attachment B: Sample of suggested survey cover letter

3