CMS-10105_Supporting_Statement_part_B_041019

Supporting Statement Part B

NATIONAL IMPLEMENTATION OF THE IN-CENTER HEMODIALYSIS CAHPS

SURVEY

(CMS-10105, OMB 0938-0926)

2. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement nationally the In-center Hemodialysis CAHPS (ICH CAHPS) Survey to measure patients’ experience of care with in-center hemodialysis (ICH) facilities under Contract Number HHSM-500-2017-00068G.

1. Potential Respondent Universe and Sample Selection Method

B.1.1. Sampling Patients for the National Implementation

Medicare-certified dialysis facilities that serve more than 30 survey-eligible sample members during a calendar year are required to contract with a CMS-approved survey vendor to collect and submit ICH CAHPS Survey data on their behalf. All approved ICH CAHPS Survey vendors are required to use standardized survey administration protocols and specifications provided by CMS. The national implementation of the ICH CAHPS survey is conducted on a semiannual basis, with sampling and data collection activities conducted as shown in

Exhibit B.1.

Exhibit B.1. Sampling Window for the ICH-CAHPS Semiannual Surveys

	Spring survey	Fall survey
Sampling window (months in which patients received ICH care)	January–April	June–September
Sample selected	May	October
Data collection period	June–August	November–January

The national implementation of the survey is fielded on a rolling semiannual basis. The results for each semiannual survey are merged with data from the immediately preceding semiannual survey for developing composite measures for public reporting. A primary issue is obtaining sufficient sample size within a facility to produce confidence intervals for point

estimates that are sufficiently narrow. Approximately 200 observations are needed per year to produce a confidence interval that has a bound of +/−0.07. Approximately 90.7% of Medicare- certified ICH facilities serve 100 or fewer unique patients a year; 9% serve between 101 and 200 patients a year, and less than 1.0% serve more than 200 patients each year.

For each semiannual wave, patients who received care during the sampling window and who meet survey eligibility criteria will either be chosen randomly or selected with certainty depending on the number of survey-eligible patients the ICH facility served during the preceding 12-month period. If a facility’s patient volume is large enough, the number of patients sampled for that facility for each semiannual survey will be sufficient to yield a minimum of 200 completed surveys over the two semiannual surveys. If a facility does not serve enough survey- eligible patients over a given 12-month period to yield 200 completed surveys from the two semiannual surveys, a census of all survey-eligible patients will comprise the sample. Most of the ICH facilities need to survey all of their eligible patients at least once during the course of a calendar year and most patients are sampled twice within a given year.

Facilities with 1–200 Unique Patients. A census of all ICH patients is conducted for facilities with fewer than 201 eligible ICH patients at each semiannual sampling wave. Thus, patients at these smaller ICH facilities are sampled twice in a given year.

Facilities with 201–400 Patients. For dialysis centers that have between 201 and 400 eligible ICH patients at the first semiannual sampling period (the Spring Survey), a simple random sample of 200 patients is selected for that sampling period to obtain 100 completed responses. For the Fall Survey, the goal is to obtain an additional 100 completed interviews while attempting to minimize patient overlap of patients between the first and second semiannual waves of sampling. To achieve this goal, we first identify all eligible patients from that facility who were not selected for the Spring Survey. If the number of eligible patients not selected in the Spring Survey is equal to or exceeds 200 then we select a simple random sample of 200 from these patients for the Fall Survey. If there are 200 or fewer patients then all of these patients are selected for the Fall Survey. To obtain 200 completed surveys, we also select a simple random sample of the appropriate size from the patients who were selected in the Spring Survey, provided that they are still receiving treatment at that facility and still meet all of the survey eligibility requirements.

2. Information Collection Procedures

Three modes of survey administration are allowed during the national implementation of the ICH CAHPS Survey to give ICH facilities options for their preferred survey administration modes, based on their goals and resources. These three modes are described below:

• Mail-only Mode: ICH CAHPS Survey data collection for the mail-only mode consists of mailing a pre-notification letter, explaining the purpose of the survey, and letting patients know that a hardcopy questionnaire will soon be sent to them via mail. A questionnaire package consisting of a cover letter, the ICH CAHPS questionnaire, and a pre-addressed, postage-paid return envelope is sent to all sample patients two weeks following the pre notification letter. A second mailing containing a questionnaire and cover letter is mailed to all sample patients who do not respond to the first mailing within four weeks after the first questionnaire package is mailed. Data collection ends ten weeks after the first questionnaire package is mailed.

• Telephone-only Mode: In this mode, all sample patients are first sent a pre- notification letter letting them know that a professional interviewer working on the ICH CAHPS Survey will soon be contacting them via telephone. All sample patients are then contacted by professional telephone interviewers who are trained on ICH CAHPS survey administration procedures, including procedures for working with dialysis patients. Telephone interviewers are trained on the appropriate response to common questions and concerns that dialysis patients might have about survey participation, and are required to offer to administer the interview in different call- backs if the sample patient indicates that he or she cannot complete the interview in one call. A maximum of 10 telephone contact attempts per patient are attempted to complete the survey. Data collection ends ten weeks after the initial telephone contact begins.

• Mixed Mode: All sampled patients included in the mixed-mode data collection sample receive a pre-notification letter letting them know that we will soon be contacting them via mail. We then send an initial mailing of a questionnaire, cover letter, and postage-paid return envelope that patients included in the mail-only sample receive. Sample patients assigned to this mode who do not respond to the mail survey within four weeks after the questionnaire is mailed will be assigned to the telephone follow- up. Telephone interviewers make up to 10 attempts to complete the interview by phone with all mail survey nonrespondents included in the mixed-mode sample. Data collection ends ten weeks after the first questionnaire package is mailed.

Survey vendors who wish to become “approved” to conduct the ICH CAHPS Survey on behalf of ICH facilities must complete the ICH CAHPS survey vendor training, which provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission.

3. Methods to Maximize Response Rate

Response rates range average 28.6% for mail-only surveys, 23.2% for phone-only surveys and 33.6% for mixed-mode surveys, for an overall average response rate of 33.1%, as shown in Exhibit B.2. Every effort will be made to maximize patient response rates while retaining the voluntary nature of the ICH CAHPS Survey.

Exhibit B.2. Response Rates for ICH-CAHPS Semiannual Surveys

	Mail	Only	Phone	Only	Mixed	Mode	Overall	Rate
	N	RR	N	RR	N	RR	N	RR
Spring 2015- Fall 2015	48,484	29.4%	17,051	22.0%	595,920	33.7%	661455	33.1%
Spring 2016- Fall 2016	32,646	27.1%	13,175	22.2%	616,657	31.8%	662478	31.4%
Spring 2017- Fall 2017	30,645	26.3%	11,714	22.9%	602,644	34.2%	645003	33.6%
Spring 2018- Fall 2018	17,264	25.2%	15,918	25.6%	589,989	32.7%	623171	32.3%
Overall*	167,439	28.6%	76,815	23.2%	2,893,424	33.6%	2,893,424	33.1%

*Overall includes Fall 2014 survey results, which were not displayed on Dialysis Facility Compare

After the sample file is downloaded, survey vendors must verify mailing addresses using a commercial address update service. Each prenotification letter envelope contains the CMS logo along with the survey vendor’s return address. Furthermore, each questionnaire mailing includes a cover letter containing information about the survey, including sponsorship and objectives, a description of how survey results will be used, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. Because some dialysis patients may be reluctant to participate because of fear of retribution from their dialysis centers, the mail survey materials contain assurances that the patients’ dialysis facility will not have access to their survey responses linked to their names or any other information that can identify the patients. In addition, CMS requires that the mail survey cover letters be printed on the survey vendor’s letterhead and signed by the survey vendor’s ICH CAHPS project manager.

We require that all mail survey vendors use current best practices in the survey materials to enhance response rates. These best practices include using a simple font no smaller than 12

point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire.¹

For sample patients included in the mail and mixed-mode data collection for the ICH CAHPS Survey, the second questionnaire mailing is expected to increase the response rate. The cover letter included in the second questionnaire package to mail survey nonrespondents contains a stronger appeal for the sample patient’s help on this survey, including indicating that the survey is an opportunity for them to provide input on the quality of dialysis care dialysis patients receive. To maximize response rates for the telephone-only mode and the telephone follow-up of the mixed-mode survey, we require that up to 10 attempts be made to reach each sample patient, with those attempts varying by day of the week and time of day. Telephone interviewers will be trained on how to answer the questions that are most frequently asked by sample patients, and to address any concerns that they may have about participating in the survey. Because some

dialysis patients may not feel well on the day that they receive dialysis treatments, telephone interviewers are instructed to offer to call back at a time that is better for the sample patient, and offer to conduct the telephone interview on two or more different calls.

4. Tests of Procedures

CMS used data from the mode experiment to assess the effects of data collection mode, patient characteristics, and nonresponse on survey results. We used the data from the mode experiment to develop models that will be used to statistically adjust survey results from the national survey to control for factors that are beyond the control of the ICH facilities. The following analyses were conducted on mode experiment data:

• Analyses of individual survey items to assess missing data and item distributions

• Statistical analysis of patient mix effects and nonresponse patterns on survey results

• Hypothesis testing to detect differences in key variables between modes.

¹ Dillman, Don A. (1999). Mail and Other Self-Administered Surveys in the 21st Century: The Beginning of a New Era. The Gallup Research Journal, 2(1): 121-140.

More information on the mode experiment and patient-mix adjustment is found in Attachment E.

5. Statistical Consultation and Independent Review

This sampling and statistical plan was prepared by RTI International. The primary statistical design was provided by Gordon Brown of RTI International. Dr. Brown has since been replaced by Scott Scheffler, who can be reached by telephone at (919) 248-8540-or by e-mail at [email protected].