CMS-10105_Supporting_Statement_part_B_

ICH CAHPS – Supporting Statement Part B

Supporting Statement Part B

National Implementation of the In-Center Hemodialysis CAHPS Survey

(CMS-10105, OMB 0938-0926)

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to continue implementing nationally the In-center Hemodialysis CAHPS (ICH CAHPS) Survey to measure patients’ experience of care with in- center hemodialysis (ICH) facilities under Contract Number GS-00F-354CA - 75FCMC20F0078.

B.1 Potential Respondent Universe and Sample Selection Method

B.1.1. Sampling Patients for the National Implementation

Medicare-certified dialysis facilities that serve more than 30 survey-eligible sample members during a calendar year are required to contract with a CMS-approved survey vendor to collect and submit ICH CAHPS Survey data on their behalf. All approved ICH CAHPS Survey vendors are required to use standardized survey administration protocols and specifications provided by CMS. The national implementation of the ICH CAHPS survey is conducted on a semiannual basis, with sampling and data collection activities conducted as shown in Exhibit B.1.

Exhibit B.1. Typical Sampling Window for the ICH CAHPS Semiannual Surveys

	Spring survey	Fall survey
Typical sampling window (months in which patients received ICH care)	October-December	April-June
Sample selected	March	September
Data collection period	April-July	October–January

The national implementation of the survey is fielded on a rolling semiannual basis. The results for each semiannual survey are merged with data from the immediately preceding semiannual survey for developing composite measures for public reporting. A primary issue is obtaining sufficient sample size within a facility to produce confidence intervals for point estimates that are sufficiently narrow. Approximately 200 observations are needed per year to produce a confidence interval that has a bound of +/−0.07. Approximately 92% of Medicare- certified ICH facilities serve 100 or fewer unique patients a year; 7.5% serve between 101 and 200 patients a year, and less than 1.0% serve more than 200 patients each year.

For each semiannual wave, patients who received care during the sampling window and who meet survey eligibility criteria will either be chosen randomly or selected with certainty depending on the number of survey-eligible patients the ICH facility served during the preceding 12-month period. If a facility’s patient volume is large enough, the number of patients sampled for that facility for each semiannual survey will be sufficient to yield a minimum of 200 completed surveys over the two semiannual surveys. If a facility does not serve enough survey- eligible patients over a given 12-month period to yield 200 completed surveys from the two semiannual surveys, a census of all survey-eligible patients will comprise the sample. Most of the ICH facilities need to survey all of their eligible patients at least once during the course of a calendar year and most patients are sampled twice within a given year.

Facilities with 1–200 Unique Patients. A census of all ICH patients is conducted for facilities with fewer than 201 eligible ICH patients at each semiannual sampling wave. Thus, patients at these smaller ICH facilities are sampled twice in a given year.

Facilities with 201–400 Patients. For dialysis centers that have between 201 and 400 eligible ICH patients at the first semiannual sampling period (the Spring Survey), a simple random sample of 200 patients is selected for that sampling period to obtain 100 completed responses. For the Fall Survey, the goal is to obtain an additional 100 completed interviews while attempting to minimize patient overlap of patients between the first and second semiannual waves of sampling. To achieve this goal, we first identify all eligible patients from that facility who were not selected for the Spring Survey. If the number of eligible patients not selected in the Spring Survey is equal to or exceeds 200 then we select a simple random sample of 200 from these patients for the Fall Survey. If there are 200 or fewer patients then all of these patients are selected for the Fall Survey. To obtain 200 completed surveys, we also select a simple random sample of the appropriate size from the patients who were selected in the Spring Survey, provided that they are still receiving treatment at that facility and still meet all of the survey eligibility requirements.

B.2 Information Collection Procedures

Three modes of survey administration are allowed during the national implementation of the ICH CAHPS Survey to give ICH facilities options for their preferred survey administration modes, based on their goals and resources. These three modes are described below:

• Mail-only Mode: ICH CAHPS Survey data collection for the mail-only mode consists of mailing a prenotification letter, explaining the purpose of the survey, and letting patients know that a hardcopy questionnaire will soon be sent to them via mail. A questionnaire package consisting of a cover letter, the ICH CAHPS questionnaire, and a pre-addressed, postage-paid return envelope is sent to all sample patients two weeks following the pre notification letter. A second mailing containing a questionnaire and cover letter is mailed to all sample patients who do not respond to the first mailing within four weeks after the first questionnaire package is mailed. Data collection ends ten weeks after the first questionnaire package is mailed.

• Telephone-only Mode: In this mode, all sample patients are first sent a pre notification letter letting them know that a professional interviewer working on the ICH CAHPS Survey will soon be contacting them via telephone. All sample patients are then contacted by professional telephone interviewers who are trained on ICH CAHPS survey administration procedures, including procedures for working with dialysis patients. Telephone interviewers are trained on the appropriate response to common questions and concerns that dialysis patients might have about survey participation, and are required to offer to administer the interview in different call- backs if the sample patient indicates that he or she cannot complete the interview in one call. A maximum of 10 telephone contact attempts per patient are attempted to complete the survey. Data collection ends ten weeks after the initial telephone contact begins.

• Mixed Mode: All sampled patients included in the mixed mode data collection sample receive a prenotification letter letting them know that we will soon be contacting them via mail. We then send an initial mailing of a questionnaire, cover letter, and postage-paid return envelope that patients included in the mail-only sample receive. Sample patients assigned to this mode who do not respond to the mail survey within four weeks after the questionnaire is mailed will be assigned to the telephone follow- up. Telephone interviewers make up to 10 attempts to complete the interview by phone with all mail survey non-respondents included in the mixed-mode sample. Data collection ends ten weeks after the first questionnaire package is mailed.

Survey vendors who wish to become “approved” to conduct the ICH CAHPS Survey on behalf of ICH facilities must submit an application and complete the ICH CAHPS survey vendor training, which provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission.

B.3 Methods to Maximize Response Rate

Response rates range average 27.8% for mail-only surveys, 23.9% for phone-only surveys and 32.2% for mixed mode surveys, for an overall average response rate of 31.9%, as shown in Exhibit B.2. Every effort will be made to maximize patient response rates while retaining the voluntary nature of the ICH CAHPS Survey.

Exhibit B.2. Response Rates for ICH CAHPS Semiannual Surveys

Overall	Survey Period	# Sampled	# Eligible	# Completed Surveys	RR
	2014 Fall	329,493	305,590	113,935	37.3%
	2015 Spring	363,181	337,316	114,847	34.0%
	2015 Fall	356,721	324,139	103,808	32.0%
	2016 Spring	363,670	339,092	107,582	31.7%
	2016 Fall	347,879	323,386	100,184	31.0%
	2017 Spring	348,024	323,185	111,851	34.6%
	2017 Fall	351,700	321,818	105,120	32.7%
	2018 Spring	332,183	307,078	98,611	32.1%
	2018 Fall	344,827	316,093	102,757	32.5%
	2019 Spring	353,703	325,690	98,868	30.4%
	2019 Fall	356,127	324,732	96,255	29.6%
	2020 Spring	29,676	26,827	8,571	31.9%
	2020 Fall	350,949	322,607	96,805	30.0%
	2021 Spring	345,448	316,014	82,987	26.3%
	All Survey Periods	4,573,581	4,213,567	1,342,181	31.9%

After the sample file is downloaded, survey vendors must verify mailing addresses using a commercial address update service. Each prenotification letter envelope contains the CMS logo along with the survey vendor’s return address. Furthermore, each questionnaire mailing includes a cover letter containing the CMS logo and information about the survey, including sponsorship and objectives, a description of how survey results will be used, and the vendor’s toll-free telephone number that sampled patients can contact if they have questions or need additional information about the survey. Because some dialysis patients may be reluctant to participate because of fear of retribution from their dialysis centers, the mail survey materials contain assurances that the patients’ survey responses are kept private and cannot be linked to their names. In addition, CMS requires that the mail survey cover letters include the CMS logo and signature of a CMS representative.

We require that all mail survey vendors use current best practices in the survey materials to enhance response rates. These best practices include using a simple font no smaller than 12 point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire.⁰¹

For sample patients included in the mail data collection for the ICH CAHPS Survey, the second questionnaire mailing is expected to increase the response rate. The cover letter included in the second questionnaire package to mail survey nonrespondents contains a stronger appeal for the sample patient’s help on this survey, including indicating that the survey is an opportunity for them to provide input on the quality of dialysis care dialysis patients receive. To maximize response rates for the telephone-only mode and the telephone follow-up of the mixed- mode survey, we require that up to 10 attempts be made to reach each sample patient, with those attempts varying by day of the week and time of day. Telephone interviewers are trained on how to answer the questions that are most frequently asked by sample patients, and to address any concerns that they may have about participating in the survey. Because some dialysis patients may not feel well on the day that they receive dialysis treatments, telephone interviewers are instructed to offer to call back at a time that is better for the sample patient, and offer to conduct the telephone interview on two or more different calls.

B.4 Tests of Procedures

CMS used data from the mode experiment to assess the effects of data collection mode, patient characteristics, and nonresponse on survey results. We used the data from the mode experiment to develop models that will be used to statistically adjust survey results from the national survey to control for factors that are beyond the control of the ICH facilities. The following analyses were conducted on mode experiment data:

• Analyses of individual survey items to assess missing data and item distributions • Statistical analysis of patient mix effects and nonresponse patterns on survey results

• Hypothesis testing to detect differences in key variables between modes.

B.5 Statistical Consultation and Independent Review

This sampling and statistical plan was prepared by RTI International. The primary statistical design was provided by Gordon Brown and refined by Scott Scheffler. Amy Couzens now serves as the Sampling Task Leader at RTI and can be reached by telephone at 919-541- 5910or by email at [email protected].

0 Dillman, Don A. (1999). Mail and Other Self-Administered Surveys in the 21st Century: The Beginning of a New Era. The Gallup Research Journal, 2(1): 121-140.

8