ICH CAHPS – Supporting Statement Part B
The Centers for Medicare & Medicaid Services (CMS) is requesting clearance from the Office of Management and Budget (OMB) to implement nationally the In-center Hemodialysis CAHPS (ICH CAHPS) Survey to measure patients’ experience of care with in-center hemodialysis (ICH) facilities under Contract Number HHSM-500-2012-00151G.
Medicare-certified dialysis facilities that serve more than 30 survey-eligible sample members during a calendar year are required to contract with a CMS-approved survey vendor to collect and submit ICH CAHPS Survey data on their behalf. All approved ICH CAHPS Survey vendors are required to use standardized survey administration protocols and specifications provided by CMS. The national implementation of the ICH CAHPS survey is conducted on a semiannual basis, with sampling and data collection activities conducted as shown in Exhibit B.1.
Exhibit B.1. Sampling Window for the ICH-CAHPS Semiannual Surveys
|
Spring survey |
Fall survey |
Sampling window (months in which patients received ICH care) |
January–April |
June–September |
Sample selected |
May |
October |
Data collection period |
June–August |
November–January |
The national implementation of the survey is fielded on a rolling semiannual basis. The results for each semiannual survey are merged with data from the immediately preceding semiannual survey for developing composite measures for public reporting. A primary issue is obtaining sufficient sample size within a facility to produce confidence intervals for point estimates that are sufficiently narrow. Approximately 200 observations are needed per year to produce a confidence interval that has a bound of +/−0.07. Approximately 81.4% of Medicare-certified ICH facilities serve 99 or fewer unique patients a year; 17.6% serve between 100 and 199 patients a year, and 1.0% serve more than 200 patients each year.1
For each semiannual wave, patients who received care during the sampling window and who meet survey eligibility criteria will either be chosen randomly or selected with certainty depending on the number of survey-eligible patients the ICH facility served during the preceding 12-month period. If a facility’s patient volume is large enough, the number of patients sampled for that facility for each semiannual survey will be sufficient to yield a minimum of 200 completed surveys over the two semiannual surveys. If a facility does not serve enough survey-eligible patients over a given 12-month period to yield 200 completed surveys from the two semiannual surveys, a census of all survey-eligible patients will comprise the sample. Depending on the data collection mode the ICH facility decides to use, most of the ICH facilities need to survey all of their eligible patients at least once during the course of a calendar year and most patients are sampled twice within a given year.
Facilities with 1–200 Unique Patients. A census of all ICH patients is conducted for facilities with fewer than 201 eligible ICH patients at each semiannual sampling wave. Thus, patients at these smaller ICH facilities are sampled twice in a given year.
Facilities with 201–400 Patients. For dialysis centers that have between 201 and 400 eligible ICH patients at the first semiannual sampling period (the Spring Survey), a simple random sample of 200 patients is selected for that sampling period to obtain 100 completed responses. For the Fall Survey, the goal is to obtain an additional 100 completed interviews while attempting to minimize patient overlap of patients between the first and second semiannual waves of sampling. To achieve this goal, we first identify all eligible patients from that facility who were not selected for the Spring Survey. If the number of eligible patients not selected in the Spring Survey is equal to or exceeds 200 then we select a simple random sample of 200 from these patients for the Fall Survey. If there are 200 or fewer patients then all of these patients are selected for the Fall Survey. To obtain 200 completed surveys, we also select a simple random sample of the appropriate size from the patients who were selected in the Spring Survey, provided that they are still receiving treatment at that facility and still meet all of the survey eligibility requirements.
Three modes of survey administration are allowed during the national implementation of the ICH CAHPS Survey to give ICH facilities options for their preferred survey administration modes, based on their goals and resources. These three modes are described below:
Mail-only Mode: ICH CAHPS Survey data collection for the mail-only mode consists of mailing a pre-notification letter, explaining the purpose of the survey, and letting patients know that a hardcopy questionnaire will soon be sent to them via mail. A questionnaire package consisting of a cover letter, the ICH CAHPS questionnaire, and a pre-addressed, postage-paid return envelope is sent to all sample patients two weeks following the pre notification letter. A second mailing containing a questionnaire and cover letter is mailed to all sample patients who do not respond to the first mailing within four weeks after the first questionnaire package is mailed. Data collection ends ten weeks after the first questionnaire package is mailed.
Telephone-only Mode: In this mode, all sample patients are first sent a pre-notification letter letting them know that a professional interviewer working on the ICH CAHPS Survey will soon be contacting them via telephone. All sample patients are then contacted by professional telephone interviewers who are trained on ICH CAHPS survey administration procedures, including procedures for working with dialysis patients. Telephone interviewers are trained on the appropriate response to common questions and concerns that dialysis patients might have about survey participation, and are required to offer to administer the interview in different call-backs if the sample patient indicates that he or she cannot complete the interview in one call. A maximum of 10 telephone contact attempts per patient are attempted to complete the survey. Data collection ends ten weeks after the initial telephone contact begins.
Mixed Mode: All sampled patients included in the mixed-mode data collection sample receive a pre-notification letter letting them know that we will soon be contacting them via mail. We then send an initial mailing of a questionnaire, cover letter, and postage-paid return envelope that patients included in the mail-only sample receive. Sample patients assigned to this mode who do not respond to the mail survey within four weeks after the questionnaire is mailed will be assigned to the telephone follow-up. Telephone interviewers make up to 10 attempts to complete the interview by phone with all mail survey nonrespondents included in the mixed-mode sample. Data collection ends ten weeks after the first questionnaire package is mailed.
Survey vendors who wish to become “approved” to conduct the ICH CAHPS Survey on behalf of ICH facilities must complete the ICH CAHPS survey vendor training, which provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission.
Results from the mode experiment indicate that the response rate will vary based on data collection mode, with a 30.4% response rate from mail-only sample patients, 30.7% for telephone-only surveys, and 41.8% for mixed-mode data collection efforts, for an overall response rate of approximately 33.6%. The mode experiment response rates are consistent with the actual mail only and overall response rates of the survey, and lower for telephone only and mixed mode as shown in Exhibit B.2. Every effort will be made to maximize patient response rates while retaining the voluntary nature of the ICH CAHPS Survey.
Exhibit B.2. Response Rates for ICH-CAHPS Semiannual Surveys
|
Mail Only |
Phone Only |
Mixed Mode |
Overall |
||||
|
N |
RR |
N |
RR |
N |
RR |
N |
RR |
Fall 2014 |
31,929 |
33.1% |
11,760 |
23.1% |
261,901 |
38.4% |
305,590 |
37.3% |
Spring 2015 |
24,734 |
31.1% |
9,047 |
21.7% |
303,535 |
34.7% |
337,316 |
34.4% |
Fall 2015 |
23,750 |
27.5% |
8,004 |
22.4% |
292,385 |
32.7% |
324,139 |
32.0% |
Overall |
80,413 |
30.8% |
28,811 |
22.5% |
857,821 |
35.1% |
967,045 |
34.4% |
After the sample file is downloaded, survey vendors must verify mailing addresses using a commercial address update service. Each prenotification letter envelope contains the CMS logo along with the survey vendor’s return address. Furthermore, each questionnaire mailing includes a cover letter containing information about the survey, including sponsorship and objectives, a description of how survey results will be used, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. Because some dialysis patients may be reluctant to participate because of fear of retribution from their dialysis centers, the mail survey materials contain assurances that the patients’ dialysis facility will not have access to their survey responses linked to their names or any other information that can identify the patients. In addition, CMS requires that the mail survey cover letters be printed on the survey vendor’s letterhead and signed by the survey vendor’s ICH CAHPS project manager.
We require that all mail survey vendors use current best practices in the survey materials to enhance response rates. These best practices include using a simple font no smaller than 12 point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire (Dillman, 2009).
For sample patients included in the mail and mixed-mode data collection for the ICH CAHPS Survey, the second questionnaire mailing is expected to increase the response rate. The cover letter included in the second questionnaire package to mail survey nonrespondents contains a stronger appeal for the sample patient’s help on this survey, including indicating that the survey is an opportunity for them to provide input on the quality of dialysis care dialysis patients receive. To maximize response rates for the telephone-only mode and the telephone follow-up of the mixed-mode survey, we require that up to 10 attempts be made to reach each sample patient, with those attempts varying by day of the week and time of day. Telephone interviewers will be trained on how to answer the questions that are most frequently asked by sample patients, and to address any concerns that they may have about participating in the survey. Because some dialysis patients may not feel well on the day that they receive dialysis treatments, telephone interviewers are instructed to offer to call back at a time that is better for the sample patient, and offer to conduct the telephone interview on two or more different calls.
CMS used data from the mode experiment to assess the effects of data collection mode, patient characteristics, and nonresponse on survey results. We used the data from the mode experiment to develop models that will be used to statistically adjust survey results from the national survey to control for factors that are beyond the control of the ICH facilities. The following analyses were conducted on mode experiment data:
Analyses of individual survey items to assess missing data and item distributions
Statistical analysis of patient mix effects and nonresponse patterns on survey results
Hypothesis testing to detect differences in key variables between modes.
More information on the mode experiment and patient-mix adjustment is found in Attachment E.
This sampling and statistical plan was prepared by RTI International. The primary statistical design was provided by Gordon Brown of RTI International. Dr. Brown has since been replaced by Scott Scheffler, who can be reached by telephone at (919) 248-8540-or by e‑mail at [email protected].
Dillman, Don A. (1999). Mail and Other Self-Administered Surveys in the 21st Century: The Beginning of a New Era. The Gallup Research Journal, 2(1): 121-140.
Dora, M., Zalai, M. D., & Novak, M. (2008). Depressive disorders in patients with chronic kidney disease. Primary Psychiatry, 15(1), 66–72.
Kurella Tamura, M., Wadley, V., Yaffe, K., McClure, L. A., Howard, G., Go, R., Allman, R. M., Warnock, D. G., & McClellan, W. (2008). Kidney function and cognitive impairment in US adults: The Reasons for Geographic and Racial Differences in Stroke (REGARDS) study. American Journal of Kidney Diseases, August 2008.
Medicare Payment Advisory Commission (MedPac). (2003, October). Report to the Congress: Modernizing the outpatient dialysis payment system. Retrieved from http://www.medpac.gov/documents/oct2003_Dialysis.pdf
U.S. Bureau of Labor Statistics (2010). National Compensation Survey: Occupational wages in the United States, July 2010. U.S. Department of Labor, Bureau of Labor Statistics. Available at http://www.bls.gov/ncs/ncswage2010.pdf. Based on average wages.
1 Based on data from the 2011 End Stage Renal Disease Facility Survey.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | The National Implementation of the Home Health Care CAHPS Survey |
| Subject | OMB Supporting Statement for Home Health Care CAHPS Survey |
| Author | Centers for Medicare & Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |