The Home Health Care CAHPS® Survey
Part B
Collection of Information
Employing Statistical Methods
TABLE OF CONTENTS
Section Page
B. Collection of Information Employing Statistical Methods 1
B.1 Potential Respondent Universe and Sample Selection Method 1
B.1 Sampling Patients for the National Implementation 1
B.1.1a National Implementation Sampling Specifics 2
B.2 Information Collection Procedures 4
B.3 Methods to Maximize Response Rate 5
B.5 Statistical Consultation and Independent Review 6
i
This PRA requests renewal of the national implementation of the HHCAHPS Survey, conducted by approved HHCAHPS survey vendors doing the survey on behalf of Medicare home health agencies (HHAs). Although no survey mode effects were found in the 2009 Mode Experiment, there are differences in results based on patient characteristics, particularly whether the patient lives alone, does the patient self-report fair to poor health, does the patient report poor mental health status, the age of the patient, educational level, whether the patient has been diagnosed with schizophrenia or dementia, whether the survey was completed by a proxy, and the language in which the survey was completed. Every time we update our public reporting file, we adjust it for the patient characteristics in that file, so the patient mix adjusters change with every file. We post the patient mix adjusters on our website, https://homehealthcahps.org along with downloadable databases of HHCAHPS data.
The sampling requirements for national implementation are described below.
For national implementation, HHAs assemble a census of their patients (both current and discharged) for the sampling window, defined as a calendar month. Every month, each HHA submits a file to its contracted survey vendor containing patient information for all patients to whom the HHA provided home care during the sampling month . The national survey is fielded on a rolling basis, and the results for each quarter are merged with data from the three immediately preceding quarters and analyzed. The sample frame for the national implementation is assembled at the level of the CMS Certification Number (CCN), and the CCNs (Medicare home health agencies) comprise the units of comparison for HHCAHPS survey results reported on the Home Health Compare website every three months, in the months of January, April, July, and October.
Each HHA’s sampling frame contains all the patient data needed for both fielding the survey and analyzing the data for public reporting. The HHA’s survey vendor reviews the frame and excludes any patients who are not eligible to participate in the Home Health Care CAHPS® Survey. Patients ineligible for the survey are those who
are receiving hospice or are discharged to hospice,
are deceased when the sample is drawn,
are under 18 years of age at the end of the sample month,
did not have at least one skilled home health visit in the sample month and at least two home health care visits during a two-month look-back period covering the sample month and the prior month,
are maternity patients,
are “no publicity” patients,
are receiving only nonskilled (aide) care,
are state-regulated patients, or
are patients who were sampled for HHCAHPS during the last 5 months.
The requirement that a patient not be sampled more than twice a year is intended to reduce burden on individual patients and to increase the probability of response.
For the national implementation of the Home Health Care CAHPS® Survey, each participating HHA sends to its contracted survey vendor a patient sample frame each month containing information about each patient who received home health care during the sample month, with sufficient information for the vendor to determine exclusions and with information needed for both fielding the survey and for patient-mix adjustment. The survey vendor removes from the sample frame patients who do not meet survey eligibility requirements and then draws a random sample of the remaining patients.
Survey vendors working under contract with HHAs are instructed to use a reliable program to generate random numbers for sampling. The Centers for Medicare & Medicaid Services (CMS) has continually recommended to survey vendors that they use the free program
RAT-STATS, available from the Department of Health and Human Services, Office of Inspector General website, or some other validated sample selection program such as SAS to select the sample. The sampling procedure recommended is simple random sampling, but disproportionate and proportionate stratified random sampling may be allowed since some HHAs may want to analyze their own data and view survey results for individual branches. HHAs that deviate from simple random sampling using disproportionate sampling are required to request an exception and to obtain approval from CMS. An exception is permitted if the minimum sample is 10 per strata and the information needed to increase weights is reported to RTI.
Although the national implementation sampling is conducted on a monthly basis (with the survey initiated for each monthly sample within 3 weeks after the sample month ends), data from four quarters are accrued, aggregated, analyzed, and publicly reported on a quarterly basis, with the data from the most current quarter replacing data from the oldest of the prior four quarters, when the data are posted on Home Health Compare on www.medicare.gov. For four calendar quarters, a minimum of 300 completed surveys is the target for each participating HHA. If an HHA’s patient population is too small to yield 300 completed surveys, a census is surveyed. The 300 completed surveys needed for analysis is derived from the formula for the precision of a proportion with the estimate at .5, the confidence interval of about +/- 0.05, and a confidence level of 95%. (Many agencies, with a substantial sampling fraction, can achieve a higher precision because of the finite population correction factor.)
In the national implementation of the Home Health Care CAHPS Survey, the number of patients needed for selection each month to yield a minimum of 300 completed surveys per year (25 per month) is determined by each HHA and its survey vendor. The mode of administration of the survey may be an important determining factor in response rates. Based on the target number of completed interviews the estimated sample sizes for HHAs participating in the national implementation of the Home Health Care CAHPS Survey are the following:
|
|
Sample Size for |
Mode |
Response Rate |
25 Responses/Month |
28.0% |
89 |
|
Phone |
25.0% |
100 |
Mixed |
32.0% |
78 |
Each agency’s survey vendor uses its experience on other surveys with home health patients and/or other similar populations, the data collection mode, and expected response rates as guides for calculating the monthly sample sizes that will be needed for the Home Health Care CAHPS Survey.
About 89% of HHAs in HHCAHPS use the Mail Mode, 6% use Phone Mode, and 5% use Mixed Mode.
The sampling rate to achieve these sample sizes indicates that HHAs with monthly frame sizes of 100 or below may need to start with a sample equal to the sample frame depending on their chosen mode. That is, all patients who meet the eligibility criteria will be included in the survey sample. For HHAs with larger sampling frames the sampling rate can be reduced, although it clearly will be higher than 50% for all modes until the frame exceeds almost 160 eligible patients per month. Monthly sample size rates should be based on the number of patients who meet survey eligibility criteria in the frames after the first test month, since that month does not have any patients who are ineligible for the survey because they would be sampled during the first month of the test file.
Three modes of survey administration are allowed during the national implementation of the Home Health Care CAHPS Survey to give HHAs options in how they would like to administer the survey, based on their goals and resources. These three modes are described below:
Mail-only mode
Mailing of the questionnaire and cover letter to all sampled patients.
Second mailing of the questionnaire with a cover letter to sample patients who do not respond to the first mailing within 3 weeks after the first questionnaire package is mailed.
Telephone-only mode
A maximum of five telephone contact attempts per patient to complete the survey.
Mixed-mode
Mailing of the questionnaire and cover letter to all sample patients.
Telephone follow-up with all sample patients who do not respond to the questionnaire mailing. A maximum of five telephone contact attempts per patient will be made to complete the survey.
Data collection for each sampled patient must be initiated no later than 3 weeks (21 days) after the close of the sample month. We do allow HHAs to apply for a late fielding request up to the 14th day in the following month. For example, if an HHA was late sending its April patient list to the vendor (between May 21st-May 26th), then the HHA and the HHA’s vendor can apply for a late fielding request, which CMS usually accepts up to June 14th (in this example). The fielding period is 42 days and there are not extensions on that time frame. Again, once data collection begins, it must be closed out within 6 weeks.
Survey vendors who wish to become “approved” to conduct the Home Health Care CAHPS Survey on behalf of HHAs complete the Home Health Care CAHPS survey vendor training, which provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission. We post the list of approved HHCAHPS survey vendors on our website, https://homehealthcahps.org.
Every effort is made to maximize patient response rates, while retaining the voluntary nature of the Home Health Care CAHPS Survey. Each questionnaire mailing includes a cover letter explaining what the survey is about, who is conducting it and why, and the name and tollfree telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. For the mail-only mode of administration, our approved HHCAHPS survey vendors must use best practices in survey materials to enhance response rates. These best practices include using a simple font no smaller than 10 point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire. The second mailing for the mail only implementation is expected to increase the response rate, as is the telephone follow-up portion of the mixed-mode implementation.
These analyses are done with each quarterly data submission from survey vendors:
Conduct analyses of individual survey items to assess missing data and item distributions.
Perform Ordinary Least Squares regression on each CAHPS measure using national level HHCAHPS data and a set of patient-mix adjusters previously identified from the HHCAHPS Mode Experiment.
Construct the patient mix adjusters to be applied to each home health agency’s HHCAHPS results.
Patient mix adjust the 5 publicly reported CAHPS measures for each home health agency.
Assess the effects of patient mix adjustment in the current quarter by comparing to prior quarters to see if results are similar to prior results.
We did not find a survey mode effect for HHCAHPS in the 2009 Survey Mode Experiment; therefore, no adjustments for mode are being made. The following patient mix adjusters were identified as significant during the field test and are analyzed and confirmed with each quarter of submitted data: self-reported overall health status, self-reported
mental/emotional status, education, age, whether the patient lives alone, whether the survey was completed by a proxy, whether the patient has dementia or schizophrenia, and the language in which the survey was completed.
This sampling and statistical plan was prepared by RTI International and reviewed by CMS. RTI continues to evaluate the data in many different analyses. Recently, RTI has evaluated the outliers in the HHCAHPS survey data results, and they have analyzed how the questions contribute to our third composite that concerns specific health issues. RTI has additionally evaluated HHAs that receive footnotes in our publicly reported data. RTI’s primary statistical point-of-contact is Mr. Harper Gordek, MPH, RTI International, Research Triangle Park, NC (919) 541-1231.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | The National Implementation of the Home Health Care CAHPS Survey |
| Subject | OMB Supporting Statement for Home Health Care CAHPS Survey |
| Author | Centers for Medicare & Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2021-01-14 |