The Home Health Care CAHPS
Survey
Part B
Collection of Information
Employing
Statistical Methods
TABLE OF CONTENTS
Section Page
B. Collection of Information Employing Statistical Methods 1
B.1 Potential Respondent Universe and Sample Selection Method 1
B.1.2b National Implementation Sampling Specifics 6
B.2 Information Collection Procedures 8
The national implementation of HHCAHPS is voluntarily sponsored implemented by home health agencies (HHAs); and each participating HHA contracts with an independent survey vendor to conduct the survey for it.
HHAs assemble a census of their patients (both current and discharged) for the samplinge windowframe, defined as a calendar month. Each HHA submits a file containing patient information for all patients to whom the HHA provided home health care during the samplinge month to its contracted survey vendor (for the national implementation). The national survey is fielded on a rolling basis, and the results for each quarter merged with data from the 3 immediately preceding quarters and analyzed. The sample frame for the national implementation is constructed will be assembled at the level of the CMS Certification Number level, which was (formerly known as the Medicare provider number and is), the unit of comparison for survey results reported on the Home Health Compare website.
The sample frame prepared by the HHA contains all the patient data needed for fielding the survey and data needed for analysis.
The HHA’s survey vendor (for the national implementation) reviews the frame and excludes any patients who are not eligible to participate in the Home Health Care CAHPS Survey. Patients ineligible for the survey are those who
are receiving hospice or are discharged to hospice,
are deceased when the sample is drawn,
are under 18 years of age at any time during their stay,
did not have at least one skilled home health visit in the sample month and at least two home health care visits during a 2-month look-back period starting with the last day of the sampled month,
received routine maternity care only are maternity patients,
are “no publicity” patients,
are residing in states that prohibit the surveying of certain patients (for example, with HIV, or other patients with protected health information status),
are receiving only nonskilled (aide) care, or
were included in an HHCAHPS sample during the last 5 months.
The requirement that a patient not be sampled more than twice a year is intended to reduce burden on individual patients and to increase the probability of response.
For the national implementation of the Home Health Care CAHPS Survey, each participating HHA will sends to its contracted survey vendor each month a patient sample frame containing information about each patient who received home health care during the sample month, with sufficient information for the vendor to determine exclusions and with information needed for both fielding the survey and for patient-mix adjustment. The survey vendor will removes from the sample frame patients who do not meet survey eligibility requirements and then draws a random sample of the remaining patients.
Survey vendors working under contract with HHAs are will be instructed to use a reliable program to generate random numbers for sampling. The Centers for Medicare & Medicaid Services (CMS) will recommends that survey vendors use the free program RATSTATS, available from the Department of Health and Human Services, Office of Inspector General website, or some other validated sample selection program such as SAS to select the sample. The sampling procedure recommended is simple random sampling, but disproportionate and proportional stratified random sampling is may be allowed since some HHAs may want to analyze their own data and view survey results for individual branches. HHAs that deviate from simple random sampling (using disproportionate sampling) are will be required to request an exception and obtain approval from CMS. An exception is will be permitted if the minimum sample is 10 per strata and the information needed to weight the datato crease weights is reported to CMSRTI.
The national survey is fielded on a rolling basis, and the results for each quarter merged with data from the 3 immediately preceding quarters and analyzed. Although the national implementation sampling is conducted on a monthly basis (with the survey initiated for each monthly sample within 3 weeks after the sample month ends), data from four quarters will be accrued, aggregated, analyzed, and reported on a quarterly basis, with the data from the most current quarter replacing data from the oldest of the four quarters. For 4 calendar quarters, a minimum of 300 completed surveys is the target for each participating HHA. If an HHA’s patient population is too small to yield 300 completed surveys, a census must will be surveyed. The 300 completed surveys needed for analysis is derived from the formula for the precision of a proportion with the estimate at .5, the confidence interval of about +/- 0.05, and a confidence level of 95%. (Many agencies, with a substantial sampling fraction, can achieve a higher precision because of the finite population correction factor.)
In the national implementation of the Home Health Care CAHPS Survey, the number of patients needed for selection each month to yield a minimum of 300 completed surveys per year (25 per month) is will ultimately be determined by each HHA and its survey vendor. These will differ by HHA. The mode of administration of the survey is will be an important determining factor in response rates. Each agency survey vendor uses its experience on other surveys with home health patients and/or other similar populations, the data collection mode, and expected response rates as guides for calculating the monthly sample sizes that are needed for the Home Health Care CAHPS Survey. Using the estimated response above, the estimated sample sizes for HHAs participating in the national implementation of the Home Health Care CAHPS Survey are the following:
Mode |
Response Rate |
Sample Size for |
30.0% |
84 |
|
Phone |
28.0% |
90 |
Mixed |
34.5% |
73 |
Each agency survey vendor useswill use its experience on other surveys with home health patients and/or other similar populations, the data collection mode, and expected response rates as guides for calculating the monthly sample sizes that are will be needed for the Home Health Care CAHPS Survey.
The sampling rate to achieve these sample sizes indicates that HHAs with monthly frame sizes of 90 or below should start with a sample equal to the sample frame. That is, all patients who meet the eligibility criteria will be included in the survey sample. For HHAs with larger sampling frames the sampling rate can be reduced, although it clearly will be higher than 50% until the frame exceeds about 180 eligible patients per month. CMS will recommend that prior to starting the national implementation, survey vendors acquire from client HHAs sample frame information for each of the 3 or 6 months prior to the beginning of the national implementation to determine an appropriate sampling rate to use during the national implementation. Monthly sample size rates are should be based on the number of patients who meet survey eligibility criteria in the frames after the first test month, since that month will not have any patients who are ineligible for the survey because they would be sampled during the first month of the test file.
Three modes of survey administration are will be allowed during the national implementation of the Home Health Care CAHPS Survey to give HHAs options in how they would like to administer the survey, based on their goals and resources. These three modes are described below:
Mail-only mode
Mailing of the questionnaire and cover letter to all sampled patients.
Second mailing of the questionnaire with a cover letter to sample patients who do not respond to the first mailing within 3 weeks after the first questionnaire package is mailed.
Telephone-only mode
A maximum of five telephone contact attempts per patient to complete the survey.
Mixed-mode
Mailing of the questionnaire and cover letter to all sample patients.
Telephone follow-up with all sample patients who do not respond to the questionnaire mailing. A maximum of five telephone contact attempts per patient will be made to complete the survey.
Data collection for each sampled patient must be initiated no later than 3 weeks (21 days) after the close of the sample month. Once data collection begins, it must be closed out within 6 weeks. These same data collection modes will be used in the mode experiment using the same protocols.
Survey vendors that arewho wish to become “approved” to conduct the Home Health Care CAHPS Survey on behalf of HHAs must complete the Home Health Care CAHPS survey vendor training, which will provides detailed guidance on the protocols and guidelines for all aspects of survey implementation, from sample selection to data collection and data submission. As of the date of this submission, CMS anticipates that the first training sessions for vendors will be offered in early summer 2009; however, this is contingent upon receiving OMB approval by the end of April 2009. The national implementation is expected to start in the summer of 2009.
Every effort is will be made to maximize patient response rates, while retaining the voluntary nature of the Home Health Care CAHPS Survey. Each questionnaire mailing will includes a cover letter explaining what the survey is about, who is conducting it and why, and the name and toll-free telephone number of a survey staff member that sampled patients can contact if they have questions or need additional information about the survey. For the mail-only mode of administration, both RTI (for the mode experiment) and survey vendors (for the national implementation) will use best practices in survey materials to enhance response rates. These best practices include using a simple font no smaller than 10 point size in the survey cover letters, allowing ample white space between questions in the questionnaire, avoiding a format that displays the questions as a matrix, using a unique subject identification number on the questionnaire rather than printing the sample member’s name, and displaying the OMB number and expiration date on the questionnaire. The second mailing for the mail only implementation is expected to increase the response rate, as is the telephone follow-up portion of the mixed-mode implementation.
Based on the evidence from the HHCAHPS Mode Experiment, RTI project staff recommends using the following variables as adjustment factors on HHCAHPS Survey results. these adjusters Home Health Care CAHPS. These adjusters are the following:
Proxy respondent used
Non-English language as the primary language spoken at home
Age (five levels: 18-49, 50-64, 65-74, 75-84, and 85 plus)
Education (five levels: less than 8th grade, 8th grade to less than high-school graduate, high-school graduate or GED, some college, and college graduate or more)
Self-reported mental/emotional health status (three levels: Excellent/Very Good, Good, Fair/Poor)
Whether a patient lives alone
ADL deficit score, and
Two mental health diagnosis groups (schizophrenia and dementia).
RTI staff will re-estimate regression and impact analyses during the Home Health Care CAHPS national implementation using data from a large number of HHAs to assess the stability of the evidence for these adjusters. RTI did not find that the mode of survey administration (mail only mode, telephone only mode, or mixed mode) had any significant impact on patient responses in HHCAHPS.
To achieve the purposes of the mode experiment the following analyses will be conducted:
Analyses of individual survey items will assess missing data and item distributions.
Hypothesis testing will detect differences in key variables between modes.
The analysis of individual items and the hypothesis testing will form the basis for constructing an adjustor to be used for telephone and mixed-mode surveys.
Because home health care is a new area for CAHPS®, it is yet to be determined (through analyses from the mode experiment) which patient-mix adjusters may be necessary for reporting of the national survey results. Data from the field test suggest that self-reported overall health status, education and age should be included in the patient mix models. One of the purposes of the mode experiment will be to explore the usefulness of these and other variables for patient-mix adjustment. RTI will evaluate whether the ranking of home health agencies differs for adjusted and unadjusted Home Health Care CAHPS® results.
This sampling and statistical plan was prepared by CMS RTI International and reviewed by RTICMS. The primary statistical design was provided by Melvin Ingber of RTI International. Dr. Ingber can be reached by telephone at (410) 730-1506 or by e-mail at [email protected].
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | The National Implementation of the Home Health Care CAHPS Survey |
| Subject | OMB Supporting Statement for Home Health Care CAHPS Survey |
| Author | Centers for Medicare & Medicaid Services |
| File Modified | 0000-00-00 |
| File Created | 2021-02-01 |