HomeHealthCAHPS_OMB_PartA-revised_july09

Home health care is an important and rapidly growing segment of the U.S. health care system. National health expenditure data show that spending for home health care was more than $47 billion in 2005―an 11.1% increase over the previous year. Public spending for home health services, which accounts for three quarters of such spending, rose 12.4% in 2005. Spending for home health care is expected to continue to increase. The Centers for Medicare & Medicaid Services (CMS) has projected that by 2010, spending for home health care will top $70 billion annually and is expected to be more than $110 billion annually by 2016. (These figures come from http://www.cms.hhs.gov/NationalHealthExpendData/. Last viewed August 27, 2007.)

Home health care is a key benefit covered under Medicare Part A. The benefit includes coverage of part-time, medically necessary skilled care (nursing, physical therapy, occupational therapy, and speech-language therapy) that is ordered by a physician. If patients are eligible for skilled services, they can also receive part-time assistance with personal care needs by a home health aide. Patients are required to be “homebound” as a condition of eligibility for Medicare home health benefits. Home health care services are delivered at home to patients who are recovering from care in hospitals or nursing homes; patients who are disabled; the frail elderly; and chronically or terminally ill persons in need of medical, nursing, or therapeutic treatment and assistance with the essential activities of daily living. There are approximately 9,000 Medicare-certified home health agencies throughout the United States. In 2006, more than 3 million beneficiaries were served, and 103,931,188 visits made. As baby boomers age, the need for patient-centered, cost-effective care will be a priority to CMS―78 million baby boomers are about to begin turning 65 (http://www.cms.hhs.gov/HomeHealthQualityInits/).

For some patients, home health care can be an alternative to more costly institutional health care. Monitoring and improving the quality of home health care, as with all Medicare services, is an important policy issue. Monitoring and ensuring home health care, in particular, can be a challenge because care is provided in patients’ residences and therefore lacks some of the oversight feasible in institutional settings. In 2001, Secretary Thompson of the Department of Health and Human Services (DHHS) announced the Quality Initiative to ensure the quality of health care for all Americans through accountability and public disclosure. The goals of the initiative are to empower consumers with quality-of-care information so they can make more informed decisions about their health care and to stimulate and support providers and clinicians to improve the quality of health care. The Quality Initiative was launched nationally in November 2002 for nursing homes and was expanded to home health agencies (the Home Health Quality Initiative) in 2003. A major gap in the information currently available regarding the quality of home health care is the lack of information from the patient perspective.

As part of the DHHS Transparency Initiative on Quality Reporting, CMS plans to implement a process to measure and publicly report patients’ experiences with home health care they receive from Medicare-certified home health agencies through the data collection effort described in this request: the Consumer Assessment of Healthcare Providers and Systems (CAHPS^®) Home Health Care Survey. The Home Health Care CAHPS Survey, which was developed and tested by the Agency for Healthcare Research and Quality (AHRQ) and is part of the family of CAHPS surveys, is a standardized survey for home health patients to assess their home health care providers and the quality of the home health care they receive. Prior to the Home Health Care CAHPS survey, there was no national standard for collecting data about home health care patients’ experience with their home health care.

In the first half of 2008, AHRQ conducted a field test of the Home Health Care CAHPS Survey to determine its length and contents and to test the reliability and validity of the survey items. After reviewing field test results with a technical expert panel consisting of home health industry experts, patient advocates, and researchers, the Home Health Care CAHPS Survey was finalized as a 34-item survey instrument. (See Appendix A for a copy of the Home Health Care CAHPS Survey questionnaire.) The survey contains questions about the patient’s interactions with the home health agency, interactions with the agency’s providers, provider care and communications, and patient characteristics. Patients will also be asked to provide an overall rating of the home health care they receive. CMS submitted the final Home Health Care CAHPS Survey to the National Quality Forum (NQF) for endorsement. The survey was endorsed March 31, 2009. The NQF endorsement represents the consensus opinion of many healthcare providers, consumer groups, professional organizations, purchasers, federal agencies, and research and quality organizations. As a result of the endorsement process, a few minor changes were made to the survey. The words “over the counter” were added to Questions 4 and 5. In Question 14, the word “important” was removed since a respondent may have difficulty determining which side effects are important. Questions regarding age and gender were removed from the survey since they are available from home health administrative data. The revised survey is 34 questions long.

Questions 1-25 on the instrument are the core survey items, and questions 26-36 are the “About You” questions. Five measures from this survey will be used for public reporting – 3 composite measures and 2 global ratings. The 3 composites cover “Care of Patients,” “Communication between Providers and Patients” and “Specific Care Issues.” The global items include the overall rating of agency care, and would you recommend this agency to friends and family.

Initially, confirmatory factor analysis (CFA) based on structural equation modeling (SEM) was conducted to see whether the field test data were consistent with the hypothesized composite structure. The CFA of the field test questionnaire revealed that the observed data did not fit this model. Following the poor CFA results, exploratory analyses were conducted to identify the structure underlying the observed responses. Analyses were conducted upon a random sample of 50% of the single-imputation data set. This enabled us to evaluate the generalizability of the final model in the other 50% of the data, as well as the data sets comprised of each of the other four imputations. An exploratory factor analysis (EFA) was conducted on the correlation matrix using the principle factor method with squared multiple correlations as initial communality estimates and oblique rotation (promax) with Kaiser normalization. The number of factors was determined by the eigenvalues, and the interpretability of the rotated factor pattern matrix.

The internal consistency reliability (alpha) (a measure of how well the items in a composite hang together) was .75 for Care of Patients, .73 for Communication between Providers and Patients and .84 for Specific Care Issues. The scaling success (a measure that summarizes the discriminant validity of the composites, that is, the degree to which each item correlates more highly with its own scale than it does with competing scales) is 88% for Care of Patients, 90% for Communication between Providers and Patients and 100% for Specific Care Issues.

The Care of Patients composite is produced by combining responses to four questions that ask:

“In the last 2 months of care, how often did home health providers from the agency seem informed and up-to-date about all the care or treatment you got at home?”
“In the last 2 months of care, how often did home health providers from this agency treat you as gently as possible?”
“In the last 2 months of care, how often did home health providers from this agency treat you with courtesy and respect?”
“In the last 2 months of care, did you have any problems with the care you got through this agency?”

The Communication between Providers and Patients composite is produced by combining responses to six questions that ask:

“When you first started getting home health care from this agency, did someone from the agency tell you what care and services you would get? “
“In the last 2 months of care, how often did home health providers from this agency keep you informed about when they would arrive at your home?”
“In the last 2 months of care, how often did home health providers from this agency explain things in a way that was easy to understand? “
“In the last 2 months of care, how often did home health providers from this agency listen carefully to you?”
“In the last 2 months of care, when you contacted this agency’s office did you get the help or advice you needed?”
“When you contacted this agency’s office, how long did it take for you to get the help or advice you needed?” {It is converted into a measure of whether the patient got help on the same day – yes/no}

The Specific Care Issues composite is produced by combining responses to seven questions that ask:

“When you first started getting home health care from this agency, did someone from the agency talk with you about how to set up your home so you can move around safely?”
“When you started getting home health care from this agency, did someone from the agency ask to see all the prescription and over-the-counter medicines you were taking?”
“When you started getting home health care from this agency, did someone from the agency ask to see all the prescription and over-the-counter medicines you were taking?”
“In the last 2 months of care, did you and a home health provider from this agency talk about pain?”
“In the last 2 months of care, did home health providers from this agency talk with you about the purpose for taking your new or changed prescription medicines?”
“In the last 2 months of care, did home health providers from the agency talk with you about when to take these medicines?”
“In the last 2 months of care, did home health providers from this agency talk with you about the side effects of these medicines?”

CMS has begun planning for a national implementation of the Home Health Care CAHPS survey, which will be conducted by multiple independent survey vendors working under contract with home health agencies. In 2008, CMS, with assistance from its contractor RTI International, is developing standardized data collection and data submission tools and procedures for survey vendors when implementing Home Health Care CAHPS on behalf of their home health agency clients. As part of this Home Health Care CAHPS coordination and implementation strategy, RTI will conduct a mode experiment to test three modes of data collection that will be used on the Home Health Care CAHPS Survey: mail, telephone, and mixed mode (mail with telephone follow-up of nonrespondents). A sample of home health agencies will be asked to participate in the mode experiment, which will be conducted in spring/summer 2009.

Recruitment and training of survey vendors that apply to become approved Home Health Care CAHPS vendors will begin in spring 2009. Home health agencies can voluntarily conduct the Home Health Care CAHPS Survey using an approved Home Health Care CAHPS survey vendor starting in summer 2009 and submit the data to CMS for public reporting. Data collection for the national survey of the Home Health Care CAHPS Survey will be conducted on an ongoing basis. Comparative results from the national Home Health Care CAHPS Survey will be publicly reported on Home Health Compare, located on the Medicare.gov website, and will be updated on a quarterly basis.

A.2 Purpose and Use of Information

The national implementation of the Home Health Care CAHPS Survey is designed to collect ongoing data from samples of home health care patients who receive skilled services from Medicare-certified home health agencies. The data collected from the national implementation of the Home Health Care CAHPS Survey will be used for the following purposes:

to produce comparable data on the patients’ perspectives of the care they receive from home health agencies,
to create incentives for agencies to improve the quality of care they provide through public reporting of survey results, and
to enhance public accountability in health care by increasing the transparency of the quality of care provided in return for the public investment.

Sampling and data collection will be conducted on a monthly basis. Survey results will be analyzed and reported on a quarterly basis, with publicly reported results based on one year’s worth of data.

As part of this information collection request for the national implementation of Home Health Care CAHPS, CMS is also requesting approval to conduct a randomized mode experiment with a sample of home health agencies. The mode experiment will compare the responses to the survey across the three proposed modes to determine whether adjustments are needed to ensure that the data collection mode does not influence the survey results. In addition, data from the mode experiment will be used to determine which, if any, patient characteristics may affect the patients’ rating of the care they receive and, if so, develop an adjustment model of those data based on those factors. CMS is working with its contractor to recruit approximately 100 home health agencies to participate in the mode experiment. The mode experiment will involve up to 23,000 home health care patients.

A.3 Use of Improved Information Technology

The national implementation is designed to allow independent survey vendors to administer the Home Health Care CAHPS Survey using mail-only, telephone-only, or mixed (mail with telephone follow-up) modes of survey administration. Experience with previous CAHPS surveys, including the field test of the Home Health Care CAHPS instrument, shows that mail, telephone, and mail with telephone follow-up data collection modes work well for respondents, vendors, and health care organizations. Any additional forms of information technology, such as web surveys, would not be feasible with this population, many of whom are expected to be ill, elderly, and lack access to the Internet.

A.4 Efforts to Identify Duplication

Some home health agencies already carry out their own patient experience of care surveys. These diverse surveys do not allow for comparisons across home health care agencies. Making comparative performance information available to the public can help consumers make more informed choices when selecting a home health care agency and can create incentives for home health care agencies to improve the care they provide. Vendors/home health care agencies will have the option to add their own questions to the Home Health Care CAHPS core questionnaire. If a home health agency/vendor plans to add their own questions, they need to add them after the core questions (questions 1 - 25). The “About You” section can be placed after the core items or following the home health agency-specific items. If a home health agency/vendor decides to add their own questions, they should pay attention to the length of the questionnaire. The longer the questionnaire, the greater the burden is on respondents.

A.5 Involvement of Small Entities

A.5.1 National Implementation

All Medicare-certified home health agencies (HHAs) can voluntarily sponsor a Home Health Care CAHPS Survey, including the many small home health care agencies. However, if they choose to participate in the CMS national implementation, agencies must contract with a survey vendor that has been approved by CMS. These approved survey vendors may include small survey firms. Survey respondents will be adult home health care patients who receive skilled home health care regardless of payer (i.e., including Medicare, Medicaid, and private payers). Each month, each HHA sponsoring a Home Health Care CAHPS Survey must prepare and submit to its survey vendor a file containing patient data on patients served the preceding month that will be used by the survey vendor to select the sample and field the survey. This file (essentially the sampling frame) for most home health agencies can be generated from existing databases with minimal effort. For some small HHAs, preparation of a monthly sample frame may require more time. However, data elements needed on the sample frame will be kept at a minimum to reduce the burden on all home health agencies.

The survey instrument and procedures for completing the instrument are designed to minimize burden on all respondents. No significant burden is expected for small agencies beyond providing their contracted vendor with a monthly file of patients served.

A.5.2 Mode Experiment

For the mode experiment, a sample of home health agencies that represent a broad range of agency sizes, including some small agencies, will be selected and asked to voluntarily participate in the mode experiment to ensure representation of patients across various types of home health agencies. Each month during a 3-month data collection period, participating HHAs will be asked to provide patient information that will be used for sample selection and fielding the mode experiment. Most participating home health agencies will be able to provide the sample frame with minimum effort. However, data elements needed on the sample frame will be kept at a minimum to reduce the burden on all agencies.

As in the national implementation, the survey instrument and procedures for completing it are designed to minimize burden on all respondents. No significant burden is expected for small agencies beyond providing RTI with a monthly file of patients served.

A.6 Consequences If Information is Collected Less Frequently

So that home health patients can assess the home health care they receive as soon as possible after a home health care visit, CMS will require that participating home health agencies provide a sample frame consisting of patients who received at least one home health visit during the sample month to their survey vendor on a monthly basis. Vendors will, in turn, be required to initiate the data collection from patients within 3 weeks after the sample month closes. Respondent burden is increased and the recall factor becomes a problem if patients are asked to recall their care experiences after longer lapses of time. Monthly sampling and continuous data collection (surveying the sample within 3 weeks after the sample window closes) will reduce the amount of time between when patients receive home health care and when they are surveyed. Respondent recall, especially with home health patients, will be enhanced, thus improving the quality of survey data and results. For this reason, CMS does not believe that a less frequent data collection period will result in the most accurate and complete data for public reporting and quality monitoring purposes. While data collection will be completed by vendors on a monthly basis, data will be submitted on a quarterly basis.

A.7 Special Circumstances

Some home health patients have chronic conditions which require long-term home health care. To reduce respondent burden, CMS proposes that home health care patients not be eligible for the survey more than once during a 6-month period.

A.8 Federal Register Notice and Outside Consultations

A.8.1 Federal Register Notice

Appendix B includes the text of the notice of this implementation in the Federal Register published for 60-day comment on January 9, 2009. Twenty three comments were received. The Home Health Care CAHPS Survey was initially discussed in the May 4, 2007, Federal Register (72 Fed.Reg. 25356, 25452). A copy of that notice is included as Appendix C.

A.8.2 Outside Consultations

AHRQ was responsible for the development and testing of the Home Health Care CAHPS Survey. As the lead agency, AHRQ worked with three grantee organizations to develop and test the survey instrument: the American Institutes for Research, the Yale/Harvard team, and RAND. An additional contractor, Westat, also participated in a supporting role. During the survey instrument development phase, AHRQ also consulted with a range of outside organizations and individuals representing state and federal government agencies and non-profit and private sector organizations. AHRQ convened technical expert panels on February 8, 2007, and July 15, 2008. Panel members for the instrument development included representatives from the following organizations:

AARP (American Association of Retired Persons)
Abt Associates Inc.
American Academy of Home Health Care Physicians
American Association for Homecare
American Association of Homes and Services for the Aging
American Hospital Association
American Occupational Therapy Association
American Physical Therapy Association
American Speech-Language-Hearing Association
Maryland Health Care Commission
National Association for Home Care & Hospice
National Center for Health Statistics, Centers for Disease Control and Prevention (CDC)
National Quality Forum
Paraprofessional Healthcare Institute
Professional Healthcare Resources, Inc.
Quality Insights of Pennsylvania
Quality Partners of Rhode Island
Veterans Health Administration
Visiting Nurse Associations of America (VNAA)

For the national implementation, CMS has worked with RTI International, a contractor operating in the role of implementation coordinator. RTI is responsible for developing the protocols required to ensure standardized administration of the Home Health Care CAHPS Survey, recruiting survey vendors, working with CMS to train multiple independent survey vendors, providing oversight of the approved vendors, and receiving and processing Home Health Care CAHPS Survey data collected and submitted by survey vendors. RTI will also be responsible for analyzing data from the mode experiment to determine the mode adjustment and the patient-mix adjustment model. During the national implementation, RTI will adjust the data for mode of survey administration, patient mix and nonresponse and provide comparative results for public reporting.

In addition, RTI has convened a technical expert panel composed of representatives from the home health industry, consumer advocacy organizations, the government, and research organizations. Members of the committee have provided guidance to RTI on the development of the design for the mode experiment and plans for the national implementation. RTI, CMS, and members of the technical expert panel met on February 21, April 15, and June 19, 2008.

The technical expert panel members who provided input and guidance to RTI for the national implementation represent the following organizations:

AARP (American Association of Retired Persons)
American Association of Homes and Services for the Aged
Center for Medicare Advocacy, Inc.
Consumer Coalition for Quality Health Care
Health Services Advisory Group
Independent Consultant, formerly of AHRQ
National Association for Home Health Care and Hospice
RAND
Service Employees International Union
Visiting Nurse Service of New York

A.9 Payments/Gifts to Respondents

No payments or gifts will be provided to respondents.

A.10 Assurance of Confidentiality

Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.

Individuals and organizations contacted will be further assured of the confidentiality of their replies under 42 U.S.C. 1306, 20 CFR 401 and 4225 U.S.C.552a (Privacy Act of 1974), and OMB Circular No.A-130. In instances where respondent identity is needed, the information collection will fully comply with all respects of the Privacy Act.

For the mode experiment, RTI will include an assurance of confidentiality of the data in the mail survey cover letters and in the interview introductory script that will be used in interviews with sampled patients included in the phone-only data collection mode and in the telephone follow-up with sample patients in the mixed-mode sample (the mail survey cover letters and the telephone interview introductory script are included in Appendices D and E, respectively).

RTI understands the privacy and confidentiality concerns regarding access to Home Health Care CAHPS Survey data. All RTI staff members who will have access to patient information will be required to sign and abide by the terms of a nondisclosure agreement, where they agree to protect the identity of patients included in the mode experiment and the data they provide. RTI has redundant security protocols to protect data and computer systems. Servers are maintained in climate-controlled environments, with restricted access. A firewall stands between the internal systems and the Internet, requiring authentication of all users requesting access. User identification and passwords are unique and changed on a regular basis. Full backups are conducted on a weekly basis, with incremental backups performed nightly. Copies of backup materials are stored offsite in a secure location in case of system failure.

RTI received a Defense Security Service rating of “Superior” for the physical security of its research center. As data are collected and assembled into databases for analysis and interpretation, RTI incorporates a number of database security safeguards to protect data from accidental or intentional access and disclosure threats. RTI’s data collection and storage security measures include the following:

Maintenance of all servers in RTI’s environmentally controlled Computer Center, where computers are located in a center constructed of masonry with an automatically locking steel door that is locked at all times; fire protection is provided by a halon system with all servers having an Uninterruptible Power Supply.
User ID and password authentication to access all systems. Where appropriate, systems are configured to support the use of Digital Security Certificates for additional user authentication.
Encrypted transmission of data.
Use of Transport Layer Security, the successor technology to Secure Socket Layer for encryption of data across the Internet.
Connection to the Internet by an Internet firewall via a high-speed T2 (6.2 MBs) line. In the event of a failure, a T1 (1.544 MBs) backup will automatically provide uninterrupted Internet connectivity. Subscription to virus-protection services from McAfee VirusScan with automated update of virus signature files on all computers.
Redundant servers with automatic switchover to ensure 24/7 availability.
Daily incremental backups of all data files, with full backups created weekly.
Offsite storage of data backups.

For the national implementation, survey vendors will submit only de-identified survey data to RTI for analysis.

Survey vendors approved to conduct a Home Health Care CAHPS survey for HHAs participating in the national implementation will be required to have systems and methods in place to protect the identity of sampled patients and the confidential nature of the data that they provide. CMS and its contractor (RTI) will review each approved Home Health Care CAHPS Survey vendor’s data security systems during periodic site visits during the national implementation.

A.11 Questions of a Sensitive Nature

There are no questions of a sensitive nature in this survey.

A.12 Estimates of Annualized Burden Hours and Costs

The estimated annual hour burden is as follows:

The length of the survey estimate of .20 hours (12 minutes) is based on the written length of the survey and AHRQ’s experience conducting the field test with a sample of home health patients. It is also based on RTI’s experience conducting other surveys of similar length and complexity.

Estimated annualized burden hours and costs for the national implementation of the Home Health Care CAHPS Survey are shown in Exhibits 1 and 2. These estimates assume that 9,000 home health agencies (the universe of Medicare-certified agencies) will sponsor a Home Health Care CAHPS Survey and that 300 patients sampled from each agency will complete the survey. Not all agencies will participate in national implementation so we have estimated the maximum burden possible.

The Bureau of Labor Statistics reported the average hourly wage for civilian workers in the United States was $19.29 in June 2006. An estimate of $20 per hours allows for inflation and represents a conservative estimate of the wages of respondents.

Exhibit 1. Estimated Annualized Burden Hours: National Implementation of the Home Health Care CAHPS Survey

Form name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Home Health Care CAHPS Survey (mail only, telephone only and mail with telephone follow-up data collection modes)	2,700,000	1	.20	540,000
Total	2,700,000	1	.20	540,000

Exhibit 2. Estimated Annualized Cost Burden: National Implementation

Form name	Number of respondents	Total burden hours	Average hourly wage rate*	Total cost burden
Home Health Care CAHPS Survey (mail only, telephone only and mail with telephone follow-up data collection modes)	2,700,000	540,000	$20.00	$10,800,000
Total	2,700,000	540,000	$20.00	$10,800,000

*Based on average wages, “National Compensation Survey: Occupational Wages in the United States, June 2006,” U.S. Department of Labor, Bureau of Labor Statistics (http://www.bls.gov/ncs/home.htm; last viewed August 27, 2007).

Estimated annualized burden hours and costs for the Home Health Care CAHPS mode experiment are shown in Exhibits 3 and 4.

Exhibit 3. Estimated Annualized Burden Hours: Mode Experiment

Form name	Number of respondents	Number of responses per respondent	Hours per response	Total burden hours
Home Health Care CAHPS Survey (mail only, telephone only and mail with telephone follow-up data collection modes)	6,000	1	.20	1,200
Total	6,000	1	.20	1,200

Exhibit 4. Estimated Annualized Cost Burden: Mode Experiment

Form name	Number of respondents	Total burden hours	Average hourly wage rate*	Total cost burden
Home Health Care CAHPS Survey (mail only, telephone only and mail with telephone follow-up data collection modes)	6,000	1,200	$20.00	$24,000
Total	6,000	1,200	$20.00	$24,000

A.13 Estimates of Annualized Respondent Capital and Maintenance Costs

Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection.

There are no direct costs to respondents other than their time to participate in the survey.

A.14 Estimates of Annualized Cost to the Government

The total cost for the contracted service will be $1,665,634 for the upcoming survey for labor hours, materials and supplies, overhead, and general and administrative costs and fees. The cost for CMS staff to oversee the project is $100,000, including benefits, for a total 1-year project cost of $1,765,634. The contracted service costs include approximately $942,869 for development of systems, protocols, and materials to manage the national implementation, and $1,665,634 for training, technical assistance, oversight of vendors participating in the first year of data collection and data analysis.

A.15 Changes in Hour Burden

This is a new collection of information.

A.16 Time Schedule, Publication, and Analysis Plans

A.16.1 National Implementation of Home Health Care CAHPS

Data collection for the national implementation of Home Health Care CAHPS survey is scheduled to begin in summer 2009 by vendors sponsored by home health agencies that wish to voluntarily participate in the survey. Sampling and data collection will be conducted on an ongoing basis by survey vendors working under contract with the sponsoring home health agencies. CMS will begin publishing results from the national implementation of Home Health Care CAHPS survey on Home Health Compare located on the Medicare.gov website when HHAs have four quarters of data available for reporting. Survey vendors will submit data to CMS’ Home Health Care CAHPS Data Center (maintained and operated by RTI) on a monthly or quarterly basis; however, results that will be posted will reflect one year’s worth of data. In each quarter, RTI will adjust the data for mode of survey administration, patient mix, and non-response, if necessary. The results posted on Medicare.gov will reflect data collected in the four most recent quarters (with data from the earliest quarter replaced by the current quarter).

A.16.1a National Implementation Analysis

Analysis for the national implementation will focus on making appropriate adjustments for mode and/or patient mix, as needed depending on the outcomes of the mode experiment. The data collected each month during the national implementation phase will be transmitted to RTI. Four quarters of data will be aggregated and analyzed for these adjustment purposes. Each quarter, the oldest data will be dropped and the newest quarter added. For each item to be reported, a mean or percentage of patients choosing a particular response will be computed. The following describes how the results of the model that will be developed as part of the mode experiment, which is described in Section A.16.2, will be applied in adjusting the raw observed national survey data to remove the influences of factors not related to the care provided (and, hence, need to be adjusted prior to public reporting of comparative results from individual home health agencies).

Hierarchical modeling techniques will not be used. Sample size permitting, there is an inherent simplicity in using fixed effect models rather than random effects models. The effect of clustering of the data within the HHAs on the standard errors can be accounted for. A model, estimated using a linear or linear probability approach, can be conceptualized as having the predictive form for a specific HHA_i_, as shown in Equations 1a and 1b below.

Equation 1a

Mean response_i = a*patient characteristics_i + b*mode_i + c*HHA_i

Equation 1b

% with response of interest_i = a*patient characteristics_i + b*mode_i + c*HHA_i

Although the model will be estimated on individuals, it will be applied at the HHA level, where a*patient characteristics_i represents the list of estimated coefficients multiplied by the percentage of patients in HHA_i with each of the characteristics or the mean of each characteristic; b*mode_i is the list of coefficients for each mode multiplied by the percentage of patients with that mode; and c*HHA_i is list of coefficients multiplied by the percentage of patients in that HHA (i.e., agency fixed effects).

In the national implementation, a home health agency can choose to use one of three data collection modes—mail, telephone, or mixed mode (i.e., mail with telephone follow-up). During any one quarter, one mode will have a value of 100% and the others 0%. It is possible that modeling will indicate that, in the case of the mixed mode, the actual response mode should be indicated. In this case, such an HHA could have a percentage in each mode.

To transform the estimation equation to an adjustment equation, all the HHA fixed-effects terms will be dropped. For each HHA, Equation 1b, for example, becomes

Equation 2

Adjustment for % with response of interest_i = – a*patient characteristics_i – b*mode_i

The estimated coefficients in the a and b lists may be positive or negative in the estimation; positive coefficients become negative adjustments and negative coefficients become positive adjustments.

While the mode coefficients will be determined by the mode experiment, the value of the coefficients for the patient characteristics will be determined quarterly using all of the data collected for the particular reporting period. The regression equation for the adjustment model will have the mode coefficients fixed and the patient-mix coefficients estimated.

In the next step, the adjustment in Equation 2 will be normalized so that it is relative to a patient whose characteristics are at the means of those characteristics in the national implementation using one year of data. When the equations are estimated, each patient characteristic factor with a 1/0 value has a coefficient magnitude representing an impact of having the characteristic (variable = 1) compared to a reference group indicated by a variable that has been intentionally omitted from the equation during estimation. The omitted group is one of convenience for interpretation. In normalization, the adjustments are converted so that they are relative to the mean of the patient characteristics of the sample. To do this, the percentages (or means) for each characteristic for the entire Home Health Care CAHPS Survey are subtracted from the percentages or means for each of the patient characteristics specific to each HHA; the normalized patient characteristic in Equation 3 is the difference: HHA mean (or percentage) of the characteristic minus the national mean (or percentage) of the characteristic.

Equation 3

Normalized Adjustment for % with response of interest_i = – a*normalized patient characteristics_i – b* mode_i

Since Equation 3 is an adjustment and not a final value for the percentage with the response of interest, one more step is needed to arrive at the adjusted response, as shown in Equation 4.

Equation 4

Adjusted % with response = raw % with response – a*normalized patient characteristics_i – b* mode_i

The form of the adjustment is similar when the dependent variable is treated as a continuous variable from 1 to 10 or from 1 to 4.

A.16.1b Individual-Level Estimation and Adjustment

The formulations for the equations above assume that linear models are being used in the model estimation phase. If the linear approximation is not deemed satisfactory, nonlinear probability models such as logit will be needed.

A.16.2 Home Health Care CAHPS Mode Experiment

Home health agencies that participate in the national implementation of Home Health Care CAHPS survey can choose to use a mail-only, phone-only, or mixed (mail with telephone follow-up of nonrespondents) data collection mode. The intent of the mode experiment is to develop a method to make the results from each sponsoring home health agency that participates in the national implementation comparable based on any differences caused by the data collection mode used. In addition, the mode experiment will be used to estimate patient characteristics that are beyond the agencies’ control and that affect patient responses to the survey. Home health agencies may treat different kinds of patients; that is, some may have more long-term Medicaid, short-term private-pay, and aged and disabled post-acute patients than others. Some may have more rehabilitative patients and some more medically ill patients. Patients may rate the care that they receive from their home health agency providers differently based on these patient characteristics and others such as age, education, and overall health status. The statistical analysis that will be conducted on data from the mode experiment will be used to determine which factors in and of themselves affect survey responses. Factors such as these have been shown to be significant in other surveys and in CAHPS surveys, in particular. The regression methods proposed, discussed in detail below, have been applied in other surveys.

CMS will use the results of the mode experiment to adjust the data collected in the national implementation of Home Health Care CAHPS based on factors that are not directly related to the home health agencies’ performance. Data collection for the mode experiment is scheduled to begin May 2009; however, this start date is contingent upon receiving OMB clearance by April 30, 2009. During the national implementation, the data will be gathered and the reported responses adjusted for the mode of survey administration and patient characteristics. The reports appearing in the Home Health Compare system will have been purged, to the extent possible, of influences not related to the character of the care being given by the HHAs.

Results from the mode experiment will be made publicly available on https://homehealthcahps.org. Project staff will investigate publishing the results in a referred-journal such as the Health Care Financing Review or Health Affairs.

A.16.2a Mode Experiment Analysis

Based on data collected for the mode experiment, RTI will conduct analysis to predict CAHPS responses and ratings as a function of the mode used to administer the survey and, additionally, will estimate the potential effects of patient characteristics outside the control of the HHA.

The regression models and variable formulations will be empirically driven. Each regression model in the mode experiment analysis will estimate a dependent variable, which will be either one of the responses or a composite of responses. The responses to the survey items of greatest interest usually have ordinal responses. Answer structures are typically yes/no, never/sometimes/usually/always, or a global rating (0, 2,…10). There is no constant linear or ratio relationship inherent in these responses. Two is not necessarily twice as good as 1, and 4 is not twice as good as 2. Such responses may be rigorously modeled using logit methods concerning the probability of selection. However, if the data approximate normality and are not strongly clustered at the extreme ends of the distributions, linear regression with numeric values 1–4 or 0–10 can be used successfully.

Generally, the linear forms of the estimation models will be

Dependent variable = sum of (coefficients*patient characteristic indicators) + sum of (coefficients*mode indicators) + sum of (coefficients*home health agency indicators).

When there are categorical patient characteristics, such as age groups, one group is omitted from the set of categories included in the model. That group is a reference category to which the effect of related categories is scaled. The home health agency variables will capture the home health agency–specific effects in order to isolate the effects of mode and other characteristics.

Each item under consideration for reporting will be tested for inclusion in the appropriate adjustment formula. The outcome of the process will be to determine the mode coefficients to be used as adjusters in the implementation phase and the patient characteristic variables that will be used as adjusters in that phase. The actual coefficients on patient characteristics used in the implementation adjustments will be re-estimated as part of that analysis.

A.16.2b Data Elements for the Mode Analysis

We anticipate requesting data for analysis variables from three sources for the mode experiment analysis: the HHA, the survey itself, and (during the national implementation) the survey vendor. Exhibit 5 contains a list of potential variables for analysis.

The dependent variables to be analyzed include all survey items (or will be derived from survey items). The independent variables may come from the survey as self-reported characteristics or may have another source (such as enrollment or other administrative data). Because the survey will be fielded to patients associated with all payers, there is no single source of data that can be used for the mode analysis. Home health agencies will need to provide data from the patient record to their survey vendor (during the national implementation) or to RTI (for agencies participating in the mode experiment). Since we do not know, a priori, which variables will be most important for this population, we may be requesting, for the mode experiment, more than we will ultimately use in our implementation analyses. We recognize, however, the importance of minimizing HHA burden. Discussions with home health agencies conducted by RTI suggest that the additional administrative data requested are available at agencies, including small home health agencies, and could be supplied with minimal burden.

A.17 Exemption for Display of Expiration Date

CMS does not seek this exemption.

Exhibit 5. List of Potential Variables for Data Analysis

Data variables	From HHA	From survey respondent	From survey vendor in implementation only
Date of birth (for experiment)	X	-	-
Gender	X	-	-
Medical Record Number	X	-	-
Payer	X	-	-
Dual Medicare-Medicaid eligibility	X	-	-
Managed care organization	X	-	-
Diagnoses (ICD-9 codes) underlying cause of care and comorbidities	X	-	-
End-stage renal disease (ESRD) patient on dialysis	X	-	-
Admission date to HHA	X	-	-
Number of visits – skilled and aide/personal care	X	-	-
Source of admission	X	-	-
Deficits in activities of daily living (ADLs)—score from OASIS	X	-	-
Overall health status	-	X	-
Mental health status	-	X	-
Race and ethnicity	-	X	-
Education level	-	X	-
Whether respondent lives alone	-	X	-
Did someone help respondent with survey	-	X	-
Potential other items on survey	-	X	-
Mode of survey administration	-	-	X
Number of prompts to elicit response	-	-	X
Agency identifier	-	-	X
Interview completion date	-	-	X

File Type	application/msword
File Title	The National Implementation of the Home Health Care CAHPS Survey
Subject	OMB Supporting Statement for Home Health Care CAHPS Survey
Author	Centers for Medicare & Medicaid Services
Last Modified By	CMS
File Modified	2009-07-02
File Created	2009-07-02