B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
The respondent universe for the patient experience survey will be high-risk Medicare beneficiaries discharged from hospitals partnering with community-based organizations (CBOs) that participate in the Medicare Community-Based Care Transitions Program (CCTP). The Centers for Medicare & Medicaid Services (CMS) define the target population as fee-for-service Medicare beneficiaries “with multiple chronic conditions, depression, cognitive impairments, or a history of multiple admissions” (CMS CCTP Solicitation for Applications). CBOs are required to apply to CMS to participate in the CCTP and will define in their applications the high-risk groups they intend to target under the program. The survey universe will be all participants receiving CCTP-funded care transition services from participating CBOs.
There will be no sampling used to select respondents. All participants receiving care transition services from CCTP-participating CBOs will be asked to complete the questionnaire.
To help CBOs monitor their performance and identify and remediate barriers to intervention success, we will report response frequencies for each survey question from the H-CAHPS and CTM. In addition, for the Care Transitions Measure (CTM-3) and Patient Activation Measure (PAM-13) instruments, we will provide a summary score for each measure, and for the PAM-13 measure, we will report summary change scores, measuring the change in patient activation levels between the initiation and the completion of intervention services. CMS will use the information presented on quarterly performance reports to determine whether to renew CBOs’ contracts after an initial two-year period of performance. It is anticipated that recipients of intervention services will experience increased patient activation as measured by the PAM-13, will report high levels of satisfaction with and understanding of information received about medication and discharge plans, and will report high CTM-3 scores. We will also analyze trends in survey outcomes over time for the same CBO to assess changes in performance measures.
Patient experience data will provide hospital and CBO-level information about care transition processes that are otherwise not directly observable from claims or other administrative data. CMS will use patient experience data to help determine which CBO program agreements will be renewed after the initial two-year award period. An evaluation contractor (to be determined) will also use patient experience data to analyze and identify interventions that lead to the greatest improvement in patient activation or result in the highest level of information transfer. A technical assistance contractor (The Lewin Group) will also use patient experience data to identify CBOs or hospitals that require special assistance or to ‘fine tune’ the assistance provided to all CBOs and hospitals. Finally, the CBOs and their partnering hospitals will use patient experience data to monitor their performance and identify changes to help them achieve their longer-term performance goals.
No specialized sampling procedures will be used to accommodate unusual problems.
The
patient experience survey data collection will be ongoing, as
participants are discharged from hospitals and start receiving care
transition services. Measures of patient activation and understanding
and knowledge of aftercare plans are key monitoring and
implementation metrics for the CCTP. CBOs will administer the survey
to all participants to examine key aspects of the care planning and
discharge process. The 21 items that make up the patient experience
survey draw from three existing instruments, as described in detail
in
Part A of this justification (five items from the Hospital
Consumer Assessment of Healthcare Providers and Systems (H-CAHPS),
the three-item Care Transitions Measure (CTM-3) and the 13-item
Patient Activation Measure (PAM-13)). Appendix B contains the
combined set of questions that will be asked within four days of
hospital discharge of Medicare beneficiaries who are receiving care
transition services from a CCTP-participating CBO. In addition, the
PAM-13 items will be asked again of the same participants at the end
of the care transition services (this could range from one week to
three months, depending on the care transition model being used).
Once it is known what transition models the CBOs are using, it is
possible that the PAM-13 will be irrelevant for participants of
programs that are process-oriented only; participants in such
programs will not be asked the second administration of the PAM
questions. It is impossible to estimate how many such cases there
will be, but we expect that most CBO applicants will have some
activation component to the care transition program and, thus, most
participants will be asked the PAM questions at two points in time.
Data collection of the patient experience measures will rely on a paper-and-pencil questionnaire administered by the CBOs’ care transition nurse, discharge advocate, intervention specialist, or other designated liaison who is providing care transition services to the participant. (We also considered the feasibility of asking the beneficiaries to fill out the survey on their own, to avoid possible reporting bias. We concluded that the logistics of CBOs doing follow-up of nonresponding participants, and the likely very low response rate, made this approach less appealing. We have added wording to the questionnaire to encourage honesty in reporting in order to mitigate self-reporting bias.) CBO staff responsible for administering the patient experience survey will be trained on administration of the survey. CBOs will be provided training and a written training manual that will be posted, along with other training materials (such as training slides, frequently asked questions and answers, and question-by-question instructions for the survey), on a program website. There will also be monthly webinars for the first six months of the program (conditional on participation and the need for training), and then quarterly for new CBO staff or for organizations wishing to refresh their understanding of the data collection and privacy procedures. All CBO survey administrators will be required to attend at least one training within the first month of their organization’s receiving a CCTP award.
CBOs will be responsible for obtaining participants’ agreement to complete the survey. As described above, CBO staff will be given extensive training and reference materials about data collection; that material will include a discussion about gaining respondent cooperation. Based on experience from CMS’ Money Follows the Person survey of program participants, we expect that nearly 9 out of 10 participants will complete the questionnaire. CMS intends to make survey administration a condition of program award, and survey completion rates will be calculated as a performance outcome and included in the quarterly performance reports.
There is some concern about having the intervention specialist administer the PAM-13 at the end of the care transition program. Participants might not feel comfortable answering honestly if they fear it will reflect badly on the interventionist who both provided services and is asking the survey questions. We solicited advice from other organizations that have used the PAM-13 about this risk and, as described above, we concluded that there are tradeoffs with each mode but that the interviewer-administered mode is the preferred choice. It is important that the two administrations of the PAM be done in the same mode in order to avoid confounding the observed change in scores between administrations with the change in administration mode.
The survey will be translated into Spanish and made available to CBOs (Appendix B includes the Spanish translation of the H-CAHPS, CTM-3, and PAM-13 items). Also, we will accept proxy interviews as needed. CBOs will be strongly encouraged to collect responses directly from the participant, but if the participant requests someone else respond instead, that proxy response will be accepted.
The survey data will not be weighted or adjusted in any way, since there is no intention to generalize to the Medicare population.
All the items in this data collection are taken from tested and known instruments. Items selected from the H-CAHPS were chosen to be consistent with the evaluation of care transition interventions by the Colorado Foundation for Medical Care (CFMC), implemented under the Quality Improvement Organization (QIO) Program care transition theme and have been demonstrated to produce valid and reliable results with similar patient populations. Responses to the CTM-3 and PAM-13 instruments also have proven validity and reliability with similar patient populations (for psychometrics on CTM-3, see Coleman et al. 2007; for the PAM-13, see Hibbard et al. 2005).
A small pretest was conducted to assess the data collection procedures and the processes for entering and uploading data. A convenience sample of three participants from each of three CBOs was selected for the pretest. The pretest mirrored the data collection strategy planned for the main data collection to the extent possible. Nine participants completed the questionnaire and the CBOs were contacted to solicit feedback about the processes that were used, with an eye towards improving and streamlining the process. As expected, the interview length at the pretest was 10 minutes for the 21-question survey. The second administration survey was not included in the pretest due to time constraints. In the pretest, no one declined to participate (nine were asked and all nine agreed to complete the questionnaire). Field staff were debriefed after the pretest, and they were generally very positive about the experience and said that the patient respondents were as well. There was a suggestion to allow more flexibility in the introduction, something we will consider when we get feedback from the pilot.
In addition to the pretest, a three-month pilot test will be conducted in order to test the data collection and feedback procedures. The pilot test will be conducted with two CBOs that have about the median number of hospital admissions eligible for CCTP services. The pilot test procedures will align with the pretest and full-scale implementation procedures. We will ask CBOs in the pilot test to record the length of time it takes to administer the survey and reason for refusal if a participant declines to take the survey. CBOs will follow the outlined procedures to data enter surveys and upload data to GENTRAN. CBOs will be debriefed two weeks after the start of the pilot test data collection to solicit feedback early in the implementation. Upon completion of the pilot test, we will submit a field operations report which will include key findings, implications, and recommendations.
The following people have contributed to the study design and to the design of the survey instruments and administration protocols:
Juliana Tiongson, CCTP Team Lead, Centers for Medicare & Medicaid Research, (401) 786-0342
Dr. Boyd Gilman, senior health researcher at Mathematica Policy Research and project director, (617) 301-8974
Dr. Samuel Simon, senior health researcher at Mathematica Policy Research and lead on performance monitoring and reporting task, (617) 301-8982
Dr. Karen Bogen, senior survey researcher at Mathematica Policy Research and lead on the survey tasks, (617) 674-8355
Dr. Eric Coleman, professor of medicine and interim head of the Division of Health Care Policy and Research at the University of Colorado at Denver, senior advisor to the project on performance monitoring issues, (303) 724-2456
Dr. Jane Brock, chief medical officer at the Colorado Foundation for Medical Care (CFMC), advisor to the project on implementation issues, (303) 695-3300
Alicia Goroski, senior project director of the care transitions program at the CFMC, advisor to the project on implementation issues, (303) 784-5788
Centers for Medicare & Medicaid Services. “Medicare Community-Based Care Transitions Program Solicitation for Applications.” Available at [http://www.cms.gov/DemoProjectsEvalRpts/downloads/CCTP_Solicitation.pdf]. Accessed June 18, 2011.
Coleman, Eric A., Carla Parry, Sandra A. Chalmers, Amita Chugh, and Eldon Mahoney. “The Central Role of Performance Measurement in Improving the Quality of Transitional Care.” Home Health Care Services Quarterly, vol. 26, no. 4, 2007, pp. 93-104.
Hibbard, Judith H., Eldon R. Mahoney, Jean Stockard, and Martin Tusler. “Development and Testing of a Short Form of the Patient Activation Measure.” Health Services Research, vol. 40, no. 6, Part I, December 2005, pp. 1918-1930.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kari Beckmann |
| File Modified | 0000-00-00 |
| File Created | 2021-01-30 |