B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
The respondent universe for the participant experience survey will be high-risk Medicare beneficiaries discharged from hospitals partnering with community-based organizations (CBOs) that participate in the Medicare Community-Based Care Transitions Program (CCTP). The Centers for Medicare & Medicaid Services (CMS) define the target population as fee-for-service Medicare beneficiaries “with multiple chronic conditions, depression, cognitive impairments, or a history of multiple admissions” (CMS CCTP Solicitation for Applications). CBOs are required to apply to CMS to participate in the CCTP and will define in their applications the high-risk groups they intend to target under the program. The survey universe will be all participants receiving CCTP-funded care transition services from participating CBOs.
There will be no sampling used to select respondents. All participants receiving care transition services from CCTP-participating CBOs will be asked to complete the questionnaire.
To help CBOs monitor their performance and identify and remediate barriers to intervention success, we will report response frequencies for each survey question from the H-CAHPS and CTM. In addition, for the Care Transitions Measure (CTM-3) and Patient Activation Measure (PAM-13) instruments, we will provide a summary score for each measure, and for the PAM-13 measure, we will report summary change scores, measuring the change in patient activation levels between the initiation and the completion of intervention services. CMS will use the information presented on quarterly performance reports to determine whether to renew CBOs’ contracts after an initial two-year period of performance. It is anticipated that recipients of intervention services will experience increased patient activation as measured by the PAM-13, will report high levels of satisfaction with and understanding of information received about medication and discharge plans, and will report high CTM-3 scores. We will also analyze trends in survey outcomes over time for the same CBO to assess changes in performance measures.
Participant experience data will provide hospital and CBO-level information about care transition processes that are otherwise not directly observable from claims or other administrative data. CMS will use participant experience data to help determine which CBO program agreements will be renewed after the initial two-year award period. An evaluation contractor (to be determined) will also use participant experience data to analyze and identify interventions that lead to the greatest improvement in patient activation or result in the highest level of information transfer. A technical assistance contractor (The Lewin Group) will also use participant experience data to identify CBOs or hospitals that require special assistance or to ‘fine tune’ the assistance provided to all CBOs and hospitals. Finally, the CBOs and their partnering hospitals will use participant experience data to monitor their performance and identify changes to help them achieve their longer-term performance goals.
No specialized sampling procedures will be used to accommodate unusual problems.
The
participant experience survey data collection will be ongoing, as
participants are discharged from hospitals and start receiving care
transition services. Measures of patient activation and understanding
and knowledge of aftercare plans are key monitoring and
implementation metrics for the CCTP. CBOs will administer the survey
to all participants to examine key aspects of the care planning and
discharge process. The 21 items that make up the participant
experience survey draw from three existing instruments, as described
in detail in
Part A of this justification (five items from the
Hospital Consumer Assessment of Healthcare Providers and Systems
(H-CAHPS), the three-item Care Transitions Measure (CTM-3) and the
13-item Patient Activation Measure (PAM-13)). Appendix B contains the
combined set of questions that will be asked within four days of
hospital discharge of Medicare beneficiaries who are receiving care
transition services from a CCTP-participating CBO. In addition, the
PAM-13 items will be asked again of the same participants at the end
of the care transition services (this could range from one week to
three months, depending on the care transition model being used).
Once it is known what transition models the CBOs are using, it is
possible that the PAM-13 will be irrelevant for participants of
programs that are process-oriented only; participants in such
programs will not be asked the second administration of the PAM
questions. It is impossible to estimate how many such cases there
will be, but we expect that most CBO applicants will have some
activation component to the care transition program and, thus, most
participants will be asked the PAM questions at two points in time.
Data collection of the participant experience measures will rely on a paper-and-pencil questionnaire administered by the CBOs’ care transition nurse, discharge advocate, intervention specialist, or other designated liaison who is providing care transition services to the participant. (We also considered the feasibility of asking the beneficiaries to fill out the survey on their own, to avoid possible reporting bias. We concluded that the logistics of CBOs doing follow-up of nonresponding participants, and the likely very low response rate, made this approach less appealing. We have added wording to the questionnaire to encourage honesty in reporting in order to mitigate self-reporting bias.) CBO staff responsible for administering the participant experience survey will be trained on administration of the survey. CBOs will be provided training and a written training manual that will be posted, along with other training materials (such as training slides, frequently asked questions and answers, and question-by-question instructions for the survey), on a program website. There will also be monthly webinars for the first six months of the program (conditional on participation and the need for training), and then quarterly for new CBO staff or for organizations wishing to refresh their understanding of the data collection and privacy procedures. All CBO survey administrators will be required to attend at least one training within the first month of their organization’s receiving a CCTP award.
CBOs will be responsible for obtaining participants’ agreement to complete the survey. As described above, CBO staff will be given extensive training and reference materials about data collection; that material will include a discussion about gaining respondent cooperation. Based on experience from CMS’ Money Follows the Person survey of program participants, we expect that nearly 9 out of 10 participants will complete the questionnaire. CMS intends to make survey administration a condition of program award, and survey completion rates will be calculated as a performance outcome and included in the quarterly performance reports.
There is some concern about having the intervention specialist administer the PAM-13 at the end of the care transition program. Participants might not feel comfortable answering honestly if they fear it will reflect badly on the interventionist who both provided services and is asking the survey questions. We solicited advice from other organizations that have used the PAM-13 about this risk and, as described above, we concluded that there are tradeoffs with each mode but that the interviewer-administered mode is the preferred choice. It is important that the two administrations of the PAM be done in the same mode in order to avoid confounding the observed change in scores between administrations with the change in administration mode.
The survey will be translated into Spanish and made available to CBOs (Appendix B includes the Spanish translation of the H-CAHPS, CTM-3, and PAM-13 items). Also, we will accept proxy interviews as needed. CBOs will be strongly encouraged to collect responses directly from the participant, but if the participant requests someone else respond instead, that proxy response will be accepted.
The survey data will not be weighted or adjusted in any way, since there is no intention to generalize to the Medicare population.
All the items in this data collection are taken from tested and known instruments. Items selected from the H-CAHPS were chosen to be consistent with the evaluation of care transition interventions by the Colorado Foundation for Medical Care (CFMC), implemented under the Quality Improvement Organization (QIO) Program care transition theme and have been demonstrated to produce valid and reliable results with similar patient populations. Responses to the CTM-3 and PAM-13 instruments also have proven validity and reliability with similar patient populations (for psychometrics on CTM-3, see Coleman et al. 2007; for the PAM-13, see Hibbard et al. 2005).
A small pretest will be conducted to assess the data collection procedures and the processes for entering and uploading data. A convenience sample of three to five participants from each of two or three CBOs will be selected for the pretest. The pretest will mirror the data collection strategy planned for the main data collection to the extent possible. Nine participants will complete the questionnaire and the CBOs will be contacted to solicit feedback about the processes that were used, with an eye towards improving and streamlining the process. The interview length at the pretest is expected to be 10 minutes for the 21-question survey and another 5 minutes to administer the PAM-13 questions at the end of the care transition program. We will ask pretesters to keep track of time in order to know if this estimate is correct.
The following people have contributed to the study design and to the design of the survey instruments and administration protocols:
Juliana Tiongson, CCTP Team Lead, Centers for Medicare & Medicaid Research, (401) 786-0342
Dr. Boyd Gilman, senior health researcher at Mathematica Policy Research and project director, (617) 301-8974
Dr. Samuel Simon, senior health researcher at Mathematica Policy Research and lead on performance monitoring and reporting task, (617) 301-8982
Dr. Karen Bogen, senior survey researcher at Mathematica Policy Research and lead on the survey tasks, (617) 674-8355
Dr. Eric Coleman, professor of medicine and interim head of the Division of Health Care Policy and Research at the University of Colorado at Denver, senior advisor to the project on performance monitoring issues, (303) 724-2456
Dr. Jane Brock, chief medical officer at the Colorado Foundation for Medical Care (CFMC), advisor to the project on implementation issues, (303) 695-3300
Alicia Goroski, senior project director of the care transitions program at the CFMC, advisor to the project on implementation issues, (303) 784-5788
Centers for Medicare & Medicaid Services. “Medicare Community-Based Care Transitions Program Solicitation for Applications.” Available at [http://www.cms.gov/DemoProjectsEvalRpts/downloads/CCTP_Solicitation.pdf ]. Accessed June 18, 2011.
Coleman, Eric A., Carla Parry, Sandra A. Chalmers, Amita Chugh, and Eldon Mahoney. “The Central Role of Performance Measurement in Improving the Quality of Transitional Care.” Home Health Care Services Quarterly, vol. 26, no. 4, 2007, pp. 93 – 104.
Hibbard, Judith H., Eldon R. Mahoney, Jean Stockard, and Martin Tusler. “Development and Testing of a Short Form of the Patient Activation Measure.” Health Services Research, vol. 40, no.6, Part I, December 2005, pp. 1918-1930.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Kari Beckmann |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |