Supporting Statement
for
Evaluation of the Medicare
Health Care Quality (MHCQ) Demonstration Evaluation: Focus
Group and Interview Protocols
The Medicare Health Care Quality (MHCQ) Demonstration was developed to address concerns about the U.S. health care system, which typically fragments care while also encouraging both omissions in and duplication of care. To rectify this situation, Congress has directed the Centers for Medicare & Medicaid Services (CMS) to test major changes to the delivery and payment systems to improve the quality of care while also increasing efficiency across the health care system. This would be achieved through several types of interventions: adoption and use of information technology and decision support tools by physicians and their patients, such as evidence-based medicine guidelines, best practice guidelines, and shared decision-making programs; reform of payment methodologies; improved coordination of care among payers and providers serving defined communities; measurement of outcomes; and enhanced cultural competence in the delivery of care.
Section 1866C of the Social Security Act, as amended by Section 646 of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (P.L. 108-173, Section 1866C(b)), requires the Secretary of the Department of Health and Human Services to establish a 5-year demonstration program under which the Secretary may approve demonstration projects that examine health delivery factors that encourage improved quality in patient care. This section also authorizes the Secretary to waive compliance with such requirements of Titles XI and XVIII of the Social Security Act (42 U.S.C. 1395 et seq.) as may be necessary for the purposes of carrying out the demonstration project.
The MHCQ Demonstration programs are designed to examine the extent to which major, multifaceted changes to traditional Medicare’s health delivery and financing systems lead to improvements in the quality of care provided to Medicare beneficiaries without increasing total program expenditures. Each demonstration site uses a different approach for changing health delivery and financing systems, but all share the goal of improving the quality and efficiency of medical care provided to Medicare beneficiaries. Focus groups and individual interviews will be conducted at 2 demonstration sites that are active in the demonstration: Gundersen Health System (GHS) and Meridian Health System (MHS).
GHS is a physician-led, not-for-profit integrated delivery system serving a regional community of 550,000 people in western Wisconsin, northeastern Iowa, and southeastern Minnesota. For the MHCQ Demonstration, GHS focuses on improving care for patients with severe illnesses through the Advanced Disease Coordination (ADC) program. GHS has recently incorporated all of the elements of its advance care planning program to create this new model for more efficient, high-quality care. The MHCQ Demonstration project is providing GHS with an opportunity for further development and evaluation of this new model of advanced disease coordination. The program features an interdisciplinary care team dedicated to providing high-quality, seamless medical care, individualized for each patient and family across all settings of care, from home to hospital. For the MHCQ Demonstration, GHS is paid by Medicare a prospectively determined per beneficiary per month (PBPM) fee to provide services under the ADC program.
MHS is a nonprofit, multihospital, integrated delivery system serving central New Jersey. For the MHCQ Demonstration, MHS is providing an expanded and integrated care program for beneficiaries facing serious and advanced chronic illnesses, including stroke, heart failure, pulmonary disease, end-stage renal disease (ESRD), end-stage liver disease, cancer, and Alzheimer’s disease. The Meridian Care Journey (MCJ) program is operating at three MHS hospitals for the MHCQ Demonstration. The MCJ program follows beneficiaries through the continuum of care and focuses on outpatient interventions to provide coordinated care and to ensure that the care received reflects beneficiaries’ preferences and values. For the MHCQ Demonstration, MHS is paid by Medicare a prospectively determined PBPM fee to provide services under the MCJ program.
This MHCQ Demonstration evaluation will include analysis of both quantitative and qualitative sources of information. This multifaceted approach will enable this evaluation to consider a broad variety of evidence for evaluating the nature and impact of each site’s interventions. This OMB application seeks approval to conduct in-person focus groups and individual interviews with beneficiaries and their caregivers to inform CMS’ evaluation of the MHCQ Demonstration at the GHS and MHS demonstration sites.
The focus groups and interviews are part of an evaluation strategy for identifying the impacts of the demonstration interventions on improving quality and efficiency of care. Mixed-methods research is well-suited for accomplishing the goals of this evaluation, as different methods yield different insights. While quantitative methods (e.g., Medicare claims data analysis) are well-suited for outcomes or summative evaluation, qualitative methods (e.g., interviews, focus groups) are necessary for process or formative evaluation (Patton, 1990 and 1996; Sofaer, 1999). The combination of these methods can provide a comprehensive understanding of the nature of the GHS and MHS interventions, their implementation, and perceived outcomes for patients, providers, and other stakeholders (Creswell, 2009). Qualitative methods are particularly useful for evaluating health policy interventions, providing a more complete understanding of the interventions themselves and the context in which they are taking place, the views of different stakeholders, the unexpected outcomes, and the state and program conditions or factors more likely to be associated with success (Ragin, 1999; Rist, 1994; Sofaer, 1999; Yin, 1999).
Section 646 of the Medicare Prescription Drug, Improvement, and Modernization Act of 2003 (P.L. 108-173, Section 1866C(b)) requires that the demonstration project “examine health delivery factors that encourage shared decision making between providers and patients” and “appropriate use of culturally and ethnically sensitive health care delivery,” which may be best evaluated through seeking beneficiary and caregiver feedback. Focus groups and interviews provide a method for better understanding patient experience, as required by the law.
The evaluation research questions cover the following domains for each site: (1) administration and infrastructure, (2) beneficiary and provider participation, (3) utilization, (4) quality, (5) health information technology, (6) cost and savings, and (7) lessons learned and implications for future programs. Given that CMS seeks to understand the demonstration’s impacts on quality of care and beneficiary participation, it is critical to understand patients’ experiences from their perspective (or from their caregiver’s perspective) and how well these intervention models are serving their needs. For this evaluation, the focus groups and interviews will provide us with answers to fundamental “what, how, and why” questions about Medicare beneficiaries’ experiences with care at the GHS and MHS intervention sites, such as:
What have beneficiaries and their caregivers experienced as the benefits and problems of care received from providers participating in the MHCQ program? Were they satisfied with the services they received?
What components of the demonstration encouraged shared decision-making between patients (or their caregivers) and providers?
How does the approach to shared decision-making influence the patients’ (or caregivers’) experiences with the delivery of health care?
Were there changes in access to care for beneficiaries?
Were GHS and MHS able to reach the populations they intended?
The information collected through these focus groups and interviews is critical to CMS in determining whether the MHCQ Demonstration is effective. While most of this data will be collected using focus group format, we will offer to conduct individual interviews in lieu of the focus groups with beneficiaries and/or caregivers if needed or preferred.
These focus groups and interviews will be used by CMS to understand:
the experiences beneficiaries and caregivers have with program services, including quality of care and their participation in decision-making about care,
beneficiaries’ and caregivers’ perspectives about how access to care and communication with providers has changed,
beneficiaries’ and caregivers’ experiences with coordination of care and care transitions.
The results will also be useful to providers, Medicare beneficiaries, and caregivers by informing them about what activities may be beneficial to enhance the quality and safety of care, efficiency of care delivery, access to care, and other outcomes. This information also will facilitate diffusion and implementation of similar interventions elsewhere by documenting beneficiary and caregiver perceptions of the care they receive.
The focus groups and individual interviews will make minimal use of information technology (IT). Skilled and experienced facilitators from RTI International will lead the discussion groups and individual interviews. A dedicated note taker will capture participant responses. Individual interviews and focus groups will be audio recorded to be used as back-up to assure the completeness and accuracy of the notes and to provide members of the research team the opportunity to review the discussions at a later date. Data will be managed and analyzed in NVivo, a powerful and widely used qualitative data analysis software program (QSR International, Doncaster, Australia; Bazeley, 2007; Richards, 2009; Sorensen, 2008). RTI research team has significant experience in managing and analyzing large qualitative data sets with this type of software.
Enrollment and participation in focus groups or individual interviews will require minimal use of information technology. Telephonic communication will be used during recruitment, but participants will not be expected to use IT during their participation.
The evaluation has been designed to comprehensively address the research questions while minimizing the burden placed on the demonstration sites, demonstration participants (e.g., providers and program staff), and Medicare beneficiaries and their caregivers.
Focus groups and individual interviews are designed to complement other primary and secondary data collection and analysis (see section A-1 for more details). That is, they will build on and fill information gaps rather than duplicate information from other sources of data. For example, information from focus groups and individual interviews can be used to provide deeper understanding of beneficiaries’ and caregivers’ experiences that cannot be understood through quantitative data alone. Focus groups and individual interviews will be conducted only when primary or secondary data cannot be obtained to fully answer the evaluation research questions.
CMS and its evaluation contractor have taken numerous steps to ensure that the information to be collected through these focus groups and individual interviews are not readily available from existing sources. We have examined secondary qualitative documents and resources publicly available and have reviewed the demonstration sites’ MHCQ applications and other documentation and communications provided to CMS. Furthermore, since interventions vary at each demonstration site, we have tailored each site’s focus group and individual interview guides to implement site-specific terminology and to reflect specific and/or unique elements of the site’s intervention model.
Thus, the information collected through the focus groups and individual interviews should not duplicate any other effort and should not be obtainable from any other source.
Focus group participants and interviewees will be limited to Medicare beneficiaries and beneficiary caregivers. There is no expected involvement for small entities including small businesses, local governments, or other small entities.
The focus groups and interviews will be conducted only once during the demonstration.
There will be no special circumstances.
The 60-day Federal Register Notice was published on August 21, 2013. There were no comments received.
Each focus group participant and interviewee will be provided with a $50 gift card for their participation in the focus group. To further facilitate participation, focus groups will be held at locations convenient to beneficiaries and caregivers (e.g., near hospitals or on a bus line) as well as during convenient times such as during lunchtime or in the evenings.
Personnel who will have access to focus group and interview data (including notes, summary reports, transcripts, audio, and video tapes) and/or individual identifiers will be trained on the significance and protection of confidentiality, particularly as it relates to controlled and protected access to these data. Further, information will be provided to potential focus group participants and interviewees describing the purpose and the voluntary nature of the focus groups interviews. We will convey the extent to which respondents and their responses will be kept confidential and pledge privacy to the fullest extent possible. We will use a file-naming convention denoting the demonstration site and type of respondent (e.g. beneficiary, caregiver); the full names of participating individuals will not be included in notes or transcripts. As previously described, NVivo is a computer software package used to analyze qualitative data. The notes and the database will be stored on a secured server and password-protected computers.
Information collected in the focus groups and interviews will be confined to participant’s experiences, opinions, and perspectives regarding their care received under the MHCQ Demonstration. Furthermore, questions regarding end-of-life preferences and personal beliefs are confined to evaluating the degree to which beneficiaries and caregivers believe the intervention programs respect their preferences and beliefs. We will not ask participants to directly share information about their preferences or beliefs. Some participants may choose to share information about their health or medical condition to illustrate how it shaped their experiences with their providers. We will ask that focus group participants not share any personal information about other participants outside of the room.
We will conduct focus groups or individual interviews with up to 12 caregivers from the GHS demonstration site and up to 24 beneficiaries and/or caregivers from the MHS demonstration site. The length of each focus group will be no more than 2 hours, including time to review the focus group processes and to obtain verbal informed consent. Individual interviews will last one hour, including time to obtain verbal informed consent. The focus group protocols will also be used for the individual interviews, as we will use the same questions for both interview types.
Participants will be selected from the list of enrolled beneficiaries at each site and in consultation with staff at each site. Eligible patients will be contacted by telephone. The final list of participants will be created to achieve the desired composition of focus group participants and interviewees. These individuals will be contacted to finalize the date and time.
To estimate the cost of burden, we used an average of 6 participants for each focus group. Wage calculations are based on the mean hourly wages as indicated in the “National Compensation Survey: Occupational Wages in the United States, May 2011,” by the U.S. Department of Labor, Bureau of Labor Statistics. The Bureau of Labor Statistics reported the mean hourly wage for civilian workers in the United States was $21.74 in May 2011.
The maximum number of focus group participants and interviewees by demonstration site is shown in Exhibit 1. A total of 36 individuals will participate in focus groups across the two sites.
Exhibit 1. Maximum number of focus group participants or interviewees by demonstration site
Demonstration Site |
Maximum number of focus groups |
Number of beneficiaries or caregivers per group* |
Total participants or interviewees |
GHS |
2 |
6 |
12 |
MHS |
4 |
6 |
24 |
Total |
6 |
- |
36 |
*Some may take part in an individual interview rather than a focus group depending on logistics.
Estimated annual time and wage burden is shown in Exhibit 2. The total estimated time burden is 108 hours. The total estimated wage burden is $2,347.92, which includes 2 hours for the focus groups and 1 hour of travel time to and from the focus group site. The time and wage burdens will likely be lower, given that we expect to hold individual interviews with some beneficiaries and caregivers that will last one hour and involve minimal travel.
Exhibit 2. Estimated respondent annual time and wage burden
Participant Type |
Number of Participants |
Length of focus group (hrs) |
Travel time to/from focus group (hrs) |
Total Burden Hours |
Mean Hourly Wage Rate* |
Total Wage Burden |
Medicare beneficiaries and/or caregivers |
36 |
2.0 |
1.0 |
3.0 |
$21.74 |
$2, 347.92 |
*Based upon the mean hourly wages, “National Compensation Survey: Occupational Wages in the United States, May 2011,” U.S. Department of Labor, Bureau of Labor Statistics. http://www.bls.gov/oes/current/oes_nat.htm#b29-0000
There are neither capital nor startup costs, nor are there any operation and maintenance costs to the focus group participants and interviewees.
Total costs associated with the focus groups and interviews are estimated to be $60,868 for planning, recruitment, focus group facilitation, participant incentives, travel, meeting notes and analysis. These costs are funded through an existing CMS contract with RTI.
Federal FTE costs are expected to be negligible. The Project Officer for the CMS contract with RTI may be required to spend 0.2% of her time on the administration of this activity (~$250 of annual salary).
This is a new data collection for CMS. The focus groups and interviews will not result in any recurrent periodic reporting or recordkeeping costs or time burden.
These qualitative results will be reported in the Final Report. The Final Report will be completed in April 2016. Additionally, the RTI team plans to develop peer-reviewed publications and conference presentations that will be reviewed and approved by CMS prior to submission.
The OMB expiration date will be displayed on all disseminated data collection materials.
Supporting Statement – Part B
Collections of Information Employing Statistical Methods
This information collection does not employ statistical methods.
Bazeley, P. (2007). Qualitative data analysis with NVivo (2nd ed.). Sage Publications Ltd.
Creswell, J. W. (2009). Research design: Qualitative, quantitative, and mixed methods approaches (3rd ed.). Thousand Oaks, CA: Sage.
Kautter, J., Sorensen, A., Bernard, S., et al. Medicare Health Care Quality Demonstration: Gundersen Lutheran Health System Advanced Disease Coordination Final Case Study Report. CMS Contract No. 500-00-024, TO #24. Research Triangle Park, N.C. RTI International, 2010.
Medicare Prescription Drug, Improvement, and Modernization Act. (2003). S. 646. (P.L. 108-173, Section 1866C(b)).
Meridian Health System. (2012) Meridian Care Journey: A Palliative Care Program. Protocol Document. Neptune, New Jersey. Meridian Health System.
Miles, M.A. & Huberman, A.M. (1994). Qualitative data analysis: An expanded source book (2nd ed.). Thousand Oaks, CA: Sage
Patton, M.Q. (1990). Qualitative evaluation and research methods (2nd ed.). Thousand Oaks, CA: Sage.
Patton, M. Q. (1996). Utilization-focused evaluation (3rd ed.). Thousand Oaks, CA: Sage.
Ragin, C.C. (1999). Using qualitative comparative analysis to study causal complexity.
Health Services Research, 34(5 Pt 2), 1225-1239.
Richards, L. (2009). Handling qualitative data (2nd ed.). Thousand Oaks, CA: Sage.
Rist, Ray C. (1994). Influencing the policy process with qualitative research. In N. Denzin & Y. Lincoln (Eds.) Handbook of Qualitative Research (pp. 545-557). Thousand Oaks, CA: Sage Publications, Inc.
Sofaer, S. (1999). Qualitative methods: What are they and why use them? Health Services Research, 34(5, part 2), 1101–1118.
Sorensen, A. (2008). Use of QSR NVivo 7 qualitative analysis software for mixed methods research. Journal of Mixed Methods Research, 2(1), 106–110.
Yin, R.K. (1999). Enhancing the Quality of Case Studies in Health Services Research, Health Services Research, 34(5 Pt 2), 1209-1224.
ATTACHMENT C
SCRIPT FOR RECRUITMENT
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Month 200X |
| Author | npepoli |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |