SUPPORTING STATEMENT
Part A
Studying the Implementation of
a Chronic Care Toolkit and Practice Coaching
In Practices Serving Vulnerable Populations
Version: December 30, 2009
Agency of Healthcare Research and Quality (AHRQ)
TABLE OF CONTENTS
A. JUSTIFICATION
A. JUSTIFICATION
A.1. Circumstances Making the Collection of Information Necessary
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
An important part of AHRQ’s mission is to disseminate information and tools that can support improvement in quality and safety in the U.S. health care community. This proposed information collection supports that part of AHRQ's mission by further refining the practice coaching delivered in conjunction with a previously developed toolkit, Implementing Integrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics. AHRQ requests that the Office of Management and Budget approve, under the Paperwork Reduction Act of 1995, AHRQ’s intention to collect information needed to determine whether practice coaching is effective in facilitating adoption of the Chronic Care Model for improving treatment and management of chronic medical conditions by primary care physicians, especially those who care for underserved populations. This project is being conducted pursuant to AHRQ’s statutory authority to conduct and support research on health care and on systems for the delivery of such care, including activities with respect to quality measurement and improvement and with respect to clinical practice, including primary care and practice-oriented research. 42 U.S.C. 299a(2) and (4). This project will be conducted by AHRQ through a contract with the University of Minnesota.
The circumstances that make collection of this information necessary are that chronic disease accounts for a huge proportion of morbidity, mortality, and health care costs in the United States, yet the acute care orientation of the U.S. health care system has not served Americans with chronic conditions well. Recognizing this, the Robert Wood Johnson Foundation (RWJF) funded “Chronic Care Initiatives in HMOs” in 1993. Under this program the Group Health Cooperative of Puget Sound produced a model for effective management of chronic disease in managed health care called the Chronic Care Model (CCM). In 1998 RWJF also instituted “Improving Chronic Illness Care” (ICIC), a national program that operates improvement collaboratives to improve care for a chronic condition, and provides technical assistance and support to organizations interested in improving chronic illness care. That same year the Health Resources and Services Administration (HRSA) launched the Health Disparities Collaboratives (HDC) to change primary health care practices in order to improve the health care provided to underserved, uninsured, and underinsured Americans and to eliminate health disparities.i The HDC initiative adapted the Institute for Healthcare Improvement’s (IHI) Breakthrough Series, creating a regional infrastructure to sustain and support dissemination of improvements in care. The ICIC also used the IHI Breakthrough Series model. Both the ICIC and HDC developed a variety of tools to assist health care delivery sites.
The implementation of the CCM through collaboratives has been evaluated and found to improve the quality of care.ii,iii,iv,v,vi,vii Participants in IHI collaboratives have also reduced waste in the form of shorter intensive care unit stays, less waiting time, and fewer doctor’s office visits, emergency room visits, and unnecessary hospitalizations.viii
Although 1500 physician practices in the U.S. and internationally have been involved in CCM quality improvement efforts, most patients still do not receive their chronic care in accordance with CCM. One factor affecting CCM implementation has been that having teams attend collaborative meetings (three two-day meetings over a nine-month period) is burdensome, especially for under-resourced providers. An attempt to use the Internet as a virtual collaborative met with disappointing results.ix Another barrier to adoption of the CCM in settings that serve vulnerable populations is the scarcity of resources to implement and sustain the CCM.
In 2006 AHRQ contracted with the RAND Corporation, Group Health’s MacColl Institute, and the California Health Care Safety Net Institute (SNI) to develop a toolkit that informs safety net providers on how to redesign their systems of care along the lines of the Chronic Care Model while attending to their financial realities. The result was Implementing Integrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics.x The Toolkit was piloted in two California safety net clinics. Recognizing that merely distributing the Toolkit was unlikely to foster adoption of CCM, the intervention included six months of practice coaching delivered by the MacColl Institute. Practice Coaches (PC) are health care or related professionals who help primary care practices in a variety of quality improvement and research activities. PCs made two site visits to each site and participated in weekly team meetings by phone. They also interacted with the sites through email and phone contact.
The lack of documentation available on coaching led to the development of a practice coaching manual, which was funded by AHRQ through a contract with the RAND Corporation. Development of the Coaching Manual entailed conducting a literature review, interviewing practice coaching experts, and incorporating evaluation results from the coaching provided in conjunction with the Toolkit. The Coaching Manual was published in the winter of 2009. The literature review and interviews revealed that there are a number of different models of practice coaching. However, knowledge is scant about how practice coaching is best performed, under what conditions practice coaching is most successful, and the costs of coaching and being coached. Pilot testing the Toolkit with a low-intensity practice coaching strategy proved insufficient to encourage practices to use the Toolkit independently. The Toolkit was subsequently streamlined based on pilot sites' reports that the initial Toolkit was not easy to use. This project, Studying the Implementation of a Chronic Care Toolkit and Practice Coaching in Practices Serving Vulnerable Populations, will explore the implementation of the revised Toolkit along with a more intensive practice coaching strategy, providing lessons on methods to improve chronic care in clinical practices that serve vulnerable populations.
This project will include the following data collections:
Key Informant Interviews with providers, staff and practice coaches from 20 safety net practices that participate in the practice coaching intervention. These will be used to describe the process and content of practice coaching, perceived changes from the coaching intervention at the practice, provider and patient levels, factors that impeded or facilitated the coaching intervention and implementation of practice changes through the coaching process, overall satisfaction with practice coaching, and recommendations for improvement (see Attachments B, C and D).
Primary Care Practice Profile (PCPP). This questionnaire will be completed by a single individual at each site, either the medical director or chief administrator and will provide an overview of each replication site that will help place intervention activities and outcomes in context for each site. It covers demographics of patients served, patient flow, disease health outcomes, most frequent diagnoses, most frequent referrals, number of staff by discipline, staff and patient satisfaction, processes of care, and organizational processes. (See Attachment E)
Physician Practice Connections-Readiness Survey (PPC-RS) – This questionnaire asks about the presence of 53 practice systems in 5 of the 6 domains of the Chronic Care Model: clinical information systems (information systems, presence of registry or organized database, and systematic monitoring of patient population); decision support (clinician reminders and alerts for lab tests, and visits or guidelines related to individual patient care), delivery system redesign (services for managing patients with chronic illness involving multiple clinicians and care between visits), health care organization (performance tracking and feedback, process of using clinical information systems to aggregate and report on key indicators, and use of data for benchmarking performance and informing QI activities), and clinical quality improvement (presence of formal processes to assess care, develop interventions, and use data to monitor the effects). (See Attachment F)
Assessment of Chronic Illness Care (ACIC) – The ACIC is contained in the Toolkit and yields subscale scores and a total score. Subscale scores reflect CCM components and include: community linkages, self-management support, decision support, delivery system design, information systems, and organization of care. (See Attachment G)
Change Process Capability Questionnaire (CPCQ) – The CPCQ assesses 30 factors and strategies that experienced quality improvement leaders ranked as most important for successful implementation. A recent validation study found good predictive validity. Items correlating with the PPC-RS were eliminated after the initial validation study so there is little to no overlap across the two measures. In addition to changes in the content of care (CCM components), these also include organizational will for change (Priority) and capacity and skill in the conduct of the actual change processes and strategies. (See Attachment H)
Primary Care Staff Satisfaction Survey (PCSSS) – This questionnaire assesses staff satisfaction with their work environment. It consists of 8 4-point likert scale items and 2 open-ended questions, and was developed by the Institute for Healthcare Improvement. (See Attachment I)
Patient Assessment of Chronic Illness Care (PACIC) – The 20-item PACIC consists of five subscales which assess components of the CCM: patient activation, delivery system design/decision support, goal setting, problem-solving/contextual counseling, and follow-up and coordination. (See Attachment J)
Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS) – This questionnaire assesses patient experiences in three areas: getting appointments and healthcare when needed; how well doctors communicate, and courteous and helpful office staff. (See Attachment K)
Chart Audits -- Chart audits will be conducted at baseline, the end of the 10 month coaching intervention, and at 3-month follow-up to assess changes in patient care quality over the course of the intervention. A chart abstraction form will be developed to collect these data (see Attachment L for an example). This data collection will be performed by the project staff and will not impose a burden on the participating sites. Therefore, OMB clearance is not required for this data collection.
This project is expected to contribute to the achievement of AHRQ’s Prevention/Care Management Portfolio goal of, “supporting the evidence base and implementation activities to improve primary care and clinical outcomes through: health care redesign, clinical-community linkages, self-management support, and integration of health IT.” Because providers that serve vulnerable populations serve a disproportionate share of minority patients, this project is also expected to contribute to the reduction of racial and ethnic disparities. According to the 2007 National Healthcare Disparities Report, Hispanics and Black/African Americans fared worse than white non-Hispanic Americans on measures of chronic care quality.
This is a mixed qualitative and quantitative study that is designed to allow us to learn lessons about the practicality and impact of implementing a Chronic Care Toolkit and practice coaching in practices that serve vulnerable populations. The qualitative methods and quantitative data collected in the study will be used to document quality improvement activities and their impact across 20 safety net practice sites that are participating in the study.
If the implementation in this study is successful, the information collected by this project can be used to improve the delivery and efficiency of chronic care through the use of the Chronic Care Model in numerous primary care practices that serve vulnerable populations by using the Integrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics and Coaching Manual combined with practice coaching. If the implementation in the study is not successful, the lessons learned will be used to inform the development of other strategies for managing chronic medical for vulnerable populations.
Specific research questions that will be addressed in evaluating whether the implementation was successful are:
1. What aspects of the CCM and/or business strategies from the Toolkit were implemented as a result of the Toolkit plus practice coaching intervention?
2. What changes do practices make to self-management support, delivery of evidence based medicine, and efficiency of care as a result of the activities undertaken as part of the practice coaching intervention?
3. Do practices’ capacity to manage and sustain change in their organizations increase?
4. Does the quality of diabetes or other practice-identified chronic care improve after practices participate in the practice coaching intervention?
5. Do adult patients report increased satisfaction with care and increases in activities related to the four CCM dimensions: patient management self-support, delivery system design, decision support, and information systems?
6. Does provider satisfaction with the organizational and care environment improve after the intervention?
7. Do practice characteristics such as size, location (urban/rural) and structure (Academic Health Center, Community Health Center, Federally Qualified Health Center, Rural Health Center) appear to be associated with acceptance and implementation of the practice coaching intervention and Toolkit, what aspects of practice coaching and the Toolkit are used, what aspects the CCM are targeted, and what types and degree of change is seen in processes and quality of care post intervention?
A.3. Use of Improved Information Technology
As described in Section A.1, there has been substantial research on the implementation of the CCM through collaboratives, where it has been evaluated and found to improve the quality of care.xi,xii,xiii,xiv,xv,xvi Further, as described earlier in Section A.1, in 2006 AHRQ contracted with the RAND Corporation, Group Health’s MacColl Institute, and the California Health Care Safety Net Institute (SNI) to develop a toolkit Implementing Integrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics.xvii, which was piloted in two California safety net clinics with the assistance of practice coaches. However, the RAND/MacColl/SNI team suggested the development of a practice coaching manual because a literature review and interviews revealed that there are a number of different models of practice coaching, but knowledge is scant about how practice coaching is best performed, and under what conditions practice coaching is most successful.
In their 2005 review, Mold and Apsy identified 47 articles on practice facilitation or coaching. Of these, 25 measured the impact of interventions involving PCs on patient care outcomes and supported the potential effectiveness of PC as a strategy for planned change. Frijiling et al (2002) found that feedback reports from PCs increased rates of two (diabetic foot and eye examinations) out of 7 patient care outcomes for diabetes. In a randomized, controlled trial involving 98 physicians, Dietrich et al (1992) found that a PC-led intervention involving assessment, training, and use of decision support tools increased office system interventions related to cancer screening and preventive services. Similarly, in Margolis et al’s (2004) RCT with 26 practices, nurse facilitators increased appropriate requests for hemoglobin tests by 99% compared to a 23% decline in control sites. Baskerville, Hogg and Lemelin (2001) tested a multi-component intervention involving the use of nurse facilitators to improve preventive care performance and found statistically significant improvements on an overall index of preventive performance (11.5%), up-to-dateness (7.2%) and service inappropriateness (4.4%). Kinsinger et al (1998) conducted a study assessing the impact of a PC facilitated intervention on rates of breast cancer screening in 62 primary care practices from rural counties in North Carolina. Interestingly, while the intervention achieved improvements in care processes, these improvements did not translate to actual increases in mammograms or clinical breast exams (CBEs). This study highlights the not uncommon experience in quality improvement efforts that process improvements may fail to translate to actual improvements in care and health outcomes. PCs can provide the type of tailored hands-on support that is needed to identify factors that may be impeding improvement in these areas.
Goodwin et al (2001) tested a practice-tailored approach to increasing delivery of preventive services in an RCT involving 77 Ohio family practices (STEP-UP) and found significant changes in some areas of preventive care but not in others (immunizations). Finally, Bryce et al (1995) evaluated the impact of an audit facilitator on pattern of diagnosis and treatment of childhood asthma in 12 practices. At 2-year follow-up, there were significant increases in asthma consultations, new diagnoses of asthma and more past diagnoses reaffirmed in intervention vs. control practices. Consistent with findings that implementing the CCM can increase costs in primary care settings, costs of care at the intervention sites increased slightly. However, hospitalization costs declined. This study illustrates the very real calculus CHCs and other safety-net sites must grapple with, where the cost savings resulting from improved care in their practices accrues to other organizations but not their own.
As can be seen from the above efforts to indentify duplication, the questions that the project is intended to address have not been definitively answered by previous information collection. In fact, the previous information collection identified the need to collect information to answer the proposed questions that will be addressed by this project in order to develop improved methods of providing chronic care that can be more widely disseminated and implemented in clinical practices, including those serving vulnerable populations.
While large safety net practices deliver care to substantial numbers of low-income and uninsured patients in the U.S., a substantial portion of patients receive their health care in small and solo practice settings. In Los Angeles County for example, more than 80% of managed MediCal patients enrolled in L.A. Care, one of the largest public health plans in Los Angeles, are cared for in small practices where there are two or fewer FTE Primary Care Providers) (Seidman, 2009, personal communication). Thus it is very important to include these small practices in this study, to better understand the impact of PC in these environments also. Small safety net practices, specifically small practices with fewer than 3 Primary Care Provides (PCPs) will be included in the study to allow examination of how the process and potential outcomes of practice coaching may vary by practice size. To reduce burden on these small entities, clinic administrators and PCPs will be consulted to arrange a data collection schedule that they believe will be minimally disruptive to their work day. Based on past experience working in these smaller practices, spreading data collection over a period of 2 to 3 days so that each encounter is of shorter duration (3 20-minute sessions rather than 1 60- minute session) and coordinating these with times during the day when there are fewer patients to the clinic (which varies by practice but is often immediately after lunch and again in late afternoon) helps to reduce or eliminate disruption and burden from data collection on the staff and PCPs.
This project is a one-time information collection effort only.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
A. Federal Register Notice
As required by 5 CFR 1320.8(d), a notice was published in the Federal Register on date, February 1st, 2010 for 60 days (See Attachment M). One comment was received and is shown below, along with AHRQ's response:
Public comment:
From:
jean public [mailto:[email protected]]
Sent: Monday,
February 01, 2010 10:12 AM
To:
Lefkowitz, Doris C. (AHRQ); [email protected]; [email protected];
[email protected]; [email protected]
Subject: PUBLIC COMMENT ON FEDERAL REGISTER
THIS DEPT PROJECT IS A CESSPOOL OF OVERSPENDING. THIS ENTIRE PROJECT IS OBVIOUSLY NOT WORKING. HOW MANY ETERNITIES SHOUDL AMERICAN TAXPAYERS FUND THIS NONSENSE. SHUT DOWN THIS ENTIRE PPOJECT AND LAY OFF THE MGT OF THIS PROJECT AND WORKERS. THIS DEPT IS SO NON PRODUCTIVE THAT IT HURTS. SHUT DOWN AND SUNSET PLEASE.
JEAN PBULIC 15 ELM ST FLORHAM PARK NJ07932
AHRQ's response:
AHRQ thanks the public for taking the time to provide comments on proposed information collections. As always, AHRQ we will supervise the project closely to maximize efficiency and ensure that the project benefits those with chronic illness.
B. Outside Consultations
There were no outside consultations by the LA-NET team. AHRQ had 3 internal reviewers comment on the research design as initially proposed. In developing this project, AHRQ also consulted with the Commonwealth Fund project officer that led the Fund’s Safety Net Medical Home Initiative.
No payment or gifts will be given to respondents.
Individuals and organizations contacted will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Information that can directly identify the respondent, such as name and/or social security number, will not be collected.
No sensitive questions will be asked in this information collection.
Exhibit 1 shows the estimated annualized burden hours for the respondents' time to participate in this 2 year study. Key informant interviews will be conducted with practice coaches at mid-point in the intervention and again at the end of the intervention. Key informant interviews will also be conducted with up to 3 primary care providers and 2 other staff members from each of the 20 practices (10 per year) prior to start of the intervention, and again at 3-month follow-up after the intervention is completed. Each interview takes about 1 hour.
The Primary Care Practice Profile will be administered once and will be completed by one staff person from each practice and takes 30 minutes to complete. The Physician Practice Connections-Readiness Survey (PPC-RS) will be completed pre, post and at 3-month follow-up by three individuals from each of the 20 practices (individuals with the appropriate knowledge to complete the survey will be identified by the medical director of each site). It takes 90 minutes to complete. The Assessment of Chronic Illness Care (ACIC) will be completed by 4 staff and 4 primary care providers per practice at pre, post and 3-month follow-up and takes 30 minutes to complete. The Change Process Capability Questionnaire (CPCQ) will be completed by 4 staff and 4 primary care providers per practice at pre, post and 3-month follow-up and takes 15 minutes to complete. The Primary Care Staff Satisfaction Survey (PCSSS) will be completed by 4 staff and 4 primary care providers per practice at pre, post and 3-month follow-up and takes 15 minutes to complete. The Patient Assessment of Chronic Illness Care (PACIC) will be completed by 3,000 adult patients (1,500 annually) with chronic illness and requires 15 minutes to complete. The Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS) will be completed by 3,000 adult patients (1,500 annually) with chronic illness and requires 45 minutes to complete. Both patient surveys will be administered to adult patients with a chronic disease who receive care at the practices during a 2-day data collection period immediately before, immediately after, and at 3-month follow-up. The surveys will be administered during the post visit period in the wait room, by a bi-lingual Spanish-English research assistant. The total annualized burden hours are estimated to be 1,984 hours.
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of Respondents |
Number of responses per respondent |
Hours per response |
Total Burden hours |
Key informant interviews with practice coaches |
2 |
2 |
1 |
4 |
Key informant interviews with providers (3 per practice interviewed twice) |
10 |
6 |
1 |
60 |
Key informant interviews with staff (2 per practice interviewed twice) |
10 |
4 |
1 |
40 |
Primary Care Practice Profile (PCPP) |
10 |
1 |
30/60 |
5 |
Physician Practice Connections-Readiness Survey (PPC-RS) (3 per practice x 3 times) |
10 |
9 |
1.5 |
135 |
Assessment of Chronic Illness Care (ACIC) (8 per practice x 3 times) |
10 |
24 |
30/60 |
120 |
Change Process Capability Questionnaire (CPCQ) (8 per practice x 3 times) |
10 |
24 |
15/60 |
60 |
Primary Care Staff Satisfaction Survey (PCSSS) (8 per practice x 3 times) |
10 |
24 |
15/60 |
60 |
Patient Assessment of Chronic Illness Care (PACIC) |
1,500 |
1 |
15/60 |
375 |
Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS) |
1,500 |
1 |
45/60 |
1,125 |
Total |
3,072 |
na |
na |
1,984 |
Exhibit 2 shows the estimated annualized cost burden associated with the respondent's time to participate in this study. The total annualized cost burden is estimated to be $60,714.
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of Respondents |
Total Burden hours |
Average Hourly Wage Rate* |
Total Cost Burden |
Key informant interviews with practice coaches |
2 |
4 |
$42.00 |
$168 |
Key informant interviews with providers |
10 |
60 |
$77.64 |
$4,658 |
Key informant interviews with staff |
10 |
40 |
$32.64 |
$1,306 |
Primary Care Practice Profile (PCPP) |
10 |
5 |
$77.64 |
$388
|
Physician Practice Connections-Readiness Survey (PPC-RS) |
10 |
135 |
$77.64 |
$10,481 |
Assessment of Chronic Illness Care (ACIC) |
10 |
120 |
$55.14** |
$6,617 |
Change Process Capability Questionnaire (CPCQ) |
10 |
60 |
$55.14** |
$3,308 |
Primary Care Staff Satisfaction Survey |
10 |
60 |
$55.14** |
$3,308 |
Patient Assessment of Chronic Illness Care (PACIC) |
1,500 |
375 |
$20.32 |
$7,620 |
Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS) |
1,500 |
1,125 |
$20.32 |
$22,860 |
TOTAL |
3,072 |
1,984 |
na |
$60,714 |
*Based upon the mean of the average wages, May 2008 National Occupational and Wage Estimates accessed on December 14, 2009 at: http://www.bls.gov/oes/current/oes_nat.htm#b29-0000National Compensation Survey: ** Average for 4 staff ($32.64/hr ) and 4 physician clinicians. ($77.64/hr).
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3 shows the estimated total and annualized cost to conduct this research. The total cost over two years is estimated to be $600,000.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$162,744 |
$81,372 |
Data Collection Activities |
$92,994 |
$46,497 |
Data Processing and Analysis (20%) |
$92,994 |
$46,497 |
Publication of Results |
$23,248 |
$11,624 |
Project Management |
$92,994 |
$46,497 |
Overhead |
$135,026 |
$67,513 |
Total |
$600,000 |
$300,000 |
A.16. Time Schedule, Publication and Analysis Plans
The following table shows the correspondence of the research questions with the data collection instruments to be used and the items within each instrument that are relevant to the research question.
Research Question |
Instruments Used |
Relevant Items |
What aspects of the CCM and/or business strategies from the Toolkit were implemented as a result of the Toolkit plus practice coaching intervention? |
Key informant interviews at practices–semi-structured interview guide |
Key informant interview for practice coaches item 3
|
What changes do practices make to self-management support, delivery of evidence based medicine, and efficiency of care?
|
PPC-RS ACIC
|
Key informant interview staff/provider item 3 Key informant interview for coaches –item 4 PPC-RS – all items ACIC - all items |
Do practices’ capacity to manage and sustain change in their organizations increase? Does provider and staff satisfaction improve? |
CPCQ Key informant interviews |
Key informant interview staff/provider item 3 Key informant interview for coaches – item 4 CPCQ all items PCSSS |
Does the quality of diabetes care or other practice-selected index condition improve after practices participate in the practice coaching intervention? |
CPCQ CAHPS-PCA Key informant interviews |
CPCQ all items CAHPS-PCA items 1-37 Chart audit |
Do adult patients report increased satisfaction with care and increases in activities related to the four CCM dimensions? |
CAHPS-PCA PACIC |
CAHPS-PCA all items PACIC all items |
Does provider satisfaction with the organizational and care environment improve after the intervention? |
CPPP |
CPPP all items |
Do practice characteristics or prior level of CCM implementation appear to be associated with acceptance and implementation of the practice coaching intervention and Toolkit what aspects of practice coaching intervention and the Toolkit are used, what aspects the CCM are targeted, and what types and degree of change is seen in processes and quality of care post intervention? |
PCPP Key informant interviews at practices–semi-structured interview guide |
Key informant interview staff-provider - all items Key informant interview for coaches – all items PCPP all items ACIC all items PPC-RS all items PACIC all items CAHPS all items CPCQ all items Chart audit checklist |
Analysis plan
The purpose of this study is to examine the acceptability, feasibility and potential effectiveness of the CCM Practice Coaching Intervention and Toolkit by studying the implementation of practice coaching in 20 safety net practices in California. It is not an experimental study to determine the efficacy of the intervention. Data analysis will focus on describing the implementation and potential effects of the intervention at the individual practice level and comparing how implementation and potential effects varied across the 20 replication sites. Quantitative data collected for the study will be analyzed at the individual practice level using either statistical or heuristic guidelines to describe whether they improved, stayed the same or got worse. If heuristic means are used, the criteria will be determined in advance and based upon clinically important change as determined by the practice in consultation with the research team. If the criterion is to be statistical and the outcome measure is expressed as a rate, then a z-test will be used for differences in proportions to determine if the change from pre to post is significant at the standard alpha. If the outcome measure is a continuous variable, independent sample t-tests will be used.
The second step of the analysis is to look at patterns of change across all 20 practices thus treating each practice as a replication. Each practice will be rated as either improved, stayed the same or got worse on their individually identified criterion. The results of these individual analyses will be assessed using a Chi Square test of Goodness of Fit where the null hypothesis would be that a third of the sample would fall into each cell if the pattern was truly random. With 20 practices this is approach does not have high power, but the patterns of change across independent practices at different times linked to similar interventions, accompanied by clear descriptions of each intervention should allow us to evaluate the overall effectiveness of our intervention. Also – specific patterns of change that are particularly relevant to this study are more likely to show significant results. The actual results of individual chi-square analyses are dependent on both how many can be assigned to the ‘got better’ cell and the distribution of results among the other 2 cells. If they are fairly even (e.g. as many stayed the same as got worse) then many more must be in the ‘got better’ cell for a significant result. If, however, only 1 or 2 practices ‘get worse’, some ‘stay the same’ and at least 10 ‘get better’, there would be a significant chi square. Therefore if the intervention at least insures that no one gets worse, most get better, and if they don’t get better – at least stay the same, there would be statistically significant results.
Descriptive Analyses. Means and confidence intervals (adjusted for clustering as needed) will be computed for all continuous study variables. Frequencies and percentages will be computed for categorical variables. These descriptive analyses will further be stratified by practice type (CHC, FQHC, RHC, small practice, AHC) or other dimensions of interest to AHRQ (size) and measurement occasion (pre, post, 3 months post).
Analysis of Change. Baseline compliance rates will be calculated for the quality of care and patient outcomes indicators at the practice level and will be compared to follow-up rates using the Z test for a single proportion where the baseline rate will define the null condition and the follow-up rate will be the alternate condition. One-sided tests will be used since there is no expectation that rates will worsen after the intervention. For continuous measures collected at the individual level (staff or patient satisfaction, adherence to clinical care model, etc.), mixed models analysis of variance will be used to assess change over time while accounting for clustering at the practice level. Practice characteristics and any unplanned variation in intervention intensity will be added to these models as covariates to assess their relationship to change in the dependent measure.
Qualitative Analysis. Interview data will be entered into NVivo and analyzed for content and theme using methods recommended by Crabtree and Miller (1999). Interview data will be used to provide a detailed description of the PC process at each site, to provide a context for the quantitative data collected at each site, to allow for a qualitative comparison of how the PC intervention at each site was similar to or different from those of the other 19, and to allow us to examine qualitatively how the PC process and associated changes in care may have varied across dimensions such as practice size and type and to generate hypotheses and research questions to guide future work in this area.
Ability to attribute causality. Because this is a non-randomized study design, causality cannot be attributed to the intervention.
Ability to detect which practice characteristics and which change strategies are associated with successful implementation and impact. The project budget it not large enough to allow us to work in a large enough number of practices to determine statistically which practice characteristics and which change strategies are associated with successful implementation and impact of the Toolkit-practice coaching intervention. However, there will be more than sufficient information to develop hypotheses related to both of these questions that can be incorporated as part of the Practice Coaching research agenda that will also be developed under this Task Order.
Analysis of Qualitative Data
Qualitative data will be analyzed for content and theme using methods recommended by Crabtree and Miller and Strauss and Corbin. Expert member checks with practice staff, coaches and providers will be conducted to assess the validity and reliability of finding.
Preliminary Dissemination plan
The following methods will be used to disseminate findings and products from this project: Post findings and reports on Web sites of organizations such as the Community and Migrant Health Centers, Migrant Clinician’s Network, Health Disparities Collaboratives, AHRQ’s Health Care Innovation Exchange and Institute for Healthcare Improvement, the National PBRN Resource Center, LA Net’s, NACHC, and other relevant organizations; Publish results in journals such as the Journal for the Poor and Underserved and the American Journal of Public Health and professional journals such as Health Care for the Homeless Research Update; Present at conferences such as the National Association of Community Health Centers, the North American Primary Care Research Group and AHRQ’s Annual PBRN conference; Promote among other PBRNs beginning with the PRIME Net, OKPRN, ePCRN and NRN meta-networks; Engage organizational partners such as the Association of American Medical Colleges, IHI, and the National Association of Community Health Centers to aid in dissemination. In addition, AHRQ proposes to deliver a pre-conference workshop on practice coaching and the CCM at the AHRQ National PBRN Conference each year for 3 years to engage conference participants in discussions around the practice and science of practice coaching and to provide opportunity for knowledge, skill building and multi-PBRN projects involving practice coaching in the future.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A – Healthcare Research and Quality Act of 1999
Attachment B – Key informant interview guide for practice coaches
Attachment C – Key informant interview guide for providers
Attachment D – Key informant interview guide for staff
Attachment E – Primary Care Practice Profile (PCPP) Questionnaire
Attachment F – Physician Practice Connections-Readiness Survey (PPC-RS)
Attachment G – Assessment of Chronic Illness Care (ACIC) Questionnaire
Attachment H – Change Process Capability Questionnaire (CPCQ)
Attachment I – Primary Care Staff Satisfaction Survey (PCSSS)
Attachment J – Patient Assessment of Chronic Illness Care (PACIC) Questionnaire
Attachment K – Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS) Questionnaire
Attachment L – Chart Audit Abstraction Form
Attachment M – Federal Register Notice
References
i See www.healthdisparities.net/hdc/html/home.aspx for more information on HRSA’s Health Disparities Collaboratives.
ii Coleman K, Austin B, Brach C, et al. Forthcoming. Evidence on the Chronic Care Model in the new millennium. Health Affairs,.
iii Vargas RB, Mangione CM, Keesey J, et al. 2004. Do collaborative quality improvement programs reduce cardiovascular risk for persons with diabetes? In Annual Research Conference. San Diego: AcademyHealth.
iv Baker DW, Asch SM, Keesey JW, et al. 2005. Differences in education, knowledge, self-management activities, and health outcomes for patients with heart failure cared for under the chronic disease model: the improving chronic illness care evaluation. J Card Fail 11 (6):405-13.
v Asch SM, Baker DW, Keesey JW, et al.. 2005. Does the collaborative model improve care for chronic heart failure? Med Care 43 (7):667-75
vi Huang E, Chin M. 2005. Health disparities collaboratives: changing practices, changing lives: assessing the impact of the HRSA health disparities collaboratives and what comes next In 2005 Community Health Institute. Miami, FL: National Association of Community Health Centers. Tsai AC.
vii Morton SC, Mangione CM, et al. 2005. A meta-analysis of interventions to improve care for chronic illnesses. Am J Manag Care 11 (8):478-88.
viii Coffey RM, Matthews TL, McDermott K. Diabetes Care Quality Improvement: A Resource Guide for State Action. (Prepared by The Medstat Group, Inc. and The Council of State Governments under Contract No. 290-00-0004). Rockville, MD: Agency for Healthcare Research and Quality, Department of Health and Human Services; September 2004. AHRQ Pub. No. 04-0072.
ix Health Care Improvement Process Made Accessible to Safety-Net Providers via Breakthrough Series on Internet Foundation, Robert Wood Johnson. 2003. Health Care Improvement Process Made Accessible to Safety-Net Providers via Breakthrough Series on Internet. Available at: www.rwjf.org/portfolios/resources/grantsreport.jsp?filename=040709.htm&iaid=142.
xIntegrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics. Rockville, MD: Agency for Healthcare Research and Quality. Available at http://www.ahrq.gov/populations/businessstrategies/
xi Coleman K, Austin B, Brach C, et al. Forthcoming. Evidence on the Chronic Care Model in the new millennium. Health Affairs,.
xii Vargas RB, Mangione CM, Keesey J, et al. 2004. Do collaborative quality improvement programs reduce cardiovascular risk for persons with diabetes? In Annual Research Conference. San Diego: AcademyHealth.
xiii Baker DW, Asch SM, Keesey JW, et al. 2005. Differences in education, knowledge, self-management activities, and health outcomes for patients with heart failure cared for under the chronic disease model: the improving chronic illness care evaluation. J Card Fail 11 (6):405-13.
xiv Asch SM, Baker DW, Keesey JW, et al.. 2005. Does the collaborative model improve care for chronic heart failure? Med Care 43 (7):667-75
xv Huang E, Chin M. 2005. Health disparities collaboratives: changing practices, changing lives: assessing the impact of the HRSA health disparities collaboratives and what comes next In 2005 Community Health Institute. Miami, FL: National Association of Community Health Centers. Tsai AC.
xvi Morton SC, Mangione CM, et al. 2005. A meta-analysis of interventions to improve care for chronic illnesses. Am J Manag Care 11 (8):478-88.
xviiIntegrating Chronic Care and Business Strategies in the Safety Net: A Toolkit for Primary Care Practices and Clinics. Rockville, MD: Agency for Healthcare Research and Quality. Available at http://www.ahrq.gov/populations/businessstrategies/
| File Type | application/msword |
| File Title | SUPPORTING STATEMENT |
| Author | AHCPR |
| Last Modified By | wcarroll |
| File Modified | 2010-03-24 |
| File Created | 2010-01-08 |