Supporting Statement Part B -- Implementing Practice Coaching and the Chronic Care Model 1-8-2010

SUPPORTING STATEMENT

Part B

Studying the Implementation of

a Chronic Care Toolkit and Practice Coaching

In Practices Serving Vulnerable Populations

Version: January 6^th, 2010

Agency of Healthcare Research and Quality (AHRQ)

TABLE OF CONTENTS

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods 1

B.2. Information Collection Procedures 6

B.3. Methods to Maximize Response Rates 10

B.4. Tests of Procedures 12

B.5. Statistical Consultants 13

B.1. Respondent Universe and Sampling Methods

The primary implementation unit for this study is the safety net practice, predominately Community Health Centers (CHCs). The population of CHCs in the U.S. is 7000 based on figures maintained by the National Association of Community Health Centers (NACHC). CHCs are health center service delivery sites defined as “community, migrant, homeless, and Public Housing Health Centers that are non-profit, community-directed health care providers serving low income and medically underserved communities” (NACHC 2009). Currently CHCs serve over 18 million people throughout the United States.

This study will use purposive sampling to select sites for this study. It is impractical to select a probability sample from all of the eligible safety-net practices in the United States, since the research budget limits the project to hiring only 4 practice coaches (PCs). The level of funding available for this data collection does not permit hiring additional PCs throughout the country or having the available PCs travel extensively. Thus, the study sites must be within reasonable traveling distance of the available PCs.

Although this qualitative study will use a purposive sample, the lessons learned from a study with 20 CHCs in California will help inform quality improvement and health disparity reduction efforts both in California and in CHCs across the country. The results are likely to have relevance to other areas in the U.S. because of the basic similarities in populations served by CHCs and resources available to CHCs throughout the country.

Ten urban/suburban CHCs and 10 rural safety net practices will participate in this study from Los Angeles (N=8), San Francisco (N=2) and Northern California (N=10). Selection was based on willingness to participate, proximity (the site is within 1 hour of the PC to allow for regular on-site visits), and practice characteristics (size -small, medium, large; location-urban, suburban, rural; structure – Community Health Centers, Federally Qualified Health Centers, Academic Health Centers, and Public Health Centers). The Table below contains the names of the 20 organizations that will participate in the study.

Participating practices

Name	Structure	Location
Los Angeles Area
Asian Pacific Health Care Venture, Inc.	FQHC	1530 Hillhurst Avenue Los Angeles, CA  90027 Urban (LA)
Cleaver Family Wellness Clinic	CHC	4368 Santa Anita Ave. El Monte, CA 91731 Suburban (LA)
Clinica Monsignor Oscar Romero	FQHC	123 South Alvarado Street Los Angeles, CA 90057 Urban (LA)
Community Health Alliance of Pasadena (CHAP)	FQHC	1855 N. Fair Oaks Ave., Suite 200 Pasadena, CA 91103 Suburban (LA)
Family Healthcare Centers of Greater Los Angeles	CHC	6501 S. Garfield Ave. Bell Gardens, CA 90201 Urban (LA)
QueensCare Family Clinics	FQHC	1300 North Vermont Avenue, Suite 1002 Los Angeles, CA 90027-0005 Urban (LA)
Saban Free Clinic	CHC	90048-3476 Urban (LA) 6405 Beverly Boulevard, Los Angeles CA
Valley Community Clinic	FQHC	6801 Coldwater Canyon Ave North Hollywood, CA 91605 Suburban (LA)
San Francisco Area
Lifelong Medical Center	FQHC	PO Box 11247 Berkeley, CA. Urban/Suburban (SF)
San Francisco General Hospital	Academic	995 Potrero Avenue, San Francisco CA 94110
Redwood Community Health Coalition
Alexander Valley Regional Medical Center	FQHC	6 Tarman Drive, Cloverdale, CA 95425
Alliance Medical Center	FQHC	1381 University Avenue, Healdsburg CA  95448
Coastal Health Alliance	CHC	3 6th Street PO Box 910 Point Reyes Station, CA 94956
CommuniCare Health Centers	FQHC	2051 John Jones Road Davis, CA 95616
Community Health Clinic OLE	FQHC	1141 Pear Tree Lane, Suite 100 Napa, CA 94558
Marin Community Clinic	FQHC	P.O. Box 1868 Novato, CA 94948-1868
Petaluma Health Center	FQHC	1301 Southpoint Blvd, Suite A Petaluma, CA 94954-6858
Ritter Center	CHC	16 Ritter St, PO Box 3517, San Rafael 94912
Sonoma County Indian Health Project	Tribal Health Council	4400 Auburn Blvd., 2nd Floor Sacramento, CA 95841
Sonoma Valley Community Health Center	CHC	430 West Napa St., Suite F, Sonoma, CA 95476

The respondent universe for this study is the staff and providers from the 20 participating safety net practices (N=10 per practice – 5 staff and 5 primary care providers, for a total of N=200 across the 20 participating practices); the practice coaches who will deliver the intervention (N=4); and patients with chronic disease who receive care from the 20 participating practices (N=150 per practice, or 3000 across all 20 practices).

Staff and provider interviews and surveys. Surveys. Within each participating site (practice location), all staff members and primary care providers who work at least 50 percent of full-time will be included in the study and will be asked to complete surveys at the beginning and end of the study period. The average number of part timer or greater staff and primary care providers per site is 5 staff and 5 primary care providers for an average of 10 per site. This gives an estimated 200 potential respondents across the 20 sites. Key informant interviews. Up to two staff and 3 providers from each of the 20 safety net practices will be asked to complete two key-informant interviews each. The first key informant interview will occur immediately before the start of the intervention and will ask them about the implementation of the chronic care model at their practices, factors impeding or facilitating its implementation, and their perception of the practice’s capacity to support practice improvement activities to increase implementation of the Chronic Care Model. The second interview will occur immediately after the intervention and will ask them about their experiences with and perceptions of the impact of the practice-coaching intervention on implementation of the Chronic Care Model in their practices. The key-informant interviews with staff and primary care providers will use semi-structured interview guides that are provided in this application and will require approximately 1-hour of respondent time to complete.

In addition, all staff and providers employed 50% time or greater in the participating practices will be asked to complete descriptive surveys of the degree to which their practice has implemented the Chronic Care Model, its organizational capacity for change, and work satisfaction. The surveys staff and providers will be asked to complete are the: Assessment of Chronic Illness Care (ACIC), Change Process Capability Questionnaire (CPCQ), and Primary Care Staff Satisfaction Survey.

Two staff and one primary care provider per site (total individuals responding N=60 across 20 sites) will be asked to complete the Physician Practice Connections-Readiness Survey (PPC-RS) at pre, post and 3-month follow-up to assess implementation of components of the patient-centered medical home.

Finally, one staff person per site (total individuals responding N=20 across 20 sites) will be asked to complete the practice profile at the start of the intervention.

Practice Coach Interviews. The four coaches who deliver the practice coaching intervention to the 20 practices will complete key-informant interviews at mid-point in the intervention (5 months) and at the end of the intervention (10 months). The interview will ask the coaches about their observations about the process of coaching and how it varied by practice (each will coach between 4 and 8 practices each over the course of the intervention), the content of the coaching and how it varied by practice, factors that facilitated or impeded implementation of the Chronic Care Model and how these varied by practice, and perceived impact of the coaching on implementation of the Chronic Care Model and organizational level support to sustain current and future improvements in patient care and how this varied by practice. The key-informant interviews with practice coaches will use semi-structured interview guides that are provided in this application and will require approximately 1-hour of respondent time to complete.

Patient surveys. A total of 150 adults receiving care for a chronic disease at each of the 20 practices will be asked to complete a short patient satisfaction survey. The surveys will be administered three times, pre-intervention, post-intervention, and 3-month follow-up. They will be administered in the waiting room over two consecutive days to any adult with chronic disease who receive care the practice during a two-day period immediately before the start of the intervention (N=50 per practice), immediately after the intervention (N=50 per practice) and at 3-month follow-up (N=50 per practice). The target accrual for each practice for each data collection period will be 50 for a total of 150 per practice or 3000 patients across all 20 practices surveyed by the end of the study. The survey will be available in both Spanish and English and a bi-lingual research assistant will be available to assist patient with low literacy levels to complete the survey. The patient survey instruments that will be used are the Patient Assessment of Chronic Illness Care (PACIC) and the Consumer Assessment of Healthcare Providers and Systems- Primary Care Adult (CAHPS-PCA).

Chart audits. Chart audits will be conducted to assess changes in patient care quality over the course of the intervention. The process(es) of care that will be assessed will be determined by each individual practice during the first 4 practice coaching sessions (weeks 1-4). For practices without electronic data, audits will be conducted of the medical records of up to 60 patients with the index condition from each practice who received care from the practice during the 12 months prior to the start of the intervention. A chart audit form specific to the index condition/practice process of each specific practice will be used to capture the data. For practices with electronic data, we will work with IT personnel at the site to obtain these same data through their electronic health record or registries. Electronic or hand abstracted data will then be entered into a project database for analysis.

B.2. Procedures for the Collection of Information

Overview. Data will be collected immediately before beginning the 10-month practice coaching intervention, immediately after the intervention, and at 3-months follow-up to evaluate the intervention. The interviews with the practice coaches will be conducted at mid-point (5 months) and the end of the intervention (10 months). IRB approval will be obtained prior to beginning the study. Appropriate informed consent procedures will be followed to conform with human subjects protection and HIPAA requirements. All data will be collected by individuals trained in qualitative interviewing, and survey administration. Bi-lingual staff will be used to collect patient satisfaction surveys. Everyone involved in data collection will be required to have current Human Subjects certification, training in HIPAA compliance and be covered by a Business Agreement with the site as appropriate to ensure compliance with IRB and HIPAA requirements.

Staff and Provider Interviews and Surveys. The study PI or research assistant trained in qualitative interview and certified in human subjects research will conduct the Key-Informant interviews with staff and providers at each site. These will be conducted in-person or by phone depending on what is most convenient and feasible for the repondent. The interviews will last approximately an hour and will be audiotaped and then transcribed for analysis. The interviewee’s name will be removed from the transcript and replaced with a subject code. The research team will be the only individuals with access to the master key connecting name and code. Appropriate consent will be obtained from staff before data collection begins.

A research assistant will coordinate administration of the surveys to staff and providers. The surveys will be made available in paper format and also when possible, electronically, using a secure on-line survey program. in a secure on-line format. Appropriate consent will be obtained from staff before data collection begins. Surveys will be anonymous and no personal identifiers will be required. Surveys will contain a practice code so the data can be linked to the appropriate practice site.

Practice coach interviews. The project PI or research assistant trained in qualitative interview methods and certified in research with human subjects will complete the key-interviews with the practice coaches at mid point (5 months) and at the end of the intervention (10 months). The semi-structure interview guide provided in this application will be used. The interviews will require approximately 90 minutes to complete, and will be audio-taped and transcribed for analysis. Personal identifiers of the coaches and the practice locations/staff mentioned will not be removed to allow for analysis of variations across practice sites. The only individuals that will have access to the data are members of the research team and data will be kept in locked/password protected files per human subjects requirements. Key informant interviews that will be conducted with practice coaches about process and variations in implementation and observed impact are not included in the burden estimates since they will be conducted with fewer than 9 individuals.

Patient survey data. Patient surveys will be conducted in the waiting room of the 20 participating practices. A bi-lingual research associate trained in human subjects and in survey administration with low-literacy populations will be placed in each waiting room for a two-day period. All adults receiving care at the practice that indicate they have a chronic disease will be surveyed. Data will be collected during 3 2-day data collection periods: immediately before the intervention, immediately after and at 3-month follow-up. A trained bi-lingual research associate will be placed in each practice during those times, in the waiting room or post-visit area per the directions of the practice, and invite patients with chronic disease to complete the survey at the end of their visit. The research associate will assist those with low-literacy levels to complete the survey. The survey will be available in paper format or on a computer located in a private area of the waiting room or post visit area and patients will be offered a choice of either paper or electronic format. The survey will be anonymous with no personal identifiers requested. Each survey will contain a code linking it to the practice to allow comparison across practices. Respondents will be provided with a privacy envelope to place their surveys in after completion and will be asked to place their paper surveys in a locked survey box that will be maintained at the site by the research staff member during the data collection period. This individual will be responsible for collecting and mailing the paper-based surveys to the research offices at LA Net in Long Beach California at the end of the 2-day data collection period via Fed Ex. Upon receipt by LA Net central office, paper surveys will be stored in a locked file drawer and entered into a password protected database on a desktop computer at the office.

Chart audits. After the 4th practice coaching visit when the practice and coach have identified the target for the practice coaching intervention, a research associate will work with the practice IT personnel to identify patients who have received care in the 12 months prior to the intervention with the target condition using billing records for patient in practices without EHRs and through the EHR for practices that do have them. In non-EHR enabled practices, the research associate will then work with the medical records staff to pull paper charts for 60 randomly selected patients from the list, and audit the chart for presence or absence of key quality indicators based on HEDIS measures. In EHR enabled practices, the research assistant will work with the practice IT staff to create an excel data file containing patient visit data on these same quality indicators. All personal identifiers for patients will be removed and a subject code assigned. The master key linking patient identifiers with care data will be kept in a separate locked and password protected file. Registry and chart data sources that will be used to evaluate improvements on quality of care indicators are not included in the estimates of respondent burden for this supporting statement, since they do not require information collection from respondents.

Variables and measurement

This is a study of the implementation of the practice coaching intervention in 20 safety net practices. The primary goal of the study is to increase the understanding of the practice coaching process and how its process, content and outcomes may vary across different practice environments, and to examine its potential value for supporting greater implementation of the Chronic Care Model in safety net practices. A list of the measures that will be used to assess each key area of interest, and the source of the measure, is provided in the table below.

All surveys that will be used in the study are already existing and validated survey tools. Key-informant interview protocols were developed by the study PI, Dr. Knox and members of the research team. Dr. Knox has more than 15 years of experience in qualitative research and developing qualitative interview protocols. A more detailed description of each tool is provided in Part A, Section 1.

Variable	Measure	Source
Practice variables
Practice characteristics (co-variate)	Practice profile	Providers/staff/admin
Components of Chronic Care Model (CCM)
Self-management support	ACIC PPC-RS	Providers/staff/admin
Delivery system design	ACIC PPC-RS	Providers/staff/admin
Decision support	ACIC PPC-RS	Providers/staff/admin
Clinical information systems	ACIC PPC-RS	Providers/staff/admin
Process and Clinical indicators
Quality of care (example condition: diabetes)	HbA1c test twice in last 12 months at least 90 days apart Eye exam in last 12 months Foot exam in last 12 months LDL cholesterol in last 12 months Microalbuminuria screening in past 12 months	Registry, electronic health record (HER), or random chart audit
Patient satisfaction and experience
Patient experiences of care and satisfaction	CAHPS-PCA, and CAHPS-Spanish PACIC & PACIC-Spanish	Patients/staff
Organizational capacity for improving medical practice
Priority of change	CPCQ	Providers/staff/admin
Change process capability	CPCQ	Providers/staff/admin
Clinician/staff/administrator satisfaction	Primary Care Staff Satisfaction Survey	Providers/staff/admin
Content and process of intervention
Intensity of intervention (co-variate)	PCEF	Practice coach
Strategies used in intervention (co-variate)	PCEF	Practice coach
Process and content of intervention by site including use of Toolkit content	PCEF Key-informant interview	Practice coach, providers/staff/admin
Barriers and facilitators to implementation of intervention and of CCM and observed impact	PCEF Key-informant interview	Practice coach, providers/staff/admin

B.3. Methods to Maximize Response Rates

This is a multiple replication study that will examine variations in implementation, process and impact of practice coaching across 20 safety net organizations. This is not an experimental study and it is not seeking to develop patient or practice level data that can be generalized to the larger population. The goal of this study is to document site-specific impacts of practice coaching on quality of care indicators, and practice, provider and patient indicators, and implementation of the intervention, and then to evaluate how these factors vary across the 20 replication sites in order to develop “lessons learned” that can be used to inform future practice coaching interventions and development of a research agenda for AHRQ on the process and effectiveness of practice coaching as a practice improvement strategy.

Safety net provider and staff interviews and surveys. Response rates to key-informant interview requests will be maximized through personal invitations, scheduling interview at times that are convenient for the respondents, and offering the option of a telephone interview. Response rates for the survey will be maximized by administering questionnaires during scheduled clinical staff meeting times, during pre-intervention and post-intervention visits by the Practice Coaches, protecting anonymity, and offering the option of both paper and on-line versions of the survey, through repeated invitations to complete the survey, by in-person or telephonic follow-ups with providers and staff who fail to complete the surveys on first and second requests. Based on prior experiences administering satisfaction and other types of surveys to clinic staff, AHRQ has consistently obtained response rates as high as 85%. With 200 potential respondents, this will yield 170 respondents (target accrual is 160) which will provide sufficient power to detect change across the outcomes of intervention within each site, and will provide adequate sample size to identify potential variations in outcomes across sites by site characteristics.

Practice coach interviews. Because these individuals are employed as part of the project we anticipate a 100% response rate from the coaches to the interviews.

Patient survey. Response rates from patients on their satisfaction questionnaire will be maximized by placing staff in the practices who will collect data until target accrual is reached. Many of the larger potential sites for this study routinely see 50 to 100 adult patients per day. The smaller ones may see 25- 50 a day. With a 65% response rate, based on prior experiences with wait room surveys in safety net settings, AHRQ requires 1-3 days per site to meet the target accrual of 50 completed questionnaires per site. We have sufficient staff to collect these surveys and do not anticipate a problem reaching our target accrual over time. Bi-lingual research assistants will available to assist with non-English speaking patients which will also increase response rate.

Chart audit. Project staff will perform chart audits of the medical records of 60 patients with the index condition selected by each practice will be conducted at baseline, at the end of the intervention, and at 3-month follow-up. At each practice, depending on the practice size, several hundred to several thousand patients have chronic health conditions including Type 2 Diabetes, Asthma, COPD, or CVF, likely foci for the practice coaching interventions; and the data that will be used for the chart audits are routinely collected during patient care and are readily accessible as part of patients’ paper base or electronic health records maintained by the practices.

B.4. Tests of Procedures

All survey tools being used in this study have been previously validated, published and used in previous research. The key-informant interview protocol was piloted with 3 CHC staff in July 2009 and information was solicited about comprehensibility and clarity of the tool. Only minor revisions were indicated by the feedback and these changes have been incorporated into the tools. For purposes of formative and qualitative interviewing, piloting with 3 respondents is sufficient to assess the appropriateness of an interview protocol.

B.5. Statistical Consultations

Dr. Pamela M. Diamond, Ph.D., Assistant Professor of Behavioral Sciences and Biostatistics at the University of Texas at Houston School of Public Health was used as the primary consultant for the quantitative analysis plan for this study. Dr. Diamond has been PI on a number of federal grants involving health and public health studies, and has collaborated with the PI for this Task Order for more than 20 years on federally funded research studies.

xv