SUPPORTING STATEMENT
Part B
The AHRQ Safety Program for Improving Antibiotic Use
Version: May 15, 2017
Agency of Healthcare Research and Quality (AHRQ)
B. Collections of Information Employing Statistical Methods 3
1. Respondent Universe and Sampling Methods 3
2. Information Collection Procedures 7
The data collection planned under this project is part of a comprehensive evaluation strategy to assess the adoption of the Comprehensive Unit-Based Safety Program (CUSP) for antibiotic stewardship (AS) in acute care, long-term care and ambulatory care settings; measure the effectiveness of the interventions in the participating facilities or units; and evaluate the characteristics of teams that are associated with successful implementation and improvements in outcomes. A key component of the AHRQ Safety Program for Improving Antibiotic Use is the recruitment of facilities within sites and delivery systems that will be supportive of and likely to spread the CUSP AS model.
This data collection request covers the activities with the Cohort 1 pilot facilities, Cohort 2 acute care hospitals, Cohort 3 long-term care facilities, and Cohort 4 ambulatory care facilities.
Recruitment Methods:
Cohort 1 (Pilot Facilities)
Facility recruitment for the Cohort 1 pilot period has focused on identifying Integrated Delivery Systems (IDSs) that have indicated a willingness to participate in the project and which meet the following criteria:
Each system should consist of regional clusters of coordinated facilities, each containing at least ten (10) facilities that will participate in the project, with at least two (2) facilities from each of the settings participating from each system: acute care hospitals, long-term care facilities, and ambulatory settings.
The recruited integrated systems should represent some geographic diversity, covering at least two (2) HHS regions.
The AHRQ Safety Program for Improving Antibiotic Use has recruited three Integrated Delivery Systems (IDSs) for the pilot period:
Geisinger Health System,
Johns Hopkins Health System (JHHS), and
Carolinas HealthCare System.
All participating IDSs meet the criteria and will commit at least 10 facilities to participate in the project, with at least two facilities from each of the three settings of interest (acute care, long-term care, and ambulatory care) At least one unit from each of the facilities will participate in the project. Exhibit B.1 below outlines some basic information about each IDS.
The 30 sites (i.e. participating units) within the three IDSs participating in the Cohort 1 pilot period of the AHRQ Safety Program for Improving Antibiotic Use include diverse locations with integrated delivery systems and vary based on type of facility, geographic location (both Northern and Southern populations), urbanity, number of beds, number of providers, type of Electronic Health Record (EHR) system, and teaching status. While the three IDSs have been selected to participate in the pilot to meet the needs stated above, AHRQ is open to considering alternative pilot facilities as recommended by OMB.
Exhibit B.1: Integrated Delivery Systems Participating in the Pilot Period
IDSs |
HHS Region |
Urbanity |
Sites |
Existing Antibiotic Stewardship Program |
Number of Sites Participating From Each Setting |
|||||||
|
|
LTC |
Amb |
Acute |
||||||||
Carolinas HealthCare System |
4 |
Primarily Urban |
Over 900 care locations, including 40 hospitals, 24 urgent care centers, 5 LTC, and 600 ambulatory care sites |
Y |
TBD |
TBD |
TBD |
|
||||
Geisinger Health System |
3 |
Primarily Rural |
14 hospitals, 7 LTC, and 40 ambulatory care sites |
Y |
TBD
|
TBD |
TBD |
|
||||
Johns Hopkins Health System |
3 |
Urban |
6 academic and community hospitals, 4 surgery centers, 10 LTC (Lorien Health Systems), 39 primary and specialty care sites |
Y |
3 |
3 |
4 |
|
||||
Cohort 2 (Acute Care Hospitals), Cohort 3 (Long-Term Care Facilities) and Cohort 4 (Ambulatory Care Facilities)
Targets. For Cohorts 2, 3, and 4, the project will recruit 250-500 acute care hospitals (at least 25 from each of the 10 HHS Geographic Regions); 250-500 long-term care facilities (at least 25 from each HHS region); and 250-500 ambulatory care facilities (at least 25 from each HHS region).
Facility Characteristics. To meet the recruitment targets the project will cast a wide net and ensure that eligibility criteria garners broader participation rather than unnecessarily excluding potential sites. In addition to ensuring coverage of all 10 HHS regions, the project will seek to ensure diversity of recruited facilities. The characteristics by setting are highlighted below:
Hospitals. In Cohort 2, all hospitals will be eligible for participation including federal, short term, and long term acute care hospitals. Other characteristics include urban and rural, academic and non-academic hospitals, small, medium and large hospitals, independent and part of larger health systems.
Long Term Care. For Cohort 3, the project will recruit LTC facilities that are both urban and rural, and vary by facility size, ownership (for-profit, not-for-profit, and government-owned facilities, and chain and independent facilities).
Ambulatory Care. For Cohort 4, the project will recruit ambulatory care facilities that are both urban and rural, small, medium and large practices, and small, independent and hospital owned.
Recruitment Strategy. The project will use a multi-pronged strategy for recruitment. Through a national roster of partners (e.g., patient safety foundations, Hospital Information Networks, QIN-QIOs, other quality-focused organizations, national health associations), the project will ensure coverage of all 10 HHS regions. Exhibit B.2 below details the organizations supporting the recruitment efforts, their respective audiences, target delivery sites, and HHS regions covered for this project.
Exhibit B.2: Cohorts 2, 3, and 4 Recruitment Partners
Organizations |
Dissemination Vehicles/Venues |
Delivery Site |
HHS Regions Covered |
American Academy of Family Physicians (AAFP) |
National Conference, CME Events, Chapter Events, AAFP News |
Ambulatory |
1-10 |
Infectious Diseases Society of America |
ISDA News, ID Week (Oct 26-30, 2016) |
Hospital, LTC, Ambulatory |
1-10 |
Joint Commission on Accreditation of Healthcare Organizations (JCT) |
CJCP Education (education modules), Conferences & Seminars, Webinars, eCourses, JCR Quality and Safety Network |
Hospital, |
1-10 |
Medical Group Management Association (MGMA) |
Annual Conference, Webinars, Online Courses, Educational Conferences |
Ambulatory |
1-10 |
National Patient Safety Foundation (NPSF) |
NPSF Annual Patient Safety Congress, NPSF Lucian Leape , Patient Safety Week, Patient Safety Blog, NPSF News, Webcasts, PS Blog and President's Corner, |
Hospital, LTC, Ambulatory |
1-10
|
SHEA |
SHEA Spring Conference, ID Week, SHEA News, SHEA-Medscape |
Hospital, LTC, Ambulatory |
1- 10
|
Society for Infectious Diseases Pharmacists |
Annual Meeting |
Hospital, LTC, Ambulatory |
1-10 |
American Medical Association (AMA) |
JAMA, National Advocacy Conference, AMA Education Center, AMA Annual Meeting |
Hospital, LTC, Ambulatory |
1-10 |
Vizient, Inc. (VHA-UHC) |
Vizient Clinical Connections Summit, Vizient Webinars |
Hospital, Ambulatory |
1-10 |
Pediatric Infectious Diseases Society |
International Pediatric Antimicrobial Stewardship Conference, ID Week, Hot Topics in Infection & Immunity in Children, St. Jude/PIDS Transplant ID Symposium & Infectious Disease Research Conference |
Hospital, Ambulatory |
1-10 |
Genesis HealthCare |
Rehab.com, GRS News |
LTC |
1-10 |
Wisconsin Hospital Association |
Physician Leadership Conference, Rural Health Conference, Seminars, Webinars (at least monthly), |
Hospital |
5 |
Michigan Hospital Association Keystone Center |
MHA Patient Safety & Quality Symposium, Corporate Sponsorship Program, MHA Major Meetings, Healthcare Leadership Academy |
Hospital |
5 |
Premier, Inc. |
Health Care Summit, Premier Annual Breakthroughs Conference and Exhibition, HFMA ANI Conference |
Hospital |
1-10 |
Additional recruitment activities will include the following:
Work with the Technical Expert Panel (TEP) members from delivery organizations to gauge their interest in participating in this program.
Leverage the AHRQ patient safety listserv of over 55,000 contacts, the Armstrong Institute (AI) listserv of 8,000 contacts that have participated in prior patient safety interventions with AI/JHU, the National Patient Safety Foundation listserv, and the Infectious Disease Society of America listserv.
Use press releases and early publications from the Cohort 1 pilot to generate national interest in the AHRQ Safety Program for Improving Antibiotic Use. The project will develop press releases on the program to disseminate in relevant email mailing lists and websites.
Use free social media to recruit acute care hospitals, including periodically posting on the Johns Hopkins School of Medicine’s (JHM) Facebook (522k likes), Twitter (430k followers), and LinkedIn (18k followers).
Host informational webinars to garner interest in the project.
Develop a website for the AHRQ Safety Program for Improving Antibiotic Use that will assist with recruitment.
Collaborate with federal partners including CDC and CMS to ensure synergistic efforts across related projects.
Respondent Selection and Sample Sizes
Cohort 1 will perform as a pilot project to test the antibiotic stewardship course design and intervention strategies. For the unit-level measures, including Structural Assessment, Team Antibiotic Review Form, and EHR data, the project will sample all the participating units. For the individual-level measure for the AHRQ Surveys on Patient Safety Culture, the project will sample all the eligible staff within each participating unit.
For the semi-structured qualitative interviews with the Cohort 1 IDS staff, the project will aim to recruit respondents from two acute care facilities, two long-term care facilities and two ambulatory care sites in each of the three IDSs for a total of 18 sites. At each institution, the project aims to sample 10 frontline health care workers and five Stewardship Leaders/Master Trainers, for a total of 270 respondents.
For Cohort 2 (acute care), Cohort 3 (long-term care), and Cohort 4 (ambulatory care), the AHRQ Safety Program for Improving Antibiotic Use will collect data from all participating units when possible.
Each facility lead will conduct the structural assessment at pre- and post- intervention in order to track changes in facility infrastructure and policy related to antibiotic use (i.e. per the survey design, the project will collect site-level data twice).
For the unit-level measures, including the Team Antibiotic Review Form and EHR data, the project will sample all the participating units.
For the individual-level measure for the AHRQ Surveys on Patient Safety Culture, the project will sample all the eligible staff within each participating unit as instructed by the AHRQ survey user guide; because unit size varies, the project estimates the average number of respondents to be 25 for each unit.
Cohorts 2, 3 and 4 will include respondents at each participating unit as detailed in Exhibit B.3 below.
Exhibit B.3: Cohort 2, 3, and 4: Respondent Universe
Setting |
Respondent Universe |
Additional Information |
Acute care setting |
There will be 3 levels of participants: 1. Antibiotic Stewardship Team 2. Unit champions (e.g., physician, pharmacist, and nurse lead) 3. Frontline staff |
|
LTC setting |
1. Antibiotic Stewardship Team 2. Team leaders 3. Frontline staff |
For LTC participating facilities without an existing stewardship team, team leaders will be trained to understand the principles of antibiotic stewardship. |
Ambulatory care setting |
The project will ask each ambulatory care participating clinic to identify a “master trainer*.” |
*A physician, nurse practitioner, and/or physician assistant. Intended for primary care providers or urgent care clinics All healthcare workers at each participating clinic will be encouraged to participate. |
We anticipate a 90% cooperation rate from the participating facilities to the Structural Assessments, submission of EHR data (e.g., antibiotic use, C. difficile rates, urine cultures, etc.), response to qualitative interviews, and completion of Team Antibiotic Review forms.
Completion rates for the AHRQ Surveys on Patient Safety Culture, based on previous CUSP interventions focusing on VAP and CLABSI), are anticipated to range between 30-50%. For those interventions, the hospital or unit response rates for the AHRQ Patient Safety surveys varied from baseline to follow-up and from project to project. The response rates averaged 55% at baseline and between 16-28% at follow-up.
Primary Data Collection
The project will collect a range of data to contribute substantively to the evaluation and facilitate analysis of progress over time. Primary data collection includes surveys, assessment tools, and qualitative discussions (see Exhibit B.4 below). The project will analyze EHR data submitted from individual sites as secondary data sources.
Exhibit B.4: Summary of Primary Data Collection Activities
Primary Data Collection Sources |
Data Collection Tools |
Target population |
Data collection frequency |
Surveys |
AHRQ Patient Safety Culture Change surveys: HSOPS, NHSOPS, MOSOPS
|
All staff at the participating sites |
|
Assessment Tools |
Structural Assessment Tool (different tools for acute care, long term care, and ambulatory care settings)
|
Unit leader at the participating sites |
|
Antibiotic Appropriateness Assessment Data |
Team Antibiotic Review Form (assessment by Stewardship Team1)
|
Patients who are prescribed antibiotics at the participating sites (data will be collected by Stewardship Team and will not contain protected health information) |
|
Qualitative, In-Person Interviews (Cohort 1 only) |
A semi-structured discussion guide modified for the different stakeholder groups (frontline staff, unit champions, and the Stewardship Team) and three different health care settings
|
|
|
Surveys. The project will administer the AHRQ Patient Safety Culture Change survey to all AHRQ Safety Program for Improving Antibiotic Use participating staff at the beginning and end of the intervention. This survey has three versions, adapted for the three types of health care settings where the AHRQ Safety Program for Improving Antibiotic Use will be implemented:
The Hospital Survey on Patient Safety (HSOPS) will be utilized to evaluate safety culture for acute care hospitals (see Attachment F).
The Nursing Home Survey on Patient Safety (NHSOPS) will be administered in long term care (see Attachment G).
The Medical Office Survey on Patient Safety (MOSOPS) will be administered in ambulatory care centers (see Attachment H).
Each survey asks questions about patient safety issues, medical error, and event reporting in the respective setting. For patient safety culture surveys, the project will reach out to all eligible providers and staff in order to collect data according to AHRQ’s User Guide. All staff on the unit that is implementing the AHRQ Safety Program for Improving Antibiotic Use will be asked to complete the survey. As unit size varies, the project estimates the average number of respondents to be 25 for each unit. Participating staff should have enough knowledge about the day-to-day activities in the unit and interact regularly with other staff working in the unit in order to provide informed answers. These staff members, who spend all or most of their time at work within the unit, may include the following:
Staff who have direct contact or interaction with patients;
Staff who may not have direct contact or interaction with patients but whose work directly affects patient care;
Assessment Tools
The project will mainly collect primary data by administering assessment tools to staff members involved with the AHRQ Safety Program for Improving Antibiotic Use at each IDS:
Structural Assessment. The project will administer the Structural Assessment tool to the Stewardship Team in all three settings at baseline (pre-intervention) and at the end of the intervention period (see Attachment A through C for assessment tools for each of the three settings). This online assessment tool will ask the Stewardship Team lead five to seven questions to understand facility infrastructure and capacity to carry out the AHRQ Safety Program for Improving Antibiotic Use.
Team Antibiotic Review Form. Additionally, the Stewardship Team will conduct a monthly assessment to achieve a better understanding of the use of antibiotics being prescribed in acute care settings (see Attachment D for the antibiotic use review form) and in long-term care settings. Every four weeks, the Stewardship Team will review at least 10 patients’ medical records and fill out the appropriateness assessment tool to see if the CUSP “four moments of antibiotic prescribing” were considered.
Secondary Data Collection
EHR Data Extracts. The AHRQ Safety Program for Improving Antibiotic Use project, upon review and approval from our Technical Expert Panel (TEP), identified the following unit-level antibiotic usage and clinical measures related to antibiotic stewardship for each type of setting, which will be used to evaluate the effectiveness of the program. To be specific, the project will use these measures to answer 1) if the AHRQ Safety Program for Improving Antibiotic Use is effective to improve antibiotic use, and 2) the extent of the improvement. The TEP identified these measures as appropriate AS measures or these measures have been used in public reporting to monitor antibiotic use (e.g., CDC NHSN AUR Module2).
Every month between baseline (the three month period prior to the beginning of intervention) and the end of the intervention (after the nine month intervention period), each onsite data coordinator (with assistance from local IT) will extract the measures via their EHR system, and submit the data to the project evaluation team. The data will include the rates of antibiotic use by class and unit; CDI lab events for acute care hospitals; rates of antibiotic use by class, rates of starts by class, and CDI events for long-term care facilities; and ICD-10 codes for ambulatory care facilities. Attachment K contains the clinical measure template and Attachment L contains the instructions for conducting the data extract.
Exhibit B.5 describes the related clinical measures for the AHRQ Safety Program for Improving Antibiotic Use (including the appropriateness of antibiotic use which will be collected using the Team Antibiotic Review Forms described above), by setting.
Exhibit B.5: Clinical Measures for the AHRQ Safety Program for Improving Antibiotic Use
Setting |
Clinical measures |
Description |
Acute care setting |
Days of antibiotic therapy (DOTs) per 1,000 days present |
A day of antibiotic therapy is defined by any amount of an antibiotic administered to a patient on a single calendar day. Each drug is counted independently. Days present are defined as the aggregate number of days patients were admitted to that hospital unit. |
Clostridium difficile laboratory event episodes per 10,000 patient days by unit (quarterly measure)
|
As prior antibiotic use is the risk factor with the greatest association for C. difficile infections, C. difficile rates will be evaluated on a quarterly basis during the base and intervention period. These data are currently reportable to CDC's National Healthcare Safety Network on a quarterly basis by acute care facilities in the United States. C. difficile laboratory event is defined as the number of all non-duplicate c. difficile toxin-positive laboratory results. Patient-days are defined as the aggregate number of days patients were admitted to that hospital unit. |
|
Appropriate use of antibiotics (through primary data collection; only during intervention period) |
The Stewardship Team will review antibiotic use with the frontline team on at least 10 patients per month. The Stewardship Team will fill out the Team Antibiotic Review Form (Appendix B) which will not contain any PHI. The team will utilize the tool to assess if the four moments in antibiotic prescribing were considered. |
|
Long term care setting |
Days of antibiotic therapy (DOTs) per 1,000 resident-days |
A day of antibiotic therapy is defined by any amount of an antibiotic administered to a patient on a single calendar day. Each drug is counted independently. Patient-days are defined as the aggregate number of days that patients resided in the long-term care setting. |
Rate of new antibiotic starts initiated in nursing home per 1,000 patient-days |
Antibiotic starts is defined as prescriptions written after the resident has been admitted to the long-term care setting. Patient-days are defined as the aggregate number of days patients resided in the long-term care setting. |
|
Clostridium difficile laboratory event episodes per 10,000 resident days by unit (quarterly measure)
|
As prior antibiotic use is the risk factor with the greatest association for C. difficile infections, C. difficile rates will be evaluated on a quarterly basis during the base and intervention period. C. difficile laboratory event is defined as the number of all non-duplicate c. difficile toxin-positive laboratory results. Patient-days are defined as the aggregate number of days patients resided in the long-term care setting. |
|
Number of urine cultures obtained per 1,000 resident-days |
Urine culture collection is defined as the number of urine cultures obtained. Patient-days are defined as the aggregate number of days patients resided in the long-term care setting. |
|
Ambulatory care setting |
Antibiotic therapy prescription per 100 patient visits |
An antibiotic therapy prescription is defined by any amount of an antibiotic administered to a patient on a single calendar day. Each drug is counted independently. Patient-visits are defined as the aggregate number of visits made by any patient to an ambulatory care setting. |
Proportion of antibiotic use by non-antibiotic appropriate respiratory condition |
Proportion of patients with ICD-10 codes for non-antibiotic-appropriate respiratory conditions (non-specific URIs, acute bronchitis, influenza) prescribed antibiotic. |
Evaluation Design
The pre-intervention baseline will be three months, and the during/post-intervention period will be nine months. The AHRQ Safety Program for Improving Antibiotic Use proposes a pre-post evaluation design, comparing the Structural Assessments and AHRQ Surveys on Patient Safety Culture data from each of the three facility settings to themselves from pre-intervention/baseline to post-intervention/end of intervention. The project will examine the change from baseline to post-intervention and investigate the impacts of other factors on those changes. For data elements that are collected monthly (i.e. Team Antibiotic Review Forms and EHR data), the project will apply an interrupted time series (ITS) analysis to investigate the performance of antibiotic usage and clinical outcomes over time and the impact from the AHRQ Safety Program for Improving Antibiotic Use, because the scope of the project neither includes selection of external comparison sites nor access to EHR data from non-participating sites.
Cohort 1 (Pilot Facilities)
The Cohort 1 pilot implementation data collection is designed to be powered at 80% to detect a decrease3 from baseline to post-intervention period4 for: 1) 61 days of antibiotic therapy (DOT) per 1,000 patient-days in acute care settings,5 2) 24 DOT per 1,000 patient-days in long-term settings,6 and 3) 2.9 antibiotic prescriptions per 100 patient-visits in ambulatory care settings.7 The following assumptions were used in the power calculations: 1) 1-sided significant level of 0.058, 2) ten units from each type of settings, and 3) within-unit correlation of 0.49.
Exhibit B.6 below provides the power and the corresponding mean difference that the project will be able to detect from the baseline to post-intervention period, given different levels of significance and within-unit correlations based on the primary outcomes for each type of settings. Please note that we have included these illustrative power calculations in the spirit of transparency. The Pilot Period is intended to help test instrumentation and feasibility of our data collection schema, which will inform our Option Period data collection/analysis design, for which we have developed pro forma power calculations.
Exhibit B.6: Power Analysis for Cohort 1 (n=10, for each setting)
Correlation within unit |
1-sided significance of 0.05 |
2-sided significance of 0.05 |
||||||
Power |
Detectable Difference |
Power |
Detectable Difference |
|||||
DOT per 1,000 patient-days for acute care setting |
DOT per 1,000 patient-days for LTC setting |
Number of antibiotic prescription per 100 patient visits for ambulatory care setting |
DOT per 1,000 patient-days for acute care setting |
DOT per 1,000 patient-days for LTC setting |
Number of antibiotic prescription per 100 patient visits for ambulatory care setting |
|||
0.2 |
60% |
53.7 |
21.5 |
2.6 |
60% |
62.6 |
25.0 |
3.0 |
70% |
61.4 |
24.5 |
2.9 |
70% |
70.3 |
28.1 |
3.4 |
|
80% |
70.3 |
28.1 |
3.4 |
80% |
79.2 |
31.7 |
3.8 |
|
90% |
82.8 |
33.1 |
4.0 |
90% |
91.7 |
36.7 |
4.4 |
|
0.4 |
60% |
46.5 |
18.6 |
2.2 |
60% |
54.2 |
21.7 |
2.6 |
70% |
53.1 |
21.3 |
2.6 |
70% |
60.9 |
24.3 |
2.9 |
|
80% |
60.9 |
24.4 |
2.9 |
80% |
68.6 |
27.4 |
3.3 |
|
90% |
71.7 |
28.7 |
3.4 |
90% |
79.4 |
31.8 |
3.8 |
|
0.6 |
60% |
38.0 |
15.2 |
1.8 |
60% |
44.3 |
17.7 |
2.1 |
70% |
43.4 |
17.4 |
2.1 |
70% |
49.7 |
19.9 |
2.4 |
|
80% |
49.7 |
19.9 |
2.4 |
80% |
56.0 |
22.4 |
2.7 |
|
90% |
58.5 |
23.4 |
2.8 |
90% |
64.8 |
25.9 |
3.1 |
|
Cohort 2 (Acute Care Hospitals), Cohort 3 (Long-Term Care Facilities) and Cohort 4 (Ambulatory Care Facilities)
The Cohort 2, 3, and 4 implementation data collection is designed to be powered at 80% to detect a decrease from baseline to post-intervention period for: 1) 12.8 DOT per 1,000 patient-days in acute care settings, 2) 5.1 DOT per 1,000 patient-days in LTC settings, and 3) 0.6 antibiotic prescriptions per 100 patient-visits in ambulatory care settings. The following assumptions were used in the power calculations: 1) 1-sided significant level of 0.05, 2) 250 units from each type of settings, 3) within-unit correlation of 0.4, and 4) response rate of 90%.
Exhibit B.7 below provides the power and the corresponding effect size the project will be able to detect from baseline to post-intervention period, given different levels of significance and within-unit correlations based on the primary outcomes for each type of settings.
Exhibit B.7: Power Analysis for Cohorts 2, 3, and 4 (n=250*0.9, for one setting*)
Correlation within unit |
1-sided significance of 0.05 |
2-sided significance of 0.05 |
||||||
Power |
Detectable Difference |
Power |
Detectable Difference |
|||||
DOT per 1,000 patient-days for acute care setting |
DOT per 1,000 patient-days for LTC setting |
Number of antibiotic prescription per 100 patient visits for ambulatory care setting |
DOT per 1,000 patient-days for acute care setting |
DOT per 1,000 patient-days for LTC setting |
Number of antibiotic prescription per 100 patient visits for ambulatory care setting |
|||
0.2 |
60% |
11.3 |
4.5 |
0.5 |
60% |
13.2 |
5.3 |
0.6 |
70% |
12.9 |
5.2 |
0.6 |
70% |
14.8 |
5.9 |
0.7 |
|
80% |
14.8 |
5.9 |
0.7 |
80% |
16.7 |
6.7 |
0.8 |
|
90% |
17.4 |
7.0 |
0.8 |
90% |
19.3 |
7.7 |
0.9 |
|
0.4 |
60% |
9.8 |
3.9 |
0.5 |
60% |
11.4 |
4.6 |
0.5 |
70% |
11.2 |
4.5 |
0.5 |
70% |
12.8 |
5.1 |
0.6 |
|
80% |
12.8 |
5.1 |
0.6 |
80% |
14.5 |
5.8 |
0.7 |
|
90% |
15.1 |
6.0 |
0.7 |
90% |
16.7 |
6.7 |
0.8 |
|
0.6 |
60% |
8.0 |
3.2 |
0.4 |
60% |
9.3 |
3.7 |
0.4 |
70% |
9.1 |
3.7 |
0.4 |
70% |
10.5 |
4.2 |
0.5 |
|
80% |
10.5 |
4.2 |
0.5 |
80% |
11.8 |
4.7 |
0.6 |
|
90% |
12.3 |
4.9 |
0.6 |
90% |
13.7 |
5.5 |
0.7 |
|
* For Cohorts 2, 3, and 4, the project plans to recruit 250 to 500 facilities for each type of setting (acute care, ambulatory care, LTC). The power analysis is based on the lower end of the projections (250 facilities per type). The analysis will be able to detect smaller change as the sample size increases.
Qualitative Analysis (Cohort 1 only). The project will also conduct semi-structured qualitative interviews during Cohort 1 to assess the factors, including site-specific social norms and prescribing behavior, which facilitate/hinder implementation of the intervention. These one-on-one in-depth discussions will occur during three one-week site visits, one at each of the IDS locations, with stewardship champions/organizational leaders and frontline staff in the units where the program is being implemented. Stewardship champions/organizational leaders will include the individuals who are charged with implementing the AHRQ Safety Program for Improving Antibiotic Use at each site as well as other facility managers or leaders as deemed relevant. Frontline healthcare workers will include physicians, pharmacists, nurse practitioners, physician assistants, nurses and others deemed relevant to prescribing by setting (e.g., we will sample more nurses in the LTC setting than in other settings). These site visits will occur during the three-month baseline period and at the end of the Cohort 1 intervention period.
The data collection planned under this project is part of an evaluation to assess the adoption of the CUSP for AS in acute care, long-term care and ambulatory care settings. The results of the evaluation will be used to inform the CUSP for AS program; they will not yield generalizable results or be used for statistical estimation purposes. The project will recruit facilities that have indicated a willingness to participate in the project and meet the inclusion criteria, and who will supportive of and likely to spread the CUSP for AS model.
To encourage the participating facilities to complete and submit the Structural Assessments, Team Antibiotic Review Forms, AHRQ Surveys on Patient Safety, and EHR data, the project will implement the following strategies:
Ensure team understands the value of data collection for their ongoing efforts to sustain their antibiotic stewardship efforts (i.e., share data reports and performance results with unit staff).
Offer technical assistance webinars and onboarding calls (including information related to submitting data) to participating sites
Prompt the on-site data coordinator and unit/facility leader for feedback from each participating unit regarding the data collection activities and AS-related performance
Work closely with each on-site data coordinator to address any data collection issues and develop site-specific data collection strategies when necessary
Discuss data management and submission methods with the participating sites early in the process
Provide sites flexibility to submit data (e.g., via a central data collection platform, via IDS’s own data collection system, or using pre-coded password-protected spreadsheet)
Provide step-by-step instructions on data sharing via the central data collection platform
Ensure that data collection tools are simple, easy to use, and succinct
Ensure missing or inaccurate data is identified early and that issues are resolved in a systematic, highly-reliable way
Share data collection strategies/best practices to simplify processes through coaching calls
The Cohort 1 pilot period will perform as a pilot project to test the program design and implementation. The AHRQ Safety Program for Improving Antibiotic Use will also conduct semi-structured qualitative interviews during Cohort 1 to assess the factors which facilitated/hindered implementation of the intervention. The information collected will allow the project to:
Gauge the feasibility of collecting the same information when the pilot scales.
Identify areas where staff wanted/needed more training.
Identify the challenges and barriers of implementing the AHRQ Safety Program for Improving Antibiotic Use. Findings will inform the implementation approach for the subsequent cohorts.
The AHRQ Surveys on Patient Safety Culture are publicly available and have been widely used to assess patient safety culture in a variety of health care settings.
Johns Hopkins University and NORC at the University of Chicago will serve as the primary consultants for statistical aspects of the design and analysis of the evaluation data.
Exhibit B.8: List of Statistical Consultants
Name |
Title and Institution |
Telephone Number |
Roy Ahn, ScD, MPH |
Associate Director, NORC |
312/759-4068 |
Pranita Tamma, MD, MHS |
Assistant Professor of Pediatrics in Division of Infectious Diseases and Director of the Pediatric Antimicrobial Stewardship Program, John Hopkins University |
410/614-1492
|
Sara Cosgrove, MD, MS, FSHEA, FIDSA |
Professor of Medicine in the Division of Infectious Diseases and Director of the Department of Antimicrobial Stewardship, John Hopkins University |
443/287-4590 |
Sai Loganathan |
Senior Research Scientist, NORC |
301/634-9346 |
Yue Gao, MPH |
Research Scientist, NORC |
301/634-9440 |
Michael Kohut, PhD |
Qualitative Researcher, John Hopkins University |
410/637-7228 |
The data will be collected by NORC at the University of Chicago.
1 Please note this measure, as a component in antibiotic prescribing appropriateness, will be collected via primary data collection. Other components, such as days of antibiotic therapy days per 1,000 patient days, will be collected via secondary data collection through EHR extraction and are included in the secondary data collection section.
3 Assuming the standard deviation is the same for baseline and post-intervention period. Based on prior studies, we set up the value as 50 for DOT per 1,000 patient-days in acute care setting, 20 for DOT per 1,000 patient-days in LTC setting and 2.4 for number of antibiotic prescriptions per 100 patient-visits in ambulatory care setting.
4 Of note, data will be kept in a longitudinal way to use all 12 repeated measures for each outcome in the analysis. We aim to detect the pre- and post- difference using interrupted time series method, which is a commonly used method in interventions for improving antibiotic use.
5 Morrill HJ, Caffrey AR, Gaitanis MM, LaPlante KL. Impact of a prospective audit and feedback antimicrobial stewardship program at a Veterans Affairs medical center: a six-point assessment. PLoS One. 2016;11(3):e0150795.
6 Daneman N, Bornskill SE, Gruneir A, et al. Variability in antibiotic use across nursing homes and the risks of antibiotic-related adverse outcomes for individual residents. JAMA Intern Med. 2015;175(8):1331-9.
7 Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of inappropriate antibiotic prescriptions among US ambulatory care visits, 2010-2011. JAMA. 2016;315(17):1864-73.
8 We assumed a 1-sided significance level given that we expect the intervention would improve the outcome (very likely to be a positive change only). However, we also presented the result with an assumption of 2-sided significance level as a standard assumption.
9 Campbell MK, Mollison J, Grimshaw JM. Cluster trials in implementation research: estimation of intracluster correlation coefficients and sample size. Stat Med. 2001;20(3):391-9. In this reference, within-patient correlation is estimated as ICC = 0.47 (95% CI 0.29 to 0.65). Based on this reference, we assumed the within-unit correlation as 0.4 and range from 0.2 to 0.6.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | SChowdhury |
| File Modified | 0000-00-00 |
| File Created | 2021-01-16 |