SUPPORTING STATEMENT
Part A
Field Test of SPPC-II Toolkit and Training Modules during Pilot Phase of SPPC-II Demonstration Project
March 27, 2019
Agency of Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 6
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 6
5. Involvement of Small Entities 6
6. Consequences if Information Collected Less Frequently 6
7. Special Circumstances 7
8. Consultation outside the Agency 7
9. Payments/Gifts to Respondents 7
10. Assurance of Confidentiality 8
11. Questions of a Sensitive Nature 8
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 9
14. Estimates of Annualized Cost to the Government 10
15. Changes in Hour Burden 10
16. Time Schedule, Publication and Analysis Plans 10
17. Exemption for Display of Expiration Date 11
List of Attachments 11
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
Maternal mortality and severe maternal morbidity (SMM) has increased significantly and continuously in the United States (US) over the past 30 years.1 A considerable proportion of these adverse events are attributable to preventable harm and unintended consequences arising from clinical practice and the system of delivering perinatal care.2 To address these alarming trends, AHRQ has developed the Safety Program in Perinatal Care (SPPC).3 During its initial phase (SPPC-I), the program was comprised of three pillars: teamwork and communication, patient safety bundles, and in situ simulations.3 Despite several promising results, the evaluation of SPPC-I revealed considerable hospital attrition due to heavy data burden and competing safety initiatives.4,5 Also, differences in the local adaptation of the SPPC-I patient safety bundles selected by implementation sites thwarted a meaningful cross-site comparison of programmatic impact.4,5
The current, second phase of the program (SPPC-II), focuses on integrating the teamwork and communication pillar into patient safety bundles developed by key professional organizations and implemented in 20+ US states with technical assistance by the Alliance for Innovation on Maternal Health (AIM)6 program and funding from the Health Resources and Services Administration (HRSA). Of note, the model used by AIM to implement these bundles is through statewide perinatal quality collaboratives (PQC) aiming to enroll all birthing hospitals in the state in the PQC.
During the Planning Phase of SPPC-II, the contractor, Johns Hopkins University (JHU), developed the SPPC-II Training Toolkit for two AIM patient safety bundles: obstetric hemorrhage and severe hypertension in pregnancy. The aim of the larger SPPC-II Demonstration Project is to field test, implement and evaluate an integrated AIM-SPPC II program that overlays the SPPC-II Training Toolkit and the AIM patient safety bundles and program infrastructure in two states -- Oklahoma (OK), currently implementing the severe hypertension bundle; and Texas (TX), currently implementing the hemorrhage bundle.
As currently constructed, the SPPC-II Training Toolkit consists of two tiers: a series of brief e-learning modules for frontline staff focusing on practical communication and teamwork tools for the clinical setting (Tier 1), and a workshop for local champions (i.e., Hospital Aim Team Leaders) focusing on patient safety and quality improvement principles specifically for the labor and delivery (L&D) setting (Tier 2). Both will be utilized and assessed during our field test. The target audience for the Tier 1 toolkit are frontline providers who spend the majority of their time working in L&D units, including nurses, midwives, advanced practitioners (e.g. PAs, NPs), and physicians (i.e. housestaff, obstetricians, hospitalists). The materials consist of 8 eModules expounding on practical teamwork and communication tools based on the validated TeamSTEPPS format. In under 15 minutes, each eModule introduces the learner to 1-2 key TeamSTEPPS principles and/or tools and demonstrates their use within the clinical context of the AIM bundle that is being implemented. The target audience for the Tier 2 workshop are local leaders/educators of L&D units (e.g. nurse educator, quality manager, unit nurse/physician director) who can be expected to be champions (i.e., Hospital AIM Team Leaders) for dissemination of QI initiatives like AIM, can facilitate understanding of the program, and lead the implementation of the Tier 1 eModules for the frontline providers in their L&D units. The 1-day in-person workshop is organized into 8 sessions covering the teamwork and communication principles targeted by the SPPC-II program, how they harmonize with the AIM clinical bundles, how to facilitate understanding of Tier 1 materials for frontline staff, and considerations for implementing the SPPC-II program within their L&D unit.
The purpose of this clearance request is to field test and trial the SPPC-II Training Toolkit in a limited number of labor and delivery units, with a focus on identifying facilitators and barriers related to the understanding and use of the SPPC-II Training Toolkit by frontline providers and L&D leaders and the proposed approach of the implementation, evaluation and collection of data in the subsequent implementation and evaluation phase.
Field testing the SPPC-II Training Toolkit among frontline providers and L&D leaders is a credible way to ensure the form and functionality of the SPPC-II Training Toolkit modules are acceptable, feasible, and usable and meet the needs of end users.
The goal of the field test is to assess the feasibility, acceptability, and usability of the SPPC-II Training Toolkit by frontline providers (Tier 1) and L&D leaders (Tier 2).
To achieve this goal, the following activities will be implemented:
Short Questionnaire: a brief electronic questionnaire incorporated at the end of each Tier 1 training eModule will be used to obtain reactions on the form and function of the eModules using a standard 5-point Likert scale (e.g., 1 = strongly disagree, 5 = strongly agree) – specific questions are included in the appendix (Attachment E). Participants in the Tier 2 workshop will be asked to complete the questionnaire at the end of each module in the workshop.
Qualitative Interviews: semi-structured interviews will be conducted individually with a sub-set of frontline providers testing the Tier 1 eModules, trying to reach at 2-3 individuals from each field test site, and with participants in the Tier 2 workshop, ensuring at least 1 participant from each field test site. Questions will be focused on issues like perception of the materials, feasibility of their administration, acceptability and understanding of their content, and usability of their teachings and materials (See Attachment B).
This field test is being conducted by AHRQ through its contractor, Johns Hopkins University, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
The field test will be conducted with participants from four (4) labor and delivery units across the Johns Hopkins Health System (JHHS), with a focus on trying to account for variations in patient populations (e.g. ethnic, socioeconomic, acuity/complexity), provider characteristics (e.g. staffing models, types of providers, ancillary services), and facility characteristics (e.g. size, type, annual admissions) to help simulate the range of L&D units across the US. JHU will ensure an adequate mix of staff from JHHS sites to ensure representation of the different demographics, work settings and practice models. Working with the JHHS Obstetric Clinical Community group, JHU will introduce the SPPC-II project and proposed field test to the leadership of the L&D units. Individual meetings will be held by the JHU project team at each L&D unit with the local nursing and physician leadership to review details of the field test. Email lists for the applicable categories of local frontline staff (i.e. nurses, midwives, advanced practitioners, physicians) working in each participating L&D unit will be obtained. Contact information for appropriate local leaders/educators to participate in a Tier 2 workshop will be secured. Field test participants will complete short questionnaires following the completion of SPPC-II training modules and workshops during the field test and a subset of participants, with oral consent, will complete one-time, voluntary qualitative interviews. The interviews will be audio-recorded, and will take about one hour to complete. The Tier 1 and Tier 2 oral consent form are provided in Attachment C and D, respectively.
The information collected will be used to refine the SPPC-II Training Toolkit and contract deliverables before future implementation and evaluation. Information gained from the interviews and questionnaires will provide insights into end user perceptions, experiences and expectations, and focus attention on areas to be enhanced within the SPPC-II Training Toolkit that can improve the future implementation and evaluation of the SPPC-II Demonstration Project.
Because training of frontline clinical staff will be online using the training e-modules developed by JHU, the Tier 1 eModules will conclude with an electronic questionnaire incorporated at the end of the eModule asking users to provide their reactions to the form and function of the eModules. With the respondent’s oral consent, the qualitative interviews will be audio recorded using a small digital recording device.
There is no other study that has been conducted to field test the SPPC-II Training Toolkit, which was designed to harmonize with the existing AIM bundles for Obstetric Hemorrhage and Severe Hypertension in Pregnancy. This is known from direct communication with AIM program leadership, which would need to be involved in any such research efforts.
The field test of the SPPC-II Demonstration Project does not involve collection of information from small entities.
Tier 1 and Tier 2 self-administered questionnaires are a one-time data collection.
Qualitative, semi-structured interviews with field test participants are a one-time data collection.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
This information collection request is being submitted under AHRQ’s generic pretesting clearance “Questionnaire and Data Collection Testing, Evaluation, and Research for the AHRQ” OMB Control Number 0935-0124 and therefore does not require publication in the Federal Register.
During the Planning Phase of SPPC-II, AHRQ and JHU consulted regularly with AIM program leadership and HRSA to obtain information about the availability of data relevant to this Demonstration Project, the proposed data collection strategies, the frequency of data collection, specific data elements to be obtained through the proposed data collection. Of note, the AIM program is a subcontractor of JHU for the Demonstration Project of SPPC-II, a relationship that involves weekly meetings between AIM and JHU, and monthly meetings with all AIM, JHU, AHRQ and, HRSA representatives. Moreover, throughout the Demonstration Project, JHU has a contractual obligation to convene four meetings of a Stakeholder Panel comprised of AHRQ, HRSA, JHU, AIM, OK-PQC, TX-PQC representatives as well as six external panelists with expertise in obstetrics, quality improvement, data and evaluation methods (Attachment F). These panelists will provide guidance on all phases of the Demonstration Project, including field test activities.
9. Payments/Gifts to Respondents
Field test participants who complete the qualitative interviews will be offered a $10 Amazon gift card as a token of appreciation for their time and participation in the field test. This decision is based on experience with other AHRQ data collection; findings from the published literature; and the need to equally compensate respondents irrespective of the manner in which they complete the surveys (online or on paper) and qualitative interviews.
For AHRQ’s Medical Expenditure Panel Survey (MEPS), a monetary gift has been offered to respondents since 1996; the current amount is $50 for the five times when contacted for the surveys – this corresponds to $10 per time spent on the MEPS at each time.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
Except for hospital names and clinical staff names and email addresses, no other personally identifiable information (PII) will be collected during the field test of the SPPC-II Demonstration Project. Names of hospitals and clinical staff will be recorded, but replaced with anonymized hospital IDs and staff IDs; email addresses for staff are needed to share information about access to training e-modules and survey questionnaires if these are to be completed online. No hospital or staff names or email addresses will be included in any reports, presentations, or other publications emerging from this project. Only aggregated, de-identified results will be displayed in any reports.
All project data, including the files linking hospital and staff names and study IDs, will be stored in a secure, password-protected electronic shared folder (JHUBox), which is widely and routinely used by JHU faculty and staff for government-funded projects given the secure access it provides. A specific folder for the SPCC-II Demonstration Project will be developed to which only project staff who need to use the data or have managerial roles will have access. Only JHU team investigators will have access to project data on the secure JHUBox folder. Moreover, access to the JHUBox SPCC-II folder with project data will only be granted to project staff who need to use the data or have managerial roles. All project staff must have an official JHU email address to be granted access to the secure JHUBox folder. Protocols for data collection, storage, and analyses will be approved by the JHU Institutional Review Board and followed accordingly. All Demonstration Project data will be destroyed 3 years following completion of the project.
Confidentiality statements will be printed on respondent materials (e.g. questionnaires) using the following text: “Your responses will be kept confidential to the extent permitted by law, including AHRQ’s confidentiality statute, 42 USC 299c-3(c).”
Also, to describe the AHRQ statute, we also added the following sentence: “That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied unless you consent to the use of the information for another purpose.”
Exhibit 1 shows the estimated annualized burden hours for respondents’ time to participate in the questionnaires and qualitative interviews of the field test. The questionnaire will be administered to about 16 persons and take 5 minutes to complete. One qualitative interview, lasting one hour in length, will be conducted with a total of 16 respondents. Exhibit 2 shows the estimated annualized cost burden associated with respondents’ time to participate in this research. The total cost burden is estimated to be $1,127. The estimates for the hourly wage of adult participants are based on the national median hourly estimate for all occupations reported in the Bureau of Labor Statistics’ Occupational Employment Statistics, May 2016 National Occupational Employment and Wage Estimates United States. (See http://www.bls.gov/oes/current/oes_nat.htm.)
Exhibit 1: Estimated annualized burden hours
Activity |
Number of respondents |
Number of responses per respondent |
Hours per response* |
Total burden hours |
Short Questionnaire |
16 |
1 |
5/60 |
1 |
Qualitative Interviews |
16 |
1 |
1 |
16 |
Total |
32 |
n/a |
n/a |
17 |
* Time is an average response per respondent
Exhibit 2: Estimated annualized cost burden
Activity |
Number of respondents |
Total burden hours |
Average hourly wage rate* |
Total cost burden |
Short Questionnaire |
16 |
1 |
$66.32 |
$66 |
Qualitative Interviews |
16 |
16 |
$66.32 |
$1,061 |
Total |
32 |
17 |
n/a |
$1,127 |
*Based upon the mean hourly wage for Compensation, Benefits, and Job Analysis Specialists occupation code 00-0000, at https://www.bls.gov/oes/current/oes_nat.htm (U.S. Department of Labor, Bureau of Labor Statistics.)
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. The only cost to the respondent will be that associated with their time to respond to the information collection as shown in Exhibits 1 and 2.
14. Estimates of Annualized Cost to the Government
Exhibit 3 shows the estimated total cost for the field test, which will last for 2 months. The total cost for the field test is approximately $14,898.
Exhibit 3. Estimated Total and Annualized Cost
Cost Component |
Total Cost |
Annualized Cost |
Project Development |
$5,557 |
$5,557 |
Data Collection Activities |
$6,947 |
$6,947 |
Analysis and reporting |
$2,364 |
$2,364 |
Total |
$14,898 |
$14,898 |
Exhibit 4: Federal Government Personnel Cost
Personnel |
Staff Count |
|
Hourly Rate |
Estimated Hours |
Cost |
Grade 14 |
2 |
|
$58.57
|
139 |
$8,141 |
Grade 15
|
1 |
|
$66.74 |
17 |
$1,134 |
Total |
3 |
|
|
|
$9,275 |
.
*Based on 2019 OPM Pay Schedule for Washington/DC area: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2019/DCB.pdf
This is a new collection of information.
As soon as OMB approval is received, the field test activities will begin. The estimated time schedule to conduct the field test is shown below:
mid-late April – May 3, 2019: Field test of SPPC-II Toolkit Trainings begin (including short questionnaire)
May 1 – 24, 2019: Qualitative interviews
May 20 – June14, 2019: Data Analysis and Report Preparation
June 17, 2019: Draft Field Test Report submitted, including recommendations for revisions to SPPC-II Training Toolkit, evaluation plan, and implementation plan.
The data management process will begin at the start of data collection so that data analysis can co-occur. Unless a participant declines, the qualitative interviews will be recorded. Interviews will be electronically transcribed by a professional transcription service. Responses from the questionnaires will be collated for collective review. Transcriptions of the qualitative interviews will be reviewed and marginal notes made to identify major themes and sub-themes. After coding all transcripts, the JHU team will re-read the content of all themes and sub-themes to check for inconsistencies, redundancies, or impreciseness. Free text responses from the training e-module and workshop assessments will also be coded into similar themes and sub-themes. The JHU team will hold interpretation and decision sessions to develop responses for identified themes, and formulate an action plan where appropriate to update the SPPC-II toolkit and training materials prior to finalization for project implementation with AIM.
At the conclusion of the field testing of the SPPC-II training and toolkit materials, a report of the learnings will be prepared by the JHU team. This will include: understanding and usability of SPPC-II toolkit by end users; feasibility of utilizing SPPC-II toolkit in the proposed manner, with suggestions for adaptation or alternative methods for dissemination/implementation as indicated by end users; and recommendations for revisions to the proposed SPPC-II toolkit, implementation, and evaluation plans.
No exemption is being requested
Attachment D – Field Test Oral Consent Form – Tier 2
Attachment E – SPPC-II Field Test - Short Questionnaire
Attachment F – List of Stakeholder Panel Members
| File Type | application/msword |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |