SUPPORTING STATEMENT
Part A
AHRQ Safety Program for Improving Surgical Care and Recovery
Version: 7/20/17
Agency for Healthcare Research and Quality (AHRQ)
Table of contents
A. Justification 3
1. Circumstances that make the collection of information necessary 3
2. Purpose and use of information 4
3. Use of Improved Information Technology 6
4. Efforts to Identify Duplication 6
5. Involvement of Small Entities 6
6. Consequences if Information Collected Less Frequently 6
7. Special Circumstances 7
8. Consultation outside the Agency 7
9. Payments/Gifts to Respondents 8
10. Assurance of Confidentiality 8
11. Questions of a Sensitive Nature 9
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 10
14. Estimates of Annualized Cost to the Government 10
15. Changes in Hour Burden 11
16. Time Schedule, Publication and Analysis Plans 11
17. Exemption for Display of Expiration Date 13
List of Attachments 13
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see https://www.ahrq.gov/policymakers/hrqa99a.html), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions. AHRQ shall promote health care quality improvement by conducting and supporting:
1. research that develops and presents scientific evidence regarding all aspects of health care; and
2. the synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
This is a quality improvement project that aims to provide technical assistance to hospitals to help them implement evidence-based practices to improve outcomes and prevent complications among patients who undergo surgery. Enhanced recovery pathways are a constellation of preoperative, intraoperative, and postoperative practices that decrease complications and accelerate recovery. A number of studies and meta-analyses have demonstrated successful results. In order to facilitate broader adoption of these evidence-based practices among U.S. hospitals, this AHRQ project will adapt the Comprehensive Unit-based Safety Program (CUSP), which has been demonstrated to be an effective approach to reducing other patient harms, to enhanced recovery of surgical patients. The approach uses a combination of clinical and cultural (i.e., technical and adaptive) intervention components. The adaptive elements include promoting leadership and frontline staff engagement, close teamwork among surgeons, anesthesia providers, and nurses, as well as enhancing patient communication and engagement. Interested hospitals will voluntarily participate.
This project has the following goals:
Improve outcomes of surgical patients by disseminating and supporting implementation of evidence-based enhanced recovery practices within the CUSP framework
Develop a bundle of technical and adaptive interventions and associated tools and educational materials to support implementation
Provide technical assistance and training to hospitals for implementing enhanced recovery practices
Assess the adoption and evaluate the effectiveness of the intervention among the participating hospitals
To achieve the goals of this project the following data collections will be implemented:
Hospital staff survey – focused on assessing patient safety culture
Patient survey – focused on assessing patients’ experiences of care
Readiness and Implementation Assessments: Semi-structured qualitative interviews
Site visits
This project is being conducted by AHRQ through its contractor, Johns Hopkins Armstrong Institute for Patient Safety and Quality (JHU), with subcontractors, American College of Surgeons (ACS) and Westat, pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a(a)(1) and (2).
Hospitals will voluntarily participate in the project for their own purpose of improving surgical care. Four surveys will be administered throughout the program to strengthen the knowledge and understanding of best practices for implementation and sustainability of this program and understand its impact on the surgical outcomes of patients. The four surveys include a safety culture survey of hospital staff, a patient experience survey of patients, a readiness assessment, and an implementation assessment. Site visits will also be conducted in a small number of hospitals per cohort to continuously improve the program’s implementation process. Data collection will be done before and after project implementation to provide feedback to participating hospitals to help with their improvement efforts and to compare results before and after project implementation. In addition to burden collection, participating hospitals will routinely share other information. Extracts of their data from an existing ongoing American College of Surgeons registry will be analyzed at the American College of Surgeons. The data extracts will enable assessment of improvement in clinical process and outcome measures to help evaluate effectiveness of the intervention.
Safety culture survey. Hospitals will assess the impact of participation in the project on perioperative safety culture by having their staff complete the safety culture survey at the beginning of the program and the end of the program. Hospitals receive their survey results and then debrief their staff on their safety culture and identify opportunities for further improvement. JHU will provide technical assistance for this effort. Participating hospitals will promote awareness of the survey among their staff, coordinate implementation of the survey, encourage staff to complete the survey and provide staff time to do so, and organize their local debrief of the reports of their hospital’s results. JHU will assist this effort by providing an electronic portal for hospital staff to anonymously complete the survey, and by analyzing the data and sending a report to the hospital. Data will also be analyzed in aggregate across all participating hospitals to evaluate the impact of the overall quality improvement effort on measured safety culture.
Patient experience survey. Hospitals will also assess the impact of participation in the project on the patient’s experience with care. AHRQ intends to assist hospitals in assessing patient experience by adapting the CAHPS (Consumer Assessment of Healthcare Providers and Systems®) Outpatient and Ambulatory Surgery Survey for use in a hospital setting and adding in selected questions adapted from other surveys, including Hospital CAHPS, the CAHPS Surgical Survey, and PROMIS (Patient Reported Outcomes Measurement Information System). The approach minimizes burden on the hospitals but will yield important information that will then be used to further drive improvements in the patient’s experience with the healthcare system. AHRQ staff coordinated with CMS regarding the timing after hospital discharge of the patient experience survey administration to avoid overlap with Hospital CAHPS administration.
A pre-implementation assessment of patient experience will be done with patients before the project is implemented at the hospital. A post-implementation assessment of patient experience will be done after the project is implemented, surveying patients that were treated on the enhanced recovery pathway at participating hospitals.
The survey will be administered by Westat. Hospitals will provide patient contact information to the project team after execution of a data use agreement. This information will be provided to Westat to send the survey to patients on behalf of the hospital. Westat will provide a summative report to each hospital with the hospital’s results to promote additional local quality improvement work.
While the primary purpose of both surveys is the hospital’s quality improvement purpose, the data will also be analyzed in aggregate across all participating hospitals to evaluate the impact of the overall quality improvement effort.
Readiness and Implementation Assessments: Semi-structured qualitative interviews. Semi-structured qualitative interviews will be conducted with key stakeholders at participating hospitals (e.g., project leads, physician project champions, etc.). These include a readiness assessment conducted after a hospital’s enrollment in the project and an implementation assessment conducted after a period of implementation. The readiness assessment will help identify which, if any, technical components of the enhanced surgical care and recovery intervention already exist at the hospital, project management and resources, clinician engagement, leadership engagement and potential barriers and facilitators to implementation. The implementation assessment will evaluate what elements of the enhanced recovery practices have been adopted, resources invested, team participation, major barriers (e.g., medications, equipment, trained personnel), and leadership participation. These assessments will help identify training needs of hospitals and inform the JHU team’s approach. In addition, the results will inform the JHU team’s understanding of local adaptations of the intervention and the degree to which intervention fidelity impacts changes in outcomes.
Site visits. Semi-structured site visits will be conducted at a subset of participating hospitals. Methods for selecting sites are to be determined. Findings will help inform the JHU’s project implementation strategy. Information from these visits will be critical in understanding if and how team and/or leadership issues may affect implementation of enhanced recovery practices, including how this may differ across surgical service lines. Interviews will help uncover misalignments in role clarity, needed time and resources, best practices, and potential enablers of and barriers to enhanced surgical care and recovery implementation. Site visits will be conducted at approximately 4 hospitals per year, and each will be 1 day long. The types of hospital personnel anticipated to be involved in part or all of the site visit include senior leadership, perioperative leadership, and patient safety and quality staff. Participating hospitals will receive a structured debriefing and brief summary report at the end of the one-day visit.
Safety culture survey. The safety culture survey will be administered to hospital staff in an electronic format via the employee email address. The participating hospital will submit a list of all perioperative staff to the project team and the project team will administer the survey and compile the results on behalf of the hospital. This approach was adopted to minimize the burden of data collection at the hospital and staff level.
Patient experience survey. Data collection will not involve the use of information technology. The survey will be administered using a paper survey that is mailed to sampled patients.
Readiness and Implementation Assessments: Semi-structured qualitative interviews. The assessment will be conducted over a telephone with the hospital’s project team.
Site visits. Data collection will not involve the use of information technology. Site visits will be conducted at approximately 4 hospitals per year, and each will be 1 day long.
The proposed data collection by hospitals does not duplicate other efforts.
For all surveys and assessments, it is unlikely that any participating hospitals will be small entities.
Surveys and assessments will be administered as a pre-and post-intervention during each cohort. We have the following processes in place to mitigate loss of participation and increase response rate:
Safety culture survey. The JHU team will provide the hospital’s project leads with training and tools to promote awareness of the survey among their staff, coordinate implementation of the survey, encourage staff to complete the survey and provide staff time to do so. The JHU team will also encourage the project leads to remind staff to check their spam folder for the email to complete the survey. Additionally, the survey platform will be programmed to send a reminder email once per week during the survey administration period to staff who have not completed the survey. Finally, the JHU team will send the hospital’s project leads updates on their response rates to encourage their staff to complete the survey.
Patient experience survey. Westat will send a second survey packet to those sample members who do not respond to the first survey.
Readiness assessment. The readiness assessment is a part of every hospital’s enrollment process into each cohort. The JHU team will send an email (see cover letter) encouraging the hospitals to sign up for their 1-hour phone call. The JHU team will follow up with a friendly reminder to those hospitals who do not sign up. Our availability will be flexible and we will offer additional times throughout the baseline data collection period of each cohort.
Implementation assessment. The implementation assessment is a part of every hospital’s sign out of the program. The JHU team will send an email (see cover letter) encouraging the hospitals to sign up for their 1-hour phone call. The JHU team will follow up with a friendly reminder to those hospitals who do not sign up. Our availability will be flexible and we will offer additional times throughout the post-data collection period of each cohort.
Site visits. The sites visits will occur with 4 hospitals every cohort (with exception of cohort 1). There is no concern for low participation in these site visits.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
8.a. Federal Register Notice
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on May 18th, 2017 on page 22831 for 60 days (see Attachment L). No comments were received.
The JHU project team is consulting with a list of experts from multiple perspectives to weigh in on the evaluation approach of both the safety culture and patient experience. The list of experts represents multiple stakeholder perspectives, including surgeons, anesthesiologists, family practitioners, nurses, certified registered nurse anesthetists, researchers, patients, quality improvement experts, and State Hospital Association leads. The list of experts provides guidance on program messaging, implementation, and evaluation. Attachment K includes the list of experts for this project.
No remuneration is proposed for organizations or individuals participating in all surveys and assessments described. Hospitals will receive separate customized feedback reports of their patient safety culture and patient experience results, with comparisons to the other hospitals, which will assist them in understanding their patient safety culture and patient experience. The readiness, implementation and site visit assessments will allow hospitals to reflect on their strengths and weakness when initiating, implementing and sustaining the project at their site.
Individuals and organizations will be assured of the confidentiality of their replies under Section 944(c) of the Public Health Service Act. 42 U.S.C. 299c-3(c). That law requires that information collected for research conducted or supported by AHRQ that identifies individuals or establishments be used only for the purpose for which it was supplied.
For the safety culture survey, only hospital staff email addresses will be collected and these will be destroyed after the survey is closed. The JHU project team will assist the survey administration efforts by providing an electronic portal for hospital staff to anonymously complete the survey, and by analyzing the data and sending a report to the hospital.
For the patient experience survey, hospitals will be unidentified using a hospital ID that only ACS and Westat will have access to. Identifiers such as patient name, address, surgery date or, hospital discharge date, and hospital name will be submitted to Westat through a secure portal for survey administration purposes. Once survey data collection is complete, the personal identifiers received by Westat will be removed from the data and destroyed.
For the readiness and implementation assessment, responses will be unidentified using a coded hospital ID and recorded in a secured database.
For the site visits, information collected will be used to strengthen our understanding of best practices and barriers to implementation and sustainability.
The information to be collected is not considered a record as defined by the Privacy Act and a System of Record Notice (SORN) is not required for this work. Also, we do not believe there are questions of a particularly sensitive nature included in the surveys and assessments.
Exhibit 1 shows the estimated annualized burden hours for the respondents’ time to participate in this project.
Safety Culture Survey.
A pre-and post-implementation safety culture survey will be administered as a web-based survey to nurses, physicians and other clinical staff. Assuming an average of 100 total staff being surveyed per hospital (50 pre-and 50 post), about 60,000 staff would be surveyed. With a 60% response rate, the safety culture survey will be completed by about 36,000 staff and requires 15 minutes to complete. Annually, 12,000 staff will be surveyed.
Patient Experience Survey
A pre-and post-implementation patient experience survey will be administered by mail to patients discharged from the hospital in the surgical specialties included in the project. Assuming an average of 30 patients being surveyed pre-and post-implementation per hospital (15 pre-and 15 post), about 18,000 patients would be surveyed. With a 30% response rate, the patient experience survey will be completed by about 5,400 patients and requires about 22 minutes to complete. Annually, 1,800 patients will be surveyed.
Readiness and Implementation Assessments
A pre-and post-assessment will be administered as a semi-structured interview with the hospital project leads (e.g. one physician, one nurse). Assuming an average of 2 staff being part of each pre- and post- interview per hospital, about 2,400 staff would be surveyed. With a 90% response rate, the readiness and implementation assessment will be completed by about 2,160 staff and requires 60 minutes to complete. Annually, 720 staff will be surveyed.
Site visits
Four site visits per cohort will be conducted. No site visits will be completed during the first cohort. Assuming an average of 10 staff being a part of each site visit, about 120 staff would be part of the site visit that will take 8 hours to complete. Annually, 40 staff will be surveyed.
Exhibit 1 shows estimated annualized burden hours, and Exhibit 2 shows the estimated annualized cost burden associated with the respondents’ time to participate in this project. The total cost burden is estimated to be $299,383 annually.
Estimated Annual Respondent Burden
Exhibit 1. Estimated annualized burden hours
Form Name |
Number of Respondents |
Number of responses per respondent |
Hours per response |
Total Burden hours |
Safety culture survey |
12,000 |
1 |
0.25 |
3,000 |
Patient experience survey |
1,800 |
1 |
0.37 |
666 |
Readiness and Implementation assessment |
720 |
1 |
1 |
720 |
Site visits |
40 |
1 |
8 |
320 |
Total |
14,560 |
N/A |
N/A |
4,706 |
Exhibit 2. Estimated annualized cost burden
Form Name |
Number of Respondents |
Total Burden hours |
Average Hourly Wage Rate* |
Total Cost Burden |
Safety culture survey |
6,000 |
1,500 |
$101.04a |
$151,560 |
Safety culture survey |
6,000 |
1,500 |
$34.70b |
$52,050 |
Patient experience survey |
1,800 |
666 |
$ $23.86d |
$15,891 |
Readiness and Implementation assessment |
360 |
360 |
$101.04a |
$36,374 |
Readiness and Implementation assessment |
360 |
360 |
$52.58c |
$18,929 |
Site visits |
20 |
160 |
$101.04a |
$16,166 |
Site Visits |
20 |
160 |
$52.58c |
$8,413 |
Total |
14,560 |
4,706 |
N/A |
$299,383 |
National Compensation Survey: Occupational wages in the United States May 2016 “U.S. Department of Labor, Bureau of Labor Statistics:” http://www.bls.gov/oes/current/oes_stru.htm
a Based on the mean wages for 29-1060 Physicians and Surgeons
b Based on the mean wages for 29-1141 Registered Nurse
c Based on the mean wages for 11-9111 Medical and Health Services Managers
d Based on the mean wages for 00-0000 All Occupations
13. Estimates of Annualized Respondent Capital and Maintenance Costs
There are no direct costs to respondents other than their time to participate in the study.
Exhibit 3a. Estimated Total and Annualized Cost
Cost Component |
Safety Culture total cost |
Patient Experience total cost |
Readiness and Implementation assessment total cost |
Site visit total cost |
Annualized Cost |
Project Development (Task 1) |
$12,000 |
- |
$362 |
$362 |
$12,724 |
Data Collection Activities (Task 3) |
$14,000 |
$113,974 |
$18,854 |
$5,492 |
$152,320 |
Data Processing and Analysis (Task 5) |
$10,000 |
$74,264 |
$1,632 |
$453 |
$86,349 |
Publication of Results |
$2,000 |
- |
$543 |
NA |
$2,543 |
Project Management (Task 6) |
$2,000 |
$52,660 |
$1,401 |
$700 |
$56,761 |
Overhead |
NA |
NA |
NA |
NA |
NA |
Total |
$40,000 |
$240,898 |
$22,792 |
$7,007 |
$310,697 |
Exhibit 3b. Federal Government Personnel Cost
Activity |
Federal Personnel |
Hourly Rate |
Estimated Hours |
Cost |
Data Collection Oversight |
Medical Officer |
$77.58 |
18 |
$1396.44 |
Review of Results |
Medical Officer |
$77.58 |
20 |
$1551.60 |
Total |
$2948.04 |
|||
Annual salaries based on 2017 OPM Pay Schedule for Washington/DC area: https://www.opm.gov/policy-data-oversight/pay-leave/salaries-wages/salary-tables/pdf/2017/DCB_h.pdf
15. Changes in Hour Burden
This is a new collection of information.
As soon as OMB approval is received, data collection activities will begin, starting with cohort 1. The estimated time schedule to conduct these activities is shown below:
Exhibit 4a. Schedule for program’s cohorts and data collection
Cohort |
Start of 12 month period* |
Safety Culture |
Patient experience |
Readiness and implementation assessment |
Site visits |
|||
|
|
Pre *One month after start date to ensure training and communication of the survey to staff |
Post |
Pre |
Post |
Pre |
Post |
Time period |
1 |
TBD |
M 2 |
M 12 |
M 1-3 |
M 13-15 |
M 1-2 |
M 12-13 |
NA
|
2 |
1/1/18 |
M 2 |
M 12 |
M 1-3 |
M 13-15 |
M 1-2 |
M 12-13 |
Varied times |
3 |
1/1/19 |
M 2 |
M 12 |
M 1-3 |
M 13-15 |
M 1-2 |
M 12-13 |
Varied times |
4 |
1/1/19 |
M 2 |
M 12 |
M 1-3 |
M 13-15 |
M 1-2 |
M 12-13 |
Varied times |
* M 1–12 indicates months 1- 12 after start of cohort
Exhibit 4b. Data analysis period for program’s cohorts and data collection
Cohort |
Start of 12 month period* |
Analysis period for all surveys and assessments |
1 |
TBD |
M 13-14 |
2 |
1/1/18 |
M 13-14 |
3 |
1/1/19 |
M 13-14 |
4 |
1/1/19 |
M 13-14 |
* M 13–14 indicates months 13-14 after start of cohort
This section outlines the analyses to be conducted on the surveys and assessments.
Safety culture survey. Hospitals will receive feedback reports with their individual hospital’s results which may be displayed as frequencies, top box scores, or means. Post-intervention reports will include comparison of pre- and post-intervention results. The reports will include benchmarks based on aggregate results from the participating hospitals. In addition, analyses of the aggregate data will be performed to determine the impact of the intervention on hospital safety culture in the participating hospitals overall. The analyses will include bivariate correlation analyses to examine the relationships between hospital response rate, hospital sample size, overall culture score, and the 8 composite dimensions. All surgery centers submitting data at either baseline or followup will be included in these initial analyses. To assess actual culture change throughout hospitals’ participation in the project, two-tailed paired samples t-test analyses will be conducted on the subset of surgical centers that submit data at both baseline and followup. Stratification analyses will be conducted to explore how baseline and followup findings differ based on key factors (e.g., occupation [nurse, physician, etc]).
Patient experience survey. Data Cleaning--Response frequencies will be run on the patient experience data to identify out-of-range values, missing variables, or other data anomalies. Records of respondents who answer only demographic items will be excluded from any analyses.
Analysis for Feedback Reports--Participating hospitals that have at least 5 completed patient experience surveys during a survey administration period (pre- or post- implementation) will receive an individual hospital feedback report of their patient experience results. Results will be case-mix adjusted as applicable to account for individual characteristics like education, age, overall health status and mental health status. Individual hospital feedback reports may display results as frequencies, top box scores, or means. The post- implementation feedback reports will compare pre- and post-intervention results.
Average item and composite-level benchmarks will be created based on results from all participating hospitals within a cohort, so participating hospitals can compare their results to other participating hospitals in the same cohort. In addition, analyses of the aggregate data will be performed to determine the impact of the program’s intervention on patient experience in the participating hospitals overall.
Readiness Assessment. This assessment will evaluate hospitals’ readiness to implement the enhanced surgical care and recovery program. In addition to giving the JHU project team information about participating hospital needs, it will also allow hospitals to reflect on their potential strengths and weakness when initiating the project at their site. Hospitals’ readiness results will be compared to the implementation results to see if readiness predicts fidelity of the intervention. Additionally, we will look at whether readiness for the intervention impacts clinical outcomes.
Implementation assessment. This assessment will evaluate hospitals’ fidelity to the intervention components. The responses will also help identify lessons learned and recommended adjustments to the program and materials for future cohorts. The results will be analyzed to see if hospitals fidelity to the implementation is related to better outcomes.
Site visits. No formal analysis will be completed. The information collected will be used to strengthen the final toolkit materials and implementation process.
AHRQ does not seek this exemption.
List of Attachments:
Attachment A – Safety culture survey_Cover letter and reminder notice
Attachment B – Safety culture survey_Data collection instrument
Attachment C – Patient Experience Survey_Cover letter and reminder notice
Attachment D – Patient Experience Survey_Data collection instrument
Attachment E – Readiness assessment_Cover letter and reminder notice
Attachment F – Readiness Assessment_Data collection instrument
Attachment G – Implementation assessment_Cover letter and reminder notice
Attachment H – Implementation assessment_Data collection instrument
Attachment I – Site visit assessment_Cover letter and reminder notice.docx
Attachment J – Data collection instrument: Site Visit Assessment
Attachment K – Program list of experts
Attachment L – 60-Day Federal Register Notice
| File Type | application/msword |
| File Title | OMB Clearance Application |
| Author | hamlin-ben |
| Last Modified By | SYSTEM |
| File Modified | 2017-07-28 |
| File Created | 2017-07-28 |