Part B Surgical Care Supporting Statement Part B 20170720

SUPPORTING STATEMENT

Part B

AHRQ Safety Program for Improving Surgical Care and Recovery

Version: 7/20/17

Agency of Healthcare Research and Quality (AHRQ)

Table of contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent universe and sampling methods 3

2. Information Collection Procedures 6

3. Methods to Maximize Response Rates 8

4. Tests of Procedures 9

5. Statistical Consultants 9

B. Collections of Information Employing Statistical Methods

1. Respondent universe and sampling methods

The purposes of the surveys and assessments addressed in statement A are to provide feedback to participating hospitals to help with their improvement efforts and to compare results before and after project implementation to evaluate the impact of the overall quality improvement effort. Also, they will help the national project team tailor the technical assistance to the needs of the hospitals. Since the goals of this project are for quality improvement in participating hospitals and not to produce national estimates, purposive sampling of hospitals will be done.

Surveys will be administered pre and post intervention in a variety of participating hospitals that vary by type and size and geographic location within the U.S. Data will be collected from 100 hospitals that are participating in the first surgical service line cohort (colorectal surgeries). A total of 500 additional hospitals are expected to participate during cohorts 2, 3 and 4 with similar variation in terms of bed size, and other characteristics. The types of surgery will vary by cohort and are anticipated to include orthopedic, gynecologic, and emergency general surgery. All hospitals in the US are eligible to participate in the quality improvement program. JHU and American College of Surgeons will recruit hospitals from their extensive contact list that includes representatives from all US hospitals.

The project aims to improve patient outcomes. As noted in Supporting Statement A, clinical outcomes will be available from data extracts from an existing ongoing American College of Surgeons registry. Sample size calculations for the outcome of surgical site infections in an analysis comparing rates in a 3-month baseline period to a 3-month follow-up period for each cohort are as follows. Assumptions include 80% power, 2-sided significance level of 95%, average of 5 patients per hospital per month with the relevant surgeries for each cohort, 100 hospitals in cohort 1 and 500 hospitals in cohorts 2-4 combined, and baseline surgical site infection rate of 15% in colorectal surgeries, and 3% in other surgeries. For cohort 1 (colorectal surgeries), a relative reduction in surgical site infections of 24% will be detectable. For cohorts 2-4 combined (non-colorectal surgeries), a relative reduction in surgical site infections of 26% will be detectable.

Safety Culture Survey.

For this project, the respondents will be perioperative staff members at hospitals participating in the project. The number of staff surveyed will vary based on hospital size but will average about 50 staff per hospital at each time period. Hospitals will be asked to form a list of all perioperative staff who should be included in the survey. The pre-implementation survey will be administered at the beginning of the cohort and then again at then again at the end of the cohort, post-implementation, to assess potential improvement. Table 1 displays the estimated number of surveys to be administered.

Table 1. Hospital staff surveyed pre and post implementation for 3-year period

Surgical Service Line (Cohort)	Number of hospitals	Number of staff surveyed pre-implementation	Number of staff surveyed post-implementation	Number of staff surveyed pre and post
Colorectal (Cohort 1)	100	5,000	5,000	10,000
Hip/knee (Cohort 2)	250	12,500	12,500	25,000
Gynecology (Cohort 3)	125	6,250	6,250	12,500
Emergency General Surgery (Cohort 4)	125	6,250	6,250	12,500
Total (3 years)	600	30,000	30,000	60,000

Total sample size	60,000
Estimated response rate	60%
Total responses	36,000

Patient Experience Survey. For this project, the patient frame is patients who have undergone specific surgical procedures including colorectal surgery, hip and knee replacement, hysterectomy, emergency general surgery, and bariatric surgery. Each participating hospital will have about 5 discharged patients who have had a specific type of surgery per month. Given small numbers of patients, a census of those patients will be surveyed which results in 15 patients over the course of a 3-month pre-implementation data collection period. Another 15 patients will be surveyed over a 3-month post-implementation data collection period. Table 2 displays the estimated number of surveys to be administered over the three-year period.

Table 2. Patients surveyed pre and post implementation for 3 year period

Surgical Service Line (Cohort)	Number of hospitals	Number of patients per service line per month	Number of months	Number of patients surveyed pre-implementation	Number of patients surveyed post-implementation	Number of patients surveyed pre and post
Colorectal (Cohort 1--Year 1)	100	5	3	1,500	1,500	3,000
Hip/knee (Cohort 2--Year 2)	250	5	3	3,750	3,750	7,500
Gynecology (Cohort 3--Year 3)	125	5	3	1,875	1,875	3,750
Emergency General Surgery (Cohort 4—Year 3)	125	5	3	1,875	1,875	3,750
Total (3 years)	600			9,000	9,000	18,000

Total sample size	18,000
Estimated response rate	30%
Total responses	5,400

Readiness and Implementation Assessments

A pre-and post-assessment will be administered as a semi-structured interview with the hospital project leads (e.g. one physician, one nurse). Assuming an average of 2 staff being part of each pre- and post- semi structured assessment per hospital, table 3 displays the estimated number of surveys to be administered.

Table 3. Hospital staff surveyed pre and post implementation for 3-year period

Surgical Service Line (Cohort)	Number of hospitals	Number of staff surveyed pre-implementation (readiness assessment)	Number of staff surveyed post-implementation (implementation assessment)	Number of staff surveyed pre and post
Colorectal (Cohort 1)	100	200	200	400
Hip/knee (Cohort 2)	250	500	500	1000
Gynecology (Cohort 3)	125	250	250	500
Emergency General Surgery (Cohort 4)	125	250	250	500
Total (3 years)	600	1200	1200	2400

Total sample size	2,400
Estimated response rate	90%
Total responses	2,160

Site visits

Four site visits per cohort will be conducted. No site visits will be completed during the first cohort. Assuming an average of 10 staff being a part of each site visit, about 120 staff would be part of the site visit that will take 8 hours to complete.

Table 4. Hospital staff surveyed in site visit for 3-year period

Site visits	# of hospitals chosen for site visit	# of staff in site visit per hospital	# of staff in site visit per cohort
Cohort 1	0	0	0
Cohort 2	4	10	40
Cohort 3	4	10	40
Cohort 4	4	10	40
Total (3 years)	12	10	120
		Total	120

Total sample size	120
Estimated response rate	100%
Total responses	120

2. Information Collection Procedures

Important note: Since this project is a demonstration/implementation project a probability sample is not practical and will not be conducted for any of our surveys or assessments.

Safety culture survey.

To successfully measure perioperative patient safety culture in this project, we will leverage JHU’s existing survey data collection, analytics, and reporting platform, the Armstrong Institute Quality Portal (AIQP). We will use the Ambulatory Surgery Center Survey on Patient Safety Culture (ASC-SOPS), adapted for the inpatient surgery setting to collect safety culture measures from participating hospitals. The ASC-SOPS provides an efficient way to assess aspects of patient safety culture most pertinent in surgical services, offers more detailed mapping of perioperative provider and staff roles, and more clearly defines the parameters of the unit under investigation.

The recommendation to prepare for and participate in the safety culture survey will be included in the enrollment procedures. More information about the survey and how to participate will be discussed as open dialogue during the readiness assessment semi-structured interview. Following the interview, project leads from each hospital will receive an email from the national team describing the minimal steps for participating in the safety culture survey. Project leads will be given ideas and recommendations on how to communicate this survey to their staff to encourage participation.

The following are the safety culture composites categories that the Ambulatory Surgery Center Survey on Patient Safety Culture are intended to measure:

• Communication About Patient Information 

• Communication Openness 

• Staffing, Work Pressure, and Pace 

• Teamwork 

• Staff Training 

• Organizational Learning – Continuous Improvement 

• Response to Mistakes 

• Management Support for Patient Safety 

• Near-Miss Documentation 

• Overall Patient Safety Rating 

• Communication in the Surgery/Procedure Room 

Hospital staff that will be a part of the program will be asked to anonymously complete the adapted ASC-SOPS at the beginning and end of their 12-month cohort. The hospital’s project team will receive their survey results and then debrief their staff on their safety culture and identify opportunities for further improvement. Participating hospitals will promote awareness of the survey among their staff, coordinate implementation of the survey, encourage staff to complete the survey and provide staff time to do so, and organize their local debrief of the reports of their hospital’s results. The national project team will analyze the data and send a report to the hospital. A hospital that has less than six survey responders, will not be included in the reports or sent reports to debrief.

Patient experience survey data collection will include these steps:

• Mailing the surveys to the individuals sampled.

• Receiving completed surveys by mail. Returned surveys will be accepted throughout the data collection period.

• Mailing thank you/reminder postcards to nonrespondents. A thank you/reminder postcard will be mailed to all sampled individuals about 7-10 calendar days after the initial survey is mailed.

• Mailing reminder survey packet to nonrespondents. A second survey will be mailed to sample members who did not return the first survey. This second survey will be mailed to nonrespondents about 3-4 weeks after the first survey is mailed.

Readiness and Implementation assessment. The readiness assessment is a part of every hospital’s enrollment into each cohort. The implementation assessment is a part of every hospital’s sign out of the program at the end of each cohort. For both assessments, the national team will send an email (see cover letter) encouraging the hospitals to sign up for their 1-hour phone call with a member from the national team. The national team will follow up with a friendly reminder to those hospitals who do not sign up. Our availability will be flexible and we will offer additional times throughout the pre- and post- data collection period of each cohort to ensure that as many hospitals as possible can participate in these assessments.

Both assessments will be conducted as a semi-structured qualitative interview with key stakeholders at participating hospitals (e.g., project leads, physician project champions, etc.). The readiness assessment will help identify which, if any, technical components of the enhanced surgical care and recovery intervention already exist at the hospital, project management and resources, clinician engagement, leadership engagement and potential barriers and facilitators to implementation. The implementation assessment will evaluate what elements of the enhanced recovery practices have been adopted, resources invested, team participation, major barriers (e.g., medications, equipment, trained personnel), and leadership participation. These assessments will help identify training needs of hospitals and inform the national team’s approach. In addition, the results will inform the national team’s understanding of local adaptations of the intervention and the degree to which intervention fidelity impacts changes in outcomes.

Site visits. Semi-structured site visits will be conducted at a subset of participating hospitals. Findings will help inform the national project implementation strategy. Information from these visits will be critical in understanding if and how team and/or leadership issues may affect implementation of enhanced surgical care and recovery practices, including how this may differ across surgical service lines. Interviews will help uncover misalignments in role clarity, needed time and resources, best practices, and potential enablers of and barriers to enhanced surgical care and recovery implementation. Site visits will be conducted at approximately 4 hospitals per year, and each will be 1 day long. The types of hospital personnel anticipated to be involved in part or all of the site visit include senior leadership, perioperative leadership, and patient safety and quality staff. Participating hospitals will receive a structured debriefing and brief summary report at the end of the one-day visit.

3. Methods to Maximize Response Rates

Surveys and assessments will be administered as a pre-and post-intervention during each cohort. Response rates are discussed in statement B, section 1. We have the following processes in place to mitigate loss of participation and increase response rate for all surveys and assessments:

Safety culture survey. We are not offering cash remuneration for respondents; however, the national team will provide the hospitals’ project leads with training and tools to promote awareness of the survey among their staff, coordinate implementation of the survey, encourage staff to complete the survey and provide staff time to do so. The national team will also encourage the project leads to remind staff to check their spam folder for the email to complete the survey. Additionally, the survey platform will be programmed to send a reminder email once per week during the survey administration period to staff who have not completed the survey. Finally, the national team will send the hospital’s project leads updates on their response rates to encourage their staff to complete the survey.

Patient experience survey. We are not offering cash remuneration for respondents; however, the national team will send a second survey packet to those sample patients who do not respond to the first survey.

Readiness and Implementation assessment. We are not offering cash remuneration for respondents; however, each assessment is a part of every hospital’s enrollment into and sign out of each cohort (readiness assessment is a part of enrollment, while implementation assessment is a part of sign out). For both assessments, the national team will send an email (see cover letter) encouraging the hospitals to sign up for their 1-hour phone call with a member from the national team. The national team will follow up with a friendly reminder to those hospitals who do not sign up. Our availability will be flexible and we will offer additional times throughout the pre- and post- data collection period of each cohort to ensure that as many hospitals as possible can participate in these assessments.

Site visits. We are not offering cash remuneration for respondents. Four site visits per cohort will be conducted. No site visits will be completed during the first cohort. Findings will help inform the national project implementation strategy as well as the hospital’s implementation strategy. National team program experts will gather information from these visits on if and how team and/or leadership issues may affect implementation of enhanced surgical care and recovery practices, including how this may differ across surgical service lines. Interviews will help uncover misalignments in role clarity, needed time and resources, best practices, and potential enablers of and barriers to program implementation. Additionally, national team program experts will share insights, best practices and strategies with participating hospitals to they too can benefit from the visit. Finally, participating hospitals will receive a structured debriefing and brief summary report at the end of the one-day visit.

4. Tests of Procedures

The procedures for this specific project have not been subjected to testing, however, Johns Hopkins Armstrong Institute for Patient Safety and Quality, American College of Surgeons, and Westat have conducted many similar projects and are using well-established research methods with this project.

5. Statistical Consultants

Mark Cohen Ph.D.

American College of Surgeons

633 North St Clair

Chicago, IL 60611-3211

[email protected]

Yea-Jean Hsu, PhD

Armstrong Institute for Patient Safety and Quality

[email protected]

Naomi Yount, PhD

Westat

1600 Research Blvd.

Rockville, MD 20878

[email protected]

9