Supporting Statement
Part A
Improving Patient Safety System Implementation for Patients with Limited English Proficiency
November 4th, 2010
Agency for Healthcare Research and Quality
Table of contents
1. Circumstances that Make the Collection ofIinformation Necessary 3
2. Purpose and Use of Information 4
3. Use of Improved Information Technology 7
4. Efforts to Identify Duplication 7
5. Involvement of Small Entities 7
6. Consequences if Information Collected Less Frequently 7
8. Federal Register Notice and Outside Consultations 8
9. Payments/Gifts to Respondents 8
10. Assurance of Confidentiality 8
11. Questions of a Sensitive Nature 9
12. Estimates of Annualized Burden Hours and Costs 9
13. Estimates of Annualized Respondent Capital and Maintenance Costs 12
14. Estimates of Annualized Cost to the Government 12
16. Time Schedule, Publication and Analysis Plans 13
AHRQ’s mission
The mission of the Agency for Healthcare Research and Quality (AHRQ) set out in its authorizing legislation, The Healthcare Research and Quality Act of 1999 (see Attachment A), is to enhance the quality, appropriateness, and effectiveness of health services, and access to such services, through the establishment of a broad base of scientific research and through the promotion of improvements in clinical and health systems practices, including the prevention of diseases and other health conditions.
According to its authorizing legislation, AHRQ shall promote health care quality improvement by conducting and supporting:
1. Research that develops and presents scientific evidence regarding all aspects of health care; and
2. The synthesis and dissemination of available scientific evidence for use by patients, consumers, practitioners, providers, purchasers, policy makers, and educators; and
3. Initiatives to advance private and public efforts to improve health care quality.
Also, AHRQ shall conduct and support research and evaluations, and support demonstration projects, with respect to (A) the delivery of health care in inner-city areas, and in rural areas (including frontier areas); and (B) health care for priority populations, which shall include (1) low-income groups, (2) minority groups, (3) women, (4) children, (5) the elderly, and (6) individuals with special health care needs, including individuals with disabilities and individuals who need chronic care or end-of-life health care.
This study is being conducted by AHRQ through its contractor, Abt Associates Inc., pursuant to AHRQ’s statutory authority to conduct and support research on healthcare and on systems for the delivery of such care, including activities with respect to the quality, effectiveness, efficiency, appropriateness and value of healthcare services and with respect to quality measurement and improvement. 42 U.S.C. 299a (a) (1) and (2).
AHRQ’s use of the TeamSTEPPS system to improve patient safety
Patient safety is an important dimension of health care quality. TeamSTEPPS is AHRQ’s program to promote patient safety and reduce medical errors in U.S. health care organizations through improved team performance. Jointly developed with the Department of Defense, TeamSTEPPS has been implemented in hundreds of health care organizations in the United States and overseas since its release in 2006. TeamSTEPPS tools and materials are available here: http://teamstepps.ahrq.gov/abouttoolsmaterials.htm
The TeamSTEPPS system consists of a core curriculum that provides structured communication tools that make it easy for health care team members of any rank (doctor, nurse, technician, etc.) to flag and address patient safety problems with one another in an assertive and respectful way. TeamSTEPPS materials include an assessment tool to gauge the readiness of a site for the training, and measurement tools (e.g., learning benchmark questionnaire, team performance observation tool) to evaluate their implementation of the training and their quality improvement efforts.
TeamSTEPPS training is delivered at health care organizations by trainers who have participated in the 2.5-day Train-the-Trainer course at one of the 5 Regional Resource Centers, or by a commercial vendor. TeamSTEPPS trainers are expected to train large numbers of staff members in their home institutions. TeamSTEPPS trainers meet annually at an implementation conference where they share implementation experiences and successes.
Additional modules are periodically developed to show how TeamSTEPPS tools can be applied in specific situations and settings. New modules are field-tested before national roll-out to identify any needed changes in the module design.
Testing needed for a new TeamSTEPPS module for patients with limited English proficiency (LEP)
AHRQ recently developed a new TeamSTEPPS module focused on improving patient safety for patients with limited English proficiency (LEP patients). This module was developed because LEP patients are at elevated risk for harm from patient safety events due to communication barriers.
AHRQ proposes to field test this module and conduct case studies of its implementation in three hospitals.
The purpose of the proposed case studies is to: (1) assess the feasibility of implementing the LEP module, (2) identify any needed changes to the LEP module, including measurement tools, and (3) assess preliminary outcomes. To accomplish these goals, AHRQ will ask the three hospitals to implement the LEP module in at least one hospital unit. Implementation consists of the following activities.
1) Complete the Readiness Assessment Survey. (See Attachment B.) The purpose of this survey is to identify hospitals that may not be ready to implement the TeamSTEPPS module. The readiness assessment will be completed by a key contact person (hospital champion) at each site. The assessment may be completed in consultation with other members of a “change team” that the hospital champion may form to support the initiative.
2) Complete the Pre-work for Train-the-Trainer sessions. (See Attachment C.) The purpose of this activity is to allow trainers from each hospital to customize the training module to their hospital’s needs. The pre-work includes a survey, process map exercise, and a request to locate the hospital’s or organization’s policy on accessing language services. The pre-work will be completed by one of the hospital staff persons selected to be trainers at each site.
3) Attend the Train-the-Trainer sessions. (See Attachment D.) The purpose of this activity is to train two staff members from each of three participating hospitals in how to teach the training module. The TeamSTEPPS system requires at least two trainers for each hospital because its implementation is a team endeavor. Trainers will be selected either by the hospital champion, or by the “change team” formed by the hospital champion to support the intervention. Trainers will be selected from among natural leaders working within the hospital unit where the training will take place. Ideally the team will include a clinician (e.g., doctor, nurse) and an interpreter. Hospital staff selected to be trainers will be required to travel to Boston to attend the Train-the-Trainer sessions.
4) Conduct staff training. (See Attachment E.) The purpose of this activity is to have newly trained trainers implement the TeamSTEPPS LEP module in each participating hospital. Training participants will be drawn from the interprofessional care team in one or more hospital units (e.g., ob/gyn, surgery, etc.). This team may include nurses, physicians, technicians, front desk staff, and interpreters. Since the training teaches team behaviors, the entire interprofessional care team in a given hospital unit will be asked to attend the training session together. The training will be conducted onsite by the hospital staff members who attended the Train-the-Trainer sessions.
5) Field measurement instruments. The purpose of this activity is to measure four types of outcomes: trainee satisfaction, learning, behavior change, and patient outcomes. Satisfaction, learning and patient outcome surveys follow the format of TeamSTEPPS measurement instruments. The behavior survey is a new type of measure and focuses on the behaviors that the module is intended to modify. The following are the instruments to measure the four types of outcomes.
Training Participant Satisfaction Survey (see Attachment F) to assess trainee satisfaction with, and perceived adequacy of, the training module. This questionnaire will be administered to all training participants at the end of the training module.
Learning Outcomes Survey (see Attachment G) to assess staff knowledge about the best way to handle situations with LEP patients. To measure the change in staff knowledge resulting from the training module, this questionnaire will be administered both before and after the training.
Pre-training Behavior Survey (see Attachment H) to assess trainee behaviors of interest is administered immediately before the training. Questions from this survey are repeated in the post-training behavior survey to assess behavior change.
Post-Training Behavior Survey (see Attachment I) to assess trainee behaviors of interest after the training (repeated from the Pre-Training Behavior Survey, Attachment H) is administered 6 weeks after the training. This questionnaire also includes additional questions to assess barriers and facilitators to implementing the TeamSTEPPS tools that were discussed during the training.
Patient Outcome Survey (see Attachments J1 and J2) to measure change in patient communication and safety outcomes resulting from the training. This survey’s target audience is all patients identified as LEP. The purpose of this survey is to measure intermediate outcomes related to LEP patients’ access to language services, comprehension, and satisfaction with services. It is administered before and after the training. The survey is translated into Spanish (most common non-English language). AHRQ will ask hospitals to offer the survey in other languages as necessary through their regular language assistance mechanisms. AHRQ will ask hospitals to follow a systematic approach to sampling LEP patients (e.g.,: first 30 LEP patients treated within a given week)
6) Analyze data from measurement instruments. Hospitals will be asked to analyze the data resulting from the surveys described above as per the analysis plan detailed further below. AHRQ will offer technical assistance, as needed, in conducting the analysis to determine preliminary outcomes.
To assess the feasibility of implementing the LEP module (including fielding and analyzing data from the measurement instruments) and to obtain feedback on the LEP module that can be used to improve it, AHRQ will ask hospitals about their implementation experiences during telephone interviews and in-person interviews conducted during site visits. The following describes this data collection from trainers and other staff involved in the implementation of the LEP module.
Semi-Structured Follow-Up Interview (see Attachment K) to assess hospitals’ experiences implementing the training module. This semi-structured interview’s target audience consists of up to two trainers or change team members in each hospital where the training module is implemented. These interviews will be conducted at the 2-week, 6-week and 10-week mark after the training.
Site Visits conducted 3 months after the training to assess the hospitals’ experiences implementing the training module. Site visits will include:
Observation of the clinical setting to identify any aspects of the environment that may have impeded or supported implementation (e.g., availability of dual handset phones, signage regarding patients’ rights to an interpreter, video interpretation equipment), and note any visible signs of LEP module implementation such as posters, buttons worn by staff members, or monitoring results through a whiteboard or other visible medium.
Semi-structured site-visit interviews (Attachment L). This semi-structured interview’s target audience consists of up to 6 persons who may include trainers, change team members, frontline staff members, or other persons designated by the “hospital champion” as persons who might provide insight into LEP module implementation and outcomes. These interviews will be conducted 3 months after the training.
No claim is made that the results from this study will be generalizable in the statistical sense. Rather, these three cases will be illustrative and informative and will generate lessons that will inform adjustments to the training materials and measurement instruments.
The surveys implemented in this project will likely achieve higher response rates if collected on paper, because most potential respondents would not have access to a computer at the time of data collection. However, we may offer support to hospitals in creating electronic data entry interfaces compatible with local teams’ computer systems and skills (using Excel, Access or Checkbox Mobile).
One portion of the data collection, the Post-Training Behavior Survey, is a likely candidate for electronic data collection through an online survey sent to staff members via e-mail link. We will offer technical support to hospitals to set up this electronic data collection system through an online survey tool such as Checkbox Mobile, if they feel it would facilitate data collection from their staff.
To help hospitals in their analyses, we will also point them to free online statistical test calculators they can use to quickly and easily analyze their data.
A rigorous environmental scan was conducted as part of the preliminary research to create this new training module, searching both the peer-reviewed and grey literature for evidence of similar training modules. While several training modules have focused on improving access to culturally and linguistically appropriate care, to date none has focused on team-based training to improve patient safety for LEP patients. Thus, previous training module evaluations cannot be used for the purposes described in item 2 above.
This project is a one-time data collection effort.
This request is consistent with the general information collection guidelines of 5 CFR 1320.5(d)(2). No special circumstances apply.
As required by 5 CFR 1320.8(d), notice was published in the Federal Register on December 9th, 2010 for 60 days (see Attachment M).
None
AHRQ will offer no honoraria or incentives to respondents in any hospital.
Individuals and organizations will be assured of the confidentiality of their replies under Section 934(c) of the Public Health Service Act, 42 USC 299c-3(c). They will be told the purposes for which the information is collected and that, in accordance with this statute, any identifiable information about them will not be used or disclosed for any other purpose.
Respondents will be
informed in the introduction to surveys and interviews that their
answers will be kept confidential to the extent permitted by law.
Participation will be entirely voluntary, and the study will conform to the requirements of the Privacy Act by omitting individuals’ names, addresses, telephone numbers and other personal identifiers in the final data file.
The firm that will coordinate the data collection, Abt Associates, has conducted numerous projects and surveys involving sensitive information; consequently, facilities and procedures have been developed to maintain respondent confidentiality. All Abt Associates staff who are in contact with human subjects data are required to complete ethical training, which includes training about maintaining the confidentiality of information. Any databases created by Abt Associates will be password-protected, with only the data administrators having write-authority over files. If electronic data transfer is necessary, the data will be transferred in an encrypted and password-protected format via a secure FTP server or by diskette or CD-ROM shipped via a bonded courier. Hospitals that conduct data collection will be asked to provide the same assurances as a condition of their participation. IRB applications prepared for participating hospitals by Abt Associates will contain this language.
The surveys and case study interview protocols do not contain any questions concerning sexual behavior and attitudes, religious beliefs, income or proprietary business information. However, surveys may elicit sensitive information that reflects negatively on staff or hospital performance related to communication with LEP patients. Respondents to the survey will be explicitly informed that their participation is voluntary, information they provide is confidential to the extent provided by law, and they may choose to withdraw from the study or not respond to specific items without penalty. We will also remove hospital names from written interview records and case study reports to maintain respondent confidentiality.
Exhibit 1 presents estimates of the reporting burden hours for this one-year data collection process. Time estimates are based on prior experiences with TeamSTEPPS testing, which correspond to what can reasonably be requested of participating hospitals.
The Readiness Assessment Survey will be completed by the key contact/project champion at each of the 3 participating hospitals (3 respondents total) and will take about 5 minutes.
The pre-work for the Train-the-Trainer sessions will be completed by the two trainers selected for each site (6 respondents total) and will take about 30 minutes.
The Train-the-Trainer sessions will be conducted with 2 staff members from each hospital (6 participants total) and will last 4 ½ hours; the burden estimate of 12.5 hours includes an average of 8 hours of travel time to and from the training site.
Staff Training will include up to 30 staff members at each hospital plus the 2 trainers who are staff members (96 participants total) and will last 1 hour.
The Training Participant Satisfaction Survey will be completed by Staff Training participants at the end of the training (90 respondents total) and takes 5 minutes to complete.
The Learning Outcomes Survey will be administered twice, before and after the training (90 participants total), and will require 10 minutes.
The Pre-Training Behavior Survey will be administered to all staff invited to the training except for interpreters (approximately 75 respondents total). It will require approximately 5 minutes. Interpreters do not complete this questionnaire because the questions relate to interpreter use.
The Post-training Behavior survey will be administered two or more weeks after the training to all staff who were invited to the training (90 respondents total), and will take approximately 7.5 minutes to complete.
The Patient Outcome Survey will be administered to 30 patients before and 30 patients after the training, for a total of 60 patients per hospital (180 respondents total). This survey requires about 10 minutes to complete.
Semi-Structured Follow-up interviews will be conducted three times over a 12-week period with two trainers or change team members from each hospital, (6 participants total). Each semi-structured follow-up interview will last for about an hour.
Semi-Structured Site visit interviews will be conducted with 6 staff members from each hospital (18 participants total) and will take an hour to complete. The total annualized burden hours are estimated to be 295 hours (rounded to the nearest hour).
Exhibit 1: Estimated annualized burden hours
Data Collection Method or Project Activity |
A. Number of respondents or participants |
B. Number of responses or iterations per respondent |
C. Hours per response |
D. Total burden hours
(A*B*C) |
1. Readiness Assessment |
3 |
1 |
5/60 |
0.25 |
2. Pre-Work for Train-the-Trainer Training |
6 |
1 |
30/60 |
3 |
3. Train-the-Trainer Training |
6 |
1 |
12.5 |
75 |
4. Staff Training |
96 |
1 |
1 |
96 |
5. Training Participant Satisfaction Survey |
90 |
1 |
5/60 |
8 |
6. Learning Outcomes Survey |
90 |
2 |
10/60 |
30 |
7. Pre-Training Behavior Survey |
75 |
1 |
5/60 |
6 |
8. Post-training Behavior Survey |
90 |
1 |
7.5/60 |
11 |
9. Patient Outcome Survey |
180 |
1 |
10/60 |
30 |
10. Semi-Structured Follow-up interview |
6 |
3 |
1 |
18 |
11. Semi-Structured Site visit interview |
18 |
1 |
1 |
18 |
TOTALS: |
660 |
na |
na |
295 |
Exhibit 2, below, presents the estimated annualized cost burden associated with the respondents’ time to participate in this research. The total cost burden is estimated to be about $6,980. Note that AHRQ will reimburse travel costs for persons participating in the Train-the-Trainer training. No other travel is anticipated for training participants.
Exhibit 2: Estimated annualized cost burden
Data Collection Method |
D. Total burden hours (from Exhibit 1) |
E. Average hourly wage rate |
Total cost burden
(D*E) |
1. Readiness Assessment |
0.25 |
$26.50 |
$7 |
2. Pre-Work for Train-the-Trainer Training |
3 |
$26.50 |
$80 |
3. Train-the-Trainer Training |
75 |
$26.50 |
$1,988 |
4. Staff Training |
96 |
$22.02 |
$2,114 |
5. Training Participant Satisfaction Survey |
8 |
$22.02 |
$176 |
6. Learning Outcomes Survey |
30 |
$22.02 |
$661 |
7. Pre-Training Behavior Survey |
6 |
$22.04 |
$132 |
8. Post-training Behavior Survey |
11 |
$22.02 |
$242 |
9. Patient Outcome Survey |
30 |
$20.90 |
$627 |
10. Semi-Structured Follow-up interview |
18 |
$26.50 |
$477 |
11. Semi-Structured Site visit interview |
18 |
$26.50 |
$477 |
TOTALS: |
295 |
Na |
$6,980 |
*The average hourly wage rate of $26.50 for readiness assessments, train-the-trainer trainings, semi-structured site visit interviews, and semi-structured follow-up interviews was calculated based on the average of the mean hourly wage rate for healthcare practitioners and medical occupations (all professions), $31.02 and the average hourly wage rate for interpreters and translators, $21.97. The average hourly rate for staff receiving training pf $22.02 was calculated based on the average of the mean hourly wage rate for healthcare practitioners and medical occupations (all professions), $31.02, mean hourly wage rate for interpreters and translators, $21.97, and mean hourly wage rate for healthcare support occupations, $13.06. The average hourly wage rate for respondents to the pre-training behavior survey of $22.04 was calculated based on the average of the mean hourly wage rate for healthcare practitioners and medical occupations (all professions), $31.02, and mean hourly wage rate for healthcare support occupations, $13.06. The average hourly wage rate for patients of $20.90 was calculated on the mean hourly wage rate for all occupations. Average hourly rate for unit staff, non-interpreter was calculated based on the average of the mean hourly rate for healthcare practitioners and medical occupations (all professions), $31.02, and occupations (all professions), $31.02, mean hourly wage rate for interpreters and translators, $21.97, and mean hourly wage rate for healthcare support occupations, $13.06. Mean hourly wage rates for these groups of occupations were obtained from the Bureau of Labor & Statistics on “Occupational Employment and Wages, May 2009” found at the following URLs: http://www.bls.gov/oes/current/naics4_622100.htm , http://www.bls.gov/oes/current/oes273091.htm http://www.bls.gov/oes/current/oes_nat.htm
Capital and maintenance costs include the purchase of equipment, computers or computer software or services, or storage facilities for records, as a result of complying with this data collection. There are no direct costs to respondents other than their time to participate in the study.
The total cost of this contract to the government is $499,978. The project extends over 4 fiscal years, although data collection will take place over the course of a single year. Exhibit 3 shows a breakdown of the total cost as well as the annualized cost for the data collection, processing and analysis activity.
Exhibit 3: Estimated Cost
Cost Component |
Total Cost |
Annual Cost |
Project Development |
$301,664 |
$75,416 |
Data Collection Activities |
$52,629 |
$13,157 |
Data Processing and Analysis |
$52,629 |
$13,157 |
Publication of Results |
$51,658 |
$12,915 |
Project Management |
$41,399 |
$10,350 |
Total |
$499,978 |
$124,995 |
This is a new information collection.
Exhibit 4 Project Timeline
|
|
Description (in chronological order) |
Due Date
|
Train hospital trainers |
August 15, 2011 |
Pre-training follow-up and technical assistance calls |
September 30, 2011
|
Train staff members |
October 15, 2011 |
Post-training follow-up and technical assistance call |
October 30, 2011 |
Site visit |
November 30, 2011 |
Complete analysis |
December 15, 2011 |
Draft field test report |
December 20, 2011 |
Final field test report |
January 15, 2012 |
Revised TeamSTEPPS LEP module |
February 30, 2012 |
Final TeamSTEPPS LEP module |
March 30, 2012 |
Publication Plan:
Study results will be disseminated through a peer-reviewed publication, professional presentations, AHRQ’s Office of Communication and Knowledge Transfer, a webinar, the Massachusetts General Hospital’s Disparities Solutions Center’s (DSC) website, and e-mails to relevant professional associations and other stakeholders. The Final TeamSTEPPS LEP module will be posted on the TeamSTEPPS web site.
Analysis Plan:
As described above, the purpose of this data collection is threefold: (1) assess the feasibility of implementing the LEP module, (2) identify any needed changes to the LEP module, and (3) assess preliminary outcomes. The data to achieve Goals 1 and 2 are only qualitative, while the data to support for Goal 3 are both quantitative and qualitative. The data analysis strategies therefore differ.
Goal 1: Assess the feasibility of implementing the LEP module
Goal 2: Identify any needed changes to the LEP module
Data collection strategy: Qualitative (telephone and in-person interviews, observation)
Data analysis strategy: Case study
For goals 1 and 2, data collection will be through telephone and in-person interviews and observation. Data will be analyzed to identify key themes. Analyses will be used to write up case studies on the implementation of the new TeamSTEPPS module at three sites. Case study reports for each of the three sites will describe each site’s experience with implementation, technical assistance needs, and recommended revisions to the LEP module (including recommendations regarding fielding and analyzing data from the measurement instruments) based on this experience. A cross-site case study report will summarize lessons learned across the three cases.
Goal 3: Assess preliminary outcomes
Data collection strategy: Quantitative (Surveys)
Data analysis strategy: Univariate tabulations and appropriate pre-post statistical tests
As described above, AHRQ will ask hospitals to conduct the quantitative analyses on data collected within their hospital. Results of these analyses will be integrated into individual case study reports for each site. For analyses requiring statistical tests (described further below), if local analysis at each hospital fails to show a statistically significant result, AHRQ may request data collected from sites and conduct a pooled analysis for the cross-site case study report.
Analysis sub-goals for each set of instruments and analysis plans are summarized in Exhibit 5, below.
Exhibit 5. TeamSTEPPS LEP Module Quantitative Data Collection: Sub-Goals and Analysis Plans
Instrument |
When administered and to whom |
Analysis sub-goal |
Analysis Plan |
Training Participant Satisfaction Survey (Attachment F) |
|
Test the acceptability of the training to trainees |
|
Learning Outcomes Survey (Attachment G) |
|
Assess whether trainee knowledge increased as a result of the training |
|
Behavior Surveys (Attachments H and I) |
|
Assess behavior change resulting from the training |
|
Patient Outcome Survey (Attachment J) |
|
Assess changes in patient perceptions of communication and patient safety outcomes resulting from the training |
|
AHRQ does not seek this exemption.
Attachment A: Healthcare Research and Quality Act of 1999
Attachment B: Readiness Assessment Survey
Attachment C: Pre-Work for Train-the-Trainer (Master) Training
Attachment D: Train-the-Trainer (Master) Training Guide
Attachment E: Staff training Guide
Attachment F: Training Participant Satisfaction Survey
Attachment G: Learning Outcomes Survey
Attachment H: Pre-Training Behavior Survey
Attachment I: Post-training Behavior Survey
Attachment J1: Patient Outcomes Survey – English
Attachment J2: Patient Outcomes Survey –Spanish
Attachment K: Semi-Structured Follow-Up Interview Guide
Attachment L: Semi-Structured Site Visit Interview Guide
Attachment M: Federal Register Notice
| File Type | application/msword |
| Author | SubramanianR |
| Last Modified By | william.carroll |
| File Modified | 2011-05-11 |
| File Created | 2011-05-06 |