TS_Primary Care Settings_OMB_Supporting Statement Part B_041217

SUPPORTING STATEMENT

Part B

Implementation of TeamSTEPPS in Primary Care Settings (ITS-PC)

OMB CONTROL NO. 0935-0222

Version: April 12, 2017

Agency for Healthcare Research and Quality

Table of Contents

B. Collections of Information Employing Statistical Methods 3

1. Respondent Universe and Sampling Methods 3

2. Information Collection Procedures 3

3. Methods to Maximize Response Rates 4

4. Tests of Procedures 5

5. Statistical Consultants 5

B. Collections of Information Employing Statistical Methods

1. Respondent Universe and Sampling Methods

The TeamSTEPPS for Office-Based Care Post-Training Survey will be distributed to all individuals who have participated in the TeamSTEPPS for Office-Based Care Training program. It is anticipated that 1,200 participants will be trained during March 2017 – December 2019. We will not employ sampling, as the Agency for Healthcare Research and Quality (AHRQ) seeks to obtain feedback from everyone who participates in the program (i.e., a census survey). A 75 percent response rate or higher is anticipated.

Some of the limitations of this study, which will also be noted in the final report, include the following:

• Retrospective analysis of post-training experiences. This is not an empirical study; therefore, neither causality nor generalizability can be established;

• Survey non-response, which can limit AHRQ’s understanding of the concerns and issues participants encountered when trying to apply TeamSTEPPS tools and concepts in their home organizations or organizations that they support;

• Limited number of interviews that can be conducted per training class due to time and budgetary constraints; and

• Inability to survey participants who have changed jobs since TeamSTEPPS for Office-Based Care Training. The survey will be administered six months after training.

Despite the limitations in this study, the data collected will provide valuable information that will help refine the TeamSTEPPS for Office-Based Care curriculum to support primary care practices’ efforts to improve patient safety. By understanding the factors that facilitate or inhibit the use of tools or the spread of knowledge, AHRQ will better understand the needs of primary care practices and will be better prepared to address their future concerns, issues, and needs for improving patient safety.

2. Information Collection Procedures

The TeamSTEPPS for Office-Based Care Post-Training Survey will be administered to all individuals who have completed the TeamSTEPPS for Office-Based Care training program using the Web-based questionnaire included in Attachment A. The participant questionnaire will be sent to all training participants via email. All correspondence to training participants is presented in Attachment B. These correspondences contain a hyperlink to enable easy access to the online questionnaire.

The TeamSTEPPS for Office-Based Care Post-Training Survey will be administered six months after the completion of training. The survey will be accessible to training participants 24 hours a day, for a minimum of 30 days. On entrance into the questionnaire, respondents will view an introduction page that explains the questionnaire objectives and stresses the importance of participation. Respondents can respond to the questionnaire items by clicking on pre-coded options for closed-ended items and typing in boxes for any open-ended items. The survey requires 20 minutes to complete.

The Health Research & Educational Trust (HRET) is leading the TeamSTEPPS for Office-Based Care online training program. HRET’s subcontractor, IMPAQ International (IMPAQ), is leading this study. Completed responses to the questionnaires will be backed up daily onto IMPAQ’s password-protected, secure data collection server. Data responses will be checked visually by researchers and analysts on a regular basis to assure that data are entered appropriately into the database.

Following data collection, questionnaire responses will be compiled and assessed formally for data quality to produce a finalized database for statistical analyses. Incomplete response data poses a substantial threat to confident interpretation of the study results. The general approach to handling incomplete response data is to salvage as much data as possible using multiple techniques for examining patterns of missing data. Given the scope of the training participant questionnaire, IMPAQ will review questionnaire items with a substantial proportion of omitted responses. The precise cutoff percentage is typically chosen once the distribution of missing data has been established. It will be determined whether responses are missing in a manner that relates to other observable values. If data are determined to be missing in a manner that affects the interpretability of the responses, descriptive statistics and point estimates of relations among variables may be adjusted to account for missing data. This may be accomplished by using multiple imputation and full information maximum likelihood estimation techniques.

3. Methods to Maximize Response Rates

To effectively bolster the response rate, time-staggered notices of the opportunity to participate in the questionnaire will be sent via email. If at any point an invited respondent refuses to participate, he or she will not be contacted again through any means.

During the data collection period, invited respondents who have yet to respond to the training participant questionnaire will be re-contacted via email reminding them of the opportunity to participate and the importance of their feedback regarding the training program. The reminder notice sent via email will provide the hyperlink to access the questionnaire, the estimated time (in minutes) it will take to respond, the impending deadline for submission of their responses, and additional information regarding privacy of responses and confidentiality of personal information. The reminder notice is provided in Attachment B.

A common way to ensure an acceptable response rate is to provide introductory information through advance notice from a known and respected source. For the purposes of this study, the AHRQ Project Officer will sign the introductory e-mail to all invited respondents to provide advance notice of the study and the importance of participation (see Attachment B).

4. Tests of Procedures

A prior version of the TeamSTEPPS for Office-Based Care Post-Training Survey was piloted. The pilot test involved: (1) a series of two to three cognitive laboratory interviews with AHRQ staff members and (2) a field test of the survey with a subsample of no more than nine potential respondents. The results were used to refine the questionnaire prior to field testing. Content changes to align this version of the questionnaire with the TeamSTEPPS for Office-Based Care curriculum have been reviewed by subject matter experts.

5. Statistical Consultants

IMPAQ International, LLC will serve as the primary consultants for statistical aspects of the design and analysis of the Web questionnaire data. Dr. David P. Baker, Executive Vice President at IMPAQ, is the primary point of contact for statistical design and analyses. He can be reached at [email protected] or 443-259-5134.