SUPPORTING STATEMENT
Part B
Pilot Test of the Proposed Workforce Safety Supplemental Item Set
For the Surveys on Patient Safety Culture™
July 28, 2020
Agency for Healthcare Research and Quality (AHRQ)
1. Potential Respondent Universe and Sample Selection Method 3
2. Information Collection Procedures 4
3. Methods to Maximize Response Rate 5
5. Statistical Consultation and Independent Review 5
Cognitive Interview Participants
Cognitive interviews will be conducted with individual respondents to test the feasibility and applicability of the workforce safety supplemental items in all of the AHRQ Surveys on Patient Safety Culture™ (SOPS®) survey settings, except community pharmacies because the items are not directly applicable to that setting. Cognitive interview participants will be selected from hospitals, nursing homes, ambulatory surgery centers, and medical offices that will vary by size and geographic location. We will recruit cognitive interview participants from various staff positions within facility types. We aim to conduct English cognitive testing with a total of 20 clinicians and staff across the range of positions found in hospitals, nursing homes, ambulatory surgery centers, and medical offices, from physicians and nurses to medical assistants and managers. Limited Spanish cognitive testing will also be conducted with up to 5 individuals from hospital settings. No special selection procedures will be used to select specific participants within these facilities, with the exception of ensuring the appropriate distribution across staff positions and facility type.
Pilot Test Study Sample
While ideally the workforce safety item set would be pilot tested in all applicable settings of care, due to limited funds, and because the SOPS Hospital Survey has the broadest adoption among the SOPS surveys, pilot test data collection will be conducted in the hospital setting only. If cognitive testing supports the feasibility and applicability of the items in nursing homes, ambulatory surgery centers, and medical offices, then AHRQ could recommend that these settings administer the workforce safety items for beta testing and internal quality improvement purposes. AHRQ can consider pilot testing in these additional settings at a future date or encourage early adopters in these settings to share their data.
While cognitive testing will be done across settings of care, the pilot test will only be conducted in hospitals given limited resources. Types of potential hospitals will be developed in consultation with AHRQ staff, representatives from the Technical Expert Panel (TEP)/Subject Matter Experts (SMEs) (see Attachment E), and will include hospitals that have participated in the SOPS Hospital Database.
The aims of the overall sample design are two-fold: (1) to obtain enough pilot test data at both the hospital site level and the individual respondent level to ensure sufficient sample size (n) for examining the psychometric properties of the data, and (2) to include a variety of hospitals that differ in type, size, and geographic region.
Since the goals are to examine the psychometric properties of the item set, not to produce national estimates, purposive sampling will be used. Purposive sampling will ensure adequate variability on important hospital characteristics given the small number of hospitals included in the pilot test. It should be noted that the reason for including hospitals of different types, sizes, and geographic regions is not to compare survey results across the types, but rather to ensure that there is representativeness of hospitals by size and other characteristics. The final item set will be publicly available for use by all types of hospitals.
As shown in Table 1, we propose that data be collected from hospitals, ranging from 3 to 6 hospitals within each of the six categories of hospitals by type (teaching or non-teaching) and size (small, medium, or large). We do not propose to evenly distribute the number of hospitals across the two types, rather, propose distributing the hospitals into these categories based on the distribution among hospitals that have submitted to the SOPS Hospital Database.
Participating hospitals will provide clinician and staff lists. We will obtain approximately 11,500 clinicians and staff from 25 hospitals (Table 2). Assuming a response rate of 50 percent, we expect a total of approximately 5,700 completed questionnaires.
For the pilot study, in order to conduct the psychometric and factor analyses, we need at least 20 respondents for each survey item. Assuming the survey will have about 25 items, we will need at least 500 total respondents answering the survey items to conduct these analyses. Given item non-response due to respondents choosing not to answer an item or not knowing how to answer an item that may not applicable to them, we aim to obtain more completed surveys than the minimum number required for psychometric analyses.
Table 1. Estimated Distribution by size and type: 25 hospitals selected for pilot test
Hospital Size (# of beds) |
Type of Hospital |
Total Pilot Hospitals |
|
Teaching |
Non-teaching |
||
Small (50-99) |
3 |
5 |
8 |
Medium (100-399) |
4 |
6 |
10 |
Large (400+) |
3 |
4 |
7 |
Total |
10 |
15 |
25 |
Table 2. Estimated distribution by size and type of 11,500 individuals* surveyed within 25 hospitals selected for pilot test
Hospital Size (# of staff, includes providers) |
Type of Hospital |
Total Individuals Surveyed |
|
Teaching |
Non-teaching |
||
Small (50-99) |
3 x 375 = 1,125 |
5 x 375 = 1,875 |
3,000 |
Medium (100-399) |
4 x 500 = 2,000 |
6 x 500 = 3,000 |
5,000 |
Large (400+) |
3 x 500 = 1,500 |
4 x 500 = 2,000 |
3,500 |
Total |
4,625 |
6,875 |
11,500 |
* Assuming a 40 percent response rate, 5,926 individuals with completed workforce safety supplemental item sets will be available for analysis purposes.
Cognitive interviews will include these steps:
Emailing the surveys to the individuals recruited
Receiving completed surveys via email
Telephone interviews with respondents to discuss responses
The pilot test survey data collection will include these steps:
Programming the surveys for web-based data collection
Emailing hospital clinicians and staff to notify them of the survey
Weekly reminder emails to nonrespondents during a 4 to 6 week data collection period
Although we are not offering remuneration for pilot test sites or respondents, we are administering a web-based survey to sites and providing a feedback report to hospitals as an incentive. We will also be sending a series of follow up reminder emails to nonrespondents to maximize response rates.
The procedures for this specific project have not been subjected to testing. However, the contractor, Westat, has conducted many similar projects and will apply standard, well-established research methods for this project.
The following Westat statistical analysts developed both the study design and analytic plan for this project:
Naomi Dyer Yount, PhD, Senior Study Director, Westat, 301-610-8842
Joann Sorra, PhD, Associate Director, Westat, 301-294-3933
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Suzanne Streagle |
| File Modified | 0000-00-00 |
| File Created | 2021-01-13 |