SUPPORTING STATEMENT
Part B
Pilot Test of the Proposed Diagnostic Safety Supplemental Item Set for the Medical Office Survey on Patient Safety Culture
March 26, 2019
Agency for Healthcare Research and Quality (AHRQ)
1. Potential Respondent Universe and Sample Selection Method 2
2. Information Collection Procedures 3
3. Methods to Maximize Response Rate 4
5. Statistical Consultation and Independent Review 4
Cognitive Interview Participants
Cognitive interviews will be conducted on the Diagnostic Safety supplemental item set. Cognitive interview participants will be selected from medical offices that will vary by specialty, size, and geographic location. We will use participant recruitment methods similar to those used for the other patient safety culture surveys, such as sending research participation flyers to medical offices. We aim to conduct cognitive testing with a total of 30 medical office clinicians and staff across the range of positions found in medical offices, from physicians and nurses to medical assistants and clerical staff. No special selection procedures will be used to select specific participants within medical offices, with the exception of ensuring the appropriate distribution across staff positions and medical office type.
Pilot Test Study Sample
A list of potential medical offices will be developed in consultation with AHRQ staff, representatives from the TEP (see Attachment F for a list of TEP members), and will include medical offices that have participated in the Medical Office SOPS database. We will recruit medical offices to participate.
The aims of the overall sample design are two-fold: (1) to obtain enough pilot test data at both the medical office site level and the individual respondent level to ensure sufficient sample size (n) for examining the psychometric properties of the data, and (2) to include a variety of medical offices that differ in specialty, size, and geographic region.
Since the goals are to examine the psychometric properties of the item set, not to produce national estimates, purposive sampling will be used. Purposive sampling will ensure adequate variability on important medical office characteristics given the small number of medical offices included in the pilot test. It should be noted that the reason for including medical offices of different specialties is not to compare survey results across the types, but rather to ensure that there is representativeness of medical offices by specialty. The final item set will be publicly available for use by all types of medical offices.
As shown in Table 1, we propose that data be collected from medical offices, ranging from 2 to 42 medical offices within each of the nine categories of medical offices by type (primary, multispecialty, or single specialty) and size (small, medium, or large). We do not propose to evenly distribute the number of medical offices across the three types, rather, propose distributing the medical offices into these categories based on the distribution among medical offices that have submitted to the SOPS Medical Office Database.
Participating medical offices will provide clinician and staff lists. We will obtain approximately 2,500 clinicians and staff from 150 medical offices (Table 2). Assuming a response rate of 60 percent, we expect a total of 1,500 completed questionnaires.
For the pilot study, in order to conduct the psychometric and factor analyses, we need at least 10 respondents for each survey item. Assuming the survey will have about 25 items, we will need at least 500 total respondents answering all survey items to conduct these analyses. Given item non-response due to respondents choosing not to answer an item or not knowing how to answer an item that may not applicable to them, we aim to obtain more completed surveys than the minimum number required for psychometric analyses.
Table 1. Estimated Distribution by size and type: 150 medical offices selected for pilot test
Medical Office Size (# of staff includes providers) |
Type of Medical Office |
Total Pilot Medical Offices |
||
Primary Care Specialty |
Multi-specialty |
Other Single Specialty |
||
Small (5-10) |
42 |
5 |
8 |
55 |
Medium (11-20) |
28 |
18 |
2 |
48 |
Large (21+) |
30 |
15 |
2 |
47 |
Total |
100 |
38 |
12 |
150 |
Table 2. Estimated distribution by size and type of 2,500 individuals* surveyed within 150 medical offices selected for pilot test
Medical Office Size (# of staff, includes providers) |
Type of Medical Office |
Total Individuals Surveyed |
||
Primary Care Specialty |
Other Single Specialty |
Multi-specialty |
||
Small (5-10) |
336 |
40 |
64 |
440 |
Medium (11-20) |
420 |
270 |
30 |
720 |
Large (21+) |
855 |
428 |
57 |
1,340 |
Total |
1,611 |
738 |
151 |
2,500 |
* Assuming a 60 percent response rate, 1,500 individuals with completed diagnostic safety supplemental item sets will be available for analysis purposes.
Cognitive interviews will include these steps:
Faxing or emailing the surveys to the individuals recruited
Receiving completed surveys via fax or email
Telephone interview with respondents to discuss responses
The pilot test survey data collection will include these steps:
Programming the surveys for web-based data collection
Emailing medical office clinicians and staff to notify them of the survey
Weekly reminder emails to nonrespondents during a 4 to 6 week data collection period
Cognitive interview participants will receive cash remuneration ($150 for providers (physicians, physician assistants, and nurse practitioners), $100 for registered nurses, and $75 for other medical office support staff (medical assistants, secretaries, office managers). Although we are not offering remuneration for pilot test respondents, we are providing a feedback report to medical offices as an incentive, and follow up reminder emails to nonrespondents to maximize response rates.
The procedures for this specific project have not been subjected to testing. However, the contractor, Westat, has conducted many similar projects and will apply standard, well-established research methods for this project.
The following Westat statistical analysts developed both the study design and analytic plan for this project:
Naomi Dyer Yount, PhD, Senior Study Director, Westat, 301-610-8842
Joann Sorra, PhD, Associate Director, Westat, 301-294-3933
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 0000-00-00 |