Form #2 Form #2 Cognitive interview guide

Attachment C – Cognitive interview guide

Cognitive interviewing guide:

Using findings from the semi-structured interviews, we will develop a first draft of a questionnaire. This questionnaire will be pretested using cognitive interviewing techniques. Cognitive interviewing is one method of pretesting a survey, consisting of one-on-one interviews using a structured questionnaire. In general, respondents describe their reactions to and thoughts about the questionnaire items. Problem areas can be easily identified after a relatively small number of tests. Cognitive interviewing can be an iterative process after adjustments to the questionnaire have been made.

Cognitive interviewing will allow questionnaire developers to understand the strengths and weaknesses of the questionnaire. For example, based on earlier work on adverse event reporting (Farley et al., 2008), we anticipate that hospitals will exhibit different reporting system characteristics. Differences between hospital adverse event reporting might be further amplified in light of recent reporting arrangements with PSOs. Differences might also exist in the nature of information that is reported, the process of reporting adverse events within hospitals, and the terminology relevant to reporting that hospitals might use. The cognitive testing process will allow questionnaire developers to finalize an instrument that is flexible enough to capture the diversity of adverse event reporting systems of US hospitals.

The final cognitive testing guide cannot be developed until the survey instrument is complete. However, we anticipate the cognitive testing process will address the following areas and domains of information:

1. Overall assessment of the questionnaire.

We will ask respondents to comment on questionnaire items as if they were answering them, and to tell us if an item was confusing, not clear or not relevant. We also will ask about proper terminology and response choices. We will use a cognitive interviewing technique of follow-up probing to pinpoint problems and revise and strengthen the questionnaire. In followup probing, the researcher identifies focal issues such as the interpretation of technical terms, making choices and response alternatives, or memory retrieval over a long period (Forsyth and Lessler, 1991). We will ask respondents to comment on items and concepts that are presented in the questionnaire for their relevance to their hospital processes and reporting arrangements. The interviewing will allow us to evaluate respondents’ understanding of the concepts and questionnaire items, and to determine the relevance of certain items and response options that would be relevant.

2. Process of Reporting:

We will ask respondents if the questionnaire items on the adverse event reporting process are appropriate and relevant to the respondent’s hospital. We expect to include questionnaire items about who submits and receives reports, who reports adverse events to PSOs, and/or to a state or federal agency. We will ask respondents if these questionnaire items are relevant for their setting. If not, we will use follow-up probes to determine how the questionnaire might better capture the reporting process in respondents’ hospitals. We will ask if the questionnaire items on changes in reporting before and after contracting with a PSO reflect the respondent’s experience. For example, the questionnaire will ask whether hospitals needed to change their process of reporting to accommodate the use of Common Formats and reporting events to PSOs. We will use follow-up probes to further understand hospital-specific reporting system changes in a number of areas, including any changes in personnel involved in the reporting process, changes in the format in which reports are received (paper vs. electronic), differences in types of events reported, and how those events are defined. We expect that the questionnaire will also address whether the hospital has multiple reporting systems for adverse events, and if so, how multiple systems interrelate. The cognitive testing process will elucidate whether items covering this issue are relevant and understood by the respondent.

3. Reporting to PSOs:

In the cognitive testing process, we will ask about whether the questionnaire captures respondents’ experiences regarding reporting adverse events to PSOs. In particular, the questionnaire will try to establish whether hospitals contract with more than one PSO for reporting purposes; whether adverse events are reported to more than one PSO, and whether the same adverse event is reported to more than one PSO. The questionnaire also will try to establish which events are reported to PSOs and which are not, and if events are reported to other systems that are not reported to PSOs. Follow-up probes will be used to understand more fully the reporting relationships of hospitals to PSOs, and whether these questions are meaningful and understood by the respondent.

4. Terms and Definitions.

We anticipate asking respondents if they have knowledge of, and their understanding of the following terms, which we anticipate using in the questionnaire:

Adverse event, incident, near-miss, unsafe condition, harm

Patient Safety Organizations

Common Formats

References:

Farley, DO; Haviland, A; Champagne, S; Jain, AK; Battles, JB; Munier, WB; Loeb. (2008) Adverse-event-reporting practices by US hospitals: results of a national survey J M. Quality and Safety in Health Care. 17:416-423

Forsyth, BH; Lessler, JT (1991). Cognitive Laboratory Methods: A Taxonomy. In Measurement Errors in Surveys, edited by Paul P. Beamer and Robert M. Groves. New York: John Wiley & Sons, Inc.