5 Attachment B - Diagnostic Safety Cog Interview Guide 7-3

AHRQ Diagnostic Safety Supplemental Item Set for the Medical Office Survey on Patient Safety Culture Pilot Test, Supporting Statement A

Attachment B: Diagnostic Safety Cognitive Interview Guide

Draft Diagnostic Safety Supplemental Item Set

Cognitive Interview Guide

7/3/19

Draft Diagnostic Safety Supplemental Item Set: Cognitive Interview Guide

Respondent ID#:	Interviewer:
Respondent Job Title:	Note Taker:
Date & Time of Interview:	Tape Recorded?

Introduction and Oral Consent

Hello, my name is _____________________________. I have [a/two] colleague(s) with me here – [NAME(S)]. We work for Westat, a private research company in Rockville, Maryland. Thank you for taking the time to complete the survey and talk with us.

Westat is developing survey questions for the Agency for Healthcare Research and Quality, Department of Health and Human Services that ask about how the culture in medical offices supports the diagnostic process, accurate diagnoses, and communication around diagnoses. You recently completed a set of draft survey questions for this survey.

I am talking with you today to find out how the survey questions worked for you – for example, were the questions easy to understand and answer? Were any of the words vague or confusing? I am interested in what you think about the questions, and I will be asking you what the questions mean to you. There are no right or wrong comments.

This is a research project and your participation is voluntary. You may skip any question you do not want to answer and you may stop the interview at any point. I expect the interview to take about 1 hour. We take many steps to keep your comments and survey responses private. I will discuss your responses only with other project team members. We will not include your name or your medical office’s name in any written findings reports. We may include quotes in our internal notes to help us understand better the phrasing you and others use to describe your experiences.

I will be happy to answer any questions you may have about this task. If you have any questions about your rights as a research participant, you may contact Westat’s Human Subjects Protections office. Would you like that phone number? (IF YES: Please call 1-888-920-7631 and leave a message with your full name, the name of the research study that you are calling about, and a phone number beginning with the area code. Someone will return your call as soon as possible).

Next, we will walk through parts of the survey and I’d like for you to tell me about how you decided on your answers and any other reactions you might have had to the questions. I’ll also ask you some specific questions to help us better understand how the survey is working. Your comments will help in identifying possible problems [IF APPLICABLE: and thank you for providing some of those comments on the survey in the boxes we provided]. Please don’t hesitate to share any thoughts, bring up problems, suggest wording changes, or say which items you prefer – the whole purpose of this interview is to improve the items and use the best ones in the survey.

Do you have a copy of your survey?

Because I want to pay close attention to what you say, I would like to tape record our interview so that I can listen to it later to see if I missed anything. Is that okay?

TURN ON THE RECORDER: I need to ask your permission again so that it is recorded: Today is mo/day/year at [time]. Do you agree to participate in this interview and to have it audio recorded? …Great, Thank you. Before we start, do you have any questions? Okay, let’s begin.

Form Approved
OMB No. XXXX-XXXX
Exp. Date XX/XX/20XX

GENERAL INTERVIEWER INSTRUCTIONS:

-START EACH QUESTION EVALUATION BY ADDRESSING ANY WRITTEN COMMENTS, THEN FOLLOW UP AS NECESSARY WITH A GENERAL PROBE ABOUT THE PARTICIPANTS ANSWER (eg – Can you tell me how you chose your answer for…)

-CONTINUE WITH PROVIDED PROBES OR EMERGENT PROBES AS NECESSARY TO EVALUATE THE PARTICIPANT’S COMPREHENSION & RESPONSE

-PAY PARTICULAR ATTENTION TO ATYPICAL RESPONSES SUCH AS DK/NA, NEITHER…, OR DISAGREE (AGREE FOR NEG. WORDED ITEMS)

Definition

Instruction and Definition	Interviewer Script, Probes, and Notes
	We are going to be asking you about the sections starting on page 8 with the heading “Diagnostic Processes in Your Medical Office”.
The following items ask about the diagnostic process in your medical office. The diagnostic process starts when a patient seeks care for a health problem and includes gathering and interpreting information, communicating the diagnosis to a patient, and revising the diagnosis over time, as needed.	Did you notice the text in bold at the top of this page? Did you read it? [IF YES] What was your reaction? What do you think of when you see that term ‘diagnostic process?’ -Who’s involved in the diagnostic process in your office? -What’s included in the process? When does it start, end? Was anything missing from the definition? Was anything unclear?

Public reporting burden for this collection of information is estimated to average 60 minutes per response, the estimated time required to complete the survey. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: AHRQ Reports Clearance Officer Attention: PRA, Paperwork Reduction Project (XXXX-XXXX) AHRQ, 540 Gaither Road, Room # 5036, Rockville, MD 20850.

Section H: Testing Process (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	Now let’s look at Section H “Testing Process.”
This office follows a standardized process for tracking patient test results.	Can you tell me how you decided on your answer for the first question? What are you thinking of when you read “tracking patient test results”? Are these tests done at your office, or by someone else, or both? IF AGREE: Can you describe the process? How is it standardized? IF DISAGREE: How does your office track patient test results? IF NEITHER: What does it mean to ‘Neither agree nor disagree’ that the process is standardized? How does your office track patient test results?
When this office doesn’t receive a patient’s test results, staff follow up.	Can you tell me how you chose your answer for the next one? -What does “staff follow up” here mean to you? Who’s following up with whom? -Who does your office receive test results from? [DO PROVIDERS FOLLOW UP, OR JUST STAFF?]
Patients are told to assume that their test results are normal if they don’t hear back from this office. (NEGATIVELY WORDED)	How about the third question? [DO THEY CATCH NEGATIVE WORDING?]
All test results are communicated to patients, even if the test results are normal.	How about the last one? -How are test results communicated to patients? Are they all communicated the same way? Who’s communicating with whom? [IS THIS THE INVERSE OF H3, OR SOMETHING DIFFERENT?]

Section I: Referral Process (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	Now let’s look at Section I “Referral Process”.
This office follows a standardized process for tracking referrals	Can you tell me how you decided on your answer for the first question? What are you thinking of when you read “tracking referrals”? IF AGREE: Can you describe the process? How is it standardized? IF NEITHER/DISAGREE: How does your office track patient test results? [Why isn’t it standardized?]
When this office refers a patient to an outside provider, we provide their test results with the referral	Can you tell me how you chose your answer for the next one? IF AGREE: How do you provide the test results?
After making an urgent referral, this office tries to confirm whether the patient actually went to the appointment	Can you tell me how you chose your answer for the next one? What does “urgent referral” mean to you in this item?

Section J: Office Communication Around Diagnosis (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	Let’s move on to the next section… Before we talk about the questions in this section, can you tell me who you were thinking of as ‘providers’ when answering this section? What about ‘staff’?
Providers in this office encourage staff to share their observations about a patient’s condition.	Can you tell me how you chose your answer for 1? IF AGREE: Can you provide an example of how they encourage staff to share their observations…? [IS THIS JUST REPORTING VITAL STATS, OR PROVIDING SUGGESTIONS ON DIAGNOSIS?]
Providers in this office seek staff input to help inform a patient’s diagnosis.	Can you tell me how you chose your answer for 2? IF AGREE: How do providers seek staff input? [HOW DO ANSWERS TO Q1 AND Q2 COMPARE?]
When staff think a provider has missed an important piece of diagnostic information, they discuss it with the provider.	Can you tell me how you chose your answer for 3? What are some examples of ‘an important piece of diagnostic information’?
When staff are concerned about the accuracy of a diagnosis, they speak up.	Can you tell me how you chose your answer for 4? IF AGREE: How often does this happen? [HOW DO ANSWERS TO Q3 AND Q4 COMPARE?]

Section K: Communication And Feedback About Diagnostic Errors (scale: Strongly Disagree – Strongly Agree)

Definition and Draft Items	Interviewer Script, Probes, and Notes
	Let’s move on to the next section…
Diagnostic errors are: The failure to establish an accurate and timely explanation of the patient’s health problem(s) or communicate that explanation to the patient. Any mistake or failure in the diagnostic process leading to a wrong, missed, or delayed diagnosis.	Did you notice the text in bold at the top of this page? Did you read it? [IF YES] What was your reaction? What do you think of when you see the term ‘diagnostic error?’ Was anything missing from the description? Was anything in the description unclear?
We are informed about diagnostic errors that happen in this office.	Can you tell me how you chose your answer to 1? IF AGREE: How are you informed about diagnostic errors? IF NEITHER/DISAGREE: Are you informed about diagnostic errors at all?
In this office, providers and staff discuss ways to prevent diagnostic errors from happening.	Can you tell me how you chose your answer to 2? IF AGREE: Can you say more about these discussions?
When providers in this office discover a diagnostic error made by another provider, they talk to that provider about it	Can you tell me how you chose your answer to 3? When answering this question, who were you thinking of as ‘another provider’? [HOW DO 1-PROVIDER OFFICES RESPOND?] [IS THIS APPLICABLE FOR NON-PROVIDERS?] [ARE THE OTHER PROVIDERS IN THIS OFFICE OR OUTSIDE THIS OFFICE OR COULD BE EITHER?]
Providers in this office are reluctant to talk with other providers about their own diagnostic errors. (NEGATIVELY WORDED)	Can you tell me how you chose your answer to 4? [CAN NON-PROVIDERS ANSWER THIS QUESTION?] [DO THEY CATCH THE NEGATIVE WORDING?]

Section L: Documentation Around Diagnosis (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	Let’s move on to the next section titled ‘Documentation Around Diagnosis’
The information for making a diagnosis is hard to find in the patient’s medical record. (NEGATIVELY WORDED)	Can you tell me how you chose your answer to 1? [IS THIS APPLICABLE TO NON-PROVIDER?] [DO THEY CATCH THE NEGATIVE WORDING?]
When providers are uncertain about a patient’s diagnosis, they document their uncertainty in the patient’s medical record.	Can you tell me how you chose your answer to 2? IF AGREE: How is uncertainty documented? IF NEITHER/DISAGREE: Is uncertainty documented? [IS THIS APPLICABLE TO NON-PROVIDER?]
Alternative diagnoses are documented in the patient’s medical record when the diagnosis is unclear.	Can you tell me how you chose your answer to 3? IF AGREE: How are alternative diagnoses documented? Are they always documented when providers are uncertain about a patient’s diagnosis? [IS THIS APPLICABLE TO NON-PROVIDER?]

Section M: Communication with Patients (scale: Strongly Disagree – Strongly Agree)

Draft Item	Interviewer Script, Probes, and Notes
If providers are unsure about a diagnosis, they discuss other possible diagnoses with the patient.	Can you tell me how you chose your answer for the question in the next section? [IS THIS APPLICABLE TO NON-PROVIDER?]

Section N: Provider to Provider Consultation (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	NON-PROVIDERS: The next section asks about provider-to-provider consultation – did you feel like you could answer the questions in this section?
Providers in this office ask other providers for advice about complicated diagnoses.	Can you tell me how you chose your answer to 1? Who were you thinking about as ‘other providers’? IF AGREE: Can you provide an example? [ARE THE OTHER PROVIDERS IN THIS OFFICE, OUTSIDE THIS OFFICE, OR BOTH?] [IS THIS APPLICABLE TO NON-PROVIDER?]
When a report from a specialist/radiologist /pathologist needs clarification, providers talk with them directly.	Can you tell me how you chose your answer to 2? Can you describe what happens when a report needs clarification? Who at your office communicates with the specialist…? IF AGREE: How often does a provider talk with the specialist…directly? [IS THIS APPLICABLE TO NON-PROVIDER?]
When a diagnosis is complicated, providers in this office encourage the patient to get a second opinion.	Can you tell me how you chose your answer to 3? What were you thinking of when reading the phrase ‘When a diagnosis is complicated’? [IS THIS APPLICABLE TO NON-PROVIDER?]

Section O: Time Pressure (scale: Strongly Disagree – Strongly Agree)

Draft Items	Interviewer Script, Probes, and Notes
	Let’s move on to the next section… NON-PROVIDERS: This section also asks questions about providers – did you feel like you could answer these?
Providers in this office have the flexibility to extend appointment times if needed.	Can you tell me how you chose your answer for 1? IF AGREE: How does this flexibility work? [HOW DO NON-PROVIDERS ANSWER THIS?]
Providers in this office have enough time during their regular workday to complete patient notes.	Can you tell me how you chose your answer for 2? What does it mean for providers to ‘have enough time during their regular work day’? When do providers typically complete their patient notes? [HOW DO NON-PROVIDERS ANSWER THIS?]
Providers in this office have enough time during their regular workday to investigate complicated cases.	Can you tell me how you chose your answer for 3? Can you say more about how providers investigate complicated cases? Does this typically happen during their regular work day? [HOW DO NON-PROVIDERS ANSWER THIS?]
Providers in this office have enough time to stop and think about alternative diagnoses.	Can you tell me how you chose your answer for 4? Can you say more about how providers stop and think about alternative diagnoses? [HOW DO NON-PROVIDERS ANSWER THIS?] [IS THIS DIFFERENT FROM 3?]

Section P: Overall Ratings on Diagnostic Processes (scale: Poor – Excellent)

Draft Items	Interviewer Script, Probes, and Notes
	The last section we want to go over is Section P.
Overall, how would you rate your medical office…: In making accurate and timely diagnoses and clearly communicating them to patients? At preventing and learning from diagnostic errors?	Can you tell me how you chose your answer to a? A asks about accurate, timely and clearly communicating diagnoses – is your rating the same for each of these? Can you tell me how you chose your answer to b? B asks about preventing and learning from diagnostic errors – is your rating the same for both of these?

CLOSING:

ADDRESS ANY WRITTEN COMMENTS IN SECTION R OR ANYTHING RELATED TO THE DEMOGRAPHIC QUESTIONS.

[FOR RS WHO DID NOT SELECT DK/DNA]: Now I have a general question about the survey. I noticed you didn’t select Does Not Apply or Don’t Know - the response option in the last column. Did you notice it?

Do you have any additional comments?

Thank you very much for participating in this interview. Your comments have been helpful.

TURN OFF RECORDER

1