Form 2 – Draft Health IT Patient Safety Supplemental Item Set C

Attachment D

Draft Health IT Patient Safety Supplemental Item Set

Cognitive Interview Guide

3-20-15

Draft Health Information Technology Patient Safety Supplemental Item Set

Round 1 Telephone Cognitive Interviews: Introduction and Oral Consent

Respondent ID#: ___________________________

Respondent Job Title: __________________________

Date of Interview: _________________________

Time of Interview: _________________________

Interviewer: __________________________

Tape Recorded? _________________________

Introduction and Consent

Hello, my name is _____________________________. I have [a/two] colleague(s) with me here – [NAME(S)]. We work for Westat, a private research company in Rockville, Maryland. Thank you for taking the time to complete the survey and talk with us.

Westat is developing a set of items which focus on Health Information Technology in relation to patient safety for the Agency for Healthcare Research and Quality, Department of Health and Human Services. You recently completed a set of draft items for this survey.

I am talking with you today to find out how the survey items worked for you – for example, were the questions easy to understand and answer? Were any of the words vague or confusing? I am interested in what you think about the questions, and I will be asking you what the questions mean to you. There are no right or wrong comments.

This is a research project and your participation is voluntary. You may skip any question you do not want to answer and you may stop the interview at any point. I expect the interview to take about 1 hour. We take many steps to keep your comments and survey responses private. I will discuss your responses only with other project team members. We will not include your name or your hospital’s name in any written findings reports.

I will be happy to answer any questions you may have about this task. If you have any questions about your rights as a research participant, you may contact Westat’s Human Subjects Protections office. Would you like that phone number? (IF YES: Please call 1-888-920-7631 and leave a message with your full name, the name of the research study that you are calling about, and a phone number beginning with the area code. Someone will return your call as soon as possible).

Because I want to pay close attention to what you say, I would like to tape record our interview so that I can listen to it later to see if I missed anything. Is that okay?

TURN ON THE RECORDER: I need to ask your permission again so that it is recorded: Today is mo/day/year at [time]. Do you agree to participate in this interview and to have it audio recorded?]

I will ask a few general questions about the survey items, and then address specific survey items. Your comments will help in identifying possible problems. Please share your thoughts and don’t hesitate to bring up problems, suggest changes, or say which items you prefer – the whole purpose of this pretest is to improve the items and use the best ones in the survey.

Do you have a copy of your survey? Good. Before we start, do you have any questions? Okay, let’s begin.

Health Information Technology (Health IT) Patient Safety Supplemental Items

Filter question for Health IT Patient Safety:

A. Do you interact with any of your hospital’s Health Information Technology (Health IT) systems—including any computer-based software or systems in your hospital, electronic health records (EHRs), electronic medical records (EMRs), electronic prescribing (e-prescribing), computer physician order entry (CPOE), or electronic decision support tools?

• Yes [IF YES, GO TO QUESTION 1]

• No [IF NO, GO TO BACKGROUND QUESTIONS IN HOSPITAL SURVEY]

If Yes, What types of Health IT systems do you work with?

If No, What types of tasks do you perform in the hospital?

Health Information Technology (Health IT) Supplemental Items

1. Training

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We are given adequate training on how to use our Health IT systems………….………………	1	2	3	4	5	9
We are trained when there are changes to our Health IT systems……………………………….	1	2	3	4	5	9

PROBES:

Q1. You answered ________ for question 1. Can you say more about how you arrived at your answer?

Who were you thinking of when you answered? [Who is the “we”?]

What does “adequate training” mean in this question?

Can you give some examples?

What Health IT systems were you thinking of when you answered?

Q2. You answered ____. Can you say more about that?

What kind of Health IT system “changes” were you thinking of when you answered this question?

What kind of training were you thinking about when you answered?

Also, who is the “we” you were thinking about when you answered?

What Health IT systems were you thinking of when you answered?

Health Information Technology (Health IT) Supplemental Items

1. Training

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We are trained on what to do when our Health IT systems are down…………………………….	1	2	3	4	5	9

PROBES:

Q3. You answered ________ for question 3. Can you say more about how you arrived at your answer?

What does is mean when your “Health IT systems are down?”

Can you give some examples of what you are trained to do?

Who were you thinking of when you answered? Who is the “we”?

Who conducts the training?

What Health IT systems were you thinking of when you answered?

Health Information Technology (Health IT) Supplemental Items

2. Ease of Use/Work Process

How much do you agree or disagree with the following statements about the Health IT systems that you personally work?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
Our Health IT systems don’t do what we need them to do. (negatively worded)	1	2	3	4	5	9
Our Health IT systems are easy to navigate…	1	2	3	4	5	9

PROBES:

Q1. You answered ________ for question 1. Can you say more about that?

When you read “our Health IT system” who were you thinking of? [Who’s Health IT system?]

When you read “don’t do what we need them to do”, what were you thinking of?

Can you give us some examples?

Can you give some examples of the things you need it to do?

Q2 You answered ____. Can you say more about how you arrived at your answer?

What does “easy to navigate” mean to you?

What makes your system [easy/not easy] to navigate? Can you give us some examples? [IF IT’S NOT EASY, CHECK TO SEE IF IT’S DUE TO LACK OF TRAINING/USER ERROR VS DIFFICULT SOFTWARE]

What “system” were you thinking of when you answered?

Health Information Technology (Health IT) Supplemental Items

2. Ease of Use/Work Process

How much do you agree or disagree with the following statements about the Health IT systems that you personally work?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We use our Health IT systems in a way that integrates well with our work flow…...	1	2	3	4	5	9
It is easy to find patient information in our Health IT systems when we need it……………..	1	2	3	4	5	9

PROBES:

Q3. You answered ________ for question 3. Can you say more about how you arrived at your answer?

What do you think of when you read “work flow”?

What does it mean to integrate your Health IT system well with the work flow? Examples?

Who is the “we” you were thinking of? Work Area/Unit? Hospital?

What Health IT systems were you thinking of when you answered?

Q4 You answered ____. Can you say more about that?

What “patient information” were you thinking of when you answered?

Can you think of a time when it was difficult to find patient information?

Examples?

What Health IT systems were you thinking of?

Health Information Technology (Health IT) Supplemental Items

2. Ease of Use/Work Process

How much do you agree or disagree with the following statements about the Health IT systems that you personally work?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
It is easy to enter information in the wrong place in our Health IT systems. (negatively worded)…	1	2	3	4	5	9
It is easy to access clinical reference materials directly from our Health IT systems…………	1	2	3	4	5	9

PROBES:

Q5. You answered ________ for question 5. Can you say more about how you arrived at your answer?

Can you say in your own words what this question is asking you?

Can you think of a time when something was entered in the wrong place? Can you give us some examples?

Q6. You answered _____. Can you tell me more about that?

This question asks about “clinical reference materials”? What were the materials you were thinking of when you answered this question?

What does it mean to be “easy to access”?

CHECK WHETHER ACCESSIBLE

CHECK WHAT ‘EASY’ MEANS

Health Information Technology (Health IT) Supplemental Items

3. Health IT Patient Safety Risks

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
Our Health IT systems lower the risk for patient harm…………….……………….. …………………….. .	1	2	3	4	5	9
There are so many alerts in our Health IT systems that we cannot pay attention to them all (negatively worded).	1	2	3	4	5	9

PROBES:

Q1. You answered _____ to question 1. Can you say more about how you arrived at your answer?

What does it mean to “lower the risk for patient harm”?

Examples?

What Health IT system were you thinking of when you answered?

Q2. You answered ____. Can you say more about that?

What kind of “alerts” were you thinking of?

Can you give some examples?

What does it mean “not to be able to pay attention to them all”?

Who is the “we” in this question?

Health Information Technology (Health IT) Supplemental Items

3. Health IT Patient Safety Risks

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We report problems with our Health IT systems.	1	2	3	4	5	9
In our unit, Health IT systems are monitored to identify problems that may lead to patient harm.	1	2	3	4	5	9

PROBES:

Q3. You answered ___ to question 5. Can you say more about why you answered that way?

Can you give me some examples of Health IT systems problems?

How do you report these problems?

Who are they reported to?

What happens after they are reported?

What Health IT systems were you thinking of?

Q4. You answered ____. Can you say more about that?

What does it mean to “monitor” your Health IT systems?

Who is responsible for monitoring problems with the Health IT systems? (Clinical staff? IT staff? Both?)

Can you give an example of some Health IT problems that may lead to patient harm?

How are these problems discovered?

Are they discovered after a mistake happens or before?

Health Information Technology (Health IT) Supplemental Items

3. Health IT Patient Safety Risks

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We are made aware of Health IT-related patient safety events in our unit. ………….……………..	1	2	3	4	5	9
We take shortcuts with our Health IT systems that may put patients at risk (negatively worded) ………	1	2	3	4	5	9

PROBES:

Q5. You answered _____ to question 5. Can you say more about that?

What is a “Health IT-related patient safety event?”

IF SA/A: How are you made aware of Health IT-related patient safety events?

Who makes you aware of these events?

Q6. You answered ____. Can you say more about that?

This question asks about “shortcuts” – what is a Health IT system “shortcut”? Can you give us some examples?

How can a shortcut with your Health IT system put patients at risk?

Who is the “we” you answered for in this question?

Health Information Technology (Health IT) Supplemental Items

3. Health IT Patient Safety Risks

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
Our Health IT systems help us provide better quality care to patients…………………………	1	2	3	4	5	9
Our Health IT systems help us provide safer care to patients……………………………………	1	2	3	4	5	9

PROBES:

Q7. You answered _____. Can you say more about why you answered that way?

What does it mean to provide “better quality care to patients”?

Can you give us some examples of a Health IT system providing better quality of care to patients?

When you read “our Health IT system” who were you thinking of? [Who’s Health IT system?]

Q8. You answered ____. Can you say more about that?

What does it mean to provide “safer care to patients”?

How does Health IT system help you provide safer care to patients?

Is “safer care” different from “quality care”?

Health Information Technology (Health IT) Supplemental Items

3. Health IT Patient Safety Risks

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
We can continue to function without risks to patient safety during downtimes of our Health IT systems…………………………	1	2	3	4	5	9

PROBES:

Q9. You answered _____. Can you say more about why you answered that way?

What does it mean to “function without risks to patient safety”? Can you give us some examples?

What are Health IT system downtimes?

What Health IT systems were you thinking about?

Health Information Technology (Health IT) Supplemental Items

4. Communication

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
Our Health IT systems allow us to effectively share patient information with: Other providers	1	2	3	4	5	9
The patient	1	2	3	4	5	9
Relevant patient registries	1	2	3	4	5	9
As a result of our Health IT systems, providers and staff have less direct interaction with one another. (negatively worded)………….…………	1	2	3	4	5	9

PROBES:

Q1. You answered _____. Can you say more your answer?

What does it mean to “share patient information effectively”?

Can you give us some examples of sharing patient information with:

Other providers?

The patient?

Relevant patient registries?

What were you thinking of when you read “relevant registries”?

Q2. You answered ____. Can you say more about that?

What were you thinking of when you read “direct interaction with one another”? Examples?

[DOES IT INCLUDE ONLY FACE-TO-FACE INTERACTIONS? WHAT ABOUT VERBAL INTERACTIONS?]

Can you say how these “direct interactions” affect patient care?

Who are the “providers and staff”?

Health Information Technology (Health IT) Supplemental Items

4. Communication

How much do you agree or disagree with the following statements?	Strongly Disagree 	Disagree 	Neither Agree nor Disagree 	Agree 	Strongly Agree 	Does Not Apply or Don’t Know 
As a result of our Health IT systems, the quality of the information communicated among providers and staff has improved.	1	2	3	4	5	9
We communicate patient care information effectively when our Health IT systems are down.	1	2	3	4	5	9

PROBES:

Q3. You answered _____. Can you say more about how you arrived at your answer?

What made you [agree/disagree] here?

What does “quality of the information communicated” mean to you?

This question mentions “among providers and staff” – who were you thinking of when you answered this question?

Q4. You answered ___ to question 4. Can you say more about that?

What does it mean to “communicate patient care information effectively”? Examples?

Who were you thinking of when you answered this question? [Who is the “we”?]

What Health IT systems were you thinking of?

Health Information Technology (Health IT) Supplemental Items

5. List of Health IT Patient Safety and Quality Issues

The following items describe things that can happen when using Health IT systems that can affect patient safety and quality of care. In the past 12 months, how often did the following things happen with the Health IT systems you personally work with?

	Daily 	Weekly 	Monthly 	Several times in the past 12 months 	Once or twice in the past 12 months 	Not in the past 12 months 	Does Not Apply or Don’t Know 
Our Health IT systems were down	1	2	3	4	5	6	9
The results from a patient’s lab or imaging test were not available in the our Health IT systems	1	2	3	4	5	6	9

PROBES:

What do you think of when you read the list of Health IT patient safety and quality issues?

Are these things that you have ever experienced?

Can you say more about that?

Q1. You answered _______ to question 1. Can you say more about your answer?

Can you give some examples of when your Health IT systems were down?

Q2. You answered _______ to question 2. Can you say more about your answer?

Is this something you experience – that the lab or imaging results were not available in the Health IT system?

Health Information Technology (Health IT) Supplemental Items

5. List of Health IT Patient Safety and Quality Issues

The following items describe things that can happen when using Health IT systems that can affect patient safety and quality of care. In the past 12 months, how often did the following things happen with the Health IT systems that you personally work?

	Daily 	Weekly 	Monthly 	Several times in the past 12 months 	Once or twice in the past 12 months 	Not in the past 12 months 	Does Not Apply or Don’t Know 
The wrong electronic health record was used for a patient………………	1	2	3	4	5	6	9
A patient’s medication or allergy list was not up-to-date in the patient’s electronic health record	1	2	3	4	5	6	9

PROBES:

Q3. Can you say more about what made you choose _______ for question 3?

Can you say more about how this happens?

Q4. You answered _______ to question 4. Can you say more about how you arrived at your answer?

Can you say more about how this happens?

How easy or difficult was it for you to come to your answers to this section?

Health Information Technology (Health IT) Supplemental Items

6. Reporting Health IT Problems

In the past 12 months, how often did you contact your hospital’s IT department/staff to discuss:	Daily 	Weekly 	Monthly 	Several times in the past 12 months 	Once or twice in the past 12 months 	Not in the past 12 months 	Does Not Apply or Don’t Know 
A technical problem you had with your hospital’s Health IT systems	1	2	3	4	5	6	9

PROBES:

Q1. You answered _______ to question 1. Can you say more about that?

Have you personally contacted the Health IT Dept?

Is it just one person’s job to contact the Health IT Dept or does everyone/anyone do that?

How responsive was the Health IT staff to handle this issue?

1