0920-1050 Healthcare Provider - Drug Overdose Moderator Guide

Confidential and Proprietary January 13, 2021

CDC DRUG OVERDOSE INFORMATION FOR PROVIDERS WEB USABILITY TEST TASKS AND TEST QUESTIONS

OMB Control No. 0920-1050

Exp. Date 05/31/2022

Public reporting burden of this collection of information is estimated to average 45 minutes per respondent. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB Control Number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia, 30333; ATTN: PRA (0920-1050).

Hello, [participant name]. My name is [moderator name] and I’m going to walk you through this session.

We’re interested in seeing how healthcare providers and public health practitioners search and use opioid guidelines and resources on CDC’s website.

I want to make it clear right away that we are not testing how well you can use the website. We’re testing the website to understand how easy or difficult it is to use. You can’t do anything wrong here. My job is to find out what’s wrong with it or where it can be improved. If you think the website is confusing or difficult to use, we need to know honestly what you think so that we can inform designers and developers of this website how to make it better.

We are interested in how you use the website, which is why we are video-recording your computer screen and asking you what’s going through your mind as you go. We are also audio-recording our conversation. The recordings will be used only to help us figure out how to improve the website, and it won’t be seen by anyone except the people working on the project.

Next, I am going to take a moment to ask you to confirm that you have been notified about your privacy, that we are recording this conversation, and that your participation is voluntary. This information was all included in the consent form that was emailed to you.

I am going to start recording our conversation now so that we have a record of your answers.

[Moderator begins recording]

For the record, can you confirm that you have been notified about your privacy, that we are recording this conversation, and that your participation is voluntary?

[Only continue conversation after the participant has provided verbal confirmation]

Do you have any questions before we begin?

1. PRE-TEST QUESTIONS (10 minutes)

A1. Have you looked for guidelines and resources related to opioid use before?

0 No (skip to Section B)

1 Yes (skip to A1a)

8 Refused to answer

9 Do not know

A1a. How do you typically look for information related to opioid overdose? (what resources do you use?)

A1b. What type of information or topics related to opioid use would be most helpful to you?

A1c. Are there any challenges that you encounter when searching for information related to opioid use?

A2. Have you ever visited the CDC website for opioid-related information before?

0 No (skip to Section B)

1 Yes

8 Refused to answer

9 Do not know

A2a. Think of a time you visited this site, were you able to find what you were looking for (did the site contain all the information you need)?

A2b. On a scale of 1 to 5, with 1 being very difficult to use and 5 being very easy to use, how would you rank the site?

2. INTERVIEWER INSTRUCTIONS

(Participants will be prompted to complete tasks independently at first, although interviewer will step in after 2 failed attempts in completing any task.)

Suggested Probing Questions:

Questions/prompts for each of the tasks

How helpful is the information provided? Why did you choose that option? What do you think should be changed? What else do you think might be useful? Was there information presented that was difficult to understand? Which parts (if any) surprised you? How easy or hard did you think this section was to use? What did you think about the layout of the information? Overall, what are your impressions? Would you use this feature in the management of your own health? Why or why not? 

Neutral prompts

That’s interesting; could you say a bit more about that? What are you thinking now? What makes you say that? Could you tell me more about that? Why do you think that? What do you think about that?

Common questions to ask after each task

How easy was it for you to complete the task? What did you think was easy about it? What did you think was difficult about it? What parts confused you?]

We are now going to ask you to share your screen and pull up a website link that we will send you.

To share your screen:

1. Look for the GoToMeeting Control Panel on your screen.

2. Windows Users: Click on the Screen icon on your toolbar. This pulls up a list of screen-sharing options for you to pick from. Click Share when you’re ready.

Mac Users: In the Screen Sharing pane, use the drop-down menu to select a screen-share option. Click the Play button above the drop-down menu to begin.

Next, we’re going to send you a website link (https://www.cdc.gov/drugoverdose/providers/index.html). [Notetaker sends link via chat.] Please pull up a browser to a blank page, like Google, and insert the link into your browser. Please do not click on anything on the page until I give you instructions to do so.

3. PARTICIPANT TASKS (30 minutes)

C1. This is the landing page for CDC’s opioid information site.

a. Please give me your initial reactions to this page and its layout.

b. What do you think of the menu?

c. Without clicking on anything, please describe the sections you see and what kind of information you would expect to see in them.

d. If you were exploring, what would you click on first?

C2. Next, I am going to ask you to complete various tasks in this site. Remember that there are no wrong answers. If a task is difficult to complete, that just lets us know that the website needs to be improved.

You can let me know when you feel you’ve accomplished the task.

I would like you to: Find quick guidance on when and how to prescribe opioids for chronic pain.

1. [PRE] Where would you normally go to find information like this? Why?

2. [PRE] What kinds of information or resources will you be looking for?

3. What was easy/difficult about this task?

4. How helpful is the information provided? Was there information presented that was difficult to understand?

5. What do you think should be changed? What else do you think might be useful?

6. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to STAY ON CURRENT PAGE]

C3. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

I would like you to: Look for training information on the risks and benefits or prescription opioids.

a. What was easy/difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to STAY ON CURRENT PAGE]

C4. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

I would like you to: Look for webinars on how to effectively communicate with patients about opioid use.

a. What was easy/difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to GO BACK TO THE LANDING PAGE]

C5. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

I would like you to: Find out how the Prescription Drug Monitoring Programs (PDMPs) help you determine a safe dosage of opioids for patients for chronic pain.

a. What was easy/difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to GO BACK TO THE LANDING PAGE]

C6. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

I would like you to: Find additional information on CDC’s role in the opioid overdose pandemic.

a. What was easy/difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

4. POST-TEST QUESTIONS

D1. System usability scale (SUS)

I am going to read several statements and I want you to tell me how much you agree or disagree with them. Please rank these statements on a scale of 1 to 5 with 1 being ‘strongly disagree’ and 5 being ‘strongly agree.’

1= strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree

	Item	Strongly disagree				Strongly agree
1*	I think that I would like to use this website frequently	1	2	3	4	5
2	I found this website unnecessarily complex	1	2	3	4	5
3*	I thought the website was easy to use	1	2	3	4	5
4	I think that I would need the support of a technical person to be able to use this website	1	2	3	4	5
5*	I found various functions in this website were well integrated	1	2	3	4	5
6	I thought there was too much inconsistency in this website	1	2	3	4	5
7*	I would imagine that most people would learn to use this website very quickly	1	2	3	4	5
8	I found this website very cumbersome to use	1	2	3	4	5
9*	I felt very confident using this website	1	2	3	4	5
10	I needed to learn a lot of things before I could get going with this website	1	2	3	4	5

SUS scores have a range of 0 to 100. To calculate the SUS score, first sum the score contributions from each item. Each item's score contribution will range from 0 to 4. For items 1,3,5,7, and 9 the score contribution is the scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus the scale position. Multiply the sum of the scores by 2.5 to obtain the overall value of SU.

D2. Outcomes

	Strongly disagree				Strongly agree
I intend to continue using this website	1	2	3	4	5
I want to continue using website rather than discontinue	1	2	3	4	5
I would recommend this website to someone who seeks my advice	1	2	3	4	5
I consider this website to be my first choice for looking up marijuana-related information	1	2	3	4	5

D3. On a scale of 1 to 5, with 1 being very difficult to use and 5 being very easy to use, how would you rank the website?

D4. Do you have any last comments about the website or your experiences with it?

That concludes our session today. Thank you for your time and feedback. Your input is very valuable in helping us improve the website.