0920-1050 Healthcare Provider - Acute Pain Moderator Guide

Confidential and Proprietary

CDC ACUTE PAIN TOOL WEB USABILITY TEST TASKS AND TEST QUESTIONS

OMB Control No. 0920-1050

Exp. Date 05/31/2022

Public reporting burden of this collection of information is estimated to average 45 minutes per respondent. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB Control Number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to CDC/ATSDR Information Collection Review Office, 1600 Clifton Road NE, MS D-74, Atlanta, Georgia, 30333; ATTN: PRA (0920-1050).

Hello, [participant name]. My name is [moderator name] and I’m going to walk you through this session.

We’re interested in seeing how healthcare providers and public health practitioners search and use opioid guidelines and resources on CDC’s website.

I want to make it clear right away that we are not testing how well you can use the website. We’re testing the website to understand how easy or difficult it is to use. You can’t do anything wrong here. My job is to find out what’s wrong with it or where it can be improved. If you think the website is confusing or difficult to use, we need to know honestly what you think so that we can inform designers and developers of this website on how to make it better.

We are interested in how you use the website, which is why we are video recording your computer screen and asking you what’s going through your mind as you navigate. We are also audio-recording our conversation. The recordings will be used only to help us figure out how to improve the website, and it won’t be seen by anyone except the people working on the project.

Next, I am going to take a moment to ask you to confirm that you have been notified about your privacy, that we are recording this conversation, and that your participation is voluntary. This information was all included in the consent form that was emailed to you.

I am going to start recording our conversation now so that we have a record of your answers.

[Moderator begins recording]

For the record, can you confirm that you have been notified about your privacy, that we are recording this conversation, and that your participation is voluntary?

[Only continue conversation after the participant has provided verbal confirmation]

Do you have any questions before we begin?

1. PRE-TEST QUESTIONS (10 minutes)

A1. Have you looked for guidelines and resources related to acute pain treatment before?

0 No (skip to Section B)

1 Yes (skip to A1a)

8 Refused to answer

9 Do not know

A1a. How do you typically look for information related to acute pain treatment? (what resources do you use?)

A1b. What type of information or topics related to acute pain management would be most helpful to you?

A1c. Are there any challenges that you encounter when searching for information related to acute pain?

A2. Have you ever visited the CDC website for acute pain treatment and information before?

0 No (skip to Section B)

1 Yes

8 Refused to answer

9 Do not know

A2a. Think of a time you visited this site, were you able to find what you were looking for (did the site contain all the information you need)?

A2b. On a scale of 1 to 5, with 1 being very difficult to use and 5 being very easy to use, how would you rank the site?

2. INTERVIEWER INSTRUCTIONS

(Participants will be prompted to complete tasks independently at first, although interviewer will step in after 2 failed attempts in completing any task.)

Suggested Probing Questions:

Questions/prompts for each of the tasks

• How helpful is the information provided?

• Why did you choose that option?

• What do you think should be changed?

• What else do you think might be useful?

• Was there information presented that was difficult to understand?

• Which parts (if any) surprised you?

• How easy was this section to use?

• How hard was this section to use?

• What did you think about the layout of the information?

• Overall, what are your impressions?

• Would you use this feature in the management of your own health? Why or why not? 

Neutral prompts

• That’s interesting; could you say a bit more about that?

• What are you thinking now?

• What makes you say that?

• Could you tell me more about that?

• Why do you think that?

• What do you think about that?

Common questions to ask after each task

• How easy was it for you to complete the task?

• What did you think was easy about it?

• What did you think was difficult about it?

• What parts confused you?

We are now going to ask you to share your screen and pull up a website link that we will send you.

To share your screen:

1. Look for the GoToMeeting Control Panel on your screen.

2. Windows Users: Click on the Screen icon on your toolbar. This pulls up a list of screen-sharing options for you to pick from. Click Share when you’re ready.

Mac Users: In the Screen Sharing pane, use the drop-down menu to select a screen-share option. Click the Play button above the drop-down menu to begin.

Next, we’re going to send you a website link (https://www.cdc.gov/acute-pain/). [Notetaker sends link via chat.] Please pull up a browser to a blank page, like Google, and insert the link into your browser. Please do not click on anything on the page until I give you instructions to do so.

3. PARTICIPANT TASKS (30 minutes)

C1. This is the landing page for CDC’s acute pain information site.

a. Please give me your initial reactions to this page and its layout.

b. What do you think of the topic categorization?

c. Without clicking on anything, please describe the sections you see and what kind of information you would expect to see in them.

d. If you were exploring, what would you click on first?

C2. Next, I am going to ask you to complete various tasks in this site. Remember that there are no wrong answers. If a task is difficult to complete, please let us know so the website can be improved.

You can let me know when you feel you’ve accomplished the task.

[Direct Content] A patient with migraine headaches that occur several times a week. They have previously been prescribed Naratriptan, but it didn’t manage their pain. What does this site suggest about prescribing opioids to manage this patient’s pain?

1. [PRE] Where would you normally go to find information like this? Why?

2. [PRE] What kinds of information or resources will you be looking for?

3. What was easy about this task? What was difficult about this task?

4. How helpful is the information provided? Was there information presented that was difficult to understand?

5. What do you think should be changed? What else do you think might be useful?

6. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to STAY ON CURRENT PAGE]

C3. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

[Not Present] A patient with a third-degree burn needs pain management, but there is a concern about prescribing high doses of opioid drugs. Are there other treatments that can be used in combination with opioids that support the patient but limit the opioid dose?

1. Where else, outside of this site could you look to help complete this task?

2. What do you think should be changed? What else do you think might be useful?

[After participant completes task and answers all questions, ask them to STAY ON CURRENT PAGE]

C4. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

[Direct Content] A teen has acute dental pain and plans to see a dentist in about a week. What risks does the site list about prescribing opioids in this case?

a. What was easy about this task? What was difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

[After participant completes task and answers all questions, ask them to GO BACK TO THE LANDING PAGE]

C5. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

[Not Present] A patient has been prescribed opioids for chronic pain and is about to undergo surgical treatment that will require pain management. Find guidelines for managing patients in this situation.

a. Can you use the site search to find this information?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

[After participant completes task and answers all questions, ask them to GO BACK TO THE LANDING PAGE]

C6. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

[Interpretation] A 25-year-old patient with acute pain in their lower left back needs pain management. How many days do these guidelines suggest they be prescribed opioids?

a. What was easy about this task? What was difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

C7. We are going to move on to a new task. Please do not start clicking on anything until I tell you to do so.

[Generalizability] A patient asks if opioids would be an effective treatment for their sprained thumb. What information from this site could you share with them? The patient is on Facebook and you have their email address.

a. What was easy about this task? What was difficult about this task?

b. How helpful is the information provided? Was there information presented that was difficult to understand?

c. What do you think should be changed? What else do you think might be useful?

d. What did you think about the layout of the information?

4. POST-TEST QUESTIONS

D1. System usability scale (SUS)

I am going to read several statements and I want you to tell me how much you agree or disagree with them. Please rank these statements on a scale of 1 to 5 with 1 being ‘strongly disagree’ and 5 being ‘strongly agree.’

1= strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree

	Item	Strongly disagree				Strongly agree
1*	I think that I would like to use this website frequently	1	2	3	4	5
2	I found this website unnecessarily complex	1	2	3	4	5
3*	I thought the website was easy to use	1	2	3	4	5
4	I think that I would need the support of a technical person to be able to use this website	1	2	3	4	5
5*	I found various functions in this website were well integrated	1	2	3	4	5
6	I thought there was too much inconsistency in this website	1	2	3	4	5
7*	I would imagine that most people would learn to use this website very quickly	1	2	3	4	5
8	I found this website very cumbersome to use	1	2	3	4	5
9*	I felt very confident using this website	1	2	3	4	5
10	I needed to learn a lot of things before I could get going with this website	1	2	3	4	5

SUS scores have a range of 0 to 100. To calculate the SUS score, first sum the score contributions from each item. Each item's score contribution will range from 0 to 4. For items 1,3,5,7, and 9 the score contribution is the scale position minus 1. For items 2,4,6,8 and 10, the contribution is 5 minus the scale position. Multiply the sum of the scores by 2.5 to obtain the overall value of SU.

D2. Outcomes

	Strongly disagree				Strongly agree
I intend to continue using this website	1	2	3	4	5
I want to continue using website rather than discontinue	1	2	3	4	5
I would recommend this website to someone who seeks my advice	1	2	3	4	5

D3. On a scale of 1 to 5, with 1 being very difficult to use and 5 being very easy to use, how would you rank the website?

D4. Do you have any last comments about the website or your experiences with it?

That concludes our session today. Thank you for your time and feedback. Your input is very valuable in helping us improve the website.