0920-20QB Usability_Testing_Discussion_Guide

OMB Control No. 0920-1050

Exp. Date 05/31/2022

The public reporting burden of this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to - CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta, Georgia 30333 ATTN: PRA (0920-1050)

CDC-034 WISQARS Discussion Guide | Prepared by Jeff Lindsay

October 2020

Discussion Guide

The objective of the Discussion Guide is to provide a loose outline of the conversation for the interviews. Questions are not meant to be read word for word, but rather a guide to ensure all relevant points are discussed during the time allotted.

Introduction and Informed Consent

Hello, my name is Jeff Lindsay.

Our discussion today is being sponsored by the Centers for Disease Control and Prevention (CDC), National Center for Injury Prevention and Control (NCIPC). Our purpose in talking with you today is to hear from you on your thoughts and opinions about a website that provides fatal and nonfatal injury, violent death and cost of injury data from a variety of sources.

Before we begin, I would like to remind you that your participation in this discussion is voluntary and you may end it at any time or feel free to skip any question you feel uncomfortable answering.

To ensure accuracy, our discussion today is being audio and video recorded. At no time will your name appear in any reports or presentations that result from this work.

Do you give your permission to be audio and video recorded today? (If participant says no, thank them and terminate the interview).

Also, some of my research team is observing our session to help me prepare my report. Do you give your permission for them to listen in on our chat? (If participant says no, thank them and terminate the interview). Thanks. A few quick questions and then we’ll get started.

1. How does data related to health and injuries play a role in what you do?

   1. What do you use the data for?

   2. What sources do you typically get it from?
      
      

2. Have you visited the CDC’s WISQARS website before? For what purpose?

   1. How often do you visit the site?

   2. When was the last time you visited the site?

WISQARS Site

Now I’d like to show you a site where you can view injury data and hear your thoughts about it. Please try to think out loud as you go, as it helps me to follow along.

[Researcher will start Respondent at the TOS page to allow the Respondent to encounter the welcome tool tip in case they wish to engage with it.]

1. What is your first impression of the site?

2. What kind of information do you expect to find on this site? Please explain. How current would you expect the information to be?

3. What would you do here first? Why?

4. Please take a few moments to look around the site. Again, please try to think out loud as you do. [Strategist will note where Respondent chooses to explore; where they go, what they look for, their reactions, etc.]

5. What word or phrase do you think best describes what you’ve seen on the site so far? Explain.

WISQARS Fatal Injury Data Visualization - Mapping

6. Let’s say you’ve come to the site because you want to see a map showing fatal injury data for the United States from 2017 to 2018. What would you do if you were looking for this information?

7. What information are you able to find? Is this information helpful? Why or why not?

8. What do you think about how the information is being displayed? How would you like to see it shared? What types of visualizations are the most useful for you?

9. If you wanted to see the data displayed as a counts instead of age adjusted rates, how would you view that data?

10. What if you wanted to see the data at the county level instead of the state level? If you wanted to see metro and non-metro counties?

11. If you wanted to change the color theme of the map from it’s current color to blue, how would you do that?

12. What if you wanted to see just the counties in the south eastern US? What if you only wanted to see counties in the state of Florida?

13. If you wanted to zoom in to a specific part of the state what would you do?

14. What do you think about the information provided by the legend on the map? What does it mean?

15. If you wanted to see the data as a table instead of the map visualization, how would you do that?

16. What do you think about your ability to narrow the data you look at? Are you able to tell what filters and selections have been applied? Are there any filters you would want to have that are not available? If yes which ones?

17. If you wanted to save the data from your search for later or use it with a different program how would you get the data from your search?

18. Could anything be done to improve the visualization or the table? If you aren’t certain what a chart or graph means how would you search for clarification or additional information?

19. If you needed assistance or had questions in general about this website or the information you find here what would you do?

20. How would you start over if you wanted to do an entirely new search?

WISQARS Fatal Injury Data Visualization – Leading Causes of Death

21. Now let’s say you’d like to see the leading causes of death across age groups in the United States. How would you look for this information?

22. What information is being shown? What do you think about how the leading cause information is displayed? How would you want to see this information visualized?

23. If you need to see specifically unintentional injury as a focus what would you do? How useful is this visualization? How should this information be displayed?

24. Now imagine you wanted to focus specifically on cancer as a cause of death. How would you view that data?

25. If you did wanted to reset the trends that are visible in the visualization right now what would you do?

26. What do the numbers in each box indicate? If you wanted to view the data as a rate instead of a count how would you change how it is being displayed?

27. If you wanted to see a bar chart of ICD codes, what would you do? What if you wanted to see it with a specific type of injury?

28. If you needed to narrow down the data to look at a specific age group, what would you do? What if you wanted to look at all deaths or all injuries?

29. What if you wanted to view the data in terms of years of potential life lost? How would you interpret this data? What options do you have when it comes to looking at years of life lost?

30. What do you think about your ability to narrow the data you look at? Are you able to tell what filters have been applied? How would you change them once they are selected? Are there any filters you would want to have that are not available? If yes which ones?

31. What do you think about the leading causes of death visualization overall? What uses would you have for this visualization? What could be done to make it better or more useful?

Debrief

32. Now that you’ve had a chance to explore the site, what do you think of it overall? What word or phrase comes to mind to describe the site?

33. What do you think the site is doing well?

34. What could the site do better?

35. Overall how is the site doing when it comes to visualizing data? Are there any visualizations you liked or didn’t like? Which ones and why?

36. Will you use this site in the future? Why or why not? If yes, how do you see yourself using it? What would bring you back to the site?

37. Would you recommend this site to others? If yes, who? What would you tell them about it?

38. Are there any needs you have when it comes to data or data visualization similar to what we looked at today that feel nothing out there addresses? If yes, what?

39. Do you have any final thoughts or questions you’d like me to pass along?

I am going to step out for a moment to check with my note taker and make sure I did not forget to ask anything.

Researcher will consult with stakeholders about any further questions to ask.

Once follow up questions (if any) are complete:

I have no more questions for you at this time - Thank you for coming in!

7