Form 1 Facilitator's Guide - Catalogue

Attachment 4. Facilitator’s Guide for NCCOR Catalogue of Surveillance System

[Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally.]

OMB No. 0925-0642

Expiry Date: 09/30/2014

Collection of this information is authorized by The Public Health Service Act, Section 411 (42 USC 285a). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will not appear in any report of the study. Information provided will be combined for all study participants and reported as summaries. You are being contacted by phone (or in person) to complete this instrument so that we can improve the website.

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Pre-Test

[Administer the informed consent and video release form.]

Introduction

Thank you for agreeing to participate in this study. We are conducting a study sponsored by the National Institutes of Health; we will be looking at a website today that was developed by the National Collaborative on Childhood Obesity Research, or NCCOR. Do you have any questions for me before we get started?

I’m going to ask you to explore the website and try to do some tasks with the site. Then I’d like to get your opinion about it. You and I will work on this together for about an hour. As we are working, some other people in another room may watch your monitor screen and help me out by taking notes on our work session.

There are two important things you should keep in mind while you work with this website:

• First, I did not design the website so don’t worry about hurting my feelings with your comments. If there are problems with the website or design, I would like to discuss them with you to find out how we can find ways to make the website better.

• Second, we are assessing the website and not you, so you cannot make any mistakes. We want the website to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

Any comments you have, either positive or negative, will help make the product better so feel free to tell me your honest opinions and comments. After we finish the tasks, I’ll ask you a few questions and then give you some time to provide your own comments or ask me questions. Do you have any questions for me before we get started?

One more thing before we get started, I’d like to let you know that as you’re using the site, if you have questions as we go along, feel free to ask. I may not answer your questions right away in order to see how well you can figure out things for yourself, but it is valuable to know if you do have questions about the site.

Background Information

We have invited people with different backgrounds to participate in this activity so I’d like you to tell me where you work and what your role is. Do you conduct research in childhood obesity? If so, which of the following do you consider your primary research area?

• Diet

• Physical activity

• Geography

• Disease Outcomes, e.g., diabetes, cardiovascular disease, cancer

• Obesity-related policy

• Other (details provided by recipient)

Tasks

[Facilitator should ask about specific webpages and website sections as they are encountered. If users explore areas listed as a task below, the facilitator should get the feedback during their exploration. Areas of the site that seem to be of interest and then are dropped should be asked about.]

[Start Participant on landing page for the Catalogue of Surveillance Systems tool – nccor.org/projects/catalogue/index.php]

1. Have you used the Catalogue before?
   
   

   1. [IF YES] When did you first access it? When did you last access it? How often do you use the site?
      
      

   2. What did you use it for? Did you find what you were looking for?

3. [IF NO] Take a moment to look around and tell me what you think you, personally, might use this for.
   
   

1. Go ahead and use this tool to try to find a surveillance system (SS) or multiple surveillance systems that might be relevant to your work.
   
   

2. [Follow-up questions on how to choose a SS after participant has narrowed the list]
   
   

   1. How would you decide which surveillance system(s) to look at?
      
      

   2. Did you notice the “Compare” feature? How would you use it?
      
      

4. Let’s take a look at one of the surveillance systems more closely
   
   

   1. What information would you need to know to help you decide whether or not the surveillance system would be useful to you?
      
      

   2. Go ahead and take a look at the different tabs and tell me if it’s clear what it’s showing you.
      
      

   3. How would you get to the actual surveillance system?

SPECIFIC TASKS TO GIVE USERS IF THEY HAVEN’T EXPLORED THE FILTERS

5. How would you determine whether there are any surveillance systems that collect physical activity data on 2-4 year olds at the local level?

Goal: Generate list of SS with multiple given characteristics

FOLLOW-UP QUESTIONS ON SPECIFIC FEATURES

6. [Follow-up questions on the filters]

   1. What do you think about the specific filters that are used here?

   2. Is it clear what they all mean? (If any of the labels are unclear, see if participant notices the question mark icon)

   3. Are there any that you would remove?

   4. Are there any that you would add?

   5. What do you think about the way the filters work?

   6. If you had several filters selected, how would you remove them all to start fresh?
      
      

7. [Follow-up questions on the search box]

   1. When would you use the search box, as opposed to the filters?

   2. What do you think is searched when you add in a search term? (e.g., the surveillance system name, the information about it, specific fields, etc.)

   3. [If a previous user] Have you used the search before, and if so, how well did it work?
      
      

8. Are there any additional features that would be useful to include in this tool?

Follow-up Questions

Next, I have a few more questions about your experience with the website.

1. What was your overall impression of the Catalogue?

2. What did you like most about the way the site looks and works?

3. What did you dislike about the way the site looks and works?

4. Was there anything about the website that was unexpected or was initially confusing to you?

5. Do you have any other comments, questions or suggestions for improving the website?

Wrap-up

This concludes the usability test. Thank you for your participation.

4