2 Facilitator's Guide - Measures

Attachment 5. Facilitator’s Guide for NCCOR Measures Registry

[Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally.]

OMB No. 0925-0642

Expiry Date: 09/30/2014

Collection of this information is authorized by The Public Health Service Act, Section 411 (42 USC 285a). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will not appear in any report of the study. Information provided will be combined for all study participants and reported as summaries. You are being contacted by phone (or in person) to complete this instrument so that we can improve the website.

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Pre-Test

[Administer the informed consent and video release form.]

Introduction

Thank you for agreeing to participate in this study. We are conducting a study sponsored by the National Institutes of Health; we will be looking at a website today that was developed by the National Collaborative on Childhood Obesity Research, or NCCOR. Do you have any questions for me before we get started?

I’m going to ask you to explore the website and try to do some tasks with the site. Then I’d like to get your opinion about it. You and I will work on this together for about an hour. As we are working, some other people in another room may watch your monitor screen and help me out by taking notes on our work session.

There are two important things you should keep in mind while you work with this website:

• First, I did not design the website so don’t worry about hurting my feelings with your comments. If there are problems with the website or design, I would like to discuss them with you to find out how we can find ways to make the website better.

• Second, we are assessing the website and not you, so you cannot make any mistakes. We want the website to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

Any comments you have, either positive or negative, will help make the product better so feel free to tell me your honest opinions and comments. After we finish the tasks, I’ll ask you a few questions and then give you some time to provide your own comments or ask me questions. Do you have any questions for me before we get started?

One more thing before we get started, I’d like to let you know that as you’re using the site, if you have questions as we go along, feel free to ask. I may not answer your questions right away in order to see how well you can figure out things for yourself, but it is valuable to know if you do have questions about the site.

Background Information

We have invited people with different backgrounds to participate in this activity so I’d like you to tell me where you work and what your role is. Do you conduct research in childhood obesity? If so, which of the following do you consider your primary research area?

• Diet

• Physical activity

• Geography

• Disease Outcomes, e.g., diabetes, cardiovascular disease, cancer

• Obesity-related policy

• Other (details provided by recipient)

Tasks

[Facilitator should ask about specific pages and sections as they are encountered. If users explore areas listed as a task below, the facilitator should get the feedback during their exploration. Areas of the site that seem to be of interest and then are dropped should be asked about.]

[Start participant on landing page for the tool – http://nccor.org/projects/measures/index.php]

1. Have you used the Measures Registry before?

   1. [IF YES] What did you use it for?

      1. How often do you use the tool?
         
         

   2. [IF NO] Take a moment to look around and tell me what you think you, personally, might use this for.
      
      

2. How would you go about finding out what measures are included in this Registry that relate to your particular work?

   1. How would you decide which measures to look at?
      
      

   2. Did you notice the “Compare” feature? How would you use it?
      
      

3. Let’s take a look at one of the measures more closely

   1. What information would you need to know to help you decide whether or not this measure would be useful to you?
      
      

   2. Go ahead and take a look at each of the tabs and tell me if it’s clear what it’s showing you.
      
      

   3. If you decided you wanted to use this measure, what would you do next?
      
      

4. How would you decide whether or not to use an already existing measure versus creating your own or adapting one to fit your particular needs?
   
   

5. If there were multiple measures available that could work for you, how would you decide between them?

   1. Is there anything that would be helpful to include that would help you make that decision?

FOLLOW-UP QUESTIONS ON SPECIFIC FEATURES

6. [Follow-up questions on the filters]

   1. What do you think about the specific filters that are used here?

   2. Is it clear what they all mean? (If any of the labels are unclear, see if participant notices the question mark icon)

   3. Are there any that you would remove?

   4. Are there any that you would add?

   5. What do you think about the way the filters work?

   6. If you had several filters selected, how would you remove them all to start fresh?
      
      

7. [Follow-up questions on the search box]

   1. When would you use the search box, as opposed to the filters?

   2. What do you think is searched when you add in a search term? (e.g., the measure names, the information about a measure, specific fields, etc.)

   3. [If a previous user] Have you used the search before, and if so, how well did it work?
      
      

8. Are there any additional features that would be useful to include in this tool? For example…

   1. Tool Kits containing recommended measures

   2. Interactive ‘how to’ guides

   3. Scores or rankings for all measures

   4. Opportunities to comment on measures

Post Test Interview Questions

Next, I have a few specific questions about your experiences with the website.

1. What was your overall impression of the Catalogue?

2. What did you like most about the way the site looks and works?

3. What did you dislike about the way the site looks and works?

4. Was there anything about the website that was unexpected or was initially confusing to you?

5. Do you have any other comments, questions or suggestions for improving the website?

Wrap-up

This concludes the usability test. Thank you for your participation.

4