OMB No.: 0925-0642
Expiration Date: 9/30/2014
Collection of this information
is authorized by The Public Health Service Act, Section 411 (42 USC
285a). Rights of study participants are protected by The Privacy Act
of 1974. Participation is voluntary, and there are no penalties for
not participating or withdrawing from the study at any time. Refusal
to participate will not affect your benefits in any way. The
information collected in this study will be kept private under the
Privacy Act. Names and other identifiers will not appear in any
report of the study. Information provided will be combined for all
study participants and reported as summaries. You are being
contacted by telephone to complete this instrument so that we can
improve the website.
Public reporting burden for
this collection of information is estimated to average 60 minutes
per response, including the time for reviewing instructions,
searching existing data sources, gathering and maintaining the data
needed, and completing and reviewing the collection of information.
An agency may not
conduct or sponsor, and a person is not required to respond to, a
collection of information unless it displays a currently valid OMB
control number.
Send comments regarding this burden estimate or any other aspect of
this collection of information, including suggestions for reducing
this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive,
MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not
return the completed form to this address.
[Note: This document is intended to serve as a guide to the facilitator. ]
Thank you, [name], for agreeing to be a part of this study.
I’d like you to know that [observer name] is on the call today to take notes. He’ll observe while you and I work together. With your permission, we’ll also record our session together, just to make sure that [observer name] and I don’t miss anything you say. Is that okay? [Wait for response]
As was mentioned when we scheduled this appointment with you, we expect the process will take between 30 minutes and an hour.
Today we’ll be looking at a website that was created for the Cancer Imaging Program, which is part of the National Cancer Institute. I’m going to ask you to explore the website and then try to do some tasks on the site.
As we go through the site, please keep in mind that we are testing the website and not you—there are no right answers or procedures. If you can’t find certain information or don’t know how to do something, it isn’t your fault. The website is supposed to be easy to use. If it isn’t easy for you to use, that tells me we need to fix something on the website.
I want you to know that our company was not involved in the development of this website or its content, so don’t worry that anything you say might hurt my feelings or offend in any way. Any comments you have, either positive or negative, will be useful, so please feel free to tell me what you think.
After we’ve finished the various tasks, I’ll give you some time to ask me anything you’d like. Do you have any questions for me now?
We have invited people from different backgrounds to help us usability test the website, so I’d like you to ask you a little bit about yourself.
Where do you work? [If an academic institution, we will determine size (using the Carnegie Classification of Institutions of Higher Education, which is based on full-time equivalent (FTE) enrollment.)]
What is your role there?
If researcher/medical practitioner/clinician: Are you currently or have you ever received any Federal funding for research?
If so, are you receiving or have you ever received any funding from the National Institutes of Health (NIH)?
Are you receiving or have you ever received any funding from the National Cancer Institute (NCI)?
If yes, have you ever received funding from the Cancer Imaging Program?
If yes, are you currently funded by CIP?
Are you a new or “junior” investigator?
If unsure, ask: Have you ever been the PI on a project that competed successfully for an NIH-supported research grant?
If not, then junior investigator.
If yes, what type of mechanism? (If F, K, L, R00, R03, R15, R21, R25, R34, R36, R41, R43, R55, R56, R90, RL5, RL9, SC2, SC3, T, or X01, then junior investigator. Otherwise, not junior investigator.)
Have you used the CIP website to find information in the past?
If no, skip to Setting Up the Usability Tasks.
If yes, what did you look for on the site?
Have you visited the site more than once?
If yes: would you say you visit the site rarely (two or three times total), occasionally (every few months), or on a regular basis?
If regularly, once a month? Once a week?
If current or former CIP grantee, did you use the site when first applying for a CIP-administered grant?
[Explain how we will be able to see their screen and what they do. Transfer control of screen to them and work with them until we are satisfied the system is working properly.]
Please go to the NCI Cancer Imaging Program (CIP) website. [Record process used, including search engine and key words.]
If not successful, direct them to cancer.gov and ask them to find it from there.
If still not successful, direct them to http://dctd.cancer.gov/ and ask them to find it from there.
I’m going to give you a few minutes to poke around on the site on your own so you can become familiar with it. Feel free to ask any questions or make comments while you do that.
[Observe and record where they go first. Observe what they do for about 3 minutes or until they seem to be ready to move on.]
What is your impression of the CIP homepage?
What prompted you to go to [the first page they visited]?
Now I’m going to ask you to use the site to answer some questions. Please note that we will be measuring time-on-task and number of clicks you make for each task. This will help us measure how much work it takes to complete each task. From this point forward, please avoid “exploring” the site beyond what is necessary for each specific task. OK?
It is important that you think aloud while you work. Your thoughts and reactions will help us know whether the site is working well and identify any possible problem spots.
If you feel lost or cannot answer a question, please let me know. Although I won’t be able to give you any suggestions or hints, I can repeat the question.
Let’s begin with the first task. [Make sure they start at the homepage.]
What is the CIP’s mission statement?
Who is the Associate Director of the CIP?
What NCI division is the Cancer Imaging Program a part of?
Does the CIP website offer information specifically for patients?
How many branches does the CIP have?
What is the name of the most recently released CIP funding initiative?
Where was the Cancer Imaging Research Camp held in 2012?
Find information about NIH funding mechanisms such as P20 exploratory grants.
What is the expiration date for PAR-11-150?
CIP provides imaging guidelines for clinical trials on the web site. How were these guidelines developed?
What is one way that cancer researchers can use the data in The Cancer Imaging Archive (TCIA)?
What are the dates and location for the 2013 midwinter meeting of the Society of Nuclear Medicine?
Name two of the working groups in the Quantitative Imaging Network.
Find one CIP publication that appeared in a peer-reviewed journal.
What group is collaborating with the CIP in the Phase 2 N01 Program?
Download a Frequently Asked Questions (FAQ) document for one of the Investigational New Drug Applications (INDs) that CIP has created.
What is the CIP’s mission statement?
Who is the Associate Director of the CIP?
What NCI division is the Cancer Imaging Program a part of?
Does the CIP website offer information specifically for patients?
How many branches does the CIP have?
Where was the Cancer Imaging Research Camp held in 2012?
CIP provides imaging guidelines for clinical trials on the web site. How were these guidelines developed?
What are the dates and location for the 2013 midwinter meeting of the Society of Nuclear Medicine?
What kinds of groups or organizations sponsor clinical imaging trials?
Show me where you would look for an imaging clinical trial in which your patients might participate.
How are imaging clinical trials and drug treatment trials different?
What is one way that cancer researchers or members of the public can use the data in The Cancer Imaging Archive (TCIA)?
How long does a virtual colonoscopy examination take to complete compared to a conventional colonoscopy?
What is the purpose of sentinel node mapping for breast cancer staging?
What is the CIP’s mission statement?
Who is the Associate Director of the CIP?
What NCI division is the Cancer Imaging Program a part of?
Does the CIP website offer information specifically for patients?
How many branches does the CIP have?
What are two of the five uses for cancer imaging described on the website?
How long does a virtual colonoscopy examination take to complete?
How is digital mammography different from conventional mammography?
Name one kind of nuclear imaging.
Can humans hear the sound waves that are used to perform an ultrasound?
Show me where you would look for help finding a clinical trials in which to participate.
What is one way that patients and members of the general public can use the data in The Cancer Imaging Archive (TCIA)?
[RETRIEVE CONTROL OF SCREEN. DISPLAY SCALE AND 10 STATEMENTS.]
I’d like to collect a bit of information from you using a standardized tool before we discuss your experience. I’m going to read a statement and ask you whether you agree or disagree, using a scale from 1 to 5 where 1 equals strongly disagree and 5 equals strongly agree. OK?
I think that I would like to use this website frequently.
I found the website unnecessarily complex.
I thought the website was easy to use.
I think that I would need the support of a technical person to be able to use this website.
I found the various functions in this website were well integrated.
I thought there was too much inconsistency in this website.
I would imagine that most people would learn to use this website very quickly.
I found the website very cumbersome to use.
I felt very confident using the website.
I needed to learn a lot of things before I could get going with this website.
Now I’d like to ask you a few specific questions about your impressions of the CIP site.
When you think about how the CIP is laid out and how it works, what is your overall impression of the site?
What did you like most about the way the website looks and works?
What did you like least about the way the website looks and works?
Was anything missing from the site that you expected to see? [Probe: content, features, functions]
Was there anything on the site that you did not expect to see? [Probe: links, pages]
Do you feel this site is current? Why/why not?
OK, we’ve finished the usability testing. Do you have any further questions or comments?
Thank you again for your participation.
| File Type | application/msword |
| File Title | Facilitator’s Guide for Usability Test |
| Author | Kathy Sedgwick |
| Last Modified By | Vivian Horovitch-Kelley |
| File Modified | 2012-12-03 |
| File Created | 2012-11-13 |