2 CCT Usability Script

OMB No.:0925- 0642

Expiration Date: 9/30/2014

Collection of this information is authorized by The Public Health Service Act, Section 411 (42 USC 285a). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will not appear in any report of the study. Information provided will be combined for all study participants and reported as summaries. You are being contacted by telephone to complete this instrument so that we can improve the website.

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Facilitator’s Guide for Usability Test

[Note: This document is intended to serve as a guide to the facilitator. ]

Introduction and Warm Up

Thank you, [name], for agreeing to be a part of this study, which is being sponsored by the National Institutes of Health (NIH).

I’d like you to know that [observer name] is on the call today to take notes. He’ll observe while you and I work together. With your permission, we’ll also record our session together, just to make sure that [observer name] and I don’t miss anything you say. Is that okay? [Wait for response]

As was mentioned when we scheduled this appointment with you, we expect the process will take between 30 minutes and an hour.

Today we’ll be looking at a website that was created for the Center for Cancer Training, which is part of the National Cancer Institute. I’m going to ask you to explore the website and then try to do some tasks on the site.

As we go through the site, please keep in mind that we are testing the website and not you—there are no right answers or procedures. If you can’t find certain information or don’t know how to do something, it isn’t your fault. The website is supposed to be easy to use. If it isn’t easy for you to use, that tells me we need to fix something on the website.

I want you to know that our company was not involved in the development of this website or its content, so don’t worry that anything you say might hurt my feelings or offend in any way. Any comments you have, either positive or negative, will be useful, so please feel free to tell me what you think.

After we’ve finished the various tasks, I’ll give you some time to ask me anything you’d like. Do you have any questions for me now?

Confirm Experience with CCT Site

I’d like you to confirm your knowledge of the website we are testing today.

Have you ever visited the CCT website?

1. If no, skip to Setting Up the Usability Tasks.

2. If yes, when did you last visit the site?

3. How often do you visit or have you visited the site?

• Once

• Two or three times total

• Occasionally (every few months)

• Monthly

• Weekly

4. If current or former training grantee/fellow/intern, did you use the site when you first applied for a cancer training grant/fellowship/internship?

Setting Up the Usability Tasks – Adobe Connect

[Explain how we will be able to see their screen and what they do. Transfer control of screen to them and work with them until we are satisfied the system is working properly.]

Findability of the Site

• Please go to the NCI Center for Cancer Training (CCT) website. [Record process used, including search engine and key words.]

  • If not successful, direct them to cancer.gov and ask them to find it from there.

  • If still not successful, direct them to http://www.cancer.gov/researchandfunding/cancertraining.

Initial Impression

I’m going to give you a few minutes to poke around on the site on your own so you can become familiar with it. Feel free to ask any questions or make comments while you do that.

[Observe and record where they go first. Observe what they do for about 3 minutes or until they seem to be ready to move on.]

• What is your impression of the CCT homepage?

• What prompted you to go to [name the first page they visited]?

Tasks

Now I’m going to ask you to use the site to answer some questions. Please note that we will be measuring time-on-task and number of clicks you make for each task. This will help us measure how much work it takes to complete each task. From this point forward, please avoid “exploring” the site beyond what is necessary for each specific task. OK?

It is important that you think aloud while you work. Your thoughts and reactions will help us know whether the site is working well and identify any possible problem spots.

If you feel lost or cannot answer a question, please let me know. Although I won’t be able to give you any suggestions or hints, I can repeat the question.

Let’s begin with the first task. [Make sure they start at the homepage.]

TASKS FOR NIH PROFESSIONALS

1. What is the CCT’s mission?

2. Who is the program contact for the Technology Transfer Center?

3. Who is the Branch Chief of the NCI Cancer Training Branch?

4. Are high school students eligible for any NCI fellowships or internships programs?

5. Name two organizations (other than NIH and NCI) that offer cancer research training opportunities.

6. Who administers the CCT extramural training programs?

7. What is the duration of (how many years) the Sallie Rosen Kaplan Fellowship Program?

8. How do you contact the Center for Cancer Training?

9. View the NCI Center for Cancer Training Train for the Future brochure.

10. What was NCI’s rank among top places for training, per results of the 2012 postdoc survey published in The Scientist?

TASKS FOR CURRENT AND PROSPECTIVE TRAINING GRANTEES, FELLOWS, INTERN

1. What is the CCT’s mission?

2. Who is the program director for the Paul Calabresi Award (K12) for Clinical Oncology?

3. Are high school students eligible for any NCI fellowships or internships programs?

4. Name two organizations (other than NIH and NCI) that offer cancer research training opportunities.

5. Find one award for which foreign nationals are eligible.

6. What is the duration of (how many years) the Sallie Rosen Kaplan Fellowship Program?

7. Name one K Award for established investigators in translational research.

8. Can people apply for a K Award electronically?

9. When is the next application deadline for T32s (Ruth L. Kirschstein NRSA Institutional Research Training Grants)?

10. What was NCI’s rank among top places for training, per results of the 2012 postdoc survey published in The Scientist?

Standard Usability Scale

[RETRIEVE CONTROL OF SCREEN. DISPLAY SCALE AND 10 STATEMENTS.]

I’d like to collect a bit of information from you using a standardized tool before we discuss your experience. I’m going to read each statement and ask you whether you agree or disagree, using a scale from 1 to 5 where 1 equals strongly disagree and 5 equals strongly agree. OK?

1. I think that I would like to use this website frequently.

2. I found the website unnecessarily complex.

3. I thought the website was easy to use.

4. I think that I would need the support of a technical person to be able to use this website.

5. I found the various functions in this website were well integrated.

6. I thought there was too much inconsistency in this website.

7. I would imagine that most people would learn to use this website very quickly.

8. I found the website very cumbersome to use.

9. I felt very confident using the website.

10. I needed to learn a lot of things before I could get going with this website.

Final Impressions

Now I’d like to ask you a few specific questions about your impressions of the CCT site.

1. When you think about how the CCT is laid out and how it works, what is your overall impression of the site?

2. What did you like most about the way the website looks and works?

3. What did you like least about the way the website looks and works?

4. Was anything missing from the site that you expected to see? [Probe: content, features, functions]

5. Was there anything on the site that you did not expect to see? [Probe: links, pages]

6. Do you feel this site is current? Why/why not?

Wrap up

OK, we’ve finished the usability testing. Do you have any further questions or comments?

Thank you again for your participation.

4

NOVA Research Company – 2/5/13