3 #5 Attachments C2, C3, C4 & C5

Needs Assessment for http://epi.grants.cancer.gov

OMB # 0925-0589-05

Attachment C2: Interviewer’s Guide

Attachment C3: Facilitator’s Guide

Attachment C4: System Usability Scale (SUS)

Attachment C5: Modified Cooper Harper questionnaires.

Attachment C2. Interview Guide

Interviewer’s Guide for

Epidemiology and Genetics Research Program

[This material is only a guide for the website usability contractor (still to be determined) as they are conducting interviews with individuals about the Epidemiology and Genetics Research Program Website to determine their information needs related to cancer epidemiology and genetics research, funding, collaborations, etc. These questions are simply a guide and may not be asked using exactly these words or in exactly this order. These questions will be used as necessary but will primarily follow the natural flow of the conversation. For example, it is possible that based on the flow of conversation, not all of the questions will need to be asked. Alternatively, some of the follow up questions may be answered naturally without being asked.]

I am ___________ and I am conducting interviews for the National Cancer Institute (NCI) with individuals who are current or potential grantees, research collaborators, or other stakeholders in cancer epidemiology and genetics research. I’d like to ask you a few questions about the website without you looking at it. This will help me to know what is so important that you really remember it. Later in the interview, you will be asked to go to the website. So, if you already have the site open, would you please minimize it for now.

1. First, are you familiar with NCI’s Epidemiology and Genetics Research Program Resources Program (EGRP)? If so, how do you use their resources? If not, what information would you potentially be interested in related to NCI’s Epidemiology and Genetics Research Program?

2. For purposes of interview completeness, please tell me your first name and estimate the number of years you’ve been involved with cancer research (or for advocates, cancer advocacy).

3. In your opinion, what is the purpose of the EGRP Website?

4. In your opinion, does it achieve this purpose?

Frequency of Use

5. How frequently do you use the EGRP website?

6. How long have you used the EGRP Website? (over what period of time) If you have not previously used the EGRP Website as a source of information, why not?

7. Do you remember the last time you were there? When was that?

Rationale for Past Website Visits and Use of Resources

8. What was the purpose of your last visit to EGRP’s website? (if applicable)

9. Did you find what you were looking for? Easily?

10. Is that a typical reason why you visit the site?

11. Are there other reasons why you visit the EGRP website?

12. For each of these reasons, what do you do with the resources (information or tools) that you find on the site?

13. Do you have a lot of experience with different resources available through the EGRP website or is your use limited to a small portion of the website? Describe which areas of the website you use.

Information Resources

14. Could you describe for me, in your own words, the type of resources that are available on the website? (If need some prompting, could ask the following):

• What types of information do you think you’ll find on the EGRP website?

• Is this website one of your primary places to look for such information?

• What are the [other] primary places you look for this type of information?

Experience with the Site

15. Tell me how you use the site. Do you always follow the same routine?

    • Let’s open up the EGRP website. Maybe you will be reminded of other issues when you’re looking at the site.

    • Is there anything else you’d like to tell me about using the site?

• Is the material on the site organized so you can find what you want easily?

• Do you ever have any trouble finding what you want?

• Is most of the material you look for under one subheading?

• Have you ever thought about the site organization? Is there an idea you have that might make it better for your work?

• Is there any type of information that is missing from the Website, the kind of information that seems to be appropriate for the site, but isn’t covered?

Other Users

16. Do you know of other individuals or groups of individuals (at your organization or elsewhere) who use the site? If so, do you know if they use the site the same way that you do?

17. Could you tell me what you know about other users and their jobs relative to the material on this website?

Anything Else

18. Is there anything else that you think about the website and your use of it that would help us understand what would make it better, easier, or more convenient for you to use?

Tasks – See Facilitator’s Guide

Thank you for the time and information that you’ve shared with me today. We’re working with the staff in EGRP to revise the website as needed to make it as helpful as possible to its users. Your insights will help us in these efforts.

Attachment C3. Facilitator’s Guide

OMB #: 0925-0589-05

Expiry Date: 05/31/2011

Facilitator’s Guide for

Epidemiology and Genetics Research Program

Public reporting burden for this collection of information is estimated to average 50 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0589-05). Do not return the completed form to this address.

[Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally. ]

Pre-Test

Administer the informed consent form.

Introduction

Thank you for agreeing to participate in this study. Do you have any questions for me before we get started?

You signed a consent form so you know that this session will be audiotaped and video of the computer screens may be recorded. I’d also like you to know that there may be observers helping me by taking notes, but don’t worry about that. You and I will work on this together and they’ll just listen.

We’re going to be looking at a website today for the National Cancer Institute’s (NCI) Epidemiology and Genetics Research Program. I’m going to ask you to explore the website and then try to complete some tasks with the site. Then I’d like to get your opinion about the site.

We have invited persons with a variety of backgrounds to participate in this activity so I’d like you to tell me where you work, how long you’ve been involved with cancer research (or for advocates, how long they’ve been involved in cancer advocacy), and what your role is at your current organization.

Since we’ll be working on a computer today, I want to let you use this computer in a way that it looks like the computer you regularly use. Tell me a website you use frequently. Now does this look too big or too small? [Facilitator adjusts the resolution until it matches for the participant’s normal experience.]

There are two important things you should keep in mind while you work with this website:

• First, I did not design it so you can’t hurt my feelings. If there are problems with the design or content, I would like to discuss them with you to see if we can find a way to make the website better.

• Second, we are evaluating the website and not you, so you cannot make any mistakes. The website is supposed to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

Any comments you have, either positive or negative, will help make the website better, so feel free to tell me whatever is on your mind. After you’re done working, I’ll ask you a few questions and then give you some time to ask me anything you’d like. Do you have any questions for me before we get started?

Initial Impression

[Open the site]

I’m going to let you explore the website on your own for a bit to get used to I (if you aren’t already), but feel free to ask any questions or make comments while you do. I may not answer your questions right away in order to see how well you can figure out things for yourself, but it is valuable to know if you do have questions about the site.

[Observe the participants and note what part of the site they go to first. Allow them to explore freely for a few minutes, but stop them if they seem to show signs of completion regardless of what parts they explore.

After their initial exploration…

• If not offered, prompt for participant’s impression of the main page.

• Ask what prompted them to explore the first section they visited.

Facilitator should ask about specific pages and sections as they are encountered. If users explore areas listed as a task below, the facilitator should get the feedback during their exploration. Areas of the site that seem to be of interest and then dropped should be asked about.]

Tasks

[Explore each of these tasks if the participant did not explore them on their own]

1. Find the following information: As you look for the following information, there may be times when you leave the EGRP website and open an outside site. If you notice that this has happened, please tell me.

   1. Information about EGRP and its staff http://epi.grants.cancer.gov/about.html

   2. EGRP-supported cancer epidemiology consortia http://epi.grants.cancer.gov/Consortia/

   3. How to collaborate with the Breast and Colon Cancer Family Registries http://epi.grants.cancer.gov/CFR/

   4. Cancer Genetics Network pages http://epi.grants.cancer.gov/CGN/

   5. How is EGRP supporting epigenetics research? (http://epi.grants.cancer.gov/epigen.html), health disparities research? (http://epi.grants.cancer.gov/ResPort/HDoverview.html), or pharmacoepidemiology and pharmacogenetics research? http://epi.grants.cancer.gov/pharm/

   6. What are some current funding opportunities related to cancer epidemiology and genetics? (http://cancercontrol.cancer.gov/funding_apply.html)

   7. Grantsmanship Resources (http://epi.grants.cancer.gov/pharm/)

   8. What are some EGRP-funded research highlights from 2009? (http://epi.grants.cancer.gov/highlights/)

   9. Information about biospecimens (http://epi.grants.cancer.gov/access_policies.html) and how to access some NCI-supported genomic datasets http://epi.grants.cancer.gov/dac/)

   10. News announcements from EGRP to the extramural cancer epidemiology and genetics community? (See “News” box on http://epi.grants.cancer.gov/ or go to http://epi.grants.cancer.gov/Bulletins/)

   11. Job information (See “Employment Opportunities” box on http://epi.grants.cancer.gov/ or go to http://epi.grants.cancer.gov/Bulletins/)

2. If you wanted to either conduct research internationally or look for existing international research, where would you look?

3. What is the relationship between EGRP, Clinical and Translational Epidemiology, Host Susceptibility Factors, Modifiable Risk Factors, and Research Methods and Technologies? (outsiders only)

4. When you have gone to the EGRP site in the past, what sorts of things have you looked for? Can you find those things on this version?

To observe during the test:

• Do users notice when switching to branch sites?

• When participants navigated to branch sites or other sections of website (e.g. Key Research Resources and Initiatives, Funding Opportunities, Our Grantees, More Info and Resources, etc., did they have problems getting back to the EGRP home page?

• Can users follow where they are as they drill down in site?

Post Test Interview Questions

I’d like to collect a bit of information from you using some standard tools before we discuss your experience.

[Administer the System Usability Scale (SUS) and Modified Cooper Harper questionnaires. See Attachments C4 and C5.]

I’d like to ask you a few specific questions.

1. On the Home page, does the organization make sense?

2. Now that you’ve had some time to work with the site, what’s your overall impression, specifically on how it’s laid out, the content, and how it works?

3. Most sites have some type of learning curve associated with them. Did this site have a learning curve? How easy or difficult do you think it is?

4. What are the three things you liked most about the way the site looks, the content, and how the site works?

5. What three things did you like least about the way the site looks, the content, and how the site works?

6. What about the site surprised you the most?

7. Is there any information or other content you would have expected to find?

8. Is there any information that you would like to see on the site, particularly on the homepage?

9. Are there any new technologies that if incorporated would make the site more useful? (If need examples, items like RSS feeds, podcasts, Twitter updates, etc).

Wrap up

OK, we’re done. Do you have any further questions or comments?

Thank you again for your participation.

Attachment C4. System Usability Scale.

Attachment C5. Modified Cooper-Harper Difficulty Rating Scale

OMB #: 0925-0589-05

Expiry Date: 05/31/2011

Public reporting burden for this collection of information is estimated to average 5 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0589-05). Do not return the completed form to this address.

EGRP #0925-0589-05, Page 9