3 Facilitator's Guides for All Groups

U ser-Centered Design, Inc. 20548 Deerwatch Place Ashburn, VA 20147 (703) 729-0998

Attachment 6.

NCI Office of Cancer Survivorship (OCS)

Website Usability Testing

Facilitator’s Guides for:

• Researcher Audience

• Advocate Audience

• Health Professionals Audience

• Public Audience

Submitted to:

Nicole Saiontz

Submitted by:

User-Centered Design, Inc.

20548 Deerwatch Place

Ashburn, VA 20147

(703) 729-2370

www.user-centereddesign.com

Attachment 6.

OCS Website Usability Testing

Facilitator’s Guide – Researcher Audience

November 2012

OMB #: 0925-0642, Expiration Date: 9/30/2014

Submitted to:

Nicole Saiontz

Submitted by:

User-Centered Design, Inc.

20548 Deerwatch Place

Ashburn, VA 20147

(703) 729-2370

www.user-centereddesign.com

[

OMB No.: 0925-0642

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as statistical summaries. You are being contacted by phone or in person to complete this instrument so that we can access the website.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change to allow the process flow naturally.]

Pre-Test

[Administer the informed consent.]

Introduction

Thank you for agreeing to participate in this study.

NCI’s Office of Cancer Survivorship (OCS) is assessing their website, and today we are conducting a usability test of its design. As a participant, you will explore the website and be given a set of tasks to do.

As we go through the tasks, feel free to offer any comments or suggestions that occur to you. We are looking for things about the design that are working well, as well as what could be improved. There are two important things you should keep in mind while you work with the website:

• First, I did not create the design so you can’t hurt my feelings with any criticisms you might have. If there are problems with the design, I would like to discuss them with you to see if we can find a way to improve the design.

• Second, we are testing the website and not you, so you cannot make any mistakes. The site is supposed to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

I’d like you to know that there are some observers with us today helping me by taking notes, but don’t worry about them. You and I will work on this together and they’ll just watch and listen.

Do you have any questions for me before we get started?

Background

We have invited persons with a variety of backgrounds to participate in this activity, so I’d like you to tell me:

1. Where do you work and what is your role there?

2. What type of research do you do? How long have you done this kind of research?

3. Have you seen the OCS site before?

   1. [If yes:] What type of information have you looked for on the website? [Probe on areas of research, funding information/opportunities, research tools (such as surveys), publications, etc. Note P’s answers to guide their exploration of the website during the tasks.]

   2. [If no:] Based on the name of the site, what kind of information do you think you would find on the site? [Note P’s answers to guide their exploration of the website during the tasks.]

Tasks

Novice Participants

1. Take a look at the OCS website [Show P http://dccps.nci.nih.gov/ocs/]. Where would go first on the website? [Note the areas where the P goes and what information they access.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. Since you are new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Expert Participants

1. You mentioned previously looking at {topic} on the OCS website. Can you show me how you accessed that information? [Note the areas where the P goes and what information they access. Repeat for each topic they mentioned earlier.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. If you were new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Follow-up Questions

1. I’d like you to reflect on the things you liked and didn’t like about the website. What were three things you liked? What were three things you disliked? What would you change about the things you disliked?

2. How easy or difficult was it for you to find information on the OCS website? What would you change about the difficult things to make them easier?

3. Was there anything missing – any information or topics that you expected to see but didn’t?

4. Is there anything about the organization of the web pages that you would change?

Wrap up

OK, we’re done. Do you have any further questions or comments?

Thanks again for your participation. [Provide P with incentive of $25.]

OCS Website Usability Testing

Facilitator’s Guide – Advocate Audience

November 2012

OMB #: 0925-0642, Expiration Date: 9/30/2014

Submitted to:

Nicole Saiontz

Submitted by:

User-Centered Design, Inc.

20548 Deerwatch Place

Ashburn, VA 20147

(703) 729-2370

www.user-centereddesign.com

[

OMB No.: 0925-0642

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as statistical summaries. You are being contacted by phone or in person to complete this instrument so that we can access the website.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally.]

Pre-Test

[Administer the informed consent.]

Introduction

Thank you for agreeing to participate in this study.

NCI’s Office of Cancer Survivorship (OCS) is acessing their website, and today we are conducting a usability test of its design. As a participant, you will explore the website and be given a set of tasks to do.

As we go through the tasks, feel free to offer any comments or suggestions that occur to you. We are looking for things about the design that are working well, as well as things that could be improved. There are two important things you should keep in mind while you work with the website:

• First, I did not create the design so you can’t hurt my feelings with any criticisms you might have. If there are problems with the design, I would like to discuss them with you to see if we can find a way to improve the design.

• Second, we are testing the website and not you, so you cannot make any mistakes while working on the tasks. The site is supposed to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

I’d like you to know that there are some observers with us today helping me by taking notes, but don’t worry about them. You and I will work on this together and they’ll just watch and listen.

Do you have any questions for me before we get started?

Background

We have invited persons with a variety of backgrounds to participate in this activity, so I’d like you to tell me:

1. Is it the case that you are a cancer advocate? Tell me about it. Have you seen the Office of Cancer Survivorship site before?

   1. [If yes (“Expert” P):] What type of information have you looked for on the website? [Probe on information to disseminate, types of resources, information on research projects and/or funding, etc. Note P’s answers to guide their exploration of the website during the tasks.]

   2. [If no (“Novice” P):] Based on the name of the site, what kind of information do you think you would find on the site? [Note P’s answers to guide their exploration of the website during the tasks.]

Tasks

Novice Participants

1. Take a look at the OCS website [Show P http://dccps.nci.nih.gov/ocs/]. Where would you go first on the website? [Note the areas where the P goes and what information they access.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. Since you are new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Expert Participants

1. You mentioned previously looking at {topic} on the OCS website. Can you show me how you accessed that information? [Note the areas where the P goes and what information they access. Repeat for each topic they mentioned earlier.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. If you were new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Follow-up Questions

1. I’d like you to reflect on the things you liked and didn’t like about the website. What are three things you liked? What are three things you disliked?

2. How easy or difficult was it for you to find information on the OCS website?

3. Was there anything missing – any information or topics that you expected to see but didn’t?

4. Is there anything about the organization of the web pages that you would change?

Wrap up

OK, we’re done. Do you have any further questions or comments?

Thanks again for your participation. [Provide P with incentive of $25.]

OCS Website Usability Testing

Facilitator’s Guide – Health Professional Audience

November 2012

OMB #: 0925-0642, Expiration Date: 9/30/2014

Submitted to:

Nicole Saiontz

Submitted by:

User-Centered Design, Inc.

20548 Deerwatch Place

Ashburn, VA 20147

(703) 729-2370

www.user-centereddesign.com

OMB No.: 0925-0642

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as statistical summaries. You are being contacted by phone or in person to complete this instrument so that we can access the website.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

[Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally.]

Pre-Test

[Administer the informed consent.]

Introduction

Thank you for agreeing to participate in this study.

NCI’s Office of Cancer Survivorship (OCS) is acessing their website, and today we are conducting a usability test of its design. As a participant, you will explore the website and be given a set of tasks to do.

As we go through the tasks, feel free to offer any comments or suggestions that occur to you. We are looking for things about the design that are working well as well as things that could be improved. There are two important things you should keep in mind while you work with the website:

• First, I did not create the design so you can’t hurt my feelings with any criticisms you might have. If there are problems with the design, I would like to discuss them with you to see if we can find a way to improve the design.

• Second, we are testing the website and not you, so you cannot make any mistakes. The site is supposed to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

I’d like you to know that there are some observers with us today helping me by taking notes, but don’t worry about them. You and I will work on this together and they’ll just watch and listen.

Do you have any questions for me before we get started?

Background

We have invited persons with a variety of backgrounds to participate in this activity, so I’d like you to tell me:

1. Where do you work and what is your role there?

2. Have you seen the OCS site before?

   1. [If yes:] What type of information have you looked for on the website?

      1. [Probe on types of information to disseminate to patients, publications, reports, guidelines, etc. Note P’s answers to guide their exploration of the website during the tasks.]

   2. [If no:] Based on the name of the site, what kind of information do you think you would find on the site? [Note P’s answers to guide their exploration of the website during the tasks.]

Tasks

Novice Participants

1. Take a look at the OCS website [Show P http://dccps.nci.nih.gov/ocs/]. Where would go first on the website? [Note the areas where the P goes and what information they access.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. Since you are new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Expert Participants

1. You mentioned previously looking at {topic} on the OCS website. Can you show me how you accessed that information? [Note the areas where the P goes and what information they access. Repeat for each topic they mentioned earlier.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. If you were new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Follow-up Questions

1. I’d like you to reflect on the things you liked and didn’t like about the website. What are three things you liked? What are three things you disliked?

2. How easy or difficult was it for you to find information on the OCS website?

3. Was there anything missing – any information or topics that you expected to see but didn’t?

4. Is there anything about the organization of the web pages that you would change?

Wrap up

OK, we’re done. Do you have any further questions or comments?

Thanks again for your participation. [Provide P with incentive of $25.]

OCS Website Usability Testing

Facilitator’s Guide – Public Audience

November 2012

OMB #: 0925-0642, Expiration Date: 9/30/2014

Submitted to:

Nicole Saiontz

Submitted by:

User-Centered Design, Inc.

20548 Deerwatch Place

Ashburn, VA 20147

(703) 729-2370

www.user-centereddesign.com

OMB No.: 0925-0642

Expiration Date: 9/30/2014

PRIVACY ACT NOTIFICATION STATEMENT

Collection of this information is authorized by The Public Health Service Act, Section 412 (42 USC 285 a-1). Rights of study participants are protected by The Privacy Act of 1974. Participation is voluntary, and there are no penalties for not participating or withdrawing from the study at any time. Refusal to participate will not affect your benefits in any way. The information collected in this study will be kept private under the Privacy Act. Names and other identifiers will be separated from information provided and will not appear in any report of the study. Information provided will be combined for all study participants and report as statistical summaries. You are being contacted by phone or in person to complete this instrument so that we can access the website.

NOTIFICATION TO RESPONDENT OF ESTIMATED BURDEN

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to: NIH, Project Clearance Branch, 6705 Rockledge Drive, MSC 7974, Bethesda, MD 20892-7974, ATTN: PRA (0925-0642). Do not return the completed form to this address.

[Note: The purpose of this document is to guide the moderator.  The questions and tasks contained herein may not be asked as written.  The facilitator often draws on participant comments and the natural flow of the testing process to determine the flow of the session.  While the facilitator will try to follow the order of the guide, many times tasks will come up ahead of time or in different order.  The facilitator may allow the order of the tasks to change in order to let the process flow naturally.]

Pre-Test

[Administer the informed consent.]

Introduction

Thank you for agreeing to participate in this study.

NCI’s Office of Cancer Survivorship (OCS) is accessing their website, and today we are conducting a usability test of its design. As a participant, you will explore the website and be given a set of tasks to do.

As we go through the tasks, feel free to offer any comments or suggestions that occur to you. We are looking for things about the design that are working well as well as things that could be improved. There are two important things you should keep in mind while you work with the website:

• First, I did not create the design, so you can’t hurt my feelings with any criticisms you might have. If there are problems with the design, I would like to discuss them with you to see if we can find a way to improve the design.

• Second, we are testing the website and not you, so you cannot make any mistakes. The site is supposed to be intuitive and easy to use. If it isn’t, that’s a problem with the website – not with you.

I’d like you to know that there are some observers with us today helping me by taking notes, but don’t worry about them. You and I will work on this together and they’ll just watch and listen.

Do you have any questions for me before we get started?

Background

We have invited persons with a variety of backgrounds to participate in this activity, so I’d like you to tell me:

1. Do you have a specific connection to cancer?

2. Have you seen the Office of Cancer Survivorship site before?

   1. [If yes:] What type of information have you looked for on the website? [Probe on information for survivors (such as coping with cancer), how to follow up after cancer treatment, survivorship statistics, survivorship research, etc. Note P’s answers to guide their exploration of the website during the tasks.]

   2. [If no:] Based on the name of the site, what kind of information do you think you would find on the site? [Note P’s answers to guide their exploration of the website during the tasks.]

Tasks

Novice Participants

1. Take a look at the OCS website [Show P http://dccps.nci.nih.gov/ocs/]. Where would go first on the website? [Note the areas where the P goes and what information they access.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. Since you are new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Expert Participants

1. You mentioned previously looking at {topic} on the OCS website. Can you show me how you accessed that information? [Note the areas where the P goes and what information they access. Repeat for each topic they mentioned earlier.]

2. You mentioned before that you were interested in {topic}. Does this website have information on this topic? Let’s explore this area. [Direct P to the area with the information.]

3. Are there any other areas on the page that interest you? [Explore all additional areas of the site the P mentions.]

4. [If the participant has not explored all areas of the site:] There are a few other areas of the website. Can you describe the other areas of the website and think about the content that might be there and the people who would be interested in that content? [After P describes one area:] Let’s look at the content now. How does this content compare to what you expected to find? [Repeat with all other major parts of the site.]

5. If you were new to the OCS website, how would you learn about what the Office does? [Note if P finds this information and probe on aspects that are most or less useful.]

6. How would you determine how current the information is that you looked at on the website? How would you determine it’s accuracy? [Note if the P seems confident or not, and follow up on their reactions if they seem uncertain.]

Follow-up Questions

1. I’d like you to reflect on the things you liked and didn’t like about the website. What are three things you liked? What are three things you disliked?

2. How easy or difficult was it for you to find information on the OCS website?

3. Was there anything missing – any information or topics that you expected to see but didn’t?

4. Is there anything about the organization of the web pages that you would change?

Wrap up

OK, we’re done. Do you have any further questions or comments?

Thanks again for your participation. [Provide P with incentive of $40.]

2