Attachment1_CDC-Index-Click-testing-Protocol_2013

Attachment 1

CDC Clear Communication Index

Website and Supporting Materials

Click Testing Protocol

November 7, 2013

Submitted to:

Cynthia Baur

Senior Advisor, Health Literacy

Centers for Disease Control and Prevention

Office of the Associate Director for Communication

1600 Clifton Rd

Atlanta, GA 30333

Submitted by:

CommunicateHealth, Inc.

26 Market Street

Northampton, MA 01060

413-582-0425

Table of Contents

Objectives 1

Participants and Recruitment 2

Methods 3

Final Report and Recommendations 7

Appendix A: Online Welcome Message 8

Objectives

To gain feedback on the drafts of 8 CDC exemplar materials, CommunicateHealth will conduct online click testing using Verify (verifyapp.com). Click testing is a technique used for gathering quick and specific feedback on materials or web designs. The Verify software captures participants’ assessment of materials, including a “heat map” of where participants expect to find specific types of information. The software can also gather participant feedback on labels, sections of content, and visual design elements.

The feedback gathered from this activity will help us improve the clarity of the exemplar materials by identifying final adjustments to be made to the content and visual design.

Participants and Recruitment

CommunicateHealth will enroll 32 participants representing 4 target audiences:

• 8 clinicians (doctors and nurses)

• 8 CDC partners

• 8 public health department staff

• 8 policymakers

Recruitment Plan

CH will use the following methods to identify and recruit participants:

• Clinicians (doctors and nurses): CH will recruit from an internal list of clinicians developed for previous studies.

• Partners: CDC will identify contacts in partner organizations. CDC will make initial contact with these partners and CH will follow up via email and phone.

• Public health department staff: CH will place a solicitation in the weekly Society for Public Health Education (SOPHE) newsletter or similar Association of State and Territorial Health Officials (ASTHO) publication with a unique email address for members to contact if they are interested in participating.

• Policymakers: CH will work closely with CDC to identify state and local level decision-makers to recruit.

Participants will be invited via email to complete a brief (30-minute) online click test using Verify, a browser-based click testing application.

Clinicians will be offered $40 for their participation.

Methods

Verify is an easy-to-use online tool that allows participants to provide feedback on designs by clicking through screens or mockups. A web link will be sent to participants with a welcome message, basic instructions and contact information in case of questions or technical difficulties. The online welcome message for participants is provided in Appendix A.

The sessions will be unmoderated, meaning participants will be able to complete the study on their own time. To log in, participants will enter their email address. This enables us to track responses of individuals recruited from each target audience. The Verify software will save data for analysis, including click speed, task success (when applicable), and participant answers to open-ended questions.

Stimulus Materials

CommunicateHealth is developing new CDC exemplar materials, which will be used for the click testing. Each participant will review two revised materials targeted to their particular audience group (Table 1).

Table 1

Audience	Material
Clinicians (doctors & nurses)	Influenza (Attachment 3)
	Wisconsin Surveillance of Autism (Attachment 4)
CDC partners	Healthcare-Associated Infections (Attachment 5)
	Heart Disease Fact Sheet (Attachment 6)
Public health department staff	Model Aquatic Health Code (Attachment 7)
	National Violent Death Reporting System (Attachment 8)
Policymakers	CDC Budget Overview (Attachment 9)
	Introduction to Epidemiology (Attachment 10)

Table 2 identifies the burden hours per material

Table 2

Audience	Material	Burden Hours
Clinicians (doctors & nurses)	Influenza (Attachment 3)	1 hour and 20 minutes
	Wisconsin Surveillance of Autism (Attachment 4)	1 hour and 20 minutes
CDC partners	Healthcare-Associated Infections (Attachment 5)	1 hour and 20 minutes
	Heart Disease Fact Sheet (Attachment 6)	1 hour and 20 minutes
Public health department staff	Model Aquatic Health Code (Attachment 7)	1 hour and 20 minutes
	National Violent Death Reporting System (Attachment 8)	1 hour and 20 minutes
Policymakers	CDC Budget Overview (Attachment 9)	1 hour and 20 minutes
	Introduction to Epidemiology (Attachment 10)	1 hour and 20 minutes

Click Testing Tasks

Participants will be asked to perform tasks and respond to questions using the revised exemplar materials. Table 3 includes a list of task types in Verify that will be used for this study.

Table 3: Verify Task Types

Annotate Task: Users can add free-form notes to a screenshot to provide feedback about the highlighted area.
Memory Task: Users can enter what they remember 5 seconds after viewing the material.
Question Task: Users can describe their general impression of the material.
Label Task: Labels point to a specific word or phrase on the material, and users can explain what they think it means.

Participants will view the revised exemplar materials and respond to prompts according to the task matrix below (Table 4). Only the content in the “Prompt” will be displayed to participants. The content in brackets will vary by material.

Table 4: Task Matrix

Task #	CDC Index #	Task Type	Prompt	Optional
1.	N/A	Memory Task	Try to remember and list as many things as you can from the screen you are about to see.
2.	N/A	Question Task	What is your first impression of the material?
3.	Q1	Annotate Task	On the following screen, click and drag your mouse to put a box around the main message of this material.
4.	Q4	Annotate Task	On the following screen, click and drag your mouse to put a box around the images that you think are relevant to the main message you just highlighted. In the blank box, tell us why it’s relevant.
5.	Q5	Annotate Task	On the following screen, click and drag your mouse to highlight the “call to action.”
6	Q7	Label Task	How would you define or explain the terms indicated by the red blinking targets? Type your answer in the blank boxes. (Markers will depend on the material.)
7.	Q7	Annotate Task	We want to know if anything written on this page would be confusing or unclear to your colleagues, or others like you. Click and drag your mouse over anything that’s confusing or unclear. In the text box, type why you think so.
8.	Q10	Annotate Task	On the following screen, click and drag your mouse to highlight the areas that give you the most important information in this material.
9.	Q12	Annotate Task	On the following screen, click and drag your mouse to highlight the areas that tell you what to do about [the problem].	Y
10.	Q13	Annotate Task	On the following screen, click and drag the mouse to highlight the areas that tell you why it’s important to follow the [behavioral recommendations].	Y
11.	Q15 & Q16	Annotate Task	On the following screen, click and drag your mouse to highlight any numbers that would be confusing or unclear to your colleagues, or others like you. In the text box, type why you think so.	Y
12.	Q18	Annotate Task	On the following screen, click and drag your mouse to highlight where it explains why [the risk in the material] is a risk.	Y
13.	Q19	Annotate Task	On the following screen, click and drag your mouse to highlight the areas that address the risks and benefits of the [behavioral recommendations].	Y
14.	Q20	Annotate Task	On the following screen, click and drag your mouse to highlight areas that explain or help you understand the [numeric probability].	Y
15.	N/A	Question Task	What do you think of the length of the material you've just reviewed? Scale: Too long, Somewhat too long, Just right, Somewhat too short, Too short
16.	N/A	Question Task	What questions do you still have that weren’t answered, or what information, if any, do you think is missing from this material?
17.	N/A	Question Task	Who do you think the primary audience is for this material?

Final Report and Recommendations

CommunicateHealth will use the Verify online software to analyze the data collected from click testing. The final report will outline measures provided by Verify including:

• Heat maps (where participants clicked the most on a screen during all of the annotate tasks)

• Responses to open-ended questions

CommunicateHealth will deliver a brief memo detailing the click testing findings and recommendations for revisions to the exemplar materials.

Appendix A: Online Welcome Message

Form Approved

OMB No: 0920-0956

Exp. Date: March 13, 2016

Online Welcome Message

Thank you for agreeing to provide us feedback on materials that have been developed based on CDC’s Clear Communication Index. Your feedback is extremely important. We anticipate that it will take you about 20 minutes to complete this questionnaire.

Your responses will be kept in a secure manner. All information will be used for evaluation purposes only.

Please note that these are draft materials and do not have final approval from CDC.

__________________________________________________________________

Public reporting burden (completion time) for this collection of information is estimated to average 20 minutes per session. This includes the time it takes to review instructions, and gather and maintain the data needed. An agency can not conduct or sponsor (and a person is not required to respond to) a collection of information unless it displays a currently valid OMB control number. Please send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden to CDC/ATSDR Reports Clearance Officer; 1600 Clifton Road NE, MS D-74, Atlanta Georgia 30333; ATTN : PRA (0920-0956)

2 6 Market Street, Northampton, MA 01060 | 413-582-0425 | www.communicatehealth.com