B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
B-1. Respondent Universe and Sampling Methods
The data collection will not use statistical methods to select respondents. The responses are intended to serve as feedback to help ensure FDA communications meet the needs of the public. FDA would consider it unacceptable for our communications to be unclear or hard to understand or find (for the Website) for more than 20 percent of its Web users or readers. A convenience sample need not be representative to indicate a need for further message refinement.
Each survey will have a desired quota of respondents, based on the website being studied. All interested in participating will be asked a series of up to 4 Demographic Questions (Attachment 5) and up to 6 Introductory Questions (Attachment 6). Responses to the screening questions will determine when to stop the survey or to not administer the survey to that type of respondent. When the desired number of respondents is obtained, further surveys will not be administered and the survey will be removed from the site.
There are many possible respondent types, including the general public, public health professionals, industry representatives, and healthcare professionals. Participants will be invited to participate through one of three possible ways:
Invited through a message and link on a FDA Web page.
Recruited nationwide from a combination of pre-existing databases of volunteers.
Recruited from lists drawn from appropriate professional organizations and personal contacts made available through FDA contacts. The participants will be telephoned, sent an e-mail, or mailed a letter and invited to join.
We estimate that approximately 20% of the total number of participants recruited will participate in the survey. This figure is based on a review of literature regarding web-based surveys and the previous results of similar surveys distributed by customer survey professionals and web survey vendors.
The survey will be provided to all participants via one or more of the methods described in Section A-3 of this document, Use of Improved Information Technology and Burden Reduction. Respondents will fill out the survey by clicking on the appropriate response, entering text for each question, or attempting to find the answer to specific tasks. The answer will automatically be stored electronically, apart from any personal identifiers.
The data collected will be stratified in different ways during the analysis. Survey data will be examined by profession, geography, frequency of Web usage, information needed, usage of information, answer to task questions, and experience on the website while performing tasks. Demographic data will be used to ensure that participants are selected from a wide range of demographics or from a specific demographic, as needed to represent the target audience for a website. Profession or role (for example, public health professional, practicing physician, mother, caregiver, or student) is one of the primary ways survey data will be stratified since FDA’s website and communications have multiple target audiences, each with different goals, wants, needs, and preferences. There will be additional ways of stratifying the data in an individual survey, based on the survey’s specific target audiences and overall goals.
B-2.
Procedures for the Collection of Information
For both remote online and in-person surveys, responses will be collected automatically by the survey software. Handwritten notes, video, and/or audiotape will be used to supplement data collected during in-person surveys. In-person surveys will be conducted on government-owned computers or FDA contractor-owned computers and will be facilitated by FDA staff or contractors. Remote online surveys will be conducted at the user’s computer and cannot be used to identify users as described in A-10.
B-3. Methods to Maximize Response Rates and Deal with Non-response
The usability surveys will be designed to minimize respondent burden. The survey will be logical, easy to read, and relatively quick to complete. The surveys are Web-based, which will allow for a visually appealing layout and remote surveying (in which participants take the survey from their own computer.)
The purpose of the web-based survey is to obtain feedback so each Web page or communication can be judged to determine how to improve information presentation and dissemination. Therefore, we do not always require a high response rate to obtain useful information. Sampling size will vary depending on the website in question and the scope of the usability survey. The lowest sampling size necessary to glean useful data will be used, and no survey will have more than 150 respondents.
B-4. Tests of Procedures or Methods to be Undertaken
CDC already performed technical testing to ensure technical quality, ease of use, and proper collection of data for existing usability survey methods. Principles of user research were used in creating the new survey questions. FDA web experts and communication specialists have reviewed the proposed survey questions. Pilot surveys were conducted with ten FDA staff. Additionally, virtually all survey questions in this package were used in previous surveys at CDC or are standard usability questions utilized by web professionals.
B-5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
We anticipate the data to be straightforward and not require complex analysis techniques. Each program will obtain input from in-house staff or vendors who are skilled in survey methodology and information systems technology. Data collection and analysis will be performed by FDA FTEs or contracted services. Such persons include Miriam Campbell, Ph.D., M.P.H. of the Risk Communications Staff within the FDA Office of Planning.
FDA COTR
Miriam K. Campbell, Ph.D., M.P.H., Social Science Analyst
Food and Drug Administration
Office of the Commissioner, Office of Planning, Risk Communications Staff
10903 New Hampshire Avenue
Building 32, Room 3276
Silver Spring, MD 20903
301-796-9108
List of Attachments
Executive Order 12862
Public Health Service Act (42 USC 241) Section 301
60-day Federal Register Notice
Consent Forms
Demographic Questions
Introductory questions
Core questions
Activity/Task questions
Follow-up questions
Example survey
Attachment 1: Executive Order 12862
Attachment 2: Public Health Service Act (42 USC 241) Section 301
Attachment 3: 60-day Federal Register Notice
Attachment 4: Consent Forms
Consent Forms
One of the following Consent Forms will be used. The bracketed portion of a Consent Form may be changed based on the particular evaluation scenario.
OPTION 1: Online Welcome Message and Consent Form
OMB No.:
Exp Date:
[Welcome! Thank you for agreeing to help the Food and Drug Administration. Your feedback is extremely important. We anticipate that it will take approximately [30 minutes] to complete these questions.]
[We are not testing your abilities in any way; we are only testing [the FDA Web site] to see how well it works. Please use [the FDA Web site] in whatever manner is comfortable and normal for you.]
[In some cases, we will collect keyboard responses, mouse clicks, audio and video data.] Your responses to all questions will be kept in a secure manner. No personal identifiers will be recorded. All information is used for evaluation purposes only, and FDA does not plan to share the data with anyone outside FDA.]
[To proceed through the survey, select your answer for each question and click [“Continue”].]
Public reporting burden of this collection of information is estimated to average [30 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA {[PRA officer name and address].
OPTION 1: Consent Form for Interview
OMB No.:
Exp Date:
[Thank you for agreeing to help the Food and Drug Administration. Your feedback is extremely important. We anticipate that it will take approximately [30 minutes] to complete these questions.]
[We are not testing your abilities in any way; we are only testing [MESSAGE] to see how well it works. Please cooperate in whatever manner is comfortable and normal for you.]
Your responses to all questions will be kept in a secure manner. No personal identifiers will be recorded. All information is used for evaluation purposes only, and FDA does not plan to share the data with anyone outside FDA.]
Public reporting burden of this collection of information is estimated to average [30 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA {[PRA officer name and address].
OPTION 3: Audiotape and Videotape Consent Form
OMB No.:
Exp Date:
[The purpose of this document is to obtain your consent to audiotape and videotape today’s usability testing session. We want to record the session in order to analyze the information collected today, in depth, at a later time.]
[The tape will be used internally within FDA, and FDA does not plan to share the tape with anyone outside FDA. It will not be broadcast or used for any other purpose. No personal identifiers will be linked to the data and your signed consent form will be stored separately from the recording.]
[If you agree with this, please sign where indicated.]
Print Name: __________________________________
Signature: ___________________________________ Date: ____________________
Public reporting burden of this collection of information is estimated to average [60 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA contact name and address].
OPTION 4: Talent and Consent Waiver
OMB No.:
Exp Date:
DEPARTMENT OF HEALTH AND HUMAN SERVICES
FOOD AND DRUG ADMINISTRATION
SILVER SPRING, MD
TALENT AND CONSENT WAIVER
TO WHOM IT MAY CONCERN:
I hereby grant full permission to the Department of Health and Human Services, Food and Drug Administration (FDA) to use, reproduce, publish, distribute, and exhibit my name, picture, portrait, likeness or voice or any or all of them in or in connection with the production of a television tape or film recording, sound track recording, motion picture film, filmstrip, still photograph, or intranet / extranet posting, in any manner for training and other purposes. I understand that portrait shots and other pictures of me will initially be posted on the FDA intranet and extranet site and that those pictures may be used in FDA's internal and external written materials, including ultimately on FDA's Internet site.
Without limitation as to time, I hereby waive all rights for compensation in connection with the use of my name, picture, portrait, likeness or voice, or any or all of them, or in connection with said television tape or film recording, sound track recording, motion picture film, filmstrip, still photograph, FDA internal and/or external written materials, or intranet/extranet/Internet posting, in whole or in edited form and any use to which the same or any material therein may be put, applied or adapted by the United States Government and others in the health field.
IN WITNESS WHEREOF I have hereunto set my hand and seal this _______
day of __________________A.D. 20____.
NAME:_____________________________________________________
(PRINT)
ADDRESS:__________________________________________________
____________________________________________________________
SIGNATURE:_______________________________________________
WITNESS:_______________________________________________
DATE:________________________________________
PROJECT NAME OR NO.________________________________
TITLE:________________________________________
Source: CDC 0.1286 (E), 8/2001, CDC Adobe Acrobat 5.0 Electronic Version, 9/2003
Public reporting burden of this collection of information is estimated to average [60 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA contact name and address].
OPTION 5: Participation Consent Form
OMB No.:
Exp Date:
[We have revised [the FDA Web site or MESSAGE] and would like you to "try out" [the new Web site or MESSAGE]. We want to know what works well for you and what does not, so that we can further improve [the Web site or MESSAGE].]
[During this session, we'll
1. Ask you about your background
2. Ask you to perform a series of tasks to find information on [the Web site or MESSAGE]
3. Ask you to give us feedback on the new [homepage or MESSAGE]
[We will keep track of your interactions with the Web site, so that we can find ways to improve the system./ We will ask you questions about your understanding of the message, so that we can find ways to improve it.] The information that is captured only will be used to make changes to [the Web site or MESSAGE]. The whole exercise will take approximately [20 minutes] to complete.]
[When analyzing and reporting the results, no personal identifiers will be linked to the data and your signed consent form will be stored separately.]
[To consent to the use and release of this information, please sign your name below.]
Name _______________________ Date: _______________________
Public reporting burden of this collection of information is estimated to average [60 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA contact name and address].
OPTION 6: Short Participant Consent Form
OMB No.:
Exp Date:
[Welcome! Thank you for agreeing to help the Food and Drug Administration (FDA) evaluate [MESSAGE]. Your feedback is extremely important. We anticipate that it will take approximately [20 minutes] to complete these questions.]
[Your responses to all questions will be kept in a secure manner. All information is used for evaluation purposes only, and FDA does not plan to share the data with anyone outside FDA.]
[To proceed through the survey, select your answer for each question and click [“Continue”].]
Public reporting burden of this collection of information is estimated to average [20 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA contact name and address].
OPTION 7: Card Sort Instructions
OMB No.:
Exp Date:
[Introduction:
We are conducting research that will help us gain a better understanding of how [MESSAGE] should be organized to make it easier to use.]
[Instructions:
Begin by reviewing the items in the left column. These items represent content on [MESSAGE].]
[Start by placing all of the items that belong together into the same group. Drag the items that belong in the same category from the left column over to the right column. When you are finished adding items to a category, click the yellow box in the middle column to name each group.]
[If an item is unfamiliar, you may create a category called “Miscellaneous”.]
[There is no correct number of groups, but make sure that you think about how the items relate to each other. If you have a group with a large amount of items, consider splitting it up.]
[You must do the exercise in one sitting. Please do not leave the browser to go to another task - you will not be able to return.]
[Thank you for taking the time to participate. We appreciate your help!]
Public reporting burden of this collection of information is estimated to average [60 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA contact name and address].
OPTION 8: In person and Video Consent
OMB No.:
Exp Date:
[Understanding Your Participation]
You are being asked to participate in a usability study for [e.g. SITE or MESSAGE]. By participating in this evaluation, you will help us improve our products. This evaluation will take [15 minutes] to complete.]
[We will observe you and record information about how you work. We may record your comments and actions using written records, audio recording, and screen recording.]
[By signing this form, you give your permission for our organization to use:
recorded voice (if you consent to this, initial here _____ )
verbal statements (if you consent to this, initial here _____ )
session recording (if you consent to this, initial here _____ ) ]
[This data will be used solely for the purposes of product design and development. Your full name will not be used during any presentation of the results of this study.]
[Any information you acquire about our products is confidential and proprietary and is being disclosed to you only so you may participate. By signing this form, you agree not to talk about this product to anyone. You may tell them that you helped to evaluate an Internet web site.]
[You may withdraw from the evaluation at any time. If, at any time, you need a break, please inform the observer immediately. If you have questions, you may ask at any time.]
[If you agree with these terms, please indicate your agreement by signing below.]
Name (Printed) ______________________________________________
[E-mail Address ______________________________________________]
[Signature ______________________________________________]
Date ______________________________________________
Public reporting burden of this collection of information is estimated to average [15 minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden to FDA [PRA name and address].
Attachment 5: Demographic Questions
Demographic Questions
(Questions can be used in intercept interviews, telephone interviews, online research, and focus group screeners.)
Gender:
Male
Female
How old are you? Are you...?
under 18 years of age
18-24 years of age
25-34 years of age
35-44 years of age
45-54 years of age
55-64 years of age
65-74 years of age
75 years of age or older
What is the highest level of education you have completed?
Grade school
Less than high school graduate/some high school
High school graduate or completed GED
Some college or technical school
Received four-year college degree
Some post graduate studies
Received advanced degree
Other: _____________________
Please tell me your race or ethnic background. Are you?
Ethnicity:
Hispanic or Latino
Not Hispanic or Latino
Race:
Asian
African
American Indian or Alaska Native
Black or African-American
Native Hawaiian or Other Pacific Islander
Haitian
South American or Central American
White/Caucasian
Other:____________________
Were you born in the United States?
Yes
No
Do you speak a language other than English in your home?
What language is that?
Chinese
Spanish
Urdu/Hindu
Arabic
Bengali
Portuguese
Russian
Japanese
German
Indonesian
Punjabi
Wu, Tibetan
Telugu
Marathi
French
Vietnamese
Korean
Tamil
Italian
Turkish
Cantonese
Pashto
Tagalog/Filipino
Gujarati
Polish
Ukrainian
Malay
Persian
Romanian
Dutch
Thai
Yoruba
Uzbek
Igbo
Nepali
Kurdish
Serbian
Greek
Zulu
Haitian Creole
Hebrew
Cambodian
Laotian
Mung
Other (specify): _____
In what state, city, and zip code do you currently live?
What is your current occupational status? Would you say…
Employed
Unemployed
Homemaker
Student
Retired, or
Disabled
Other:_______________
What is your current job title? What term would you use to describe the profession you are in? ______________________________________________
How long have you been worked in this profession? ______________ [years/months]
Are you a [health professional]? Yes/No
What is your [clinical specialty]? ____________
What is your [office setting]?
[solo practice]
[group practice]
[acute-care hospital]
[long-term-care hospital]
[rehabilitation hospital]
[freestanding clinic]
[visiting nurse]
[nursing home]
[pharmaceutical company]
[OPTION
What is your marital status?
Married
Living as married
Divorced
Widowed
Separated, or
Single, never been married
Which of the following categories best describe your total, annual household income after taxes?
Under $20,000/year
$20,001 - $40,000/year
$30,001 - $60,000/year
$60,001 - $80,000/year
$80,001 - $100,000/year
Over $100,000/year
Number of children (under age 18) living in the household:
None
1-2 children
3-4 children
5 or more children
Do you currently rent or own your home?
Own
Rent
Occupied with paying monetary rent
Introductory Questions
(Questions can be used in intercept interviews, telephone surveys, online surveys, and focus group discussion guides.)
Have you ever looked for health information from any source?
Yes
No
When you look for health information, where did you go?
Books
Brochures, pamphlets, etc.
Organization
Family
Friend/Co-Worker
Health care provider
Search the Internet via search engines such as Google
Participate in online communities or blogs
Visit specific websites (please specify: ________________)
http://www.fda.gov
Library
Magazines
Telephone information number (1-800-Number)
Complementary or alternative practitioner
Other: _________________________
Do not look or go anywhere
The most recent time you looked for information was it for
Yourself
Someone else you care for or take care of , such as a spouse/partner, child, parent, relative, or friend
Both?
In general, how much would you trust information about health or medical topics from a doctor or health care professional? Would you say a lot, some, a little, or not at all?
A lot
Some
A little
Not at all
In general, how much would you trust information about health or medical topics? Would you say a lot, some, a little, or not at all?
How about from family and friends?
How about in newspapers or magazines?
How about on the radio?
How about on the Internet?
How about on http://www.fda.gov?
How about on television shows such as [specific programs by name or program types such as news or Discovery Health channel]?
How about from charitable organizations?
How about religious organizations and leaders?
How about from [government health agencies or specific agency or agencies]?
[IF NEEDED: Government health agencies include the Food and Drug Administration (FDA), National Institutes of Health (NIH), the Centers for Disease Control and Prevention (CDC), and state and local health departments.]
Think about the most recent time you looked up information on [INSERT health topic or behavior] from any source. About how long ago was that?
Days ago
Weeks ago
Months ago
Years ago
How frequently do you look up information on [INSERT health topic or behavior]?
Daily
Weekly
Monthly
Less than monthly
What type of information were you looking for in your most recent search?
Organizations that focus on the [INSERT health condition/disease/syndrome/injury/
disability/injection/medication/medical device]
Side effects for [INSERT health condition/disease/syndrome/injury/disability/injection/medical/medical device]
Causes for health condition/disease/risk factors for [INSERT health condition/disease/syndrome/injury/disability/injection/medication/medical device]
Coping with health condition/disease/syndrome/dealing with [INSERT health condition/disease/syndrome/injury/disability/injection/medication/medical device]
Diagnosis of [INSERT health condition/disease/syndrome/injury/disability/medication/medical device]
Information on [INSERT health condition/disease/syndrome/injury/disability/medication/medical device]
Paying for medical care/insurance
Prognosis/recovery from [INSERT health condition/disease/syndrome/injury/disability/injection/medication/medical device]
Screening/testing/early detection of [INSERT health condition/disease/syndrome/injury/disability]
Symptoms of [INSERT health condition/disease/syndrome/injury/disability]
Treatment/cures for [INSERT health condition/disease/syndrome/injury/disability]
Where to get medical care for [INSERT health condition/disease/syndrome//injury/disability]
Information on complementary alternative or unconventional treatments for [INSERT health condition/disease/syndrome/injury/disability]
Other: __________________________
The next few questions are about various ways you might get health information.
Some newspapers or general magazines publish a special section that focuses on health. In the past 12 months, have you read health sections of the newspaper or of a general magazine?
Yes
No
Some local television and radio news programs include special segments of the newscasts that focus on health issues. In the past 12 months, have you watched or listened to health segments on the local news?
Yes
No
Some people notice information about health on the Internet, even when they are not trying to find out about a health concern they have or someone in the family may have. Have you read such health information on the Internet in the past 12 months?
Yes
No
Here are some ways people use the Internet. Some people have done these things, but other people have not. In the past 12 months, have you done the following things while using the Internet?
Bought medicine or vitamins online?
Participated in an online support group for people with a similar health or medical issue?
Used Email or the Internet to communicate with a doctor or a doctor’s office?
Used a Web site to help you with your diet, weight, or physical activity?
Looked for a healthcare provider?
Downloaded a health-related app to a Smartphone?
Viewed a health-related website using a mobile device, such as an iPhone or Droid?
Visited a “social networking” site, such as “My Space” or “Facebook”?
Wrote in an online diary or “blog” (i.e., Web log)?
Kept track of personal health information, such as care received, test results, or upcoming medical appointments?
About how often do you use the Internet?
Several times a day
About once a day
3-5 days a week
1-2 days a week
Every few weeks
Less often
Never
Don’t know/refused
How do you use your cell phone?
Browse the Internet
Use mobile applications (apps)
Text messaging
Sending email
Taking photos
Other (please specify) ___________________
As I read the following list of items, please tell me if you, personally, happen to have each one, or not.
A desktop computer
A laptop computer
A cell phone
A personal digital device, like a Sidekick, Palm Pilot , iPhone, Google Droid, or Blackberry
An iPod or other MP3 player
Attachment 7: Core questions
Core Questions
(Questions can be used in intercept interviews, telephone interviews, individual in-depth interviews [cognitive interview], online research, and focus group discussion guides.)
Comprehension
1. In your own words, what is the main idea that this [communication type] is trying to get across, in your own words?
2. How well do you think the main ideas come across?
3. Is it trying to get people to do something? [Probe: What action would the message prompt you to take?]
4. Were there any words that were unusual or unfamiliar?
5. What other words would you use to replace them?
6. Is there anything confusing, unclear, or hard to understand?
Impressions
1. How would you sum up in just a few words your first impression of this [communication type]? [Probes: Do you like it? Not like it? What makes you say that?]
2. What feelings do you have in reaction to this [communication type]? [Probe: Anything positive? Anything negative?]
3. Was your reaction to this was positive or negative?
4. What positive [images/thoughts] do you associate with “[INSERT message/phrase]?”
5. What negative [images/thoughts] do you associate with “[INSERT message/phrase]”?
6. Is this an appealing [communication type]? [Probe: What makes the [communication type] appealing or unappealing?]
7. How does it make you feel?
8. Was this a new idea or something that you’ve heard before?
9. Do you strongly agree with any part of this [communication type]? If so, what?
10. Do you strongly disagree with anything in this [communication type]? If so, what?
11. Is this [communication type] believable or not? Why or why not?
12. What additional information would you need in order to more strongly believe this [communication type]?
Audience and Personal Relevance
1. Who would you say we are trying to reach with this [communication type]?
2. Does it seem like this [communication type] is talking to you, and people like you? Or someone else?
3. What in the [communication type] suggested it was talking to [you and people like you or someone else]?
4. Do you see yourself doing this...or something like it? Why/why not?
5. Who do you believe would benefit most from [MESSAGE]?
Content and Wording
1. Do you like the way it is written? [Probe: tone, language/style, etc.] Is it easy to read?
2. Is there anything you want to know that this [communication type] does not tell you?
3. If you saw a poster or brochure with this message, would the idea get your attention? Why or why not?
4. Are there any words or phrases here that you think are especially attention-getting or appealing?
5. Are there any words or phrases that bother you?
6. Are there are words that you think should be said differently?
7. How could this [communication type] be improved?
8. Is there a way to say this differently that would make you personally more likely to notice and think about the message?
9. Thinking back to the information we are trying to convey, is there anything else you would add?
Efficacy
What do you think this [communication type] is recommending that you do?
How likely is it that you would try to do this? Please explain.
Would you consider doing this? Why or why not?
What makes it hard to do this?
Which of these options would be the easiest to incorporate into your life?
What makes that option is the easiest?
What do you think of this idea?
How appealing is it to you as a way to control [INSERT health topic or behavior]?
Do you think it could work in your home? Why or why not?
Would you consider doing this at home?
How would you try it out in your home?
How feasible would this be to try at home? Explain.
What would make this easier for you to do?
How would you try this at work or when not at home?
How feasible would this be to try at your workplace?
As you look at these [communication types], does any one of these stand out as the one that you would prefer? Which one? Why?
Is there any more information that you would need that would help you decide whether to [USE, EAT] this [PRODUCT TYPE]?
How easy are these [GUIDELINES, LABELS] for you to follow and understand?
What, if anything, makes the [GUIDELINES, LABELS] difficult to follow?
How might this be presented in an easier way?
What are the good things about trying this?
What makes it hard to do this?
Who in your household would be against your trying this? Why?
[SHOW SHEET WITH DESCRIPTIONS FOR DRUG, MEDICAL DEVICE, or FOOD PRODUCT] Please review these descriptions and mark those descriptions that you feel make you most likely to [USE, CONSUME] the [PRODUCT TYPE].
Identify the two or three descriptions that you think are most important.
Which of these is most important to you?
Why did you choose that description?
Placement
Do you like the idea of having printed materials to take with you into the doctor's appointment or to take home from the doctor’s office?
Do you think you would take one with you when you go to see the doctor or leave it in the waiting room? Why?
Does your answer to (2) affect the chance that you will read the information? How?
Does your answer to (2) affect your interest in asking your doctor a question about it? How?
Where would you expect to see these printed materials?
Where should they be placed so that you pay attention to them?
In what form do you prefer to see medical information presented (posters, brochures, fliers, online)?
What types of medical information would you find useful and work best for your needs?
something small enough to carry with me
an updatable list for the medicines I take
a device or smart phone application that warns me about problems mixing medicines and supplements
a device or smart phone application that compares medicines for a specific diagnosis
Channels
Where do you get your information about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
Probe for sources such as media, family, friends, or church
Has your doctor talked with you about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
What did he/she tell you?
What are some of the ways you have gotten information about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT] prior to today? [Probe: role of media, word-of-mouth, other.]
Is [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT] a topic on which you seek out information? If so, how?
What type of information are you seeking?
Where might you seek out information?
Where would you turn first for information?
How often, if at all, do you pick up information about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
What are the sources of this information?
When it comes to [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT], are there any organizations that you would really trust as a reliable source of information?
What makes them a trusted source of [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT] information?
What do you think [INSERT organization name] needs to know about you? How would you want them to be involved with you? Do you see ways of partnering with them? How would you like to receive the information?
How do you usually learn about [topic]l? Who or what is your main source of trusted information about this issue? Is there adequate information out there? How do these sources compare to [INSERT organization name]?
How would you rank your level of knowledge and understanding of information you have received from [INSERT organization name]?
5: knowledge is extensive
4: above average
3: average
2: have some knowledge
1: no knowledge
How could this information be conveyed more effectively?
What types of information would you like to receive regarding [topic]?
What would be the most effective way or format to provide this on information [INSERT health topic or behavior]? (Probes: Video? Educational pamphlets? Public meetings? Internet?)
What are some places where you might notice messages like these?
Are there some places in particular that you would be most likely to notice and pay attention to these messages?
Source of Information
1. If you are trying to put together a group of people to deal with [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT] in a comprehensive fashion, who are the people you want at the table?
• Probe: community based organizations, health care practitioners, and policy m makers
• Probe: religious leaders/faith based organizations
• Probe: others with the same medical needs as yourself?
• Probe: What is it about these people that make you want them there?
• Probe: Once you get these people in the room, what is the conversation going to sound like?
2. What topics do you want to cover? What will be the outcome of this conversation, what kinds of things are going to happen as a result?
3. How do you feel about FDA as the source of this information?
Spokespeople/ Sponsor
1. Who do you think would be a good spokesperson to use to convince you and your friends to [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
2. Who would have the ability to influence you?
Would it be a physician, celebrity, religious or spiritual leader, or someone like you?
3. If you were trying to make up your mind about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT], who would influence you?
4. If you were trying to influence a friend about [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, FOOD PRODUCT], what would you say?
Probe for benefits and ways to overcome barriers
5. Have you ever heard of an organization called the Food and Drug Administration or FDA?
6. What if the FDA was to say something like this? Would that change the way you look at these statements? Would it make any of them more or less believable? More or less appealing? Motivating?
7. What if the FDA and [INSERT partner name] said something like this? Would that change the way you look at these statements? Would it make any of them more or less believable? More or less appealing? Motivating?
Comparison of Concepts/ Messages/ Materials
1. Do you think one is more appealing than the others? Which? Why/why not?
2. Is one more likely than another to get your attention?
• To make you think about it afterwards?
• If yes, what is it about that one that “works” for you (or someone else you care for or take care of, such as a spouse/partner, child, parent, relative, or friend)?
3. Now that you’ve seen all of these concepts, which one catches your attention the most?
4. Looking over all of the different messages we have discussed, which two or three are the most effective?
Probe: What makes it most effective?
5. Which one was most inspiring or motivating for you personally?
6. Which one is most believable?
7. Now, looking at just the top two or three messages, which one message is the most important to you to help you with [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT] and that you have the greatest likelihood of doing?
8. What makes this message most effective?
9. What about this particular one is most engaging?
10. What item is least effective?
11. Which of these is the worst? Why? What, specifically, do you dislike about it? What do you mean by "worst?" What criteria did you use?
• Least likely to attract attention?
• Least likely to read?
• Least likely to act upon?
12. Did any of the concepts not motivate you at all? Why not?
13. Did any of the concepts turn you off? What was it about the statement/s that turned you off?
14. Would any of the statements make you feel opposed to [INSERT NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
15. What could be changed to make it more effective?
16. What information would most motivate you to ask your doctor about [NAME OF DRUG, MEDICAL DEVICE, VACCINE, or FOOD PRODUCT]?
17. What is the most motivational format for this information?
Questions for parents/children
1. How does this concept fit relative to your everyday challenges raising your teen (or child) with [INSERT NAME OF DRUG or MEDICAL DEVICE]?
Probes: Dealing with teachers/administrators and academic obstacles?
2. What do parents need to know about being effective advocates for their children?
3. In the context of this concept, how would you complete the following sentence, “As a parent raising a teen (or a child or baby) who uses [INSERT NAME OF DRUG or MEDICAL DEVICE], be prepared for________.”
4. What challenges do you face for which you would like to receive tips from other parents?
5. What coping strategies have you learned from other parents?
6. If you had three key pieces of advice for other parents, what would they be?
7. How relevant is this concept for parents of teens (or children) who use [INSERT NAME OF DRUG or MEDICAL DEVICE]?
8. How does this concept fit relative to giving teens (or children) greater independence and responsibility?
9. What are your concerns with maintaining your teen’s (or child or baby’s) safety while giving him/her room to self-manage his/her use of [INSERT NAME OF DRUG or MEDICAL DEVICE] and be more independent?
10. How does providing your teen (or your child) with greater independence affect your family?
Probe: Does it increase or decrease family stress?
11. What are the most important lessons that you have learned with regard to your teen’s (or your child’s) independence?
12. If you could give other parents advice about giving teens (or children) more independence, what would you say?
13. What have your teens (or children) told you about managing their use of [INSERT NAME OF DRUG or MEDICAL DEVICE] that has been helpful?
14. What have your teens (or your children) said to you about managing their use of [NAME OF DRUG or MEDICAL DEVICE] that has been difficult for you to hear?
15. What have you heard from your teens (or your children) concerning facing typical teen (or child) challenges?
16. What have you learned from your teen (or your child) about disclosing their use of [INSERT NAME OF DRUG or MEDICAL DEVICE] amid the stigma surrounding [INSERT NAME OF DRUG or MEDICAL DEVICE]?
17. What topics would you like to hear more about from your teen (or child)?
18. What resources do parents need when they feel alone and unsupported in raising their teens?
19. How important is it to stay connected to other parents of teens (or children) with [INSERT NAME OF DRUG or MEDICAL DEVICE]?
Questions about influenza
1. What is an influenza pandemic?
2. What the chances there will be another influenza pandemic?
3. Is the United States prepared for an influenza pandemic?
4. Why are public health officials worried about an influenza pandemic?
5. Will the measures people were told to take during the flu vaccine shortage in 2009-10 protect people from pandemic influenza (hand washing, covering your mouth when coughing)?
6. If pandemic influenza comes into the United States, who is likely to get it first?
7. What should people do in the event of an influenza pandemic?
8. Can a vaccine be made to protect against pandemic influenza?
9. How long will it take to make enough pandemic influenza vaccine for everyone?
10. Who decides who will get vaccine first and how will they decide?
11. Are there available medicines that can prevent or treat influenza if a pandemic breaks out? How effective are they?
12. What numbers of people (by age, medical condition, and socioeconomic class) are likely to get influenza in a pandemic? How many will die? What are your assumptions?
13. Who will get pandemic influenza vaccine if there is not enough for everyone?
14. Why aren’t young children and critical service workers higher on the list of priority groups for pandemic influenza vaccination?
Internet Searches
1. How did you find the [webpage]?
• [Search engine]
• [Referral or links from other sites]
• [Media/news story]
• [Received information from my doctor]
• [OPTION]
• Other:_________
2. Did you find what you were looking for [on the FDA website today]?
• [Yes]
• [No]
• [Somewhat]
• [Other:___________________]
3. If you answered no to Question 2: What were you looking for
4. Would you like [options for interacting electronically with the website]? [Freeform Answer]
5. Which of the following [technologies] would you use if available [on the Web site]?
• [Download Podcasts or other Audio/Visual presentations]
• [Watch webinars]
• [Sign up for mobile alerts]
• [Sign up for email alerts]
• [Make general comments/suggestions in a forum-style manner]
• [Other:___________________]
6. How often do you [use SITE]?
• [Daily]
• [Weekly]
• [Monthly]
• [A few times year]
• [Never]
7. How would you rate [your overall satisfaction with SITE]?
• [Very satisfied]
• [Satisfied]
• [Dissatisfied]
• [Very dissatisfied]
8. What can we do to [improve your satisfaction with SITE]? [Freeform answer]
9. What is the #1 reason you [usually visit SITE]?
[Freeform answer]
10. How do you usually [find information on SITE]?
• [I go to the [SITE home page]
• [I have the information bookmarked / I know the web page address ]
• [I use the [A-Z Index]
• [I use the [search engine]
• I use the databases
• [I use another search engine (such as Google.com, etc.) ]
• [Other: _________________]
11. Please rate [your level of agreement with the following]:
• [Information is easy to find.]
• [Web pages are well organized.]
• [Content is useful. ]
• [Information is current.]
• [Search results are useful].
• ['Look and feel' is pleasing].
12. If you could make one change to the website, what would it be?
13. With what aspect of the website are you most satisfied?
14. With what aspect of the website are you least satisfied?
15. What are the [SITE, e.g. top 3 other (non-FDA) Web sites] you [visit to find similar information for work-related purposes]? [Freeform answer]
16. What features have you seen on [SITE, e.g. these Web sites] that are helpful to you or that you would like to see incorporated into [SITE]? [Freeform answer]
17. [Please review the following audiences. Rank these audiences in order of importance for [SITE], with "1" being the MOST important audience and "10" the LEAST important. Each audience should be given a different number.]
• [Citizens/consumers/general public]
• [Manufacturers]
• [Educators or teachers]
• [Healthcare providers]
• [Journalists]
• [Policymakers, legislators, or staff]
18. Who do you think is [the main audience for SITE]? [Freeform answer]
19. Briefly describe the #1 reason that [AUDIENCE visit SITE, in your opinion.] [Freeform answer]
20. In your opinion, how [effective is SITE, e.g. current Web site in meeting the needs of AUDIENCE]?
• [Very effective]
• [Somewhat effective]
• [Somewhat ineffective]
• [Very ineffective]
21. Additional comments about [SITE] or suggestions for [improving user-friendliness]? [Freeform answer]
22. What kinds of problems would [AUDIENCE have using SITE]? [Freeform answer]
23. Describe the [SITE]. [Freeform answer]
24. I think [SITE] is [easy to use].
25. What aspects of [SITE] did you [find difficult to understand]?
26. Describe any improvements we could make to [SITE to better meet the needs of AUDIENCE]. [Freeform answer]
27. Please list any additional audiences [that SITE should be targeting]: [Freeform answer]
28. Choose the top [INFO, e.g. 2 options that you think will have the greatest impact on improving SITE].
• [OPTION]
• [OPTION]
• [OPTION]
• [OPTION]
29. Do you think some [AUDIENCE, e.g. people] [would have problems using SITE]? [Freeform answer]
30. What kinds of [AUDIENCE, e.g. people] [would have problems using SITE]? [Freeform answer]
31. What is your typical purpose [when searching for info on the Web]?
• [Specific [INFO] for myself]
• [Specific [INFO] for someone else (loved one, family member, patient, client)
• [Other :____________]
32. Where do you typically acquire [INFO]?
• [Internet/Web]
• [Radio]
• [TV]
• [Blog]
• [Don’t consult sources]
• [Other: _______]
33. Which do you use [most]?
• [Internet/Web]
• [Radio]
• [TV]
• [Blog]
• [Don’t consult sources]
• [Other: _______]
34. Do you use [search engines] more often than [the navigation links on a Web site]? [Freeform answer]
35. How do you typically [access info on the Internet/Web]?
• [Search engine (Examples: Google, Yahoo.)]
• [Specific [SITE, e.g. Web sites/pages/applications]
• From links in emails I receive
• From links in social media (for example, Facebook, Twitter)
• [Online Newsletters of Listservs]
• [Other:________]
36. What [SITE] do you visit for [INFO]?
37. Which [SITE] do you find useful for [acquiring INFO]?
• [HHS (Department of Health and Human Services)]
• [FDA]
• [CDC]
• [NIH (National Institutes of Health)]
• [WebMD]
• [Yahoo! Health]
• [Other: ________]
38. How do you usually find and select [Web sites - and specific pages on those Web sites] when [researching INFO]?
• [I choose specific Web sites that I am already familiar with]
• [I choose Web sites that appear on search engines (e.g., Yahoo, Google) ]
• [I look at Web sites mentioned in TV, news, or radio reports]
• [I look at Web sites mentioned in publications I read]
• [I look at Web sites I hear about from friends, family or co-workers]
• [Other: _________________ ]
39. Have you ever [visited SITE before]?
• [Yes]
• [No]
• [I’m not sure]
40. How often do you [visit SITE]?
• [Daily]
• [Weekly]
• [Monthly]
• [Never]
41. Is there any other way you have [obtained INFO, e.g. information from FDA] besides [using their Web site]? [Freeform answer]
42. Does the [depth of information on SITE] provide:
• [Too much detail]
• [Just the right amount of detail]
• [Not enough detail]
43. In your own words, [what is the main purpose of SITE]? [Freeform Answer]
44. Describe the [SITE, e.g. FDA home page]. [Freeform Answer]
45. How would you rate [SITE on the ease of finding information]? [Freeform Answer]
46. How likely are you to [visit this Web site again]? [Freeform Answer]
47. How [easy was] it for you to [find the program’s contact information]? [Freeform Answer]
48. What would make it [easier for] you to [find the program’s contact information]? [Freeform Answer]
49. How do you generally use [INFO from SITE]?
• [Read online]
• [Print for reference]
• [Print for hand-out]
• [Email to others]
• [Refer others to the Web site]
• [Other ________________]
50. [Please rate the following statements [Strongly disagree], [Disagree], [Neither agree nor disagree], [Agree], or [Strongly agree].]
• [Statement]
• [Statement]
• [Statement]
51. I think [SITE or INFO] is:
• [Credible]
• [Scientific]
• [Up-to-date]
• [Action-oriented]
52. Have you ever visited [SITE] before?
Have you been affected by [INFO, e.g. insert public health emergency or health topic] [in the past 2 years]?
• [Yes]
• [No]
How have you been affected by [INFO, e.g. insert public health emergency or health topic] [in the past 2 years]?
• [OPTION #1.]
• [OPTION #2.]
• [OPTION #3.]
• [OPTION #4.]
• [Other:______________]
Which of the following, if any, do you [associate with INFO]?
• [Credible]
• [Current / up-to-date information]
• [Easy to use]
• [Friendly]
• [Helpful]
• [Entertaining ]
• [Good use of my time]
• [Other: _________________]
What were your first impressions of [SITE or INFO]? [Freeform answer]
What aspects of [SITE or INFO] did you find [DESCRIPTOR]? [Freeform answer]
Overall, how satisfied were you with [SITE] after [exploring the SITE's design, content, and features]?
• [Very Satisfied]
• [Somewhat Satisfied]
• [Neutral]
• [Somewhat Satisfied ]
• [Very Satisfied]
How likely is it that you will [still be using the Web site 2 weeks from now]?
• [Definitely will still be using TIME from now]
• [Probably will still be using TIME from now]
• [Might or might not be using TIME from now]
• [Probably will NOT be using TIME from now]
• [Definitely will NOT be using TIME from now]
What [INFO], if any, would you like [added to SITE]? [Freeform answer]
[Now we would like your feedback on SITE or INFO]. [Without clicking anywhere, please spend as much time as you would in real life learning about all the different INFO that SITE offers.] [When you have a good understanding of INFO that SITE offers, please press 'BUTTON']
How would you rate [INFO on SITE]?
• [Very [DESCRIPTOR]
• [Somewhat [DESCRIPTOR]
• [Neutral]
• [Somewhat [DESCRIPTOR]
• [Very [DESCRIPTOR]
What content or information, if any, do feel is [missing from SITE or INFO]? [Freeform answer]
How [likely] are you to [ACTION]?
• [Very Likely]
• [Somewhat Likely]
• [Neutral]
• [Somewhat Likely]
• [Very Likely]
Which of the following best describes [what you attempted to do on SITE today]?
• [Learn about [SITE]]
• [OPTION]
• [OPTION]
• [OPTION]
• [OPTION]
• [None of the above ]
• [Other: _________________]
How [DESCRIPTOR, e.g., .helpful] was [the information provided about SITE or INFO]? [Freeform answer]
Which of the following, if any, problems or frustrations did you experience while [ACTION, using] [SITE or INFO]?
• [Navigating from page to page was difficult ]
• [Pages would not load / Technical Issues]
• [Site was slow]
• [Too many categories or sections]
• [Too few categories or sections ]
• [Product and service categories were not useful ]
• [Organization of the site was confusing ]
• [Labels / content used were unclear ]
• [I could not find the product / service / information I was looking for]
• [I got lost on the site]
• [I did not encounter any problems or frustrations]
• [Other: _________________]
68. What [INFO] were you looking for that you [could not find]?
[Freeform answer]
69. What [INFO] could be [DESCRIPTOR, e.g. more clear]?
[Freeform answer]
70. How [DESCRIPTOR, e.g. easy] was it for you to [determine INFO]?
• [Extremely [DESCRIPTOR]
• [Somewhat [DESCRIPTOR]
• [Neutral]
• [Somewhat [opposite of DESCRIPTOR]
• [Extremely [opposite of DESCRIPTOR]
71. What would make it [easier] for you to [ACTION, e.g. find] [INFO]? [Freeform answer]
72. How [DESCRIPTOR] did you find [INFO]?
• [Extremely [DESCRIPTOR]
• [Somewhat [DESCRIPTOR]
• [Neutral]
• [Somewhat [opposite of DESCRIPTOR]
• [Extremely [opposite of DESCRIPTOR]
73. How could [the information provided about INFO] be more [DESCRIPTOR]? [Freeform answer]
74. What [INFO], if any, do you feel [was missing]? [Freeform answer]
75. How would you rate the [ease of finding information about INFO]?
• [Very [DESCRIPTOR]
• [Somewhat [DESCRIPTOR]
• [Neutral ]
• [Somewhat [opposite of DESCRIPTOR]
• [Very [opposite of DESCRIPTOR]
76. After reviewing [SITE], what additional questions, if any, do you still have [about SITE]? [Freeform answer]
77. Please indicate how likely you are, if at all, to do the following
• [Definitely would [ACTION]
• [Probably would [ACTION]
• [Probably would not [ACTION]
• [Definitely would not [ACTION]
78. Based on your experience today, [how DESCRIPTOR were you] [with your ability to find INFO you needed]?
• [Not at all [DESCRIPTOR]
• [Neutral]
• [Extremely [DESCRIPTOR]
79. Which of the following best describes how you [navigate while looking for health information on your mobile device]?
• I have never [ACTION][SITE or INFO] online before
• [OPTION]
• Other: _________________
80. Which of [the following tools] do you [frequently use when looking for test clinics online]?
• [OPTION]
• [OPTION]
• [OPTION]
• Other: _________________
81. How often do you [look for drug/medical device/medical product/animal medical product information online]?
• [I have never looked online before]
• [Once per year or less]
• [A few times per year]
• [Once per month]
• [A few times per month]
• [Once per week or more often]
82. How likely are you to do the following [within the next 24 hours]?
• [OPTION]
• [OPTION]
• [OPTION]
83. What about [SITE or INFO], made a lasting impression on you? [Freeform Answer]
84. How would you [have contacted the FDA to ask your question or obtain the information you were looking for]?
[Send an email ]
[Browse the Website]
[Call FDA's 800 number]
[Write a letter to FDA]
[Would not have contacted FDA]
[Other:____________]
85. If you plan on [following up with FDA to clarify the answer you received during this site visit,] [how would you do that]?
[Send an email]
[Call FDA’s 800 number]
[Write a letter to FDA]
[Return to the Website at a later time]
[Other:____________]
86. Did you [call the Call Center]?
[Yes]
[No]
87. Was [your problem or question] resolved? [Freeform answer]
88. If you were [dissatisfied with Call Center's resolution of your problem or question], what was the reason?
[Nothing was unsatisfactory]
[On hold too long]
[Took too long to receive e-mail response]
[Staff was not knowledgeable about my question/problem]
[Did not receive a return e-mail or follow-up call]
[Problem cannot be resolved at this time]
[OPTION]
[OPTION]
[Other:____________]
89. How [satisfied] were you with [the availability of on-site help]? [Freeform answer]
90. In the [last 30 days] how many times have [you visited the FDA’s Web site instead of calling the call center]? [Freeform answer]
91. Please rate [the degree to which the Call Center representative(s) spoke clearly]. [Freeform answer]
92. If we made the changes you have mentioned would you be [more or less likely to use it]?
• [Much more]
• [Maybe a little more]
• [About the same]
• [Less likely]
93. Please rate [the accuracy of information on this site]. [Freeform Answer]
94. Please rate [the degree to which the Call Center representative(s) seemed interested in your concern, suggestion, or report]
95. What is your overall satisfaction [with this site]? [Freeform Answer]
96. How well does [this site meet your expectations]? [Freeform Answer]
97. How does [this site compare to your idea of an ideal website]? [Freeform Answer]
98. How likely are you [to return to this site]? [Freeform Answer]
99. How did you [find this site]?
• [Search engine]
• [Referral or links from other sites]
• [Media/news story]
• [Other, please specify:]
100. Do you have any accessibility concerns with this site, [such as small font sizes, color or contrast, or issues with your screen reader]? [Freeform Answer]
101. What other [Web sites] do you use to find health information]? [Freeform Answer]
102. In general, how much would you trust [information about health or medical topics from a doctor or health care professional]?
• [A lot]
• [Some]
• [A little]
• [Not at all]
103. Some [newspapers or general magazines publish a special section that focuses on health]. In the past 12 months, have you [read health sections of the newspaper or of a general magazine]?
• [Yes]
• [No]
104. Some people notice [information about health on the Internet], even when they are not trying to find [out about a health concern they have or someone in the family may have]. [Have you read such health information on the Internet in the past 12 months?]
• [Yes]
• [No]
105. About how often do you use [the Internet]?
• [Every few weeks]
• [Less often]
• [Never]
• [Don’t know/refused]
106. As I read the following list of items, please tell me if you, personally, happen to have each one, or not.
• [A desktop computer]
• [A laptop computer]
• [A Smart phone such as the iPhone or Android]
• [A cell phone]
• [A personal digital device, like a Sidekick, Palm Pilot or Blackberry]
• [An iPod or other MP3 player]
107. Have you heard about [health topic/campaign]? [Freeform Answer]
108. What can you tell me about [health topic/campaign]? [Freeform Answer]
109. Do you like the idea of having [material to take with you into a doctor's appointment or to take home]? [Freeform Answer]
110. How do you prefer to see [health information presented]? [In what form (probe: posters, brochures, fliers)?] [Freeform Answer]
111. What kinds of promotional items would you use? [Freeform Answer]
112. What are some of the ways you have gotten information about [health behavior] prior to today? [Freeform Answer]
113. When it comes to [INSERT health topic or behavior], are there any organizations that you would really trust as a reliable source [of information]? [What makes them a trusted source of health topic information?] [Freeform Answer]
114. How would you rank your level of knowledge and understanding of [information you have received from FDA]?
• [knowledge is extensive]
• [above average]
• [average]
• [have some knowledge]
• [no knowledge]
115. Do any [companies or organizations say] something like this now? Which ones? [Freeform Answer]
116. Do you think one is [more appealing] than the others? [Which? Why/why not?] [Freeform Answer]
117. Now that you’ve seen [all of these concepts], which one [catches your attention the most]? [Freeform Answer]
118. What is the most [motivational] format for [this information]? [Freeform Answer]
119. How relevant is this concept for [parents of teens (or children) who [are worried about smoking]? [Freeform Answer]
120. What resources do [parents need when they feel alone and unsupported in raising their teens]? [Freeform Answer]
121. Looking at the image, would you say it [fits in with what they are trying to convey]? [Freeform Answer]
122. What is the main message of [this illustration]? [What does it tell you?] [Freeform Answer]
123. Picturing this image on [a poster, brochure, print ad, or other material,] how likely is it [that an image like this will help draw your attention and curiosity]? [Freeform Answer]
124. Are there other [images] that might [convey this idea better]? [Freeform Answer]
125. How do you feel about [the images] used in this concept? [Are they helpful/engaging? Why/why not]? [Freeform Answer]
126. Is the color [appropriate]? [Freeform Answer]
127. What do you think about the [length of the content on the Web page]? [Freeform Answer]
128. Would you change anything about your current routine after [seeing this]? [Freeform Answer]
129. Before [being contacted for this study], had you ever heard of the [INSERT organization name]?
[Yes]
[No]
Attachment 8: Activity/Task questions
[*There will be no more than 10 activity / task questions relevant to the specific Website/page/ application. Respondents will be asked to click through the specific Website/page/application to complete the activities. For each activity question, there will be a series of follow up questions to evaluate the participant’s ease or difficulty in finding the answer to the question. An example of a task question and follow up questions is shown below. All questions will be either radio button, freeform, or rating scale.*]
Note: You must include some Activity/Task questions. You may change the entire text in the following example Activity / Task questions, as needed, for each test to focus on the key areas of the Website/page/application.
“GENERAL” USABILITY ACTIVITY / TASK QUESTIONS
Instructions
Example Activity 1 of 10
[If you live in an area where a food you regularly eat has been identified by the FDA as unsafe to eat, and you have not been asked to stop eating it, what steps would you take?]
Example Activity 2 of 10
[If you have a food in your house that you have heard was contaminated, would you throw it out?]
[Continue with up to 9 more activity questions and follow up questions.]
FIRST-CLICK TESTING
Instructions
example 1: [For the following question, please make [insert number] click(s) in an attempt to find the answer. You do not need to find the actual answer to the question.]
example 2: [For the following question, please visit [insert number] page(s) in an attempt to find the answer. You do not need to find the actual answer to the question.]
Task
example 1: [Where would you find information about ______?]
[Were would you find information about visiting the FDA Webpage?]
example 2: [Where would you find _______?]
[Where would you find a Drug Safety Communication?]
CARD SORTS
Instructions
example: [We are conducting research that will help us gain a better understanding of how our Website should be organized and make it easier to use.]
Task
example:
[Review the items in the left column. These items represent content on our current Website.]
[Place all of the items that belong together into the same group. Drag the items that belong in the same category from the left column over to the right column. When you are finished adding items to a category, click the yellow box in the middle column to name each group.]
[If an item is unfamiliar, you may create a category called “Miscellaneous”.]
[There is no correct number of groups, but make sure that you think about how the items relate to each other. If you have a group with a large amount of items, consider splitting it up.]
[You must do the exercise in one sitting. Please do not leave the browser to go to another task - you will not be able to return.]
(Questions can be used in intercept interviews, telephone interviews, individual in-depth interviews [cognitive interview], online research, and focus group discussion guides.)
Testing Images/Visuals/Illustrations
Looking at the image, would you say it fits in with what they are trying to convey?
What is the main message of this illustration? What does it tell you?
Picturing this image on a poster, brochure, print ad, or other material how likely is it that a image like this will help draw your attention and curiosity - that’s you personally, no one else
For you, are there other images that might convey this idea better?
How do you feel about the images used in this concept? Are they [helpful/engaging]? Why/why not?
What is your general reaction to the way these look?
How appealing is this image to you personally?
How do you feel about the colors and graphics? What do you like/dislike about them?
Would it catch your attention if you saw it somewhere?
Would you take the brochure (print ad, etc.) to view the additional information? Why/why not? Is there anything that you would say should be changed about the way this looks that would help make someone like you think about this after you’ve looked at it?
What are there things about the pictures that you think are particularly ATTENTION-GETTING? APPEALING?
Things about the pictures that BOTHER you in any way?
What makes it stand out?
Are you able to see the headline and text easily?
What do you think about the type?
Is it easy/difficult to read? [Probe: Font too big? Too small? Too dense?]
Is the color appropriate?
What do you think about the length of the piece?
Testing Logos
If you had to give this image a grade, like in school, where "A" is the best and "F" is a failing grade, then what grade would you give this logo as a choice for a health education campaign?
Would you think of having the logo of a governmental organization, such as the Food and Drug Administration or FDA, on them? Have you ever heard of the FDA?
What would you think of having the logo of the [INSERT organization name] on these concepts?
How about having the logo of a charitable organization such as [INSERT organization name] on these concepts?
Of all these different sponsors we've discussed, to you, which would be best to put on materials?
Which would make the information most believable?
Which would make the information most motivating?
Testing Media Messages (e.g., radio advertisements, print advertisements, etc.)
|
Strongly Agree |
|
|
|
Strongly Disagree |
Overall, I liked this ad |
1 |
2 |
3 |
4 |
5 |
I liked the people in this ad |
1 |
2 |
3 |
4 |
5 |
I liked the voices in this ad |
1 |
2 |
3 |
4 |
5 |
This ad was easy to listen to (to read) |
1 |
2 |
3 |
4 |
5 |
I learned something new by listening (by reading) this ad |
1 |
2 |
3 |
4 |
5 |
I was interested in listening to (reading) this ad |
1 |
2 |
3 |
4 |
5 |
This ad was made/written for a person like me |
1 |
2 |
3 |
4 |
5 |
This ad was easy to understand |
1 |
2 |
3 |
4 |
5 |
I liked the sound effects in this ad |
1 |
2 |
3 |
4 |
5 |
I would look at (read) this ad if I saw it |
1 |
2 |
3 |
4 |
5 |
I am interested in this ad’s topic |
1 |
2 |
3 |
4 |
5 |
I like the way this ad looks |
1 |
2 |
3 |
4 |
5 |
I liked the colors in this ad |
1 |
2 |
3 |
4 |
5 |
I trust the information in this ad |
1 |
2 |
3 |
4 |
5 |
I can do what this ad suggests |
1 |
2 |
3 |
4 |
5 |
I will do what this ad suggests |
1 |
2 |
3 |
4 |
5 |
I trust the information in this ad |
1 |
2 |
3 |
4 |
5 |
What other comments would you like to make about this ad?
What was the main message of this ad?
Is there anything you especially liked about this ad?
Is there anything you would change about this ad?
Where would you want to see an ad such as this one?
Testing Brochures/Booklets
First look (without reading). On a scale from 1 to 5, where 1 indicates that you strongly disagree, and 5 indicates that you strongly agree, please tell me the number which indicates how much you agree or disagree with each statement.
|
Strongly Agree |
|
|
|
Strongly Disagree |
I would pick up this brochure if I saw it |
1 |
2 |
3 |
4 |
5 |
I would read this brochure if I saw it |
1 |
2 |
3 |
4 |
5 |
I am interested in this brochure’s topic |
1 |
2 |
3 |
4 |
5 |
I like the way this brochure looks |
1 |
2 |
3 |
4 |
5 |
Second look (after reading). Using the same scale as before, tell me the number which indicates how much you agree or disagree with each of the following statements.
|
Strongly Agree |
|
|
|
Strongly Disagree |
|
|
|
|
|
|
Overall, I liked this brochure |
1 |
2 |
3 |
4 |
5 |
I liked the pictures in this brochure |
1 |
2 |
3 |
4 |
5 |
This brochure was easy to read |
1 |
2 |
3 |
4 |
5 |
I learned something new by reading this |
1 |
2 |
3 |
4 |
5 |
I was interested in reading this |
1 |
2 |
3 |
4 |
5 |
This was written for women like me |
1 |
2 |
3 |
4 |
5 |
This brochure was easy to understand |
1 |
2 |
3 |
4 |
5 |
I liked the colors in this brochure |
1 |
2 |
3 |
4 |
5 |
I trust the information in this brochure |
1 |
2 |
3 |
4 |
5 |
I can do what this brochure suggests |
1 |
2 |
3 |
4 |
5 |
I will do what this brochure suggests |
1 |
2 |
3 |
4 |
5 |
What other comments would you like to make about this brochure?
What was the main message of this brochure?
Is there anything you especially liked about this brochure?
Is there anything you would change about this brochure?
Where would you want to see a brochure such as this one?
Please indicate how much you agree or disagree with the following statements about the message included in the (INSERT FORMAT - print ad, brochure, video, etc.). Use this scale to respond to the following questions:
|
Strongly Disagree |
|
|
|
Strongly Agree |
This [INSERT format] was convincing. |
1 |
2 |
3 |
4 |
5 |
This [INSERT format] said something important to me. |
1 |
2 |
3 |
4 |
5 |
I liked this [INSERT format] overall. |
1 |
2 |
3 |
4 |
5 |
This [INSERT format] grabbed my attention. |
1 |
2 |
3 |
4 |
5 |
This [INSERT format] told me something I didn’t already know. |
1 |
2 |
3 |
4 |
5 |
This [INSERT format] gave me good reasons to [listen to (INSERT FDA]) |
1 |
2 |
3 |
4 |
5 |
This [insert format] was confusing. |
1 |
2 |
3 |
4 |
5 |
This [insert format] spoke to me. |
1 |
2 |
3 |
4 |
5 |
I do not like this [insert format]. |
1 |
2 |
3 |
4 |
5 |
Persuasiveness
First, please the persuasiveness of the [INSERT FORMAT – print ad, video, etc] you just saw. Be honest, how persuasive was it?
|
Strongly Disagree |
|
|
|
Strongly Agree |
The message that I saw about [insert TOPIC] was compelling. |
1 |
2 |
3 |
4 |
5 |
The message about [insert TOPIC] was persuasive. |
1 |
2 |
3 |
4 |
5 |
The message was dumb. |
1 |
2 |
3 |
4 |
5 |
The message was weak. |
1 |
2 |
3 |
4 |
5 |
Attachment 9: Follow up questions
Follow-Up Questions
(Questions can be used in intercept interviews, telephone interviews, individual in-depth interviews [cognitive interview], online research, and focus group discussion guides.)
Does this message make you want to take action?
Would it make you think more about the importance of [safely using NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT, or FOOD PRODUCT]?
Would it make you want to [safely use NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT or FOOD PRODUCT]?
Would it make you likely to do something about increasing [your safe use of NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT or FOOD PRODUCT]?
Would you change anything about your current routine after seeing this?
Do you plan to follow its advice?
Doctors are interested in what they can say to motivate patients to [safely use NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT, or FOOD PRODUCT]. What is your advice to them? That is, what should doctors tell people if they want them to [safely use NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT, or FOOD PRODUCT]?
Was there any advice that you would not follow or comply with? Explain.
Is there anything that could be changed to make it more likely you would [safely use NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT, or FOOD PRODUCT]?
If you saw a story on [NAME OF DRUG, MEDICAL DEVICE, VACCINE PRODUCT, ANIMAL DRUG PRODUCT or FOOD PRODUCT safety] (on television, in newspapers, or on the Internet], how likely would you be to pay close attention to it:
Talk about it with your family in the next week or so
Talk about it with your friends in the next week of so
Bring up the issue the next time you visit your healthcare provider
Behavioral Intentions
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1 1 1 1
|
2
2 2 2 2
|
3
3 3 3 3
|
4
4 4 4 4
|
5
5 5 5 5
|
Involvement
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1
1
1
1
1
1
1
1
1
1
|
2
2
2
2
2
2
2
2
2
2
2
|
3
3
3
3
3
3
3
3
3
3
3
|
4
4
4
4
4
4
4
4
4
4
4
|
5
5
5
5
5
5
5
5
5
5
5
|
Self-efficacy scale
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1
1
1
1 |
2
2
2
2
2 |
3
3
3
3
3 |
4
4
4
4
4 |
5
5
5
5
5 |
Attitudes
The following questions ask about your opinion on [INSERT health condition/behavior/disease/syndrome/injury/disability]].
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1
1
1
|
2
2
2
2
|
3
3
3
3
|
4
4
4
4
|
5
5
5
5
|
Response efficacy scale
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1
1
1
|
2
2
2
2
|
3
3
3
3
|
4
4
4
4
|
5
5
5
5
|
Self-protection motivation
|
Strongly Disagree |
|
|
|
Strongly Agree |
|
1
1
1
1
|
2
2
2
2
|
3
3
3
3
|
4
4
4
4
|
5
5
5
5
|
Final Follow-up Questions
[Before being contacted for this study, had you ever heard of…] FDA or the Food and Drug Administration?
Yes
No
[Before being contacted for this study, had you ever heard of…] the Department of Health and Human Services?
Yes
No
Before being contacted for this study, had you ever heard of the [INSERT organization name]?
Yes
No
What is the main purpose [of MESSAGE]? [Freeform answer]
How well do you think the main ideas come across [in this MESSAGE]? [Freeform answer]
How could this [MESSAGE] be improved? [Freeform answer]
Did you learn anything new [from this MESSAGE]? [Freeform answer]
Do you plan [to ACTION as a result of reading this MESSAGE?
[Yes]
[No]
Please describe [the steps you plan to take as a result of reading this MESSAGE]. [Freeform answer]
If there were/was a/an [recall/warning] about a drug, what would you do to find out more about it? [Freeform answer]
If there were/was a/an [recall/warning] about a medical device, what would you do to find out more about it? [Freeform answer]
If there were/was a/an [recall/warning] about a food that you eat, what would you do to find out more about it? [Freeform answer]
Was http://www.fda.gov organized in a way that [made it easy for you to find related publications]?
Describe your experience using [this Web site previously]. [Freeform Answer]
What do you think about the layout of this Web site? [Freeform Answer]
Have you ever [visited a site like this before]? [Freeform Answer]
What do you think of [the process of interaction with http://www.fda.gov on particular topics]? [Freeform answer]
What would you tell someone else about the FDA web site? [Freeform answer]
What do you think about [Podcasts/Facebook] as a way to get information about health from the FDA]? [Freeform answer]
[Please take a minute or two and read this page about the FDA website. How would you describe this website to a friend?
[Freeform answer]
After using [the FDA website today], how likely are you to do one of the following activities:
Talk to a friend or family member about the topic
Email the information to a friend or family member
Share the information online via social media such as Facebook, Twitter, Blogs, etc.
Print the information and share it
Bookmark this page for later use
Do additional research online
Order a publication from FDA.gov
How satisfied are you with [the services FDA provided]? [Freeform Answer]
What did you learn from your [call/website visit] to FDA that you did not know before? [Freeform Answer]
Did what you learned make you want to [change what you do]?
[Yes]
[No]
Please tell me more about [what changes you are thinking about.] [Freeform Answer]
Have you actually [changed any behaviors]?
[Yes]
[No]
Overall, how satisfied are you [with the quality of the health information provided by FDA]? [Freeform Answer]
Thinking about [your entire call experience], how satisfied are you [with FDA’s electronic information]? [Freeform Answer]
How well did the information provided [by FDA] [answer your questions]?
○ [Very well]
○ [Fairly well]
○ [Not very well]
○ [Not at all well]
Did the [response] you received from [FDA] [show that they]:
○ Clearly understood your needs]
○ [Somewhat understood your needs]
○ [Did not understand your needs very well]
○ [Did not understand your needs at all]
How quickly was [FDA] able to [respond to your questions]?
○ [Within 24 hours]
○ [Within 2-3 days]
○ [Up to a week]
○ [More than a week]
How satisfied are you with [the services FDA provided]?
○ [Very satisfied]
○ [More satisfied than dissatisfied]
○ [More dissatisfied than satisfied]
○ [Very dissatisfied]
What is the reason you [were not satisfied]? [Freeform Answer]
Did you get [the information] you [asked for]?
[Yes]
[No]
Did [FDA’s mobile web site] provide [the information you needed]?
[Yes]
[Somewhat]
[No]
Did the information you received] make you want [to change something you do]?
[Yes]
[No]
Thinking about your overall experience ordering a publication, how satisfied are you?
[Very satisfied]
[More satisfied than dissatisfied]
[More dissatisfied than satisfied]
[Very dissatisfied]
Who are you looking for [information] for? [Freeform Answer]
Has the information that you have received so far [been easy to understand]? [Freeform Answer]
What [information] do you still need? [Freeform Answer]
SUS: SYSTEM USABILITY SCALE
Please rate the following statements [Strongly agree], [Agree], [Neutral], [Disagree], or [Strongly disagree]:
I would recommend this site to others.
I found the website unnecessarily complex.
I thought the website was easy to use.
I think I need help sometimes to be able to effectively use this website.
I found that the various functions in this website worked well together.
I thought there was too much inconsistency in this website.
I would imagine that most people would learn to use this website very quickly.
I found this website very awkward to use.
I felt very confident using this website.
I will need to learn a lot about this website before I could effectively use
PROBES
A probe is a question largely based upon answers to the survey questions and issues that seem worth pursuing-- for example, when noticing that a respondent seems confused or spent additional time answering something. A probe may be needed at any point during a remote or in person test. The bracketed portion of the following Probes may be changed, to address individual participant actions, issues, concerns, etc during the usability test.
[Please share any additional comments you have about this Web site/page/application.]
[Could you tell me what the term “prevention” means to you?]
[Why did you answer that way?]
[In your own words, could you tell me what you think this question is asking?]
[Was this question easy or hard to answer? Why?]
[How sure are you about your answer?]
[How did you decide where to look for health information?]
[How difficult was it for you to figure out where to go next from this point?]
[Why did you search where you did?]
Example Survey
Welcome Screen
Welcome! Thank you for agreeing to help the Food and Drug Administration (FDA) evaluate their website. Your feedback is extremely important. We anticipate that it will take approximately 20 minutes to complete this questionnaire.
The first few questions will be about your purpose for using FDA.gov. Then we will ask you to perform some tasks on the website. We are not testing your abilities in any way; we are only testing the FDA website to see how well it works. Please browse the website in whatever manner is comfortable and normal for you.
Your responses to all questions will be kept in a secure manner. No personal identifiers will be recorded. All information is used for evaluation purposes only and does not involve sales of any kind. FDA does not plan to share the data with anyone outside FDA.
Click “Continue” to begin. To proceed through the survey, select your answer for each question and click “Next.”
Survey Questions
Demographic and Introductory Questions
We would like to start by asking you some introductory questions. After these questions you may be asked to do a few tasks on the CDC website.
We would like to get some demographic information about you. This information is for evaluation purposes only, and all responses will be kept in a secure manner. No personal identifiers will be stored with or linked to this information.
Are you male or female?
Male
Female
How old are you?
Under 18 years of age
18-24 years old
25-34 years old
35-44 years old
45-54 years old
55-64 years old
65-74 years old
75 years old or older
What is the highest level of education you have completed?
Grade school
Less than high school graduate/some high school
High school graduate or completed GED
Some college or technical school
Received four-year college degree
Some post graduate studies
Received advanced degree
Other: _____________________
What is your current occupational status? Would you say…
Employed
Unemployed
Homemaker
Student
Retired, or
Disabled
Other:_______________
About how often do you use the Internet?
Several times a day
About once a day
3-5 days a week
1-2 days a week
Every few weeks
Less often
Never
Don’t know/refused
As I read the following list of items, please tell me if you, personally, happen to have each one, or not.
A desktop computer
A laptop computer
A cell phone
A personal digital device, like a Sidekick, Palm Pilot , iPhone, Google Droid, or Blackberry
An iPod or other MP3 player
Core Questions
A search engine such as Google or Yahoo
Specific website (please specify __________________)
From links in emails I receive
From links in social media (for example, Facebook or Twitter)
Online Newsletters or Listservs
Other:________
Did you find what you were looking for on FDA,gov today?
Yes
No
Somewhat
Other:___________________
Which website do you find useful for finding information about health or illness?
HHS (Department of Health and Human Services)
FDA
CDC
NIH (National Institutes of Health)
WebMD]
Yahoo! Health
Other: ________
Have you ever visited the FDA website before?
Yes
No
I’m not sure
Activity/Task Questions
Now we are going to ask you to perform three tasks on the website. For each task, the page presented will be reset to the FDA.gov Home page. Please start your task from this page. Click “Next” to proceed to the first task.
Activity 1: Use FDA.gov to find information on Food Labeling.
I was able to complete the task by using the website.
I was able to complete the task by recalling prior knowledge or using another website.
I was not able to complete the task.
I think I was able to complete the task but I’m not sure.
W
Please rate the USEFULNESS of the information on this Web page scale of 1 to 10.
Not useful 1 2 3 4 5 6 7 8 9 10 Extremely Useful
Please rate the EASE OF FINDING of the information on this Web page scale of 1 to 10.
Not easy 1 2 3 4 5 6 7 8 9 10 Extremely Easy
What difficulties and frustrations, if any, did you encounter in completing this task?
Link labels were unclear
Information was unclear
I was lost on the website
The task took too long too complete
There were too many link choices
Links did not go where I expected them to go
The information was not where I thought it would be
I could not find the information
The search function was difficult to use
The information was available but not complete
Information on a topic was located in more than one place on the website
I had no difficulties or frustrations in completing this task
Other (please specify): _______________________
Activity 2: Use FDA,gov to find out how to use acetaminophen (for example, Tylenol) safely.
I was able to complete the task by using the website.
I was able to complete the task by recalling prior knowledge or using another website.
I was not able to complete the task.
I think I was able to complete the task but I’m not sure.
What is the maximum dosage of acetaminophen that you should take?
3 mg
35 mg
325 mg
500 mg
Please rate the USEFULNESS of the information on this Web page scale of 1 to 10.
Strongly disagree 1 2 3 4 5 6 7 8 9 10 Strongly Agree
Please rate the EASE OF FINDING of the information on this Web page scale of 1 to 10.
Strongly disagree 1 2 3 4 5 6 7 8 9 10 Strongly Agree
What difficulties and frustrations, if any, did you encounter in completing this task?
Link labels were unclear
Information was unclear
I was lost on the website
The task took too long too complete
There were too many link choices
Links did not go where I expected them to go
The information was not where I thought it would be
I could not find the information
The search function was difficult to use
The information was available but not complete
Information on a topic was located in more than one place on the website
I had no difficulties or frustrations in completing this task
Other (please specify): _______________________
Activity 3: Use the CDC website to find information about ionizing radiation.
I was able to complete the task by using the website.
I was able to complete the task by recalling prior knowledge or using another website.
I was not able to complete the task.
I think I was able to complete the task but I’m not sure.
How do you get exposed to ionizing radiation?
medical imaging exams
using your cell phone
watching television
using the toaster
Please rate the USEFULNESS of the information on this Web page scale of 1 to 10.
Strongly disagree 1 2 3 4 5 6 7 8 9 10 Strongly Agree
Please rate the EASE OF FINDING of the information on this Web page scale of 1 to 10.
Strongly disagree 1 2 3 4 5 6 7 8 9 10 Strongly Agree
What difficulties and frustrations, if any, did you encounter in completing this task?
Link labels were unclear
Information was unclear
There were too many link choices
Links did not go where I expected them to go
The information was not where I thought it would be
Information on a topic was located in more than one place on the website
I had no difficulties or frustrations in completing this task
Other (please specify): _______________________
Follow up Questions
We'd like to conclude the survey by asking about the experience you just had on the website.
Did you learn anything new from visiting FDA,.gov today?
Overall what did you like MOST about the website?
Overall what did you like LEAST about the website?
What information and/or functions would you like to see included or highlighted on this website/page/application?
D
Please rate the following statements on a scale of 1 to 10.
Strongly disagree 1 2 3 4 5 6 7 8 9 10 Strongly Agree
The health information on this website/page/application was presented in a manner that I could understand. _____
It was easy to find the information I was looking for on this website. _____
This website was visually interesting. _____
The website contained useful information. _____
I would use this website as my primary source for health or medical information. _____
I trust the information I found on this website. _____
Did you encounter difficulties with the Search feature, and if so, what was the primary issue?
I did not encounter any difficulties with the search functionality
Too many results
I was not sure what words to use in my search
The descriptions/abstracts were not helpful
The results were not relevant to my search terms
The search returned no results
The results were not relevant to my needs
Search speed was too slow
Other: ________
P
Thank you Message
Thank you! You have completed the survey. We appreciate your participation in this evaluation. We value your input and look forward to using your feedback to help us in our ongoing effort to improve our website/page/application.
| File Type | application/msword |
| Author | PHPPO_User |
| Last Modified By | miriam.campbell |
| File Modified | 2011-06-06 |
| File Created | 2011-06-06 |