Revised (Renewal) OMB Package for Usability Evaluation - Appendix 11

Updated - Revised (Renewal) OMB Package for Usability Evaluation - Appendix 11.doc

CDC Website and Communication Channels Usability Evaluation

Revised (Renewal) OMB Package for Usability Evaluation - Appendix 11

OMB: 0920-0735

Document [doc]
Download: doc | pdf

Appendix 11: New Questions

NEW Demographic Questions (part of Appendix 5)

The bracketed portion of the following Demographic Questions may be changed to suit a particular evaluation scenario.


Form Approved

OMB No.: 0920-0735

Exp Date: March 31, 2010


Are you male or female?

Male

Female



What is your age? [Freeform Answer]


What year were you born? [Freeform Answer]


How would you describe yourself?

  • [Hispanic or Latino] 

  • [Black or African American] 

  • [White]

  • [Asian] 

  • [American Indian or Alaska Native]

  • [Native Hawaiian or Other Pacific Islander] 

  • [Prefer not to answer]



What is your race and ethnicity? [Freeform Answer]


Please tell me your race or ethnic background. [Are you?]

    • Ethnicity:

      • [Hispanic or Latino]

      • [Not Hispanic or Latino]

    • Race:

      • [White/Caucasian]

      • [Black or African-American]

      • [American Indian or Alaska Native]

      • [Native Hawaiian or Other Pacific Islander]

      • [Asian]

        • [Vietnamese]

        • [Cambodian]

        • [Filipino]

        • [Japanese]

        • [Korean]

        • [Chinese]


What is your country of residence?

  • [USA] 

  • [Canada]

  • [Afghanistan] 

  • [Albania] 

  • [Algeria] 

  • [American Samoa] 

  • [Andorra] 

  • [Angola] 

  • [Argentina] 

  • [Armenia]

  • [OPTION]



What is/are your [city, state, zip code]? [Freeform Answer]



Were you born in the United States?

  • [Yes]

  • [No]


What is your current occupational status? [Would you say…?]

  • [Employed]

  • [Unemployed]

  • [Homemaker]

  • [Student]

  • [Retired]

  • [Disabled

  • [Other:_______________]


What term would you use to describe your current profession? [Freeform Answer]


What is your marital status?

  • [Married]

  • [Living as married]

  • [Divorced]

  • [Widowed]

  • [Separated]

  • [Single, never been married]


Which of the following categories best describe your total, annual household income?

  • [Under $20,000/year]

  • [$20,001 - $30,000/year]

  • [$30,001 - $40,000/year]

  • [$40,001 - $50,000/year]

  • [$50,001 - $60,000/year]

  • [$60,001 - $80,000/year]

  • [$80,001 - $100,000/year]

  • [Over $100,000/year]


Number of children [(under age 18)] [living in the household]:

  • [None]

  • [1-2 children]

  • [3-4 children]

  • [5 or more children]


How many children do you have and what are their ages?

  • Infants (less than 1 year)

  • Toddlers (1–4 years)

  • Youth (5–12 years)

  • Teens (13–18 years)

  • Adults (over 18 years)


How long have you worked in your current profession?

  • Less than one year

  • 1 year

  • 2 – 5 years

  • 6 – 10 years

  • More than 10 years


Do you currently rent or own your home?

  • [Own]

  • [Rent]

  • [Occupied without paying monetary rent]


What is your current relationship status?

  • [Single]

  • [Married to a man]

  • Married to a woman]

  • [In a relationship with a man]

  • [In a relationship with a woman]

  • [Divorced or Widowed]


Have you ever had [an HIV test]?

  • [Yes]

  • [No]


What was the result of your last [HIV test]?

  • [Positive]

  • [Negative]

  • [Don’t know]


When was the last time you [had an colorectal cancer screening test]? [Freeform Answer]


Are you the parent or guardian of a [boy/girl, ages INSERT range years]?

  • [Yes]

  • [No]


Are you or have you ever been [sexually active]?

  • [Yes]

  • [No]


Do you feel comfortable reading materials that require [a 7th grade reading leve]l?

  • [Yes]

  • [No]


Do you, or does any member of your household or immediate family work for:

  • [a market research company]

  • [an advertising agency or public relations firm]

  • [the media (TV/radio/newspapers/magazines)]

  • [as a healthcare professional (doctor, nurse, pharmacist, dietician, etc.)]


Have you [participated in a focus group, intercept interview, telephone survey, and/or online survey in which you were asked your opinions regarding a product, a service, or advertising within the past six months]?

  • [Yes]

  • [No]


How many of these have you attended [in the past six months]? [Freeform Answer]

What was/were the topics [discussed]?


NEW Introductory Questions (Part of Appendix 6)

The bracketed portion of the following Introductory Questions may be changed to suit a particular evaluation scenario.


Form Approved

OMB No.: 0920-0735

Exp Date: March 31, 2010


What is the highest level of education you have completed?

  • [High school or less] 

  • [Some college] 

  • [College degree]

  • [Advanced degree]

  • [Prefer not to answer]



Describe your work environment: [Freeform Answer]


What is your primary specialty? [Freeform Answer]


Do you have a subspecialty? [What is it?] [Freeform Answer]


Which best describes [your role at CDC]?

  • [Leadership or Program-specific]

  • [Web-related: e.g. Web site manager, developer/programmer, Web site content coordinator, Web site graphics designer, usability specialist, information architect, etc].

  • [other CDC employee: uses www.cdc.gov for a job that is NOT Leadership, Program-specific, or Web-related]

  • [Administrative or building management staff ]

  • [Center, division, or branch management ]

  • [Public health professional (epidemiologist, health communicator, health education, program analyst, etc.) ]

  • [Healthcare provider – Doctor]

  • [Healthcare provider – Nurse, Nurse practitioner, Physician’s Assistant ]

  • [Researcher, scientist, or laboratorian ]

  • [Writer, editor, or library scientist ]

  • [Other: _________________]


Are you..?

  • [Individual interested in health issues] 

  • [Patient, friend or family member of an individual interested in health issues] 

  • [Physician] 

  • [Nurse, physician's assistant, nurse practitioner] 

  • [Other, please specify:]


What [industry sector] do you work in?

  • [Agriculture, Forestry and Fishing]

  • [Construction]

  • [Healthcare and Social Assistance]

  • [Manufacturing]

  • [Mining]

  • [Services]

  • [Transportation, Warehousing and Utilities]

  • [Wholesale and Retail Trade]


Where do you [work]?

  • [Federal government]

  • [State government]

  • [Local government]

  • [Academia]


What does [workplace health and safety] mean to you? [Freeform Answer]


Please tell me [what kinds of workplace safety and health procedures] you use in your [workplace]? [Freeform Answer]


Where do you currently [get your information on workplace safety and health concerns]?

  • [NIOSH site: Which site do you use:__________________________]

  • [OSHA site: Which site do you use: _________________________]

  • [Other workplace health and safety site(s): Please list ___________________________]


What is the size of [the facility where you work]?

  • [Large (500 or more employees)]

  • [Medium (100-499 employees]

  • [Small (1-99 employees)]


What do you do [in your job]? [Freeform Answer]


Which of the following best describes [the reason for your visit today]?

  • [Health information for myself]

  • [Health information for someone else (loved one, family, friend)]

  • [Other, please specify:]



Which of the following items do you [use on the CDC.gov website]?

  • [Bookmark and share (to tag content for social bookmarking sites like Digg or Del.icio.us)] 

  • [Blogs] 

  • [Video or CDC-TV] 

  • [Other, please specify:]


Have you ever [viewed CDC's Web site on your mobile phone or device]?

  • [Yes]

  • [No]


How frequently do you [use the CDC.gov Web site]?

  • [Daily] 

  • [Weekly] 

  • [Monthly] 

  • [This is my first time]


What is your current job title? [Freeform Answer]


How did you find out about [CDC-INFO]?

Was this the first time you [e-mailed the CDC]?

[Yes]

[No]


Do you consider [CDC] to be a reliable source of [information]?

  • [Yes]

  • [No]


What do you think is the best way to [get the word out about Immunizations in your community]?

  • Billboard 20 TV

  • Bus ad (outside or inside)

  • Can’t think of anything

  • Childcare/day care

  • Clinic News program

  • Community agency Commercial

  • Doctor/other health care professional

  • Storyline on existing program

  • Flyer

  • Friend/family member [Fill in name of show]:

  • Grocery Store 21 Nutrition Program for Women, Infants & Children (WIC)

  • Hospital 22 Other [Fill in]

  • Information line 23 Refused to answer

  • Internet/Website

  • Magazine

  • Newspaper

  • Public Event

  • Public Health Department

  • Radio

  • School


From your perspective, what recent improvements have you seen [at CDC to help those looking for free publications]? [Freeform Answer]


Please share highlights of a recent [call or e-mail] [where you feel you were particularly helped]. [Freeform Answer]


In [the past 12 months], [who have you contacted], when you needed [information about HIV testing]? [Freeform Answer]



NEW Core Questions (Part of Appendix 7)

The bracketed portion of the following Core Questions may be changed to suit a particular evaluation scenario.


Form Approved

OMB No.: 0920-0735

Exp Date: March 31, 2010


Did you find what you were looking for [on the CDC website today]?

  • [Yes]

  • [No]

  • [Somewhat]

  • [Other:___________________]

   

Would you like [options for interacting electronically with the website]? [Freeform Answer]


Which of the following [technologies] would you use if available [on the Web site]? [Download Podcasts or other Audio/Visual presentations]

  • [Making comments to blog entries]

  • [Listservs/RSS feeds to receive up-to-date news on….]

  • [Making general comments/suggestions in a forum-style manner]

  • [Other:___________________]



Additional comments about [SITE] or suggestions for [improving user-friendliness]? [Freeform answer]


What kinds of problems would [AUDIENCE have using SITE]? [Freeform answer]


Describe the [SITE]. [Freeform answer]


I think [SITE] is [easy to use].


What aspects of [SITE] did you [find difficult to understand]?



What was your [knowledge of the workplace safety and health issues] prior to [using this Web site]?

  • [Knew nothing]

  • [Knew a little bit.]

  • [Knew an average amount]

  • [Knew a lot]


What is your [knowledge about the workplace safety and health issues] now that you [have used the workplace safety and health resource]?

  • [Know nothing]

  • [Know a little bit.]

  • [Know an average amount]

  • [Know a lot]


What areas of [workplace safety and health] would you like [to learn more about that we could include on our site]? [Freeform answer]


Why do you [search for workplace safety and health information on our site]?

  • [For employees]

  • [For self]

  • [For friends]

  • [For colleagues]


What kind of [information are you looking for]?

  • [Workplace safety information]

  • [Workplace health information]

  • [Funding information]

  • [Prevention]

  • [Fact Sheets]

  • [NIOSH numbered publications]

  • [Where to get more information]

  • [Standards]

  • [Recommendations]

  • [Best practices]

  • [Other]

What is your [preferred resource for occupational safety and health information]?

  • [TV]

  • [Radio]

  • [Newspaper]

  • [Newsletter]

  • [Social Network]

  • [Twitter]

  • [YouTube]


How do you [access workplace safety and health information on the web]?

  • [NIOSH Website]

  • [Printed materials]

  • [Science Blog]

  • [Twitter]

  • [Facebook or MySpace]

  • [Other______]


How could we [improve our workplace safety and health resource]? [Freeform Answer]


Do you think [workplace safety and health resource] is [useful for making workplace safety and health decisions]?

  • [Strongly disagree]

  • [Disagree]

  • [Neither agree or disagree]

  • [Agree]

  • [Strongly agree]


Do you think you will [use the workplace safety and health information you have learned from this resource in your workplace]?

  • [Strongly disagree]

  • [Disagree]

  • [Neither agree or disagree]

  • [Agree]


What changes would [improve the site]?


What do you like [least] about [this workplace safety and health resource]? [Freeform Answer]


If we made the changes you have mentioned would you be [more or less likely to use it]?

  • [Much more]

  • [Maybe a little more]

  • [About the same]

  • [Less likely]


Please rate [the accuracy of information on this site]. [Freeform Answer]


What is your overall satisfaction [with this site]? [Freeform Answer]


How well does [this site meet your expectations]? [Freeform Answer]


How does [this site compare to your idea of an ideal website]? [Freeform Answer]


How likely are you [to return to this site]? [Freeform Answer]


How did you [find this site]?

  • [Search engine] 

  • [Referral or links from other sites] 

  • [Media/news story] 

  • [Other, please specify:]


Do you have any accessibility concerns with this site, [such as small font sizes, color or contrast, or issues with your screen reader]? [Freeform Answer]


What other [Web sites] do you use to find health information]? [Freeform Answer]


What do you [like about other Web sites] that you would [like to see on CDC.gov]? [Freeform Answer]


Is t[he Siebel system] similar to [the system used in your previous job]? [Freeform Answer]


What did you like better about [that system]? [Freeform Answer]



Please describe [a typical day in your job]. [Freeform Answer]



What is the [most enjoyable part? What tasks feel the most efficient or effective?] [Freeform Answer]



Describe an experience you had [on the job where you did not feel adequately trained]. [Freeform Answer]



Please describe [the most common workarounds you use in your job. How often do you use them?] [Freeform Answer]



What tasks [do you feel rushed on]? [Freeform Answer]



What features [do you feel rushed using]? [Freeform Answer]



What part of [this process] [works well for you]? [Freeform Answer]



How would you redesign [the Web site]? [Freeform Answer]



How would you [find a Prepared Response on the site]? [Freeform Answer]



What tips / tricks do you use when [entering search terms]? [Which work well?] [Freeform Answer]



Please show me how you [keep you’re a – Z list up-to-date.] [Freeform Answer]



Have you used [“Help”] before? [Freeform Answer]



When is the last time you used [“Help”]? [Freeform Answer]



Did they know there is [“Help”]? [Freeform Answer]



What you would do if[ the Intranet Web site web down]? [Freeform Answer]



If you were [designing a help manual for you and your teammates to use], [what items would you include in it]? [Freeform Answer]



What aspects of [the Siebel application] did not [seem logical to you when you started using it]? [And now?] [Freeform Answer]



Is [the Siebel application] labeled and organized in a way that made sense to you when you first started using it? [And now?] [Freeform Answer]



Is there anything else you’d like to tell me about what you like and don’t like about [the systems you use to complete your job tasks]? [Freeform Answer]

Where did you find out about [this text messaging program]? [Freeform Answer]


How would you rate [your level of satisfaction with this pilot]? [Freeform Answer]

Was [this text messaging pilot] what you expected? [Freeform Answer]

How did [it] fall short or exceed your expectations? [Freeform Answer]


Was [the signup process] consistent with [other text messaging programs you’ve participated in]? [Freeform Answer]


Was [the timing of] [BLANK] ideal? [Freeform Answer]


Was it [easy or hard] to [TASK read texts, re-read / save, respond]? [Freeform Answer]


Did you need help [to participate in this pilot]? [Freeform Answer]


Was [using the CDC mobile Web site to find information] the [correct level of complexity]? [Freeform Answer]

How complex was [using your phone to get the information you needed]? [Freeform Answer]


How many hours a day do you [have your mobile phone with you / turned on]? [Freeform Answer]


Do you get [other health content on your phone]? [Freeform Answer]


Would you be interested in [future health-related text messages from CDC]? [Freeform Answer]


What other [health topics] would you like to [receive text messages about from CDC]? [Freeform Answer]


Would you / Do you like getting [health information via your mobile phone]? [Freeform Answer]

Would you prefer [mobile, web, email, phone call, or something else]? [Freeform Answer]


Would you describe yourself as someone [who stays current on technology]? [Freeform Answer]


Did you get [these messages from any other sources, as well]? [Freeform Answer]

Which do you like or dislike about [BLANK]?

  • [OPTION]

  • [OPTION]

  • [OPTION]


What unanswered questions do you still have about [H1N1]? [Freeform Answer]


Who else do you think would benefit from [health-related text messages]? [Freeform Answer]


What would you change about [this text messaging pilot]? [Freeform Answer]


The most recent time you [looked for information on your mobile device] was it for

  • [Yourself]

  • [Someone else you care for or take care of, such as a spouse/partner, child, parent, relative, or friend]

  • [Both?]


In general, how much would you trust [information about health or medical topics from a doctor or health care professional]?

  • [A lot]

  • [Some]

  • [A little]

  • [Not at all]


Some [newspapers or general magazines publish a special section that focuses on health]. In the past 12 months, have you [read health sections of the newspaper or of a general magazine]?

  • [Yes]

  • [No]


Some people notice [information about health on the Internet], even when they are not trying to find [out about a health concern they have or someone in the family may have]. [Have you read such health information on the Internet in the past 12 months?]

  • [Yes]

  • [No]


About how often do you use [the Internet]?

  • [Every few weeks]

  • [Less often]

  • [Never]

  • [Don’t know/refused]


As I read the following list of items, please tell me if you, personally, happen to have each one, or not.

  • [A desktop computer]

  • [A laptop computer]

  • [A cell phone]

  • [A personal digital device, like a Sidekick, Palm Pilot or Blackberry]

  • [An iPod or other MP3 player]


Have you heard about [health topic/campaign]? [Freeform Answer]


What can you tell me about [health topic/campaign]? [Freeform Answer]


Do you like the idea of having [material to take with you into a doctor's appointment or to take home]? [Freeform Answer]


How do you prefer to see [health information presented]? [In what form (probe: posters, brochures, fliers)?] [Freeform Answer]


What kinds of promotional items would you use? [Freeform Answer]


What are some of the ways you have gotten information about [health behavior] prior to today? [Freeform Answer]


When it comes to [INSERT health topic or behavior], are there any organizations that you would really trust as a reliable source [of information]? [What makes them a trusted source of health topic information?] [Freeform Answer]


How would you rank your level of knowledge and understanding of [information you have received from CDC]?

  • [knowledge is extensive]

  • [above average]

  • [average]

  • [have some knowledge]

  • [no knowledge]



Do any [companies or organizations say] something like this now? Which ones? [Freeform Answer]


Do you think one is [more appealing] than the others? [Which? Why/why not?] [Freeform Answer]


Now that you’ve seen [all of these concepts], which one [catches your attention the most]? [Freeform Answer]


What is the most [motivational] format for [this information]? [Freeform Answer]


How relevant is this concept for [parents of teens (or children) who [are worried about smoking]? [Freeform Answer]


What resources do [parents need when they feel alone and unsupported in raising their teens]? [Freeform Answer]


Looking at the image, would you say it [fits in with what they are trying to convey]? [Freeform Answer]


What is the main message of [this illustration]? [What does it tell you?] [Freeform Answer]


Picturing this image on [a poster, brochure, print ad, or other material,] how likely is it [that an image like this will help draw your attention and curiosity]? [Freeform Answer]


Are there other [images] that might [convey this idea better]? [Freeform Answer]


How do you feel about [the images] used in this concept? [Are they helpful/engaging? Why/why not]? [Freeform Answer]


Is the color [appropriate]? [Freeform Answer]


What do you think about the [length of the content on the Web page]? [Freeform Answer]


Would you change anything about your current routine after [seeing this]? [Freeform Answer]


Before [being contacted for this study], had you ever heard of the [INSERT organization name]?

[Yes]

[No]


NEW Activity / Task Questions (Part of Appendix 8)

Respondents may be asked to click through the specific Web site, social media, mobile-based or other electronic communication channels to complete an activity / task. For each activity question, there may be a series of followup questions to evaluate the participant’s ease or difficulty in finding the answer to the question. Examples of activity / task questions and follow up questions are shown below. The actual activity / tasks used for each usability evaluation will be created to suit a particular evaluation scenario.


Form Approved

OMB No.: 0920-0735

Exp Date: March 31, 2010


Example Activity 3

[Where would you find information about visiting the CDC museum?]


Example Activity 4

[Where would you find a journal that CDC develops called the MMWR?]


Example Activity 5

[Please group this list of items from the Web site into categories that make sense to you.


Example Followup Question to Activity 5

[Please give each category you create a label.]



Example Activity 6

[What is the main idea that this Web page is trying to get across, in your own words?


Example Followup Question to Activity 6

[Does this page make sense to you?]




NEW Followup Questions (Part of Appendix 9)


GENERAL” QUESTIONS

The bracketed portion of the following “General” Follow up Questions may be changed to suit a particular evaluation scenario.

Form Approved

OMB No.: 0920-0735

Exp Date: March 31, 2010



How well do you think the main ideas come across [in this page/content]? [Freeform answer]


How could this [page/content] be improved? [Freeform answer]


After using [the CDC website today], how likely are you to do one of the following activities:

  • Talk to a friend or family member about the topic 

  • Email the information to a friend or family member 

  • Print the information and share it 

  • Bookmark this page for later use 

  • Do additional research online 

  • Order a publication from CDC.gov 


How satisfied are you with [the services CDC provided]? [Freeform Answer]


What did you learn from your [call visit to CDC] that you did not know before? [Freeform Answer]


Did what you learned make you want to [change any behaviors]?

  • [Yes]

  • [No]


Please tell me more about [what changes you are thinking about.] [Freeform Answer]


Have you actually [changed any behaviors]?

  • [Yes]

  • [No]


Overall, how satisfied are you [with the quality of the health information provided by CDC]? [Freeform Answer]

Thinking about [your entire call experience], how satisfied are you [with CDC’s electronic communication channels]? [Freeform Answer]

How well did the information provided [by CDC] [answer your questions]?

[Very well]

[Fairly well]

[Not very well]

[Not at all well]


Did the [response] you received from [CDC] [show that they]:

[Clearly understood your needs]

[Somewhat understood your needs]

[Did not understand your needs very well]

[Did not understand your needs at all]



How quickly was [CDC] able to [respond to your questions]?

[Within 24 hours]

[Within 2-3 days]

[Up to a week]

[More than a week]


How satisfied are you with [the services CDC provided]?

[Very satisfied]

[More satisfied than dissatisfied]

[More dissatisfied than satisfied]

[Very dissatisfied]


What is the reason you [were not satisfied]? [Freeform Answer]


Will you distribute [the materials] to [school children]?

  • [Yes]

  • [No]


Did you get [the information] you [asked for]?

  • [Yes]

  • [No]


Did [CDC’s mobile web site] provide [the information you needed]?

  • [Yes]

  • [Somewhat]

  • [No]


Di d the information you received] make you want [to change your behavior]?

  • [Yes]

  • [No]


Thinking about your overall experience ordering a publication, how satisfied are you?

  • [Very satisfied]

  • [More satisfied than dissatisfied]

  • [More dissatisfied than satisfied]

  • [Very dissatisfied]


What did you learn from [your call to CDC-INFO] that you did not know before? [Freeform Answer]


Who are you looking for [information] for? [Freeform Answer]


Where are you getting [information about this event] for? [Freeform Answer]


Has the information that you have received so far [been easy to understand]? [Freeform Answer]


What [information] do you still need? [Freeform Answer]


Please rate the following statements [Strongly agree], [Agree], [Neutral], [Disagree], or [Strongly disagree]:


  • I would recommend this site to others.

  • I found the website unnecessarily complex.

  • I thought the website was easy to use.

  • I think I would need help sometimes to be able to effectively use this website.

  • I found that the various functions in this website were well integrated.

  • I thought there was too much inconsistency in this website.

  • I would imagine that most people would learn to use this website very quickly.

  • I found this website very awkward to use.

  • I felt very confident using this website.

  • I will need to learn a lot about this website before I could effectively use












File Typeapplication/msword
AuthorPatricia L. Nelson
Last Modified Bycqj0
File Modified2010-02-09
File Created2010-02-09

© 2024 OMB.report | Privacy Policy