Feedback Sessions for OnGuardOnline.gov

Fleishman-Hillard
The Federal Trade Commission (FTC)
On Guard Online - Moderator Guide:
English
Last Updated: 4/1/13

Prepared by:
Walky Goode
Senior User Experience Researcher
SPARK Experience Design
[email protected]
Phone: 202.215.6962

Prepared for:
Federal Trade Commission
Fleishman-Hillard

Introductions & Background Questions
Why are we here
I’d like to thank you for spending part of your day talking to us. We’ll spend about an hour together. We
work for a company called SPARK Experience Design and we are helping our client to improve their website
in order to make it more helpful and easier to use.

Some ground rules
I want to mention a few ground rules before we get started.
1. Everything you share with us today will be completely anonymous. We promise not to attribute
anything you say or write down with you personally. All information gathered in this process will be
combined with other data to give us a big picture of what everyone had to say.
2. We want you to feel free to speak your mind. This isn’t a test where there is a right or wrong
answer. We are here to listen to what you have to say and talk about your opinions. You can ask
questions at any time.
3. We will be recording the session for research purposes. We can’t possibly remember everything
that you say during the session, so we will be recording today’s session. We promise not to share
the recordings with anyone except those directly involved with this project.
Do you have any questions before we begin?
Great. Let’s begin!

Getting to know you
I would like to find out more about you so that we can have a better understanding of how this website
might help you.
1. In a typical week, about how many hours do you spend using the Internet?
Less than 3 hours
3-5 hours
6-15 hours
15-30 hours
More than 30 hours

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

2

2. As I read each of the following, please tell me if this is something you do online using the Internet:
Yes

Yes

Email / Chatting

Search

Shopping / Travel

Get news

Watch videos

Use social network sites

Financial Transactions

Job search

Visit a local, state or federal
government website

Other

3. [IF PARENT]: How many children do you have in each of the following age groups?
Less than 7 years old
8-12 years old
More than 12 years old

4. [IF IT EDUCATOR]: In what capacity do you teach computer skills to others?

5. [FOR SPANISH VERSION ONLY, IF IT EDUCATOR]: Do you teach in Spanish? Do you use Spanish or
bilingual materials?
Teach in Spanish
Materials in Spanish/Bilingual

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

3

Task Performance
Activity introduction
Next we are going to have a look at a website together and get your opinions on it.

Homepage
I’m going to pull up the webpage now and I would like you to tell me your first impressions, whatever
comes to mind. You can feel free to scroll up and down on the page, but please refrain from clicking on
anything right now.
Open browser and go to: ENGLISH: www.onguardonline.gov/, SPANISH: www.alertaenlinea.gov/
1. What is your first impression of this site? What word would you use to describe this site?
2. Who is this website designed for?
3. What types of information do you think this site covers?
4. Where would you go to receive updates from, this website? Probe to see if participant notices “Stay
Connected: Get Email Updates”.
5. Would you trust this website to offer credible information? Which aspects of this website make it
appear credible? Which aspects make it appear less credible? Who’s responsible for this site?
6. [ENGLISH]: How would you find Spanish information on this site?
[SPANISH]: How would you find English information on this site?
7. Is there anything else that you would expect to have on the homepage that you don’t see here?
8. Have you visited this website before?
9. If yes, what tasks did/do you most frequently perform on the website?

Activity introduction
Now you will have the opportunity to try out the website by performing a few tasks.
Study Flow: 
Start at homepage. Read task instructions first, have user interact with the web page then ask
post-task questions.
Observe:
1. Where user clicks first and sequence of clicks
2. Which path user follows?
3. Errors
4. Task status:
a) Passed with no difficulty
b) Passed with difficulty

c) Failed

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

4

Task 1a: Phishing – Part 1
You've heard about fake emails that look like they're from a trusted source, like your bank or a
government agency, but are actually from people trying to get you to reveal your personal information.
Where do you look to learn more?
Expected Path:
A. Homepage >> Avoid Scams >> Phishing
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult?
3. Is there anything in particular that you would change the design to make this easier?

Task 1b: Phishing- Part 2
You want to send the article in Spanish to a friend. How would you do that?
Expected Path:
A. Gray Toggle in upper right of article body area (underneath Email, Print, Share)
Post-task Questions:
1. What are your impressions of this task? Easy? Difficult?
2. Is there anything in particular that you would change the design to make this easier?

Task 2: Cookies
You’ve heard of online tracking and behavioral advertising. You know it has something to do with
websites tracking your online activities. You want to know if there’s anything you can do about it. Find
information about online tracking.
Expected Path:
A. Home ->Be Smart Online -> Cookies: Leaving a Trail on the Web
B. Home ->Online Tracking Slider
Post-task Questions:
3. What are your impressions of this task? Easy? Difficult?
4. Is there anything in particular that you would change the design to make this easier?

Task 3: Using Free Wi-Fi
Browsing the Internet on your laptop at the local coffee shop, you overheard someone say that he’d
never use his laptop there because he doesn’t want other users to snoop into his files. Find out if this is
possible and how to protect your information.

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

5

Expected Path:
A. Homepage >> Be Smart Online >> Tips for Using Public Wi-Fi Networks
B. Homepage >> Video and Media >> Public Wi-Fi Networks
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is there anything in particular that you would change in the website to make this easier?

Task 4: Changing the Security Settings on a Wireless Router
Your cable company just installed wireless Internet at your home and recommended that you set up a
password to share within your household. Find information that walks you through the steps on how to do
this.
Expected Path:
A. Homepage >> Secure Your Computer >> Securing Your Wireless Network
B. Homepage >> Secure Your Computer >> [Right Rail] Tutorials
C. Homepage >> Video and Media >> Tutorials
D. Homepage >> Video and Media >> Wireless Security
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is there anything in particular that you would change in the website to make this easier?

Task 5: Malware
Your computer is running super-slow and seems to crash a lot. A friend suggests that maybe you have a
virus. Where would you look to learn more?
Expected Path:
A. Home >> Secure Your Computer >> Malware
B. Home >> Secure Your Computer >> Computer Security >> Malware link in 2nd body paragraph
C. Home >> Secure Your Computer >> Computer Security >> [Right Rail} Malware
D. Homepage >> Video and Media >> Protect Your Computer from Malware
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is there anything in particular that you would change in the website to make this easier?
Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

6

Task 6: Resources for Kids’ Online Safety
You’ve been asked to share some tips about kids’ online safety at a workshop for parents. Find some
resources to help you at the workshop. For example: Handouts to give out, something that you can present
on a computer, etc.
Expected Path:
A. Homepage >> Just For You >> Educators (Netcetera Toolkit, Living Life Online, [Right Rail] Any video)
B. Homepage >> Just For You >> Educators >> [Right Rail] Netcetera Online Only >> Links within text
C. Homepage >> Just For You >> Parents ([Right Rail] Any video)
D. Homepage >> Just For You >> Parents >> [Right Rail] Netcetera Online Only >> Links within text
E. Homepage >> Just For You >> Parents >> [Right Rail] Just For Kids >> Heads Up! >> [Right Rail] PDF
F. Homepage >> Just For You >> Kids (Living Life Online, [Right Rail] Any video, Do You Live Life Online?)
G. Homepage >> Just For You >> Kids >> Heads Up! >> [Right Rail] PDF
H. Homepage >> Protect Kids Online >> [Right Rail] Any video
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is there anything in particular that you would change in the website to make this easier?
If participant found a video, otherwise probe to find a video:
4. What did you learn from watching this video?
5. Use one word or phrase to describe the video you just saw.
6. Is the video content valuable? Use a scale from 1 to 7, where 1 is not valuable at all and 7 is extremely
valuable.

Task 7: Online Reviews
You want to buy a tablet online. You want to get a good deal and make sure that the tablet is nice. Find tips
and information that might help you.
Expected Path:
A. Homepage >> Be Smart Online >> Comparing Products Online
B. Homepage >> Be Smart Online >> Shopping Online Infographic
C. Homepage >> Be Smart Online >> [Right Rail] Video: Online Reviews and Recommendations
D. Homepage >> Video and Media >> Video: Online Reviews and Recommendations
Post-task Questions:
1. How confident are you that you clicked on the right area? Use a scale from 1 to 7, where 1 is not
confident at all and 7 is extremely confident.
2. What are your impressions of this task? Easy? Difficult? Use a scale from 1 to 7, where 1 is extremely
difficult and 7 is extremely easy.
3. Is the information provided useful? Use a scale from 1 to 7, where 1 is not useful at all and 7 is
extremely useful.
4. Is there anything in particular that you would change in the website to make this easier?
Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

7

Blog Evaluation
1. Did you notice the blog?
2. If participant did not notice, point at blog and ask: What type of information do you expect to see in
this blog?
3. What is your impression of this blog?
4. Would this be something that you would read in a regular basis?
5. What type of content would cause you to sign up for blog updates?
6. Do you subscribe to other blogs? If YES, how do you get email updates? Does the email show the entire
post or a summary of the post?

Debrief
1. Did you find anything particularly interesting or useful about the resources that the website
provides?
2. Let’s talk about the navigation on the website:
a. Did the links seem to be logically organized?
b. Would you organize them differently?
c. Were there any links that you were unsure about?
3. What did you like most about the site?
4. What did you like least about it?
5. If you could change anything about it, what would you change?
What would make you visit the site again?

Post Experience Evaluation
Based on your experience with the site, please rate your agreement with the following statements. Circle
the level of agreement that applies (1 means strongly disagree, 4 means neither disagree nor agree, and 7
means strongly agree).

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

8

Emotional Evaluation

Strongly
Disagree

Neutral

Strongly
Agree

I feel confident that I can use the website to find what I am
looking for

1-------2-------3-------4-------5-------6-------7

I would be curious to explore the website further to discover
things I might also be interested in

1-------2-------3-------4-------5-------6-------7

It feels like this website is designed for someone like me

1-------2-------3-------4-------5-------6-------7

Visual Evaluation

Strongly
Disagree

I thought the website had a professional look and feel

1-------2-------3-------4-------5-------6-------7

The site does not look too busy or cluttered

1-------2-------3-------4-------5-------6-------7

The site has a consistent look and feel

1-------2-------3-------4-------5-------6-------7

Functional Evaluation

Strongly
Disagree

The site provides information that is useful to me

1-------2-------3-------4-------5-------6-------7

The site’s navigation was always clear & easy to use

1-------2-------3-------4-------5-------6-------7

The content on the site was easy to skim or read

1-------2-------3-------4-------5-------6-------7

The website presents information that I didn’t realize the
FTC offered

1-------2-------3-------4-------5-------6-------7

Based on my experience today, I would likely use the web
site in the future

1-------2-------3-------4-------5-------6-------7

Neutral

Strongly
Agree

Neutral

Strongly
Agree

6.
That brings us to the end of the session.
We really appreciate the time you have taken to participate in this study your participation will assist us in
developing a better site.
Thank you once again!

Fleishman-Hillard - FTC: On Guard Online – Moderator Guide: English

9