Category B Sample

Internet Enumeration – Usability/Focus Group
Moderator Guide
03/23/09
I.

Ground Rules, Introductions and Warm-up – 5 minutes

II.

Step 1 – Exploratory Phase - 15 minutes


Before we get started, I’d like to take the “internet temperature” of the group.
How often and for what purpose(s) do you use the Internet?



Has anyone used the internet to conduct government business? What kind?

Social Security is enhancing the ways that the public can do business with us using
the internet. We have invited you here to get your impression of one of these
services. We would like you to break into separate spaces to review this online
service. Afterwards we’ll come back together to talk about your experiences.
III.
#

Usability Testing – one-on-one sessions – 45-60 minutes

Scenario Name
Starting from the
Home Page—

1

Name Change —

2

Replace a lost card—

3

Auto /
Design Aspects Tested
Non-Auto
You have heard you can get a
N/A
 Robustness of home
new Social Security card
page labeling and
online. Please go to the Social
pathing
Security Administration
 Initial reaction to
home page and find where
entry page language
you can do this.
and layout
You have recently married
Non-Auto  User orientation in
and would like to change
the system
your last name on your Social
 Labels and label
Security Card. Please update
proximity to fields
your Social Security Card
 General usability,
records.
instruction, visibility
and comprehension
Scenario Description

You lost your Social Security
card during a move and you
need to get a new one for an
upcoming job interview.

Internet Enumeration Discussion Guide

Auto



User orientation in
the system
 Labels and label
proximity to fields
 General usability
 Instruction visibility
and comprehension

1

#

Scenario Name
Edit missing
information—

4

5a

5b

Respond to mismatch
notice (recover) —

Respond to mismatch
notice (discard) —

Tracking —
6

Scenario Description
You’re reviewing your
information but got a
message saying you need to
change some things first.
Please identify what you need
to change, and make those
changes using the system.
You submitted your
information but got this
message. Please tell us what
you think you should do next.
You submitted your
information but got this
message. Please tell us what
you think you should do next.
You have submitted your
application but would like to
check the status. Please walk
us through that process.

Auto /
Design Aspects Tested
Non-Auto
Non-Auto  Error visibility and
handling

Non-Auto 
/ Auto

Response to notice
about mismatch

Discard



Response to notice
about mismatch

N/A



User orientation in
the system
Labels and label
proximity to fields
General usability




Next Steps—

7

IV.

You finished entering your
information and submitted it
to SSA. What do you think
you need to do now, if
anything? What happens
next?

NonAuto



Instruction
comprehension



User stamina for
reading



Reaction at end of
system

Focus Group Sessions - Shared Impressions and Experiences – 45-60
minutes
1.

What is your overall impression of the online application for a Social
Security card?

2.

What is your impression of the design and format of the welcome and
instruction pages?

3.

How long did it take to fill out the online application?

4.

How comfortable were you with providing the information that the website
used to verify your identity?

Internet Enumeration Discussion Guide

2

There were two ways that a person could complete the application for a
replacement Social Security card.
5.

Describe for me the two methods for completing the application.

6.

Can you explain why the two methods are needed?

Automated Process Questions
So, now let’s talk about the process where you could complete the application
online.
1. How comfortable were you with inputting your personal information into the
system?
2. Did the questions make sense? Why or why not?
3. Were the questions in the right order (sequence)?
4. Would you recommend this service to others? Why or why not?
5. On a scale of 1-5 with 1 being ‘Does Not Meet’ and 5 being ‘Fully Meets’,
how does this process meet your expectations of the level of service that a
government website should provide?
Non-Automated Process Questions
Ok, now let’s talk about the process where you could not complete the application
online.
1. How easy was it to find the instructions and screening questions?
2. How did you feel about not being able to complete the application online?
3. Would you have visited a Social Security office with your proofs, or would
you have chosen to mail them to the Social Security office?
PROBE: How comfortable are you with mailing your documents to Social
Security?
4. What do you see as possible benefits to using this process?
5. On a scale of 1-5 with 1 being ‘Does Not Meet’ and 5 being ‘Fully Meets’,
how does this process meet your expectations of the level of service that a
government website should provide?

Internet Enumeration Discussion Guide

3

General Questions
1. Of the two methods, which would you prefer, completing the application online,
or visiting the Social Security field office or mailing your proof of identity
documents? Please explain
2. Do you have any security concerns with either of these processes? Please explain.
To ensure the security of personal information that you submit to Social Security over
the Internet, we will need to verify your identity. There are various ways that we can
do that. We would like to get your reaction to a few options.
I will describe some options to verify your identity. After I have described them,
please rank them in order of preference with 1 being your most favorite and 8 being
your least favorite. (Use Flip Chart to record this information.)









Receive a PIN/Password from Social Security in the mail _______
Receive a code/password from Social Security via email ______
Verify your identify with another government agency, such as the Dept of
Motor Vehicles _____
Receive a code/password via a cell phone text message _____
Provide a credit card number to Social Security online _____
One-Time Password Token _____
Onscreen Keyboard _____
Green Address Bar _____

3. There are companies that verify financial accounts and other personal information
for businesses and government agencies. For instance, if you’ve ever requested
a FREE Credit Report online, you were asked questions like, “In the past 5 years,
at which of the following addresses did you live?”
4. How do you feel about Social Security using such a company to verify your
identity?

V. Recommendations – 10 minutes
How can we improve this online service?

Internet Enumeration Discussion Guide

4

NOTE TO MODERATOR: Read this at the completion of the usability session/focus
group
Paperwork Reduction Act Statement – This information collection, cleared under
OMB No. 0960-0526, meets the requirements of 44 U.S.C. §3507, as amended by section
2 of the Paperwork Reduction Act of 1995. We estimated that it would take 60/90
minutes (NOTE TO MODERATOR: choose appropriately depending on whether you
are leading the usability session or the focus group) to participate in this session. You
may send comments on our time estimate to: SSA, 6401 Security Blvd., Baltimore, MD
21235-6401. Send only comments relating to our time estimate to this address.

Internet Enumeration Discussion Guide

5

Welcome Screen (on page load)

Internet SS-5 Usability Testing Screen Shots
1

Welcome Screen (on track status click)

Internet SS-5 Usability Testing Screen Shots
2

Application Status

Internet SS-5 Usability Testing Screen Shots
3

Section 1

Internet SS-5 Usability Testing Screen Shots
4

Section 2

Internet SS-5 Usability Testing Screen Shots
5

Section 3

Internet SS-5 Usability Testing Screen Shots
6

Section 3

Internet SS-5 Usability Testing Screen Shots
7

Next Steps

Internet SS-5 Usability Testing Screen Shots
8