National Analysts Worldwide Study #800
February, 2008
U.S. Mint Web Usability Research
Discussion
Guide
Customers/Non-Customers
I. INTRODUCTION
Ground Rules: Video-taping, confidentiality, candor, cell phones, pagers, etc.
Background & Objectives: The purpose of today’s discussion is to: 1) improve your ability to successfully and quickly find information on the U.S. Mint’s website; and 2) increase your satisfaction with the website.
Introduction:
Feedback is important to help improve the site
Will perform a few activities/tasks on the site
Something you might do in your home or office
Will give the tasks to you one at a time, and ask you to read them aloud
Think aloud; tell us what you are doing and why
Tell us what you expect to happen
Comments are important so will be taking notes
Should take about one hour
Following tasks, we will ask you to tell us about your experience
Questions before we begin?
II. USER IMPRESSION OF THE WEBSITE (allow users to explore website)
What is your initial impression of this website?
Specifically, what type of information would you expect to find on this site? Is anything missing?
Specifically, what types of things would you expect to be able to do on this site?
Who do you think this site is for?
III. WEBSITE SCENARIOS/TASKS
Tasks will be provided to users one at a time, in random order. Tasks may vary depending on users’ experience with the U.S. Mint website and history with purchasing coins online. Following each scenario, the facilitator may probe on issues that arise during the usability scenario.
(Presented in Random Order)
1. You are interested in purchasing a 2007 U.S. Mint Uncirculated coin set ($16.95). How would you do this?
I was able to purchase this product
I was not able to purchase this product
2. You grew up in Iowa and would like to find out more about the Iowa State Quarter. What is on the IA Quarter?
A picture of corn stalk
A picture of the Mississippi River
A picture of schoolhouse
Not sure
3. A relative of yours has just passed a very valuable set of coins onto you. The set is very special and you would like take care of it properly. What is the best way to store your coins?
In a place that is consistently cool and dry
In their original holders
In 2x2 cardboard holders
All of the above
Not sure
4. You would like to purchase the 25 coin roll of James Madison Presidential $1 Coins. How much does it cost?
$30.95
$35.95
$25.95
Not sure
5. You are new at coin collecting and keep hearing the term “proof”. What is a proof?
A specially produced coin often struck more than once to accent the design
The part of a coin's design that is raised above the surface
The blank piece of metal on which a coin design is stamped
Not sure
(Additional scenarios to be added by the U.S. Mint team as needed.)
IV. QUESTIONS DURING WEBSITE TASKS (IF APPLICABLE)
Where are you looking on the screen? OR I see you looking around on the screen, what are you looking for?
Why did you click on [insert element]?
What do/did you expect to find when you click on [insert element]?
I noticed that you chose [insert element A] and not [insert element B] just now. What would you have expected to see if you had chosen [insert element B]?
What would you do next?
V. FOLLOW-UP QUESTIONS TO WEBSITE TASKS
Tell me more about… OR What do you think about… [the links, the categories of information on the site, the categories of products, the search feature and results pages, the website design, the ‘look and feel’ of the website, etc,]
Tell me about your experience with this task.
Which aspects of the website [related to this task] were valuable?
Which aspects of the website [related to this task] need improvement?
How would you use [insert specific feature of the website]?
VI. OVERALL REACTION TO USING THE WEBSITE
Overall, how would you describe the site? What is your general reaction?
What were/are your expectations, and to what extent does the website meet these?
What three things did you like best about the website?
What three things did you like least about the site?
If you were the developer and could make only one change to the website, what would that be?
Is there anything you feel is missing from the site?
If you were to describe this site to a friend, what would you say?
VII. USER RATING OF SATISFACTION AFTER USING THE WEBSITE
The System Usability Scale (SUS) will be administered at the end of the usability test.
System Usability Scale (SUS)
Current Non-Users
|
Strongly |
|
|
Strongly |
|
|
1 |
2 |
3 |
4 |
5 |
I think that I would like to use this website frequently. |
|
|
|
|
|
I found the website unnecessarily complex. |
|
|
|
|
|
I thought the website was easy to use. |
|
|
|
|
|
I think that I would need the support of a technical person to be able to use this website. |
|
|
|
|
|
I found the various functions in this website were well integrated. |
|
|
|
|
|
I thought there was too much inconsistency in this website. |
|
|
|
|
|
I would imagine that most people would learn to use this website very quickly. |
|
|
|
|
|
I found the website very cumbersome to use. |
|
|
|
|
|
I felt very confident using the website. |
|
|
|
|
|
I needed to learn a lot of things before I could get going with this website. |
|
|
|
|
|
Prior Users
|
Strongly |
|
|
Strongly |
|
|
1 |
2 |
3 |
4 |
5 |
I think that the design of the website invites me to use this website frequently. |
|
|
|
|
|
I find the website unnecessarily complex. |
|
|
|
|
|
I think the website was easy to use. |
|
|
|
|
|
I feel that I would need the support of a technical person to be able to use this website. |
|
|
|
|
|
I find the various functions in this website to be well integrated. |
|
|
|
|
|
I feel there is too much inconsistency in this website. |
|
|
|
|
|
I would imagine that most people would learn to use this website very quickly. |
|
|
|
|
|
I find the website very cumbersome to use. |
|
|
|
|
|
I felt very confident using the website. |
|
|
|
|
|
I needed to learn a lot of things before I could get going with this website. |
|
|
|
|
|
DEMOGRAPHIC AND WEB USAGE QUESTIONS
1. What is your gender? (CHECK ONE)
Male
Female
2. What is your age? (CHECK ONE)
18 to 24 years
25 to 34 years
35 to 44 years
45 to 54 years
55 to 59 years
60 to 64 years
65 years or older
3. What is the highest level of education you have completed? (CHECK ONE)
High school or less
Some college
College degree
Advanced degree
4a. Are you Hispanic or Latino?
Yes
No
4b. What is your ethnicity / race?
American Indian and Alaska Native
Asian
Black or African American
Native Hawaiian and Other Pacific Islander
White
Other
5. How much time do you spend using a computer each week? (CHECK ONE)
Less than 2 hours
3-5 hours
6-10 hours
10-14 hours
15 or more hours
6. Approximately, how often do you use the Internet? (CHECK ONE)
At least once a day
3-5 times a week
1-2 times a week
Every few weeks
7. Excluding email, how much time do you spend on the web each week? (CHECK ONE)
Less than one hour a week
1 – 10 hours a week
11-20 hours a week
21 or more hours a week
8. Have you ever purchased coins or other merchandise directly from the U.S. Mint? (CHECK ONE)
Yes
No
9. If so, have you ever purchased coins from the U.S. Mint website? (CHECK ONE)
Yes
No
10. How often do you use the U.S Mint website? (CHECK ONE)
Daily
Weekly
Monthly
Every few months
Never
-
| File Type | application/msword |
| File Title | BR Discussion Guide |
| Author | Beth Rothschild |
| Last Modified By | Yvonne Pollard |
| File Modified | 2008-01-16 |
| File Created | 2008-01-16 |