Website and Mobile Application Usability Focus Groups FY19

NAXION Study #XXX

Month/Year

U.S. Mint Web Usability Research
Discussion Guide
Customers/Non-Customers

I. INTRODUCTION

According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB number. The valid OMB control number for this information collection is 1505-0231.

• Ground Rules: Video-taping, confidentiality, candor, cell phones, pagers, etc.

• Background & Objectives: The purpose of today’s discussion is to: 1) improve your ability to successfully and quickly find information on the U.S. Mint’s website; and 2) increase your satisfaction with the website.

• Introduction:

• Feedback is important to help improve the site

• Will perform a few activities/tasks on the site

• Something you might do in your home or office

• Will give the tasks to you one at a time, and ask you to read them aloud

• Think aloud; tell us what you are doing and why

• Tell us what you expect to happen

• Comments are important so will be taking notes

• Should take about one hour

• Following tasks, we will ask you to tell us about your experience

• Questions before we begin?

II. USER IMPRESSION OF THE WEBSITE (allow users to explore website)

• What is your initial impression of this website?

• Specifically, what type of information would you expect to find on this site? Is anything missing?

• Specifically, what types of things would you expect to be able to do on this site?

• Who do you think this site is for?

III. WEBSITE SCENARIOS/TASKS

Tasks will be provided to users one at a time, in random order. Tasks may vary depending on users’ experience with the U.S. Mint website and history with purchasing coins online. Following each scenario, the facilitator may probe on issues that arise during the usability scenario.

(Presented in Random Order)

1. You are interested in the annual coin set and would like to buy one for your collection. How would you begin?

Follow-up:

• When will the annual coin sets be released?

  • Is it possible to have a reminder sent to you so that you don’t forget to return to the website?

2. Let’s pretend that your daughter had a baby this year and you would like to give your new grandchild a memorable birthday gift. Using this website, what item(s) would you consider giving the new baby?

Follow-up:

• (Once users finds the birthday set): Let’s go ahead and purchase this set. How would you do this?

3. Let’s pretend that you collect the American Eagle Silver Proof Coins. How would you find this coin using the website?

Follow-up:

• What type of information would you like to know about the American Eagle Silver Coins? Does this web page provide the information that you’re most interested in?

• How would you see a close-up on the image?

• Let’s pretend that you’ve decided not to buy this coin today, but that you might like to buy it in the future. Is there a way to “save” this item?

(Additional scenarios to be added by the U.S. Mint team as needed.)

IV. QUESTIONS DURING WEBSITE TASKS (IF APPLICABLE)

• Where are you looking on the screen? OR I see you looking around on the screen, what are you looking for?

• Why did you click on [insert element]?

• What do/did you expect to find when you click on [insert element]?

• I noticed that you chose [insert element A] and not [insert element B] just now. What would you have expected to see if you had chosen [insert element B]?

• What would you do next?

V. FOLLOW-UP QUESTIONS TO WEBSITE TASKS

• Tell me more about… OR What do you think about… [the links, the categories of information on the site, the categories of products, the search feature and results pages, the website design, the ‘look and feel’ of the website, etc,]

• Tell me about your experience with this task.

• Which aspects of the website [related to this task] were valuable?

• Which aspects of the website [related to this task] need improvement?

• How would you use [insert specific feature of the website]?

VI. OVERALL REACTION TO USING THE WEBSITE

• Overall, how would you describe the site? What is your general reaction?

• What were/are your expectations, and to what extent does the website meet these?

• What three things did you like best about the website?

• What three things did you like least about the site?

• If you were the developer and could make only one change to the website, what would that be?

• Is there anything you feel is missing from the site?

• If you were to describe this site to a friend, what would you say?

VII. USER RATING OF SATISFACTION AFTER USING THE WEBSITE

The System Usability Scale (SUS) will be administered at the end of the usability test.

Strongly disagree Strongly agree

I would use this website again	1 2 3 4 5
I thought the website was complex	1 2 3 4 5
I thought the website was easy to use	1 2 3 4 5
I thought there was too much inconsistency in the website	1 2 3 4 5
I thought the website was well integrated and cohesive	1 2 3 4 5
I thought the website was difficult to use	1 2 3 4 5
I think most people would learn to use this website quickly	1 2 3 4 5
I thought the website was awkward to use	1 2 3 4 5
I felt confident using the website	1 2 3 4 5
I need to learn a lot before I could get going with this website	1 2 3 4 5

Scoring System Usability Scale
SUS SCORE 0-60 GRADE = F
SUS SCORE 60-70 GRADE = D
SUS SCORE 70-80 GRADE = C
SUS SCORE 80-90 GRADE = B
SUS SCORE 90-100 GRADE = A

- 1 -