Homepage Visual Concept Usability Testing research plan

OMB Control Number: 3170-0024

Expires: XX/XX/XXXX

Paperwork Reduction Act Notice

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to a collection of information unless it displays a valid OMB control number. The OMB control number for this collection is 3170-0024. It expires on XX/XX/XXXX The time required to complete this information collection is estimated to average approximately 30 minutes per response. Comments regarding this collection of information, including the estimated response time, suggestions for improving the usefulness of the information, or suggestions for reducing the burden to respond to this collection should be submitted to the Consumer Financial Protection Bureau (Attention: PRA Office), 1700 G Street NW, Washington, DC 20552, or by email to [email protected].

Homepage visual concept and usability testing

Date: June 17, 2021

Summary

In this brief, we have detailed a plan to (1) test 3-4 visual concepts for the homepage, and (2) iteratively test the usability of the content and design of the final version of the homepage. Testing the homepage in this manner will help us to help make a more informed decision on the most useful and appropriate design and content for the homepage.

Goals

The home page is our front page, and the most visited page on the website. Given the economic crises many face as the result of the pandemic, providing the right information to visitors to solve their problems is critical. Past usability testing has revealed usability issues with the home page that prevent visitors from finding the info they need. Our goal is to redesign the home page before the new director is confirmed so that these issues are resolved and consumers are able to quickly find the information that they need.

Methods

The El Camino team within Design & Development will conduct task-based, unmoderated first impression, desirability, and usability testing with the general population. Using this methodology, we will measure and gauge:

• Initial impressions: People's "gut reaction" to the design variations before they can react to content or details.

• Desirability: Measure aesthetic appeal and inform design direction. This will give people time to articulate their preferences and why they feel the way they do.

• Usability testing: Assess the appeal, navigability, and usability of the home page.

Logistics and equipment

Testing sessions will be conducted remotely based on the availability of participants. UserZoom Go, a software procured and cleared for use by the DIG, will be used to capture the participants’ interactions and reactions via screen sharing and audio recording. Participants’ faces will not be recorded and their names will not be shown. The tool will be used to facilitate the recruitment, administering of the test, and dispensing of an incentive.

No additional equipment will be needed.

(Privacy: This is identical to the tool and recording methods used for first impression, navigability, and usability tests performed under Quick Turn-around Iterative COVID/Housing User Research #DI00873.)

Study design

(1) Test one: First impressions and desirability test (15-30 min total)

• Participants will take a series of four brief tests (10-15 seconds each) in which they are cycled through the mockups of the homepage designs in random order.

• Once the 10-15 seconds has passed, participants will be presented with a series of tasks, listed below in the First impression and desirability testing tasks section.

• After completing the tasks, participants will be shown a list of word choices and be asked describe out loud the images they saw using the word choices listed on the screen.

• The participant will go through this cycle until they've viewed all of the mockups.

• To conclude, participants will be asked to verbally provide their preference of mockups.

(2) Test two: Usability testing (15-30 min total)

• Participants will be presented with a series of tasks, listed below in the Usability testing tasks section.

Target audience

• No target audience – we are recruiting general population.

Recruitment

• Recruitment of general population will be done through UserZoom Go’s participant panel.

Quota

Test name	Mode	Number of participants
Test 1: First impression and desirability testing	Desktop	8
Test 2: First impression and desirability testing	Mobile	8
Test 3: Usability testing	Desktop	16
Test 4: Usability testing	Mobile	16
TOTAL		48

 

First impression and desirability testing tasks

 

Introduction 
Please take a moment to help us improve our website. There are no right or wrong answers. This should take no more than 15-20 minutes of your time. Remember to talk out loud as you go through the tasks. Thank you!  

We will ask tasks similar to the following:

• You’re going to briefly see a page from a government website that provides financial education and policy information.

• What is the purpose of the page you just saw?

• What is the first thing on the page that caught your eye?

• Please choose 3–5 words that describe this image. Say the words out loud then describe why you made those choices. Be sure to move or minimize the task button so you can see the whole image.

• Which of these four versions do you prefer? Why?

• Which one do you like the least? Why?

Usability testing tasks

Introduction 
Please take a moment to help us improve our website. There are no right or wrong answers. This should take no more than 15-20 minutes of your time. Remember to talk out loud as you go through the tasks. Thank you!  

We will ask tasks similar to the following:

• What is this page about? What can you do here?

• What is the first thing that caught your eye?

• Read the hero text. In your own words, what do you think this organization does? How do you think it can help you?

• Look at the hero image. Does this image appeal to you? Does the image fit the rest of the page? Why or why not?

• Where on this page would you click if you wanted to find out where to get a copy of your credit report? (Do not choose search for this task)

• Let's say you want some information on how to respond to a debt collector. How would use this page as a starting point to find that information?

• Let’s say you have discovered an unauthorized transaction on your bank account, and you want to file a complaint. How do you think this agency can help you?

• Look at the section titled “X”. How do you think this could help you?

• Do you trust the information on this page?

• Do you trust this organization?

• What are your impressions of how this page looks?