Financing an Auto - External testing

MODERATOR’S GUIDE
FA User Testing Website
Data Collection: [insert dates]

The Bureau does not intend to use the work described in this document, or the information obtained
from it, in any manner that would contravene the provisions set forth in 12 U.S.C. § 5519.

Section 1: Introduction
Thank you for participating in this study today. My name is
, and I am a researcher with
the Fors Marsh Group User Experience Team. Today we will evaluate a website that has
information about auto finance and auto finance shopping. The pages you will interact with
are still in development, so we ask you to interact with the [paper copies/simple prototype]
as if they were a website.
I will ask you questions and give you some tasks to complete using the website. Your
comments and feedback will help the design team improve the site. I am a researcher and
did not create the site, so please do not feel like you have to hold back on your thoughts to
be polite to me. I’m interested in both your positive and negative reactions. Difficulties you
may run into reflect issues with the design of the sites, not your skills or abilities. Please
keep in mind that I am not testing you or your knowledge. Rather, you are helping us to see
how we can improve this site.
The entire session will last about 60 minutes. In accordance with the Papework Reduction Act
of 1995, this study as been approved by the Federal government’s Office of Management and
Budget (OMB) under OMB control number [Insert control number]. Without this approval we
could not conduct this study. Do you have any questions so far?
Let’s cover a couple things before we get started.
• We are making a video recording of the computer screen during this session, but your
name and personal information will not be associated with the recordings or your
responses.
• I am interested in your thoughts and reactions as we proceed. This is important
because I can see what you are doing, but I don’t know why you are doing it. So I
need your help. While you are working, I would like you to think aloud. I would like
you to:
o Tell me what you are thinking
o Describe the steps that you are taking.
o Tell me what you are doing, what you are going to do, and why.
o Tell me why you clicked on a link and what you expect to happen next.
o This will give me insight into what you’re thinking about as you’re using the site.
• There are no right or wrong answers, and your comments and opinions will only be
used in combination with the feedback that we get from other people.
• Please make sure that you have reviewed the Privacy Act Statement has been
provided in hard copy, if you haven’t already.
• Any questions? Ok, we are ready to begin. I would like you to start by asking you some
background questions.

Section 2: Questions and Tasks
Item 1: Pre-task interview

Goal: To get to know participants and explore their relevant past behavior.
1a. What types of electronic devices do you use to visit websites?

Item 2: Planning around auto finance shopping

Goal: To explore participants’ behavior around auto finance shopping.
2a. You’ve indicated you’re planning on financing a car. Can you tell me about your
experience so far with the auto shopping process? Where in the process are you at right now?
2b. Today we'd like to talk to you specifically about shopping for an auto loan. Have you
started to think about when and how to get an auto loan?
o [IF YES] What have you done so far?
o [IF YES] Can you tell me about any information you’ve looked at (such as visiting a
website or reading a magazine) to help you research and make decisions?
o [IF NO] Why do you think it hasn’t come up?
2c. What comes to your mind when you are thinking about shopping for—and negotiating—an
auto loan? [Alternatively] How do you feel about it?

Item 3: First impressions of auto tools web page
Starting Page: Auto finance landing page
Goal: To evaluate consumers’ first impressions.

Please take a few minutes to use and interact with this web page the way that you would if you
were at home or work browsing the page on a desktop or laptop.
3a. What comes to mind when you are looking at this page?
3b. What is the first thing that caught your eye?
3c. What can you do here?
3d. What is the purpose of this site?
3e. What would you do first here? Why?
3f. Let’s say you’re preparing to shop for an auto loan, could you show me how you would
use the site to help you with the process?
3g. Let’s say you are planning to go next week to finance an auto purchase, could you show
me how would use the site to prepare for it?

Item 4: Interaction with auto finance pages
Starting Page: Auto finance landing page

Auto Finance Usability Test  3

Goal: To evaluate the overall usability, learnability, and usefulness of the auto finance
website.
4a. [If they haven’t indicated downloading a form] Imagine you are ready to go to
purchase an auto and you’re looking for a worksheet that will help you with the
purchasing process. Please download a worksheet to help you with the process when
you are financing you auto purchase.
4b. Looking at the site, tell me, what are some things you should do before going to buy
and finance a car?
4c. Can you find specific things you can negotiate to pay less for your loan overall?
4d. Find what information you’ll need to be pre-approved for financing.
4e. Find information that might help you decide what to do with your current auto?
4f. What are some very specific ways you can pay less overall for your car loan?
4g. Are there any potential pitfalls or danger areas in getting a loan that you are aware
of?

Auto Finance Usability Test  4

Item 5: Collection of usability and marketing metrics

Goal: To numerically evaluate the usability, learnability, and of the site

System Usability Scale (SUS)

Please circle the numbers that most appropriately reflect your impressions about using this
website.
1. I think that I would like to use this auto finance website one or more times if I was in the process
of purchasing an auto loan:
Strongly
Strongly
Disagree
Disagree
Neutral
Agree
Agree
1

2

3

2. I found this auto finance website unnecessarily complex:
Strongly
Disagree
Disagree
Neutral
1

2

5

Agree

Strongly
Agree

4

5

Agree

Strongly
Agree

4

5

3

3. I thought this auto finance website was easy to use:
Strongly
Disagree
Disagree
Neutral
1

4

2

3

5. I found the various functions in this auto finance website were well connected to
each other:
Strongly
Strongly
Disagree
Disagree
Neutral
Agree
Agree
1

2

3

4

5

6. I thought there was too much inconsistency in the content of this auto finance website:
Strongly
Disagree
1

Disagree
2

Neutral

Agree

Strongly
Agree

3

4

5

7. I would imagine that most people would learn to use this auto finance website very
quickly:
Strongly
Strongly
Disagree
Disagree
Neutral
Agree
Agree
1

2

3

Auto Finance Usability Test  5

4

5

8. I found the auto finance website very cumbersome to use:
Strongly
Disagree

Disagree

1

2

Neutral

Agree

Strongly
Agree

3

4

5

Net Promoter Score (NPS)

Please circle the numbers that most appropriately reflect your impressions about using this
website.
On a scale of 1 to 10, with 10 being more likely, how likely is it that you would recommend this website to
a friend, colleague, or family member as a place to consult about auto finance? Why?

1

2

3

4

5

6

7

Auto Finance Usability Test  6

8

9

10

Item 6: Review of auto finance pages

Goal: To evaluate the participant’s perception of the webpages.
6a. Do these pages match your expectations for what you would need when
planning to finance a car? Why or why not?
6b. As you read through the information, were there any parts that were confusing or hard to
understand? Why? Tell me about these text and visuals.
6c. Which information, if any, did you find most valuable? Why?
6d. What could be done to improve the content of these pages?
6e. What could be done to make these pages more engaging?
6g. After looking at this site, do you feel like you’ll be able to get a better price on your auto
financing? Why or why not?
6h. What do you feel is most important to focus on to get a better price for your auto
financing? Tell us a little more about why you feel this is important.
6i. Who do you think produced this webpage? Do you think it comes from a trustworthy
source? Why or why not?
6j. Tell me, what cost—or costs—would you compare when financing a car? Is there one you’d
pay more attention to? Why?
6k. Was there anything missing that you were expecting to find?
6l. What type of electronic device would you typically read this kind of content on (mobile,
desktop, laptop, tablet)? Where would you be while reading?

Item 7: Worksheet review

Goal: To evaluate the usability, learnability, and usefulness of the worksheet.
Please take a quick look at the worksheet if you have not already. Please let me know when
you are finished.
7a. How would you use a worksheet like this?
7b. Would you take this worksheet with you to track offers when going to get a loan? Why or
why not?
o
[IF YES] How would you take it? In which format? Would you print it out?
7c. Would you feel comfortable asking a salesperson and lender to tell you the different items
on this form? Why or why not?
7d. What, if any, items on this worksheet are most valuable?
7e. What, if any, items on this worksheet are hard to understand?
7f. How do you feel about the organization of the information on this sheet?
7g. How do you feel about the amount of information on this worksheet? Is there too much
information on the worksheet or not enough? What would you add? What would you get rid
of?

Auto Finance Usability Test  7

7h. Do you think this worksheet could help you get a better price for your auto financing?
Why or why not?
7i. Who do you think produced this worksheet? Do you think it comes from a trustworthy
source? Why or why not? What makes/will make it trustworthy?
7j. Is there anything missing from this worksheet that you would want to know?

ebriefing Questions

Item 8: Debriefing questions

(NOTE: Some are used as follow-ups to actions during session and to SUS or NPS responses;
participants do not receive all debriefing questions.)
1. What stands out to you about your experience reviewing these pages? Tell me about your
overall experience using this website. What worked well; what did not work well? [Probe
further as necessary]
2. [Follow up on satisfaction questionnaire responses.]
3. Did you learn any new information on the website that you were unaware of before, or
that was a surprise to you?
4. Earlier, we asked you if you had thought about planning for auto finance. After going
through this website, is that answer still the same? Why or why not?
5. Was there anything missing from the website shown to you that you expected to see, or
expected to have access to, that you haven’t called out already?
6. If you could name this site, what would you call it?
7. If you could give this worksheet a title, what would you choose?
Extra questions
7. Where would you go or what would you do after using this site?
8. What can this organization help you with? What can’t they help you with?
9. [Follow up to NPS] What could be changed to increase the likelihood that you would
recommend this site to a friend?
10. [Follow up to SUS1] What could be improved on the site that would increase
the likelihood that you would use it in the future?
11. What did you like most about the site?
12. What did you dislike the most about the site?
Moderator’s Guide: Auto Finance  0

13. If you could change one thing on the site, what would it be?
14. Do you have any other feedback/input that we did not cover?
15. Would you walk me through your thought process of reading through the text and tips you
found on this page?

Moderator’s Guide: Auto Finance  1