Owning a Home Project User Experience Testing – Round 1

MODERATOR’S GUIDE

Owning a Home
Know the Process - Content
UX Test (OaH3) - DRAFT
Data Collection: [Insert Dates]

DRAFT

Introduction (5 minutes)

Thank you for participating in this study today. My name is _____, and I am a researcher with
the Fors Marsh Group User Experience Team. Today we will be evaluating a prototype of a
website.
I will ask you questions and give you some tasks to complete using the website. Your
comments and feedback will help the team improve the site. I did not create the site, so
please do not feel like you have to hold back on your thoughts to be polite to me. I’m
interested in both your positive and negative reactions. Difficulties you may run into reflect
issues with the design of the site, not your skills or abilities. Please keep in mind that I am
not testing you or your knowledge. Rather, you are helping us to see how we can improve
this site.
The entire session will last about 60 minutes. In accordance with the Papework Reduction
Act of 1995, this study as been approved by the Federal government’s Office of
Management and Budget (OMB) under OMB control number 3170-0024. Without this
approval we could not conduct this study. Do you have any questions so far?
Let’s cover a few things before we get started.
You might have noticed the cameras. We are recording the session. The recording is used as
a memory aid for me so I can recall what happened during the session. Please speak openly
about your opinions and experiences. We want to learn from you, so it is important that you
share your honest opinions.
I am interested in your thoughts and reactions as we proceed. This is important because I
can see what you are doing, but I won’t know why you are doing it. So I need your help. While
you are working, I would like you to think aloud. I would like you to:
•
•
•
•
•

Tell me what you are thinking
Describe the steps that you are taking.
Tell me why you are doing what you are doing, what you are going to do, and why.
Tell me why you clicked on a link and what you expect to happen next.
Basically, just tell me everything you are thinking as you work.

There are no right or wrong answers, and your comments and opinions will only be used in
combination with the feedback that we get from other people.
What we talk about here is private. That means your name will not be associated with
anything you say in our reports and your responses will not be linked to your identity in any
way.
Likewise, we want to respect everyone’s privacy in this room and not share any of our
discussion from today with others who are not actively working on this project.
Owning a Home Journey Content – Moderator Guide  1

DRAFT

Please also make sure to review the Privacy Act Statement provided to you in hard copy.
[Provided in hard copy.]
5 U.S.C. 552(a)(e)(3)
The information you provide through your responses to Fors Marsh will assist the study
sponsor, the Consumer Financial Protection Bureau (“CFPB”), in participating in usability
testing that evaluates consumers’ experiences with a financial website.
The CFPB will obtain and access identifying information from Fors Marsh about study
participants by observing recordings made during this usability testing.
Information collected on behalf of the Bureau by Fors Marsh will be treated in accordance
with the System of Records Notice (“SORN”), CFPB.022,
https://www.federalregister.gov/articles/2012/11/14/2012-27582/privacy-act-of-1974as-amended. This information will not be disclosed. The Routine Uses for the SORN
specifies how we may use this information. Direct identifying information will only be used by
Fors Marsh to facilitate the usability testing and will be kept private except as required by
law.
This collection of information is authorized by Pub. L. No. 111-203, Title X, Sections 1013
and 1022, codified at 12 U.S.C. §§ 5493 and 5512.
Participation in this study is voluntary, you are not required to participate or share any
personally identifying information with Fors Marsh during the usability testing, including
name, age, email address, mobile number, alternate number, race, ethnic origin, gender,
city and state of residence, education level, household income, current employment status,
and occupation. You may withdraw from participation at any time.
•
•

Any questions so far?
Ok, we are ready to begin. I would like you to start by asking you some background
questions.

Owning a Home Journey Content – Moderator Guide  2

DRAFT

Section 1: Interview about steps taken in the homebuying/mortgage
process (3 minutes)

Section 1 Goal: Understand the stage each participant in the homebuying process.
You had indicated that you are [in the process of buying a home/preparing to buy a home].
I’m going to ask you a few general questions about your experience with this process.
1.1. Would you tell me about your experience so far? What has been your experience with
looking for a mortgage?
1.2. What are the major challenges you have been facing as you’ve been shopping? What
have you been struggling with?
1.3. What resources have you consulted (friends or tools/resources)? What tools or
resources do you wish you had access to but didn’t?

Section 2: First Impressions of the Landing Page (5 minutes)

Goal: Assess users’ first impressions of the site
2.1. Let’s say that you are looking for more information about the homebuying process and
you arrive at this page. What are your initial impressions?
Goal: Assess what users would do first when visiting the site
2.2. What would you do first if you came to this page with the goal of getting more
information about [insert their stage in the homebuying process]?
Goal: Assess users’ understanding of the purpose of the site
2.3. In your own words, how would you describe the purpose of this page to a friend?
Goal: Asses susers’ understanding of the way information is organized on the landing page
2.4. How would you describe how it is organized? [If necessary] What is the overall structure
for the information on the page? Why do you think the page is divided into these four
sections?

Section 3: Know the Process Phase Selection (5 minutes)

Please take some time to look at each of the sections listed here on the main part of the
page.
Goal: Assess how users understand the Know the Process phases
3.1 Do these separate sections make sense to you? Why or why not?
Goal: Assess how easily users can identify the phase that best matches where they are in
the homebuying process
3.2. Now I’ll ask you to look at each of the sections listed here and tell me which section
best fits where you are in your homebuying process.
Owning a Home Journey Content – Moderator Guide  3

DRAFT
3.3. Before you click on the link to the page, please explain why you chose that section. [If
necessary] Was there particular information that popped out to you? Is there a reason why
particular information appealed to you?
Goal: Assess how users understand the content of each phase based on the description on
the landing page
3.4. Can you tell me what you think will be covered in this section, based on what you’ve
read in the description?
Goal: Assess whether anything is confusing in the description
3.5. Is there any language in the description that’s confusing?
Moderator note: Each participant will review one page of the Know the Process content. Go
through questions for the page that the participant provides in section 3, and then continue
to the action step questions in section 4.

Section 4: Know the Process Phase Walkthrough (10-15 minutes)
Moderator note: Questions can be asked as needed if not covered in participants’
responses.

Goal: Assess how users understand the Know the Process pages overall, including the
content and organization
4.1. Can you explain what you think the purpose of this page is, in your own words?
4.2. Can you explain to me how it’s organized? [If needed] What do you think these “action
steps” refer to?
4.3. Please take a minute to explore the information on this page.
4.4. Is there anything that you’ve found confusing?
Goal: Assess how users understand the introduction section
Now I’ll ask you to look at the [Insert phase name] section.
4.5. What do you think about the introduction?
4.6. Is there anything confusing or unclear about it?
Goal: Assess how users respond to the title “Know the process”
4.7. What do you think about the title “Know the process”, near the top left of the page?
Does this title fit?
Goal: Assess how users understand the “goals for this phase” section
Now I’ll ask you to look at “Goals for this phase” section.
4.8. Can you tell me what these goals are, in your own words?
Goal: Assess how well the content aligns with their mental model
Owning a Home Journey Content – Moderator Guide  4

DRAFT
4.9a. [If previously bought a home or refinanced] Does this process match what you went
through when you were buying a home and shopping for a mortgage? (If not) What was
different?
4.9b. [If currently shopping for a home] Does this process match what you expect to happen
as you are preparing to shop? [If not] What is different?
Goal: Assess users’ understanding of the action steps presented; assess how users
understand the way information is organized within each action step
4.10. Can you give me a quick overview of what these action steps cover?
4.11. Do you find the titles of these action steps easy or difficult to understand?
4.12. What do you think about the steps listed on this page? Does the order make sense to
you? (if not) How would you rearrange it?
Goal: Assess how easily users can locate the action step that best matches where they are
in their homebuying process
4.13. Out of the action steps listed here, which one do you think best matches where you
are in your homebuying process? [Moderator note: Proceed to section 3b for the action step
selected.]

Section 5: Know the Process Phase – Action Step Walkthrough (10
minutes)

Moderator note: Only ask these questions for the action step covered during the session.
Questions specific to each action step are listed in the following sections. This section can
be repeated if there is time to review a second action step.
Goal: Assess users’ understanding of the action step content.
5.1. Please take a few minutes to read this section for the action step: [insert title]. As you’re
reading, please let me know if you find anything confusing or unclear.
Goal: Assess whether there is any information in the section that users don’t agree with.
5.2. Is there any information that is presented that you don’t agree with?
5.3. What are your thoughts on the way things are organized under the action steps as
“What to do”, “What to know”, and “How to avoid pitfalls”? Do you like this way of organizing
this information? [if not] How would you organize it?
Goal: Assess users’ understanding of the bolded heading titles within the action steps.
5.4. How clear are the bolded headings listed under this action step? Why? Are there any
that you would change?
Goal: Assess if there’s any specific vocabulary/concepts that confuse users.
5.5. Are there any specific words or concepts used that are confusing?
Owning a Home Journey Content – Moderator Guide  5

DRAFT
Goal: Assess users’ overall comprehension of the content under the action step; assess
their understanding of what they’re being asked to do.
5.6. Based on what you’ve read here, can you give me a brief overview of what the site is
suggesting that you do when [insert action step title]?
Goal: Assess whether there was unnecessary information on the page
5.7. Is there any information in this action step that you feel isn’t necessary, or seems out of
place?

Section 6: Know the Process Phase – Post-Review Action Step Questions
(10 minutes)
[Moderator note: Ask these questions after reviewing the action step chosen.]
Goal: Discover what participants would do next
6.1. What would you do next after reading this information?

Goal: Assess whether users want more information
6.2. Now, thinking about the overall content, do you feel like this is enough information to
move forward in a mortgage process, or do you feel like you would need more information
and tools?
6.3. [if yes] What kinds of information would you want? What kinds of tools would be
helpful?
Goal: Assess what content on the page users thought was the most helpful
6.4. What specific information on the page did you find particularly helpful?
Goal: Assess users’ response to the tone of the content
6.5. [If necessary] Was there anything about the tone of the content that you found
discomforting or off-putting?
Goal: Assess how users feel about the length of the content presented
6.6. How do you feel about the length of the section(s)?
Goal: Assess readability of the content
6.7. How readable did you find the content on the page?
Goal: Assess what next steps users would take after reviewing the information
6.8. After reading through the information on this page, what would you click on next?
Goal: Assess how confident users are in being able to complete the action step reviewed,
based on the information presented
6.9. On a scale of 1-5, with 1 being “not at all confident” and 5 being “extremely confident”,
how confident do you feel that you could do [action step]?

Section 7: Trust (5 minutes)
Owning a Home Journey Content – Moderator Guide  6

DRAFT
Goal: Assess whether users trust the information they receive through this tool
7.1. On a scale from 1-5, with 1 being not trustworthy at all and 5 being extremely
trustworthy, how much do you trust the information on this site? Why or why not?
Goal: Assess whether the process outlined on the site matches the users’ mental models
7.2. Does the process on this page match your own understanding of the process? What is
the same? What is different? How does this affect how much you trust the information?
Goal: Assess what factors affect that trust (or lack thereof): other sources? Info from people
they know? Etc.
7.3. Is there any information on this site that you have also received from other sources? Is
there any information that contradicts information you’ve previously received? How does
that difference/similarity affect your level of trust?
7.4a. [if they trust the site] Can you explain what makes you trust the information on this
site?
7.4b. [if they don’t trust the site] Is there anything that could be added to the site that would
make you more likely to trust this site?

Section 8: Behavior Change (3 minutes)

Goal: Assess whether users would use this tool repeatedly; assess when they use it and
under what circumstances
8.1. Do you think you would access the information on this site during your mortgage
process? When do you think you would do so?
8.2. [if more than once] How many times do you think you would refer to it: repeatedly, or
once in a while?
Goal: Assess whether users’ approach to buying a home and getting a mortgage reportedly
changes after using this tool
8.3. Do you feel like your approach to buying a home and getting a mortgage would change
after visiting this site and using these tools? How so?

Owning a Home Journey Content – Moderator Guide  7