Consumer Complaint Database 4.1 User Research & Usability Testing

MODERATOR GUIDE

User Research with Consumers:
Consumer Complaint Database

Consumer Complaint Database  1

Introduction (5 minutes)
Thank you for participating in this study today. My name is _____ and I work for Fors Marsh
Group. We are a private research firm, and we are here helping the CFPB gather information
to understand how consumers interact with financial companies.
Today we will be talking about your experience with financial products and services as well as
their providers. We are interested in both your positive and negative feedback. Your
comments and feedback will help improve the services CFPB can provide consumers like
you.
The entire session should last about 60 minutes. Do you have any questions so far?
Let’s cover a couple things before we get started.
We are making an audio recording of this session. Only those of us associated with this
project will hear the recordings, and we will not share your name or personal information. The
recordings are used as a memory aid for me so I can go back later to recall what happened
during each session.
There are some people from my team and from the CFPB who are in a viewing room to
observe, take notes, and record your comments. Your candid feedback will be very valuable,
so even though people are observing, please speak openly about your opinions and
experience. Transcripts will be maintained for future research use; however, responses will
not be linked to you personally. Your name and other personally identifiable information will
be removed from the transcripts and you will not be associated with any particular comment
or statement.
The discussion guide in front of me has all of the questions to cover during our session. I may
need to break off a conversation early in order for us to cover all of the questions. Please
keep in mind that we are not testing you or your knowledge and the questions I’ll be asking
are designed to learn from you.
Do you have any questions about the paperwork you reviewed when you came in? Do you
have any other questions about the session?

Consumer Complaint Database  2

Interview
Task order:
•
•

A, B (odd-number participants)
B, A (even-number participants)

A: Shoppers
A.1. “In the market” consumers
Goal: Learn about the current stage in their decision-making process. (3 minutes)

A.1.1.

You had mentioned on the questionnaire you completed earlier that you are
considering a new [Insert financial product]. Could you talk about where you are
at in your decision-making process?

Goal: Learn more about the person’s decision-making process during their journey. (10 minutes)

A.1.2.

Could you walk me through your overall shopping process for this [financial
product]? [If necessary] How are you going about shopping for a [financial
product]?
i. What led to your decision to begin shopping for a new [Insert financial
product]?
ii. [If necessary] What did you do first when you made the decision?

A.1.3.

What comes next in your process? [Continue to ask as necessary]

A.1.4.

What are the decisions you’ll be making along the way before making that final
decision of which product and provider to select?

A.1.5.

How did you learn about this financial product and its different options?

A.1.6.

How did you learn about the [financial provider(s)] you are considering?

A.1.7.

What additional information about this [financial provider] will you look for before
making this decision? [Continue to ask as necessary]

A.1.8.

Will there be or are there any resources that you (will be)/(are) using during this
phase in your search? [Continue to ask as necessary]

Goal: Learn more about how the participant will make a final decision. (5 minutes)
[Ask questions if that information has not already been provided]

Consumer Complaint Database  3

A.1.9.

How do you plan to make a final decision about which financial provider to
select?

A.1.10. Will you look for any additional information about the provider or product after
you make your financial decision?
Goal: To go into more detail about the resources the person is seeking during their shopping journey.
(5 minutes)

A.1.11. Have you sought information or plan to seek information about other people’s
experiences with that provider?
i. If yes, where would you go to get that information?
ii. [If necessary] If this information was available, would it be helpful? Would you
use it?
A.1.12. What information would you be looking for when reviewing information about
other people’s experiences?
A.1.13. How would you use the information you find about other people’s experiences?
A.2. “Potential switchers” consumers
Goal: Learn about the current stage in their decision-making process. (3 minutes)

A.2.1.

You had mentioned on the questionnaire you completed that you are considering
switching providers for [insert financial product]. Could you talk about where you
are in your decision-making process?

Goal: Learn more about the person’s decision-making process during their journey. (10 minutes)

A.2.2.

What led to your decision to switch providers for [insert financial product]?
i. [If necessary} What did you do first when you decided to switch providers?

A.2.3.

Could you walk me through your overall shopping process for switching
providers? [If necessary] How are you going about shopping for a different [insert
financial product]?

A.2.4.

What comes next in your process? [Continue to ask as necessary]

A.2.5.

What are the decisions you’ll be making along the way before making that final
decision to switch providers?

A.2.6.

How did you learn about the [insert financial provider(s)] you are considering?
Consumer Complaint Database  4

i. What have you learned so far about [financial provider(s)]?
A.2.7.

What additional information about this [financial provider] will you look for before
making your final decision to switch providers?

A.2.8.

What resources would you use to make this decision to switch [financial
product]?

Goal: Learn more about how the participant will make a final decision. (5 minutes)
[Ask questions if that information has not already been provided]

A.2.9.

How do you plan to make a final decision about which [financial provider] to
select?

A.2.10. Will you look for any additional information about [financial provider] after
deciding to switch?
Goal: To go into more detail about the resources the person is seeking during their shopping journey.
(5 minutes)

A.2.11. Have you sought information or plan to seek information about other people’s
experiences with that provider?
i. If yes, where would you go to get that information?
ii. [If necessary] If this information was available, would it be helpful? Would you
use it?
A.2.12. What information would you be looking for when reviewing information about
other people’s experiences?
A.2.13. How would you use the information you find about other people’s experiences?
B. Empathy-building
B.1. “I want others to know”
Goal: To learn more about the person’s financial issue and his or her current stage in resolving the
issue. (10 minutes)
B.1.1.

You had mentioned on the questionnaire that you completed earlier that you have
experienced an issue with [financial product or service]. Could you talk about that issue?

B.1.2.

Did you take any steps to follow up or resolve the issue?
i. Could you share with me what you did to follow up or resolve the issue?
Consumer Complaint Database  5

B.1.3.

Have you officially submitted a complaint about your issue?
i. [If yes] To whom did you submit your complaint and how?
ii. [If yes] Did your complaint help to resolve the issue?
iii. [If yes] Did you receive a response or action from the company after you submitted a
complaint?

Goal: To learn about the participant’s motivation for sharing with others. (10 minutes)
B.1.4.
i.
ii.
iii.
iv.
v.
vi.
vii.
viii.

Have you shared your experience with anyone?
Why?
[If yes] What drove you to share your experience with others?
[If yes] Who did you share your experience with?
[If yes] How did you share your experience with others?
[If yes] What was the specific information you were comfortable sharing?
[If yes] Did you have any concern with what you were sharing?
[If yes] Was it helpful to share your complaint with other consumers?
[If no] Is there anything you are concerned about that is leading to you not sharing
your experience with others or are you just not interested?
a. [If interested in sharing] How would you inform other consumers about a
financial issue you experienced?

B.2. “Finding other consumers like you”
Goal: To learn more about the participant's attitude toward finding people like them and learn about
their issues. (10 minutes)
B.2.1.
i.
ii.
iii.
iv.
v.
vi.
B.2.2.

B.2.3.
B.2.4.

Have you tried to connect or find information from other people with the same or similar
issue?
Why?
[If yes] How did you find these people and how you did you find or connect with
them?
[If yes] What were you looking for when you reached out to other people?
[If no] Why were you not interested in reaching out to others with a similar issue?
[If no] Would you be interested in learning about people who have the same issue?
[If no] How do you think you could find others who have the same or similar issue?

Would it be helpful to have access to complaints made by other consumers about a
financial issue?
i.
Why?
How would you use the information provided by other consumers?
Would you use the information provided by other consumers to make decisions about
which financial product and provider you would select?
i.
Why?
Consumer Complaint Database  6

[Repeat B.1.1 through B.2.4 for each issue]

C. Database User Experience:
C.1. Live Database
Goal: To learn about participants’ expectations for the CCDB.
C.1.1.
C.1.2.
C.1.3.

C.1.4.
C.1.5.

C.1.6.

Imagine that there is a database that includes consumer complaints. What would you
expect it to look like?
What types of information are you interested in finding about others’ complaints?
The Consumer Financial Protection Bureau is a government agency that helps
consumers. Every week thousands of consumers submit complaints about their
financial issues to CFPB’s website, and if they consent to share, their stories will be
shared in a database that is available to the public. What are your thoughts about this
database?
Would you be willing to share information on this database?
Would you be concerned about the privacy of your information? [If they express
concerns about sharing their information] How could they assure you that your
information would be kept private?
Would you be interested in using this database to learn about people who have had
issues with a financial product or service?

Goal: To receive feedback about the landing page of the database.
[Open landing page of the database]
C.1.7.
C.1.8.

Here is the home page for the site, what are your first impressions of it?
Please take a minute to interact with the page. What is your experience with using the
page?
C.1.9.
[Database landing page] How would you find complaints about a [financial product]?
C.1.10. [Database landing page] How would you find more complaints about a [financial
product]?

Goal: To receive feedback about the design of the database.
[Open database]
C.1.11.
C.1.12.
C.1.13.
C.1.14.
C.1.15.
C.1.16.

[Database interface] What do you think about this page?
How could you filter this information?
Does what you see meet your expectations? Why?
What is the most important information here?
Is there anything missing from this page?
Do you expect to be able to save this information? How?

C.2. Wireframe
Goal: To compare the design of the live database to the alternative designs.
Consumer Complaint Database  7

C.2.1.
C.2.2.
C.2.3.
C.2.4.
C.2.5.

Here’s a mock-up of an alternative design of the database. What do you think about this
design?
Does it meet your expectations?
What do you like or dislike about this version compared to the previous?
Which one do you prefer, the live database or the alternative design? Why?
Is there anything missing from this version?

That concludes the session. Thank you very much for your feedback. Do you have any questions for
me about our discussion?

Consumer Complaint Database  8