Submit a Complaint - Small Business Intake User Experience Test

Moderator’s Guide
Consumer Response – Small Business Intake

Introduction

Thank you for participating in this study today. My name is _____, and I work with the User
Experience Team at Fors Marsh Group. Today we will be discussing the process of submitting a
complaint about a financial institution. You will first be asked to use a website to submit a complaint
using a scenario that I’ll provide. When we get to a certain point, I’ll show you a redesigned page that
the team is considering. We’ll then talk about your experience with the process. Your comments and
feedback will help the team improve the complaint submission process. I did not create the website
and page you’ll be interacting with, so please do not feel like you have to hold back on your thoughts
to be polite to me. We’re interested in both your positive and negative reactions. Difficulties you may
run into reflect the design of the website, not your skills or abilities. Please keep in mind that I am
not testing you or your knowledge. Rather you are helping us to see how we can improve the website
and process.
The entire session should last about 60 minutes. Do you have any questions so far?

Let’s cover a couple things before we get started.
•

•

•
•

We are making an audio and video recording of the computer screen. Only those of us
associated with this project will see the recording, and we will not share your name or
personal information. The recording is used as a memory aid for me so I can go back later to
recall what happened during the session.
I am interested in your thoughts and reactions as we proceed. This is important because I’ll
be able to see what you are doing, but I won’t have insight into what you’re thinking. When
we are using the website, I’d like you to think aloud. Please let me know what you’re thinking
thoughout – including discussing terms, language, and the organization of information.
I have a guide in front of me has all of the questions to be covered during the session. Since
we have only 60 minutes, I may need to break off a conversation so that we can move on to
the next question in the guide.
If you do not want to answer any of my questions, you do not have to. Please just say so, and
we will move on. And of course, you can stop at any time.

Did you have any questions about the Privacy Act Statement?
[Provided in email]

Small Business Intake – Moderator Guide  1

5 U.S.C. 552(a)(e)(3)
The information you provide through your responses to Fors Marsh Group will assist the survey
sponsor, the Consumer Financial Protection Bureau (“CFPB”), in evaluating a form that consumers
can complete to file a complaint about a financial institution.
The CFPB will not obtain or access any directly identifying information from Fors Marsh about study
participants. The agency will only obtain and access de-identified results and aggregated analyses of
those results.
Information collected on behalf of the Bureau by Fors Marsh Group will be treated in accordance
with the System of Records Notice (“SORN”), CFPB-005 – CFPB Consumer Response System, 79 FR
21440. Direct identifying information will only be used by Fors Marsh Group to facilitate the study
and will be kept private except as required by law.
This collection of information is authorized by Pub. L. 111-203, Title X, Sections 1011, 1012,
1013(b)(3), 1021, 1034, codified at 12 U.S.C. 5491, 5492, 5493(b)(3), 5511, 5534.
•
•

Do you have any other questions before we begin?
Ok, I would like you to start by asking you some background questions.

Summary of protocol:
1. The moderator will ask participants background questions about their business.
2. Participants will be provided with a simple complaint to submit using the website.
o Participants will begin on the Submit a Complaint home page.
o When the participant arrives on the page that will include the additional small
business intake questions, the moderator will present the participant with a
mock-up of the proposed page. The presentation order of the two mock-ups
(Verions A and B) will be alternated for each participant.
3. Participant will be asked to complete a short questionnaire about impressions of the
1st mock-up.
4. The moderator will ask follow-up questions about their experience using the 1st
mock-up.
5. The moderator will provide participants with the alternative mock-up and ask them to
walk through the completion process again.
6. Participant will be asked to complete a short questionnaire about impressions of the
2nd mock-up.
7. The moderator will ask follow-up questions about their experience using the 2nd
mock-up.
8. The moderator will close the interview with debriefing questions about their overall
experience.

Small Business Intake – Moderator Guide  2

Section 1: Pre-task interview. Steps taken with a complaint

1a. One of the reasons why we asked you to participate is because you’re the owner of a
small business. Could you provide a short description of your business?
1b. [If indicated during the recruitment process] Have you had any complaints about a
financial institution you’ve worked with? [If necessary] For example, a complaint could be
about how a financial institution changed the terms on a company credit card or loan. [If
necessary] How did you address the problem? What has your experience been like when
submitting a complaint using an online form? What do you find works well and does not
work well when submitting a complaint online?

Section 2: Complaint Submission (1)

2a. Now let’s move into a scenario. Let’s say that you had taken out a new business credit
card to cover some unexpected short-term expenses. You received an introductory 0%
APR for the first 90 days but unfortunately you were late on the second payment. The
credit card company immediately imposed a 24% penalty APR. You reviewed the terms
and agreements and you can’t find any information that authorizes them to impose such
a high rate. You submit a complaint to the bank that issued the credit card but despite
an email and a follow-up call they’re not responding. After some internet searching, you
learn that you can submit the complaint to the Consumer Financial Protection Bureau.
2b. [Provide participant with the link to navigate to the Submit a Complaint website.] You
then arrive at this site with the goal of submitting this complaint. Could you show me
what you would do? Please remember to think aloud as you work.
2c. [Immediately before the participant clicks on Next to go to Step 3 (My information), direct
participant to one version of the mock-up – rotating between the two versions.] I’m
going to next show you a mock-up of the page that would come next that the team is
still working on. This page won’t be clickable but I’d like you to walk me through how you
would complete it.
2d. [If participant indicates that they would select “yes” to the question: “Is this complaint
related to a small business you own?” or check the box: “A small business owner”, then
pull up the second mock-up in the progression for the same version (“yes, selected”).]
When you select that button/check that box, the page would expand and it would look
like this. Please continue to walk me through what you would do next.
2e. What would you expect to happen next after clicking review and then eventually submit?
Would you expect to hear back from CFPB? When would you expect to hear back? What
would you expect from them when you do hear back?

Small Business Intake – Moderator Guide  3

Section 3: Satisfaction Questionnaire (1)
Satisfaction Questionnaire (SATQ)

Please select your responses for each of the questions that most appropriately reflect your
impressions of the “Who are the people involved?” page.

1. Please rate the clarity of the information presented to you on the page:
Not Clear
At All
1

Slightly
Clear
2

Moderately
Clear
3

Very
Clear
4

Extremely
Clear
5

2. Please rate the organization of the information on the page:
Not Organized
At All

Slightly
Organized

Moderately
Organized

Very
Organized

Extremely
Organized

1

2

3

4

5

Flowed Very
Well

Flowed
Extremely
Well

4

5

3. Please rate the flow of the information on the page:
Did Not Flow
Well
At All

Flowed Slightly
Well

Flowed
Moderately
Well

1

2

3

Section 4: Follow-ups (1)

4a. What was your experience like using this page?
4b. What did you think about the information that was being requested? [If necessary] Were
the questions appropriate?
4c. Do you have any concerns with the questions that were asked of you?
4d. How clear were the questions that were asked of you?
4e. How well was the information organized?
4f. How did the information and questions flow?

Section 5: Complaint Submission (2)

Small Business Intake – Moderator Guide  4

5a. [Provide participant with the alternative version of the page.] Next I’d like to show you
another version of the same page that the team is still working on. This page also won’t
be clickable but I’d like you to walk me through how you would complete it.
5b. [If participant indicates that they would select “yes” to the question: “Is this complaint
related to a small business you own?” or check the box: “A small business owner”, then
pull up mock-up number two for the same version (“yes, selected”).] When you select
that button/check that box, the page would expand and it would look like this. Please
continue to walk me through what you would do next.
5c. What would you expect to happen next after clicking review and then eventually submit?
[If necessary] Would you expect anything different than before?

Section 6: Satisfaction Questionnaire (2)
Satisfaction Questionnaire (SATQ)

Please select your responses for each of the questions that most appropriately reflect your
impressions of the “Who are the people involved?” page.

1. Please rate the clarity of the information presented to you on the page:
Not Clear
At All
1

Slightly
Clear
2

Moderately
Clear
3

Very
Clear
4

Extremely
Clear
5

2. Please rate the organization of the information on the page:
Not Organized
At All

Slightly
Organized

Moderately
Organized

Very
Organized

Extremely
Organized

1

2

3

4

5

3. Please rate the flow of the information on the page:
Did Not Flow
Well
At All

Flowed Slightly
Well

Flowed
Moderately
Well

Flowed Very
Well

Flowed
Extremely
Well

1

2

3

4

5

Section 7: Follow-ups (2)

7a. What was your experience like using this page?
7b. What did you think about the information that was being requested? [If necessary] Were
the questions appropriate?
7c. Do you have any concerns with the questions that were asked of you?
Small Business Intake – Moderator Guide  5

7d. How clear were the questions that were asked of you?
7e. How well was the information organized?
7f. How did the information and questions flow?

Section 8: Debrief

8a. How do the two versions of the page compare?
8b. Which version do you prefer? Why?
8c. Are there any additional questions that you think should be included? [If necessary,
explain the purpose of the Submit a Complaint website so participant can determine
whether any other questions should be asked.] [If minority-owned or women-owned
business] Are there any additional questions that you think should be asked that would be
unique to a [women-owned or minority-owned] business?
8d. Did you notice on one of the early pages that you had the opportunity to share? What do
you think about that option? [If necessary] Would that be something you’d be comfortable
doing?
8e. How effective do you think this page will be at collecting the information necessary to
help small business owners with their complaints? [If necessary] Is there anything else we
haven’t talked about that you think should be improved to help accomplish that goal?

Small Business Intake – Moderator Guide  6