Consumer Focus Groups and Usability Testing for Proposed Telephone Arbitration Survey

Generic Clearance for the Collection of Qualitative Feedback on the Service Delivery of the Consumer Financial Protection Bureau

2014_01_17_Attachment_C_Usability_User_Guide

Consumer Focus Groups and Usability Testing for Proposed Telephone Arbitration Survey

OMB: 3170-0024

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 3170-0024 can be found here:

Document [pdf]

Download: pdf | pdf

Attachment C: Request for Approval under the “Generic Clearance for the Collection of Qualitative
Feedback on the Service Delivery of the Consumer Financial Protection Bureau”
(OMB Control Number 3170-0024)

Credit Card Arbitration Telephone Survey:
Interviewer Guide for User Testing

Introduction (5 minutes)
My name is ___________ and I work for a company called ICF International. We are a research
company that conducts interviews and focus groups on behalf of different organizations across
the country. For this specific project, we are working on behalf of the Consumer Financial
Protection Bureau. The Consumer Financial Protection Bureau is an agency of the Federal
government whose role is to make markets for consumer financial products and services work
for Americans — whether they are applying for a mortgage, choosing among credit cards, or
using any number of other consumer financial products.
Before we begin the interview, I would like to inform you of a few things.


Your participation in this interview is totally voluntary. You can choose not to answer
any questions you do not want to answer.



Your name will not be used in any written reports for this project.



You will receive an incentive of $40 to compensate you for your time. The interview will
take approximately 60 minutes. You may discontinue participation at any time without
penalty or loss of benefits.



This research study has been approved by the US Government Office of Management
and Budget and has been assigned OMB Control Number 3170-XXXX.

Do you still wish to participate in this interview?
The purpose of this interview is to test potential items for a national telephone survey that is
going to be conducted to learn more about Americans’ credit card accounts. Before they
conduct the survey, the researchers want to test the questions to make sure that they are as
easy to understand as possible. In today’s interview, I am going to have you answer some of
the survey questions so that we can make sure that they are clear. Therefore, if anything about
the questions is confusing or unclear to you, please let me know.
Do you have any questions before we begin?

Section I: Initial Administration of Survey Questions (10 minutes)
At this point I’m going to ask you the questions on the telephone survey. As you answer each
question, please describe the thought process behind your answers. Let me know if there are
any words or phrases that you don’t understand, or if you find the question difficult to answer
for any reason.
After we have completed the whole survey, I will go back to the beginning and ask you some
follow-up questions about the questions that I asked.
Interviewer should administer survey as drafted, recording both the participant’s answers to all
questions and any notes related to the usability of the questions. Interviewer should not provide
any clarifications, or ask any follow-up questions, until the entire survey is complete.

Section II: Follow-Up Questions (45 minutes)
That completes the questions that are on the telephone survey. Now I’d like to go back to the
beginning of the survey, and ask you some additional questions to determine how clear the
questions are and how they could be improved.
Question 1
The next question on the survey was [reread Question 1]. Your answer to this question was
[repeat answer].
1. Was there anything that you found confusing about this question?
2. Did you have any difficulty answering this question for any reason? If so, why?
3. The question stated not to include “debit cards” and “prepaid cards” in your answer.
What do you think is the difference between a credit card and a debit or prepaid card?
Question 2
The next question on the survey was [reread Question 2]. Your answer to this question was
[repeat answer].
4. Was there anything that you found confusing about this question?
5. Did you have any difficulty answering this question for any reason? If so, why?
6. Did you have any difficulty deciding which credit card to think of when answering this
question? If so, why?

7. If necessary: Could you explain why you answered Yes/No/Don’t Know? How were you
involved in the decision to sign up for the card?
8. This question asked whether you were involved in the decision to “sign up” for this
credit card. Was it clear what was meant by “signing up” for a card?
a. If the question had instead asked whether you were involved in the decision to
“apply for” this credit card, would your answer have been different?
b. Which phrase do you think is easier to understand—“signing up” for a credit card
or “applying for” a credit card?
Question 3
The next question on the survey was [reread Question 3]. Your answer to this question was
[repeat answer].
9. Was there anything that you found confusing about this question?
10. Did you have any difficulty answering this question for any reason? If so, why?
11. Again, your answer to this question was [repeat answer]. Is there anything else you
would want to add to that answer, after having more time to think about it?
Questions 4 and 5
The next question on the survey was [reread Question 4]. I then gave you eight different
features of credit cards to consider, and for any features that you indicated played into your
decision, I asked how important that feature was. I’m going to go through each of those eight
features, and ask you about each in turn.
Repeat following questions for each of the eight credit card features.
12. When I asked about [feature], your answer was [answer]. [If applicable: You then said
that the feature was [importance] in your decision to sign up for the card. Did you have
any difficulty answering these questions for any reason? If so, why?
13. In your own words, what do you think I meant when I asked you about [feature]?
After asking about all eight features, ask participant to explain any clear contradictions between
their answer to Question 3 and their answer to Questions 4 and 5—for example, if participant
mentioned a feature in Question 3 but answered “No” to the same feature in Question 4, or if

participant indicated that a feature was important in Question 5 but did not mention it at all in
Question 3.
Question 6
The next question on the survey was [reread Question 6]. Your answer to this question was
[repeat answer].
14. Was there anything that you found confusing about this question?
15. If we described considering other cards as “comparison shopping,” would that make it
more clear?
16. Did you have any difficulty answering this question for any reason? If so, why?
17. In your own words, what do you think this question was asking you?
18. If participant answered “Yes”: Please describe how you compared this card to other
credit card options. What information did you use, and where did you get that
information?
Question 7
The next question on the survey was [reread Question 7]. Your answer to this question was
[repeat answer].
19. Was there anything that you found confusing about this question?
20. Did you have any difficulty answering this question for any reason? If so, why?
21. Did you look at your credit card when answering this question?
Question 8
The next question on the survey was [reread Question 8a]. Your answer to this question was
[repeat answer].
22. Was there anything that you found confusing about this question?
23. Did you have any difficulty answering this question for any reason? If so, why?
24. In your own words, what do you think this question was asking you?

25. If necessary: Could you explain why you answered Yes/No/Don’t Know?
The next question on the survey was [reread Question 8b]. Your answer to this question was
[repeat answer].
26. Was there anything that you found confusing about this question?
27. Did you have any difficulty answering this question for any reason? If so, why?
28. In your own words, what do you think this question was asking you?
a. In your own words, what does the phrase “class action” mean?
29. If necessary: Could you explain why you answered Yes/No/Don’t Know?
Questions 9/9a/9b
The next question on the survey was [reread Question 9]. Your answer to this question was
[repeat answer]. If Q9=Yes: I then asked you to list any important features of arbitration, and
your answer was [repeat answer].
30. Was there anything that you found confusing about these questions?
31. Did you have any difficulty answering these questions for any reason? If so, why?
32. If Q9 = Yes: Where have you heard of arbitration as a way of resolving disputes before?
In your own words, what is arbitration?
Questions 9c/9d
The next question on the survey was [reread Question 9c]. Your answer to this question was
[repeat answer]. If Q9c=Yes: I then asked you what the “opt out” meant, and your answer was
[repeat answer].
33. Was there anything that you found confusing about these questions?
34. Did you have any difficulty answering these questions for any reason? If so, why?
35. If Qc = No: In your own words, what do you think it means to say “opt out”?
36. If Q9c = Yes: Please describe when your credit card company gave you an opportunity to
opt out of binding arbitration requirements. Did you choose to opt out? Why or why
not?

Question 10
The next question on the survey was [reread Question 10a]. Your answer to this question was
[repeat answer].
37. Was there anything that you found confusing about this question?
38. Did you have any difficulty answering this question for any reason? If so, why?
39. In your own words, what do you think this question is asking?
40. How did you come up with your answer to this question?
The next question on the survey asked [reread Question 10b]. Your answer to this question was
[repeat answer].
41. Was there anything that you found confusing about this question?
42. Did you have any difficulty answering this question for any reason? If so, why?
43. How did you come up with your answer to this question?
44. Your answer to this question was [larger than, smaller than, the same as] your answer to
the previous question. Could you explain why that is?
The next question on the survey asked [reread Question 10c]. Your answer to this question was
[repeat answer].
45. Was there anything that you found confusing about this question?
46. Did you have any difficulty answering this question for any reason? If so, why?
47. How did you come up with your answer to this question?
Question 11
48. Was there anything that you found confusing about this question?
49. Did you have any difficulty answering this question for any reason? If so, why?
50. [Regarding 11d] How did you come up with your answer to this question?
51. When you were answering this question, did you think about how long it might take you
to complete this form?

a. If so, how long did you think it would take you to complete? On what were you
basing that assumption?
Question 12
The next question on the survey was [reread Question 12]. Your answer to this question was
[repeat answer].
52. Was there anything that you found confusing about this question?
53. Did you have any difficulty answering this question for any reason? If so, why?
54. If Q12 = Yes: If you feel comfortable doing so, could you describe the situation that you
were referring to when you answered Yes to this question? Probe to assess whether the
situation warranted a “Yes” answer. If participant expresses any reluctance to discuss
the situation, move on to next question.
a. The next question on the survey asked whether you think the process was fair,
and your answer was [repeat answer]. Could you explain why you gave that
answer?
Question 13
The next question on the survey was [reread Question 13]. Your answer to this question was
[repeat answer].
55. Was there anything that you found confusing about this question?
56. Did you have any difficulty answering this question for any reason? If so, why?
57. In your own words, what does it mean to “file a claim”?
58. If Q13 = Yes: If you feel comfortable doing so, could you describe the situation that you
were referring to when you answered Yes to this question? Probe to assess whether the
situation warranted a “Yes” answer. If participant expresses any reluctance to discuss
the situation, move on to next question.
a. The next question on the survey asked whether you think the process was fair,
and your answer was [repeat answer]. Could you explain why you gave that
answer?
Question 14
The next question on the survey was [reread Question 14]. Your answer to this question was
[repeat answer].
7

59. Was there anything that you found confusing about this question?
60. Did you have any difficulty answering this question for any reason? If so, why?
61. If Q14 = Yes: If you feel comfortable doing so, could you describe the situation that you
were referring to when you answered Yes to this question? Probe to assess whether the
situation warranted a “Yes” answer. If participant expresses any reluctance to discuss
the situation, move on to next question.
a. The next question on the survey asked whether you think the process was fair,
and your answer was [repeat answer]. Could you explain why you gave that
answer?
Note to Interviewer: It is not necessary to ask about Q15 (gender).
Question 16
The next question on the survey was [reread Question 16]. Your answer to this question was
[repeat verbatim answer, rather than code].
62. Did you have any difficulty answering this question for any reason? If so, why?
63. If interviewer found answer difficult to code for any reason, this should be noted.

Question 17
The next question on the survey was [reread Question 17]. Your answer to this question was
[repeat answer].
64. Did you have any difficulty answering this question for any reason? If so, why?
Question 18
The next question on the survey was [reread Question 18]. Your answer to this question was
[repeat answer].
65. Did you have any difficulty answering this question for any reason? If so, why?
66. If interviewer found answer difficult to code for any reason, this should be noted.
Note to Interviewer: It is not necessary to ask about Q19 (age).
8

Question 20
The next question on the survey was [reread Question 20].
67. Was there anything that you found confusing about this question?
68. Did you have any difficulty answering this question for any reason? If so, why?

File Type	application/pdf
Author	Amy Newell
File Modified	2014-01-23
File Created	2014-01-23