2014_09_03_CLEAN REV Part A vs3

CONSUMER FINANCIAL PROTECTION BUREAU
INFORMATION COLLECTION REQUEST – SUPPORTING STATEMENT A
TELEPHONE SURVEY EXPLORING CONSUMER AWARENESS OF AND
PERCEPTIONS REGARDING DISPUTE RESOLUTION PROVISIONS IN
CREDIT CARD AGREEMENTS
(OMB CONTROL NUMBER: 3170-XXXX)
The Consumer Financial Protection Bureau (the “CFPB”) seeks approval from the
Office of Management and Budget (“OMB”) to conduct a national telephone survey
of 1,000 credit card holders as part of its study of mandatory pre-dispute arbitration
agreements, which is required under Section 1028(a) of the Dodd-Frank Wall Street
Reform and Consumer Protection Act, Public Law No. 111-203, Title XIV (the “Dodd
Frank Act”).
The survey will explore (a) the role of dispute resolution provisions in consumer
card acquisition decisions and (b) consumers’ default assumptions (meaning
consumers’ awareness, understanding, or knowledge without supplementation from
external sources) regarding their dispute resolution rights vis-à-vis their credit card
issuers, including their awareness of their ability, where applicable, to opt-out of
mandatory pre-dispute arbitration agreements. 1 The survey will not gather data
regarding respondents’ post-fact satisfaction with arbitration or litigation
proceedings, given the difficulty in finding consumers that have had personal
experience with both forums.
A.

JUSTIFICATION

1.

Circumstances Necessitating Collection of Information

The CFPB seeks approval from OMB to conduct a national telephone survey of
1,000 credit card holders 2 exploring consumer awareness of and perceptions
regarding dispute resolution provisions in credit card agreements. The study will be
conducted in English and Spanish and include both residential landlines and
cellphones. 3 Participation will be voluntary.
1 Part B of this Supporting Statement provides a more detailed discussion of the survey’s key
estimates, including the required precision. Similarly, Appendix B describes how each proposed
survey question relates to the key survey estimates.
2 The survey will focus on general purpose “bank” credit cards (including co-branded cards) and
exclude proprietary credit cards.

Bilingual surveys and the use of both landlines and cellphones are intended to mitigate potential
biases due to non-response and noncoverage of the sampling frame.
3

1

The survey will help the CFPB meet its obligations under Section 1028(a) of the
Dodd Frank Act to conduct a study of, and provide a report to Congress concerning,
the use of agreements providing for arbitration of any future dispute between
entities offering or providing certain consumer financial products or services
(“covered persons”), on the one hand, and consumers, on the other hand, in
connection with the offering or providing of consumer financial products or services.
The CFPB is the sole proposed potential user, but data from the survey will be
publicly available in at least aggregate form. The CFPB is contemplating release of
survey microdata, after taking appropriate disclosure protections, such as collapsing
response cells and not returning specific variables (such as the identity of
respondents’ credit card providers).
Although the CFPB’s overall study of mandatory arbitration provisions will extend
to multiple product markets, the proposed telephone consumer survey will be
limited to credit cards. This is because credit cards offer strong penetration amongst
consumers across the nation. (In 2009, the U.S. Census estimated that 160 million
Americans would own credit cards by 2012.) Also, limiting the study to credit cards
allows the CFPB to verify the accuracy of consumers’ default assumptions regarding
their dispute resolution rights, by consulting the agreements of the entities
identified by the consumers as their credit cards’ issuing banks.
The CFPB proposes that the survey sample will be selected through random-digit
dialing of residential and cellular telephones, rather than an Internet-based survey.
Certain of the survey’s primary estimates relate to consumers’ awareness of their
contract rights with their credit card issuers. Unlike an Internet-based or mailed
survey, a telephone call requires real-time interaction, which minimizes the
possibility that respondents will supplement their knowledge before responding to
survey questions by searching the Internet or reviewing their credit card
agreements.
The key estimates to be derived from the survey are:
•

The percentage of credit card users that compare multiple credit cards (e.g.,
comparison shop) before acquiring a credit card;

•

The percentage of credit card users that volunteer in response to an openended query that dispute resolution provisions 4 were a consideration when
acquiring a credit card;

This submission uses the term “dispute resolution provisions” to reference contractual provisions
that address formal dispute resolution options. This would include agreements providing for
4

2

•

The percentage of credit card users that state that dispute resolution
provisions were a consideration when acquiring a credit card after specifically
being queried about dispute resolution provisions and seven other factors;

•

Relative tiers of importance of the credit card features that consumers
identify as being relevant to their card acquisition decisions, drawn from the
aforementioned open and closed-set queries;

•

The percentage of credit card users that affirmatively volunteer in response
to an open-ended query that they would consider bringing formal claims
against their credit card issuers (i.e., filing a complaint in small claims court;
consulting a lawyer regarding state or federal litigation; or filing a notice of
arbitration) after exhausting informal dispute resolution options;

•

Consumers’ default assumptions regarding their dispute resolution rights
(i.e., do they believe they have a right to have claims adjudicated by a judge
or jury when they may, in fact, be subject to a mandatory arbitration
provision);

•

The accuracy of consumers’ default assumptions regarding their dispute
resolution rights;

•

Consumers’ awareness of mandatory predispute arbitration as a means of
resolving disputes. 5

The CFPB proposes using probability sampling to obtain approximately 1,500
respondents aged 18 or older (1,000 credit card holders and an estimated 500 noncredit card holders). 6 Based on an expected response rate of 15-20%, the CFPB
proposes screening roughly 10,500 individuals with credit cards in order to reach
the 1,500 respondent sample. The sample will be selected to provide broad
geographic coverage of the United States. The observable demographic
arbitration of future disputes, often referred to as “mandatory pre-dispute arbitration agreements.”
The term does not encompass informal dispute resolution options, such as customer service
telephone help lines or covered persons’ internal complaint-reporting functions.
5 Part B of this Supporting Statement provides a more detailed discussion of the survey’s key
estimates, including the required precision. Similarly, Appendix B describes how each proposed
survey question relates to the key survey estimates.
6 The U.S. Census estimates that there were approximately 243 million Americans aged 18 or over in
2013. See http://quickfacts.census.gov/qfd/states/00000.html (last visited Aug. 22, 2014). Assuming
that 160 million American hold credit cards, approximately one out of every three respondents would
not own a credit card.

3

characteristics and information about measured differences across respondents will
be used to construct sample weights so that estimates from the survey are
representative of the overall U.S. population. The CFPB anticipates an estimation
error of +/- 3.1% for most of the survey questions.
The CFPB has retained ICF International to assist with administration of the
telephone survey, should OMB approve this proposal. As described herein, ICF
International assisted with this submission to OMB, including conducting consumer
focus group testing and user testing reflected in OMB Control No. 3170-024.
This proposal reflects extensive revisions compiling learnings from the engagement
of ICF International; review of nearly two-dozen written comments; two
consultation rounds with interested parties representing both financial service
organizations and consumer groups; focus group testing with two sets of consumer
panels; and extensive user testing.
2.

Use of the Information

The results of the survey will ultimately be communicated to the public as part of
the CFPB’s Dodd Frank Section 1028(a) report to Congress, to ensure an
equivalent, timely release.
In that regard, Section 1028(a) of the Dodd Frank Act states that the CFPB “shall
conduct a study of, and shall provide a report to Congress concerning, the use of
agreements providing for arbitration of any future dispute between covered persons
and consumers in connection with the offering or providing of consumer financial
products or services.”
Section 1028(b) establishes further CFPB authority, stating that the Bureau “by
regulation, may prohibit or impose conditions or limitations on the use of an
agreement between a covered person and a consumer for a consumer financial
product or service providing for arbitration of any future dispute between the
parties, if the Bureau finds that such a prohibition or imposition of conditions or
limitations is in the public interest and for the protection of consumers.” The section
specifies that “[t]he findings in such rule shall be consistent with the study
conducted under subsection (a).”
The CFPB released preliminary results of its study of arbitration provisions in
December 2013. 7 The 2013 preliminary results described the CFPB’s preliminary
work regarding the incidence of arbitration provisions, focusing on the credit card,
checking account, and general purpose reloadable pre-paid card markets. The 2013
7The

Preliminary Results are available here:
http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminary-results.pdf.

4

preliminary results also provided a typology of disputes filed with the American
Arbitration Association from the years 2010-2012, relating to the credit card,
checking account, payday loan, and general purpose reloadable prepaid card
markets. Finally, the 2013 preliminary results explored the use of small claims
courts by consumers and companies that provide consumer financial products and
services. The 2013 preliminary results, subject to updates or other revisions, will be
included in the final 1028(a) report.
In addition, as described in Section 6 of the 2013 preliminary results the CFPB has
a number of other phases of work that are underway or that are under
consideration for inclusion in the required 1028(a) report. They include work such
as:
•
The proposed telephone survey;
•
Exploring the incidence of arbitration clauses and outlining their key
features;
•
Reviewing arbitration case data concerning disputes relating to consumer
products and services;
•
Reviewing litigation case data concerning the same;
•
Attempting to assess the possible impact of arbitration clauses on the price of
consumer financial products; and
•
Examining the interrelationship between public enforcement and private
aggregate enforcement.
For its part, the proposed telephone survey will provide information about (a) the
role of dispute resolution provisions in consumer card acquisition decisions and
(b) consumers’ default assumptions (meaning consumers’ awareness,
understanding, or knowledge without supplementation from external sources)
regarding their dispute resolution rights vis-à-vis their credit card issuers,
including their awareness of their ability, where applicable, to opt out of mandatory
pre-dispute arbitration agreements. This will be used to put other information
gleaned from the broader arbitration study in context.
To provide an illustration of this interaction, the 2013 preliminary results showed
that approximately half of credit card debt is associated with credit card issuers
that utilize pre-dispute arbitration clauses. The results of the proposed telephone
survey could help explain whether consumers deliberately choose cards with or
without the clauses – or if consumers are even aware of the distinction. To provide
another example, when the Bureau interprets data it has separately gathered
regarding the prevalence of contract provisions that allow consumers to opt out of
mandatory arbitration provisions, the proposed telephone survey will help the
Bureau better understand if a consumer’s failure to opt out reflects a conscious
decision or, instead, reflects a lack of awareness of the option (or what the option
means).
5

Subsequent use by the CFPB of its 1028(a) report will consider all of its different
focus areas, including when the Bureau considers whether regulations relating to
pre-dispute arbitration provisions would be appropriate pursuant to Dodd Frank
Section 1028(b). The proposed telephone survey will be one of multiple empirical
inquiries within the 1028(a) report that will be read together. Thus, the proposed
telephone survey and its findings are part of a larger, broader analysis informed by
the entirety of the CFPB’s 1028(a) report, including the 2013 preliminary results. 8
3.

Use of Information Technology

ICF International will collect data electronically through the use of computerassisted telephone interviewing software. Such software will help reduce errors by:
•
•
•
•
•

Providing uniform question sequencing;
Automatically skipping questions, where appropriate, based on prior answers
to questions;
Randomizing how certain questions, or groups of questions, are presented;
Ensuring that some questions cannot be skipped; and
Rejecting invalid responses or data entries.

Additionally, the contractor may collect data on the length of interviews, measures
of the effectiveness of specific interviewers, and unit and item non-response rates.
This type of information can be used to improve the data collection process.
4.

Efforts to Identify Duplication

The proposed consumer survey will not duplicate empirical research that the CFPB
has identified to date. In that regard, the CFPB has been unable to identify prior
empirical studies exploring the role of dispute resolution provisions in consumer
credit card acquisition decisions or consumer default assumptions regarding their
dispute resolution rights vis-à-vis their credit card issuers.
Regarding card acquisition, the closest comparison the CFPB has identified is a
September 2011 report by Mercator Advisory Group, which explored consumer
credit card acquisition decisions, but did not report on dispute resolution provisions

Accordingly, the CFPB’s 1028(a) report will, among other topics, provide a detailed discussion of the
survey’s findings and methods. In addressing the survey, the CFPB will endeavor to provide survey
documentation that will allow the public to understand how to properly analyze data from the
survey, as well as evaluate and potentially recreate each key estimate. The report will also provide
an evaluation of the quality of the data and areas for potential improvement.

8

6

as a potential factor. 9 Mercator fielded an Internet-based consumer survey of
approximately 1,000 adults. Mercator’s survey asked consumers to identify “One
Main Reason for Selecting Credit Card at Last Application.” “Dispute resolution
provisions/rights” was not included in the fourteen reported measures. “Some Other
Reason,” which would likely subsume such concerns, was the third-largest category
for applications made within the last year, trailing only rewards and “no annual
fee” (“Some Other Reason” tied for second-largest category for selection criteria
relating to applications prior than a year old). Mercator’s survey also asked
consumers to identify the “Most Important Reason for Selecting Credit Card To Use
The Most.” Again, “dispute resolution provisions/rights” was not included in the
twelve reported measures. But again, “Some Other Reason” was one of the largest
categories – fourth for applications within the year, third for applications from prior
years.
Theoretically, empirical work regarding consumer awareness of arbitration
provisions could inform the proposed survey’s exploration of how those provisions
factor into consumers’ credit card acquisition decisions, as well as consumers’
default assumptions. The arbitration-awareness research identified to date by the
CFPB, however, does not extend to the issues the CFPB seeks to estimate. The
majority of prior awareness research focuses on whether consumers read contracts
generally. Such research largely has not specifically focused on consumer awareness
of arbitration provisions, which are frequently formatted differently than the rest of
the consumer contract. In that regard, courts that have enforced arbitration
agreements frequently observe that arbitration provisions may be subject to
different formatting (e.g., typeface, font size, bold, or underlining) than other
contractual provisions. 10

9 U.S. Credit Cardholders: Waiting for a Rebound, Mercator Advisory Group (Sept. 2011)
(CustomerMonitor Survey Series, Insight summary Report Vol. 3, Report 1).
10 See, e.g., Murea v. Pulte Group, Inc., 2014-Ohio-398 (Ct. App. Ohio Feb. 6, 2014) (“The arbitration
provisions in both agreements were clearly marked in capital letters. The arbitration clause in the
purchase agreement as conspicuously written in bold print, and the signature line of the agreement
expressly warned Murea to ‘make sure that all provisions are read and understood before signing.’”).
Forest Hill Nursing Ctr., Inc. v. McFarlan, 995 So. 2d 775, 785 (Miss. Ct. App. 2008) (noting bold, all
capital letters); Lorene Park, Be loud, clear, and fair in arbitration provisions or be prepared to
litigate, Released Aug. 7, 2012 available at
http://www.employmentlawdaily.com/index.php/2012/08/07/be-loud-clear-and-fair-in-arbitrationprovisions-or-be-prepared-to-litigate (advising employers feature arbitration agreements in a
separate document, or at least in its own paragraph set off with spacing, and to “[u]se a 12-point font
or larger; and use a bold, underlined, and capitalized heading that includes the words “Arbitration
Agreement.”) (emphasis in original).

7

The awareness studies that do focus on arbitration provisions generally do not focus
on consumer financial products and services, much less credit cards. 11 Further,
many of these studies were too limited in size or geographic diversity to draw
conclusions about the country as a whole. 12 For example:
•

In 2009, the Employee Rights Advocacy Institute for Law & Policy and Public
Citizen conducted a nationwide survey of 800 voters, which explored the
respondents’ awareness of the existence of arbitration provisions in their
employment agreements and certain agreements for goods and services. 13 In
particular, the study polled respondents about their expectations regarding
their ability to sue their employers in court as well as whether the
respondents recalled executing mandatory arbitration provisions in terms of
employment or terms of agreements for goods and services. The study,
however, did not discuss the use of arbitration provisions in connection with
the offering or providing of consumer financial products or services.

•

In contrast, the Pew Charitable Trusts focused on checking accounts in its
2012 survey of 603 consumers. 14 The study, however, did not describe data or
survey questions relating to the extent of consumer awareness of arbitration
provisions. 15 It similarly did not discuss how consumers weighed dispute

11 See, e.g., Yannis Bakos et al., Does Anyone Read the Fine Print? Testing a Law and Economics
Approach to Standard Form Contracts 1-26, N.Y.U. Law & Econ. Research Paper Series, Working

Paper No. 09-04, 2009 (studying on-line browsing habits of 45,091 households relating to online
software merchants).

See, e.g., Amy Schmitz, Access to Consumer Remedies in the Squeaky Wheel System, 39
Pepperdine L. Rev. 279-366 (2012) (describing panel of 306 Colorado consumers); Shmuel I. Becher &
Esther Unger-Aviram, The Law of Standard Form Contracts: Misguided Intuitions and Suggestions
for Reconstruction, 8 DePaul Bus. & Com. L.J. 199 (2010) (surveying 147 consumers); Debra
Pogrund Stark & Jessica M. Choplin, A License to Deceive: Enforcing Contractual Myths Despite
Consumer Realities, 5 N.Y.U. J.L. & Bus. 617, 619-23 (2000) (surveying 91 students fulfilling a
course requirement; 106 approached in a public location; and 101 law students).
12

See, e.g., National Study of Public Attitudes on Forced Arbitration: Findings from a Survey of 800
Likely 2010 Voters Nationwide. The Employee Rights Advocacy Institute for Law & Policy and
Public Citizen. Released 04/09 (research funded by the Public Welfare Foundation) available at
13

http://www.citizen.org/documents/lake-research-national-study-of-public-attitudes-forcedarbitration.pdf.
14

Banking on Arbitration: Big Banks, Consumers, and Checking Account Dispute Resolution. The

Pew Charitable Trusts. Released 11/26/12 available at
http://www.pewstates.org/uploadedFiles/PCS_Assets/2012/Pew_arbitration_report.pdf.

The Pew study references a lack of consumer awareness regarding dispute resolution provisions,
but stops short of discussing the data underlying any such inquiry. (“Pew found that many
consumers are unaware that account agreements restrict their options if they have a dispute with
15

8

resolution provisions when evaluating financial products. Instead, the study
focused on (a) the prevalence of arbitration provisions in the account
agreements of the largest 100 retail banks and credit unions by deposit
volume and (b) consumers’ preferences after they were presented with
possible features of arbitration provisions or contrasting statements about
arbitration provisions.
•

In 2003, the Institute for Advanced Dispute Resolution surveyed 1,036
Americans, seeking information about consumer awareness and preferences
regarding arbitration provisions. The survey, however, focused on
respondents’ perspectives regarding voluntary arbitration proceedings, rather
than arbitration proceedings that were required under pre-dispute contracts.
And it has not been updated in ten years. There is significant controversy
regarding whether the Institute for Advanced Dispute Resolution was or was
not an arbitration advocacy organization. 16

•

And in 2008, the Center for Responsible Lending commissioned a study from
Macro International as part of its regular CARAVAN® survey. The survey
was conducted among a national probability sample of 1,007 adults,
concluding that 68% of consumers that financed cars from dealerships were
unaware if their contract had a “forced arbitration” clause after the clause
was described to them. The study, however, did not list the questions used in
the survey. Further, while the survey featured over a thousand respondents,
only 27% of the survey population used a loan at a dealership to purchase a
car or truck. Accordingly, the sample size for the arbitration awareness
questions was only 268 respondents. 17

their financial institution.”) It similarly does not detail particular questions relating to such an
inquiry.
Roper ASW, 2003 Legal Dispute Study: Institute for Advanced Dispute Resolution (2003) cited in
Public Citizen, The Arbitration Debate Trap, at 20-22 (2008) (discussing controversy) available at
http://www.citizen.org/documents/ArbitrationDebateTrap(Final).pdf. Also, the study focused on
arbitration provisions generally, as opposed to the financial sector.
16

Joshua M. Frank, Center for Responsible Lending, Stacked Deck: A Statistical Analysis of Forced
Arbitration (2009) available at http://www.responsiblelending.org/credit-cards/research-

17

analysis/stacked_deck.pdf.

Further, the Center for Responsible Lending could not verify whether consumers’ recollections about
the presence of arbitration agreements was correct. They found that 17% of respondents said that
there was never an arbitration clause in their auto loan agreement. The Center for Responsible
Lending relied on “[c]onversations with experts familiar with these contracts suggest that many of
the people who thought there was no arbitration in their contract likely were mistaken.” In so doing,
the Center for Responsible Lending assumes that the full 17% was mistaken (an assumption that
was used in certain regression analyses). In contrast, the CFPB will be able to look to relevant bank

9

While there is a significant volume of other arbitration-focused empirical work, it
does not duplicate the proposed survey’s estimates: how dispute resolution rights
factor into consumers’ credit card acquisition decisions and consumers’ default
assumptions regarding their ability to sue the credit card issuer in a court of law.
For example, other empirical studies, including the CFPB’s own release of
preliminary results relating to its overall study of arbitration provisions, have
studied the prevalence of arbitration provisions in various types of markets. 18 Other
studies have explored arbitration outcomes, measuring consumer satisfaction after
a dispute 19 or the likelihood that consumers will achieve success in an arbitration
proceeding, taking into consideration both win rates and award sizes. 20
customer agreements, available independent of the respondents, to verify the accuracy of consumers’
perceptions.
18 See, e.g., Consumer Financial Protection Bureau, Arbitration Study: Preliminary Results (Dec.
2013) available at: http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminaryresults.pdf; Checks and Balances 2014 Update. The Pew Charitable Trusts. Released 04/09/2014
available at http://www.pewstates.org/uploadedFiles/PCS/ContentLevel_Pages/Reports/2014/Checks-and-Balances-Report-2014.pdf.; Christopher R. Drahozal & Peter
B. Rutledge, “Contract and Choice,” 2013 B.Y.U. L. Rev. 1 (2013) (studying the prevalence of
arbitration agreements in credit card agreements); Michael L. Rustad, An Empirical Study of
Predispute Mandatory Arbitration Clauses in Social Media Terms of Service Agreements, 34 U. Ark.
Little Rock L. Rev. 643 (2012) available at http://ualr.edu/lawreview/files/2013/01/Rustad-Normal.pdf
(studying the prevalence of arbitration agreements in social media service agreements and privacy
terms); Christopher R. Drahozal & Quentin R. Wittrock, Is There a Flight from Arbitration?, 37
Hofstra L. Rev. 71, 90-94 (2008) (studying the prevalence of arbitration agreements franchise
agreements); Theodore Eisenberg, Geoffrey P. Miller & Emily Sherwin, Arbitration’s Summer

Soldiers: An Empirical Study of Arbitration Clauses in Consumer and Nonconsumer Contracts
(2008), available at http://lsr.nellco.org/nyu_lewp/136; Florencia Marotta-Wurgler, “Unfair” Dispute
Resolution Clauses: Much Ado About Nothing?, in Boilerplate: The Foundation of Market Contracts

45, 47-48 (Omri Ben-Shahar ed., 2007) (studying the prevalence of arbitration agreements in enduser software license agreements); Linda J. Demaine & Deborah R. Hensler, “Volunteering” to
Arbitration Through Predispute Arbitration Clauses: The Average Consumer Experience, 67 Law &
Contemp. Probs. 55, 73-74 (2004) (studying the prevalence of arbitration agreements in consumer
contracts, generally); Elizabeth Rolph, Erik Moller & John E. Rolph, Arbitration Agreements in
Health Care: Myths and Reality, 60 Law & Contemp. Probs. 153, 171 (1997) (studying the
prevalence of arbitration agreements relating to California physicians and hospitals).
19 Arbitration: Simpler, Cheaper, and Faster than Litigation, Harris Interactive Survey, conducted
for U.S. Chamber Institute for Legal Reform (Apr. 2005), available at http:
http://www.adrforum.com/rcontrol/documents/researchstudiesandstatistics/2005harrispoll.pdf (19%
of survey respondents had entered arbitration proceedings because they were required by contract;
the other 81% voluntarily opted into the arbitration proceedings); Outcomes in Arbitration: An
Empirical Study of Consumer Lending Cases, Ernst & Young (2004) (studying outcomes in 226 by
the National Arbitration Forum in lending-related consumer-initiated cases).

Other studies have looked to consumer satisfaction in specific types of arbitration proceedings. See,
e.g., Gary Tidwell, Kevin Foster & Michael Hummel, Party Evaluation of Arbitrators: An Analysis of
Data Collected from NASD Regulation Arbitrations 3 (1999) cited in Michael Perino, Report to the

10

The CFPB will continue to monitor empirical research and related work by Federal
regulatory agencies and other researchers to ensure that the CFPB’s research
techniques reflect the current knowledge and best practices. 21

Securities Exchange Commission Regarding Arbitrator Conflict Disclosure Requirements in NASD
and NYSE Securities Arbitrations (2002), available at http://www.sec.gov/pdf/arbconflict.pdf

(describing substantial level of satisfaction amongst respondents, but cautioning that findings were
limited by a response rate of just 10-20%, which may reflect selection bias problems).

See, e.g., Alexander J.S. Colvin, An Empirical Study of Employment Arbitration: Case Outcomes
and Processes, 8 J. Empirical Legal Stud. 1 (2011); Christopher R. Drahozal & Samantha Zyontz,
Creditor Claims in Arbitration and in Court, 7 Hastings Bus. L.J.77 (2011) (comparing the outcomes
20

of debt collection cases brought by creditors in arbitration as compared to outcomes of debt collection
cases brought in court); Christopher R. Drahozal & Samantha Zyontz, An Empirical Study of AAA
Consumer Arbitration, 25 Ohio St. J. on Disp. Resol. 843 (2010) (summarizing prior empirical work
and discussing the results of a study of 301 consumer arbitrations administered by the American
Arbitration Association in 2007); Peter B. Rutledge, Whither Arbitration?, 6 Geo. J.L. & Pub. Pol’y
549 (2008) (summarizing prior empirical work); Alexander J.S. Colvin, Empirical Research on
Employment Arbitration: Clarity Amidst the Sound and Fury?, 11 Employee Rts. & Employment
Pol’y J. 405, 418 (2007); Mark Fellows, The Same Result as in Court, More Efficiently: Comparing
Arbitration and Court Litigation Outcomes, 14 Metropolitan Corp. Counsel 32 (2006) (study
conducted by legal counsel for National Arbitration Forum, regarding 2003-2004 National
Arbitration Forum consumer arbitration awards from California) available at
http://www.metrocorpcounsel.com/articles/6988/same-result-court-more-efficiently-comparingarbitration-and-court-litigation-outcomes; Lisa B. Bingham & Shimon Sarraf, Employment

Arbitration Before and After the Due Process Protocol for Mediation and Arbitration of Statutory
Disputes Arising Out of Employment: Preliminary Evidence that Self-Regulation Makes a
Difference, in Alternative Dispute Resolution in the Employment Arena: Proceedings of the New

York University 53rd Annual Conference on Labor 303, 320-28 (Samuel Estreicher & David Sherwyn
eds. 2004); Michael Delikat & Morris Kleiner, Comparing Litigation and Arbitration of Employment
Disputes: Do Claimants Better Vindicate Their Rights in Litigation?, 6 A.B.A. Litig. Sec. Conflict
Mgmt. 1, 10 (2003); Theodore Eisenberg & Elizabeth Hill, Arbitration and Litigation of Employment
Claims: An Empirical Comparison, 58 Disp. Resol. J. 44, 48 (Nov. 2003-Jan. 2004); Elizabeth Hill,

Due Process at Low Cost: An Empirical Study of Employment Arbitration Under the Auspices of the
American Arbitration Association, 18 Ohio St. J. Disp. Resol. 777, 806 (2003).

21 The Information Collection Request Search Tool maintained by the OMB’s Office of Information
and Regulatory Affairs returned only one document in response to an April 11, 2014 search for
“arbitration survey.” Available at https://www.reginfo.gov. That document, OMB Control No. 3170024, was a reference to the CFPB’s submission to OMB regarding the use of focus groups and
usability testing relating to this proposed telephone survey.

11

5.

Efforts to Minimize Burdens on Small Entities

The data collection will not burden small entities because the survey will only
collect information from individuals. 22
6.

Consequences of Less Frequent Collection and Obstacles to Burden
Reduction

Each surveyed individual will only participate once.
If the survey was not implemented, the CFPB would be limited in its ability to
provide an analysis of how dispute resolution provisions are used in connection with
the offering or providing of financial products or services to consumers. For
example, to the extent that CFPB intends to study the consumer experience with
arbitration, first-order questions would include whether consumers even know
whether they are in an arbitration environment and, if so, whether they made a
conscious decision to be subject to arbitration requirements.
7.

Circumstances Requiring Special Information Collection

No special circumstances require the collection to be conducted in a manner
inconsistent with the guidelines set forth in 5 C.F.R. 1320.5(d)(2).
8.

Consultation with Individuals Outside of the Agency on Availability of Data,
Frequency of Data, Frequency of Collection, Clarity of Instructions and
Forms, and Data Elements

On April 27, 2012, the CFPB made a “Request for Information Regarding Scope,
Methods, and Data Sources for Conducting Study of Pre-Dispute Arbitration
Agreements.” (77 F.R. 82.) The CFPB received over sixty comment letters in
response.
Certain commentators suggested that the CFPB focus on consumer satisfaction
after resolving a dispute in arbitration. The CFPB, however, does not propose that
the survey include such an exploration. Such research would ideally be conducted
with consumers that had personal experience with both litigation and arbitration,
but as many commentators highlighted, this would be a very small sample set. 23
(As the U.S. Chamber of Commerce later noted in a comment letter to OMB relating
to this proposal, “[v]anishingly few respondents will have sufficient background
22 Given that the finances of small businesses are often intertwined with their owners, the survey
instrument will instruct respondents to exclude cards used primarily for business purposes when
responding to survey questions about credit card use.

12

information to assess the comparative merits of arbitration and litigation, because
it is highly unlikely that more than a very few individual respondents will have had
the experience of pursuing similar disputes in each forum. . . . .And the number of
respondents with experience concerning both arbitration and litigation is likely to
be extremely small – indeed , it is likely to be zero.”) (emphasis in original). 24
Commentators also made a number of other suggestions, including that the CFPB
assess consumer awareness and understanding regarding dispute resolution
provisions. 25 By probing consumers’ default assumptions regarding their dispute
resolution rights, the proposed survey would provide data relevant to consumer
awareness and understanding.
On June 7, 2013, the CFPB published a Federal Register Notice which notified the
public of the CFPB’s intent to collect this survey, and provided a 60-day comment
period (Vol. 78, No. 110, Page 34352). The Federal Register Notice also included a
draft proposed survey (the “Initial Questionnaire”), which included more questions
than the current proposal, exploring additional estimates than the version included
in this submission such as estimates of consumer valuation and perception of
transaction costs relating to dispute resolution alternatives. The draft was included
to help facilitate discussion and frame the ultimate survey, but made clear that the
CFPB expected to revise the questionnaire “based on the experience and knowledge
of the contractor, comments from the public notice process, cognitive testing, and
pretesting of the survey questions.” Further, in accordance with 5 CFR
1320.5(a)(1)(iv), the Bureau has published a notice in the Federal Register allowing
23 From 2010 to 2012, there were approximately 344 arbitration filings a year with the American
Arbitration Association regarding credit card issues. Consumer Financial Protection Bureau,
Arbitration Study: Preliminary Results (Dec. 2013) available at:
http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminary-results.pdf. Not all of
the filings proceeded to adjudication by an arbitrator.
24

(U.S. Chamber of Commerce, Aug. 6, 2013, pp. 10, 16.)

25 Professor Alexander J.S. Colvin, ILR School, Cornell University (also suggesting that the CFPB
assess satisfaction); June 22, 2012 letter from American Financial Services Association (among
numerous suggestions, suggesting that the CFPB examine how well-informed consumers are
regarding the benefits of arbitration programs); June 23, 2012 letter from Jill I. Gross, Pace
University School of Law (“For the consumer-level study, researchers should survey consumers’
perceptions of fairness of arbitration clauses of sample companies as well as perceptions of fairness
of the arbitration process with those companies . . . . “); June 22, 2012 letter from MFY Legal
Services (among other suggestions, “[t]he CFPB should survey consumer knowledge about
arbitration clauses and how arbitration works. Do consumers understand what it means to sign an
arbitration clause?”); June 23, 2012 letter from National Association of Consumer Advocates and The
National Consumer Law Center (“The CFPB should also study why consumers may be passing on
available arbitration options. For example, the CFPB should examine how well-informed consumers
are as to the benefits of a fairly designed and administered arbitration program.”).

13

the public 30 days to comment on the submission of this information collection
request to the Office of Management and Budget.

Consultation with Individuals Outside of the Agency, July 2013
On July 8, midway through the 60-day comment period, the CFPB hosted two
roundtables seeking consultation with individuals outside of the agency regarding
the proposed survey. Meeting participants included leading business federations,
banking trade associations, and non-profit consumer advocacy organizations.
In each session, the CFPB described the estimates it sought to obtain from the
proposed survey and responded to questions from the participants. Among other
suggestions, one participant suggested that the CFPB include a “closed ended”
question to probe consumer knowledge of arbitration provisions, rather than an
open-ended question (which is reflected in the revised proposed script in Appendix
A). Many commentators raised concerns about the length and complexity of the
proposed survey; questions seeking answers to hypothetical scenarios were a
particular focus of these stated concerns. Certain commentators also questioned
whether consumers had sufficient knowledge of dispute resolution provisions to
either (a) estimate the amount of money they would spend to obtain the dispute
resolution rights that they preferred; or (b) estimate how much they would need to
be harmed before initiating an arbitration or litigation or participate in a class
action litigation. As discussed below, the questions relating to those issues –
Questions 8 and 11 in the Initial Questionnaire – have been stricken from the
proposed survey, which has been shortened and otherwise simplified.

Comments Received in Response to 60-Day Notice
The CFPB received approximately twenty-two responses regarding its 60-Day
Notice.
Commentators included:
•
•

•

The American Bankers Association (together with Consumer Banker
Association and the Financial Services Roundtable, the “ABA
Commentators”);
Americans for Financial Reform; Center for Justice and Democracy;
Consumer Action; Consumer Federation of America; Consumers for Auto
Reliability and Safety; Citizen Works; D.C. Consumer Rights Coalition;
Homeowners Against Deficient Dwellings; National Association of Consumer
Advocates; National Consumer Law Center (on behalf of its low income
clients); National Consumers League; Public Citizen; U.S. PIRG; and
Workplace Fairness;
the American Financial Services Association (“AFSA”);
14

•
•
•
•
•
•
•
•
•
•
•

the Credit Union National Association;
Thomas Durkin;
Jennifer Lee;
Alessandro Machi (three comments);
the Michigan Credit Union League and Affiliates;
the Missouri Credit Union Association;
National Association of Consumer Advocates and the National Consumer
Law Center, on behalf of its low income clients;
Kenneth Sheehan;
Donna Shestowsky and Nancy Welsh (two sets of joint comments);
Public Justice; and
The U.S. Chamber of Commerce.

Certain commentators (e.g., the ABA Commentators, AFSA, and the U.S. Chamber
of Commerce) expressed that the Bureau should not proceed with the survey. Many
of their concerns, however, related to concerns about the Bureau’s overall study of
mandatory arbitration studies (the proposed survey is just one of many different
parts of the overall study).
The ABA Commentators, for example, stated that “the Bureau’s information
collection should be focused on a comprehensive review of the disputes that may
arise about consumer financial services and how they can be resolved.” (ABA
Commentators, Aug. 6, 2013). The United States Chamber of Commerce similarly
asserted that the Bureau should study issues outside of the survey, such as the
nature of consumer claims, outcomes in arbitration versus litigation, class
proceedings (which they also framed as “trade-offs”), cost burdens and accessibility
of arbitration and litigation, and post-dispute arbitration. (U.S. Chamber of
Commerce, Aug. 6, 2013, pp. 3-4).
The CFPB has made clear that it is addressing several, if not all, of these other
lines of inquiry in its overall study of mandatory arbitration provisions. In that
regard, the Bureau released preliminary results relating to its study of mandatory
arbitration provisions in December 2013. 26 The Bureau’s preliminary results
included, among other things, a typology of consumer disputes filed with the
primary arbitration services provider for consumer financial products, the American
Arbitration Association, relating to consumer credit card, checking account, and
payday loan issues. The Bureau reported several data points, including the number
of filings relating to each type of product, the legal claims that they cover, claim
amounts, initial fee assessments, whether the parties were represented by counsel,
Consumer Financial Protection Bureau, Arbitration Study: Preliminary Results (Dec. 2013)
available at: http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminary26

results.pdf.

15

and the demographic distributions of the zip codes relating to consumers in those
arbitrations. The Bureau explained that it intends to address the “back-end” of
formal disputes in subsequent work: what happens to claims, how long proceedings
take to resolve, and at what cost. The Bureau highlighted that its future work will
include a review of filings in certain federal courts and, where possible, selected
state courts. It also explained that it intends to study consumer class actions,
including attempting to assess consumer benefits and transaction costs in consumer
class actions involving consumer financial services. And, the Bureau stated that it
will attempt to analyze whether consumer financial class actions exert improper
pressure on defendants to settle meritless claims. Accordingly, many of the concerns
raised in commentators’ responses relating to the proposed survey have already
been addressed by the release of the preliminary results or will be addressed with
the future work described therein.
Other commentators asked that the Bureau be mindful of increasing regulatory
burdens to credit unions. For example, the Credit Union National Association
expressed that it had no “formal objections to a survey by the CFPB,” but asked that
the Bureau be sensitive to the burden to credit unions posed by the inquiry. (Credit
Union National Association, Aug. 2, 2013.) Likewise, the Missouri Credit Union
Association stated that it “recommends that the CFPB seek to determine
consumers’ understanding of dispute resolution, particularly arbitration, while at
the same time requesting that the CFPB be cognizant of tremendous regulatory
burdens that credit unions are already facing.” (Missouri Credit Union Association,
Aug. 6, 2013.) C.f., (Michigan Credit Union League & Affiliates, Aug. 6, 2013) (“As
an industry, credit unions continue to feel the burden of significant compliance costs
for new regulations that are in response to actions they were not a party to.”) The
CFPB is cognizant of the concerns raised by credit union commentators and will be
mindful of the differences they have raised when analyzing and presenting data.
For example, the Bureau highlighted the distinction between credit unions and
other types of credit card issuers in its release of the preliminary results of the
arbitration study, noting that only nine of the 275 agreements it studied relating to
credit cards issued by credit unions contained arbitration provisions – just 3.3%
compared to approximately 58% of large bank issuers. 27
As referenced above, certain commentators raised concerns about Questions 8 and
11 on the Initial Questionnaire, which explored (a) how much consumers would
need to be harmed before initiating an arbitration or litigation or participating in a
class action litigation; or (b) the amount of money they would spend to obtain the
dispute resolution rights that they preferred. The U.S. Chamber of Commerce
highlighted what it described as the primary tension inherent in the estimates.
Consumer Financial Protection Bureau, Arbitration Study: Preliminary Results (Dec. 2013)
available at: http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminaryresults.pdf.
27

16

Without providing a description of arbitration and a description of litigation,
“consumers are unlikely to provide anything but an un-, or at a minimum, underinformed guess.” On the other hand, “[a]ny such description would to a very
significant degree influence consumers’ answers (because it would be their sole
source of information) and therefore produce wholly unreliable results.” (U.S.
Chamber of Commerce, Aug. 6, 2013 p.18.) The ABA Commentators similarly
expressed that consumers would lack sufficient information to make meaningful
assessments or comparisons about arbitration or litigation unless they had been
involved in either. (ABA Commentators, Aug. 6, 2013) (focusing on former Question
11, in particular). 28 The Michigan Credit Union League & Affiliates, as well as
individual commentator Thomas Durkin, similarly raised concerns about Questions
8 and 11 in the Initial Questionnaire, which they describe as asking consumers to
answer a hypothetical question. As mentioned above, those questions – former
Questions 8 and 11 – have since been stricken from the proposed survey.
A few commentators argued that the results of the proposed survey’s questions
regarding card acquisition and default assumptions were predictable. None of these
commentators, however, identified specific empirical work that had not already
been considered by the Bureau that would potentially duplicate the proposed case
work. Instead, the commentators cite to general studies of contract terms. Had
these commentators raised specific studies that would be duplicative of the specific
inquiries regarding consumer credit card acquisition decisions and default
assumptions, the CFPB would have closely reviewed the studies in the course of the
other works referenced herein. But, after a 60-day comment period and two sets of
in-person meetings with many of the commentators, the CFPB has not identified
any such prior work.
In particular, the Bureau submits that general studies regarding consumer contract
terms may not apply to arbitration provisions, which are frequently formatted
specifically to draw consumers’ attention to them. As described above, and in the
Bureau’s original OMB submission, arbitration agreements are frequently
highlighted in headers at the top of consumer agreements. And, in the body of the
consumer agreements, they are often set aside in their own paragraphs and
emphasized with larger font sizes and text that is bolded, underlined, or both. At
the same time, arbitration agreements describe complex issues that may be difficult
for consumers to understand. In that regard, the Bureau’s December 2013
preliminary results found that arbitration clauses are generally more complex and
28 The U.S. Chamber of Commerce also agreed that “[v]anishingly few respondents will have
sufficient background information to assess the comparative merits of arbitration and litigation,
because it is highly unlikely that more than a very few individual respondents will have had the
experience of pursuing similar disputes in each forum.” (U.S. Chamber of Commerce, Aug. 6, 2013, p.
10, 16) (“And the number of respondents with experience concerning both arbitration and litigation
is likely to be extremely small – indeed, it is likely to be zero.”) (emphasis in original).

17

written at a higher grade level than other provisions in the same contracts, using
the Flesch readability score (a widely used standard in plain language analysis).
Accordingly, arbitration agreements have relatively unique characteristics that may
impact consumers’ ability to identify them, on the one hand, but also to understand
them, on the other.
Many commentators raised concerns about the length and complexity of the
proposed survey. The revised survey is significantly streamlined, compared to the
original proposal. While responsive to commentator suggestions, streamlining of the
original proposal was included in the design of the survey. The proposed survey was
intentionally broad and released without the contributions of the survey consultants
retained by the CFPB. Focusing solely on question text read to consumers (not
including demographic questions, which were not included in the Initial
Questionnaire), the Initial Questionnaire had a Flesch readability score of
approximately 68 and an eighth grade level, respectively. 29 Focusing similarly on
non-demographic question text read to consumers, the new draft has about onethird fewer words, with approximately a 78 readability score and a sixth grade
level. 30

Consultation with Individuals Outside of the Agency, March 2014
On March 11 and 12, 2014, the CFPB met individuals outside of the agency in three
additional meetings regarding the proposed survey. Meeting participants again
included leading business federations, banking trade associations, and non-profit
consumer advocacy organizations.
The CFPB discussed a revised approach to the Initial Questionnaire, informed by
two focus group panels held by ICF International on February 18. The CFPB
explained that it would be proceeding with user-testing of a revised survey
instrument shortly after the meetings and would incorporate feedback from the
consultation in the version of the survey that would be tested.
Commentators again discussed proposed survey questions concerning consumers’
perceptions of the relative transaction costs of various types of dispute resolution.
Many commentators raised concerns about the hypothetical nature of such
questions. As discussed above, questions exploring relative transaction costs have
29 The Flesch readability score ranges from 0.0 to 100.0, with a higher number indicating greater
readability. With respect to grade level, on the other hand, a lower level indicates greater
readability.
30 These comments are referenced as examples only and should not be construed as an exhaustive
list of the comments considered by the CFPB in revising the proposed survey. The CFPB considered
many other specific comments, as well as other factors such as focus group testing and user testing,
when revising the proposed survey.

18

since been stricken from the proposed survey. Other commentators raised questions
about whether the results of the survey were predictable. As discussed above, no
commentator has identified a study or other empirical work that indicates that the
survey’s exploration of dispute resolution provisions in consumer card acquisition
decisions or default assumptions regarding dispute resolution are, in fact,
predictable – particularly in light of the special formatting often used for dispute
resolution provisions or the CFPB’s recent data indicating the differences in
readability of the provisions, compared to the rest of consumers’ credit card
agreements.
9.

Payment or Gifts to Respondents

No payment or gifts will be offered to participants in the proposed survey.
10.

Assurance of Confidentiality

In the survey’s introduction, respondents will be told about the study’s purpose, the
authority under which the data are being collected, that cooperation is voluntary,
and that direct identifying information will not be provided to the CFPB or to any
other party. 31 Respondents will be informed by both the pre-notification letter and
the introduction to the survey that their response will be kept private except where
required by law.
At the completion of the data collection, each respondent’s telephone number will be
separated from the answers to the survey questionnaire. At the completion of the
project, the ICF International will transfer or destroy all direct identifying
information according to security procedures and a data management plan to be
approved by the CFPB. 32

31 The survey’s OMB control number will be listed in pre-notification letters (Appendix C), as well as
a statement that the CFPB cannot conduct an information request until it displays a valid OMB
control number.

Consumers will be told that the average interview length is expected to be ten minutes, and that
respondents should let interviewers know if they feel that this projection was inaccurate.
32 The data management plan will provide, at least, that the CFPB’s internal archived portion of the
survey documentation contain the information set forth in Guideline 7.3.2 of the OMB Standards
and Guidelines for Statistical Surveys (Sept. 2006), where appropriate.

19

11.

Justification for Sensitive Questions

Respondents will be asked to identify the financial institution that issued their
most recently-obtained credit card. The CFPB can consult documentation relating
to that financial institution, to verify and report on the accuracy of consumers’
default assumptions regarding their dispute resolution rights. The CFPB will also
use this information to help mitigate potential sampling bias issues. In that regard,
when it interprets demographic data for sample weighting (described below) the
CFPB will be able to identify potential weighting issues by comparing publiclyavailable data sources 33 describing the general market share of credit card issuers
against respondents’ answers to Question 8, which asks consumers to identify the
“name of the bank, credit union, or company to which you make payments”
regarding the credit card the respondent uses most often for personal use. The
question specifies that the consumer can check the actual card to answer the
question.
Consumers will be asked about their prior experiences with arbitration, litigation,
class litigation, or small claims court. This information is gathered solely to help
mitigate potential sampling bias issues, as opposed to primary reporting that would
make estimates regarding consumers’ experiences in these forums. The actual
disputes will not be identified. This information will be collected, so that any
correlation between past experiences and consumer responses can be identified and,
if appropriate, controlled for.
Respondents will also be asked basic demographic information regarding education,
race/ethnicity, age, income, and gender. This information will be used to gauge and
potentially weight the sample so that the survey reflects representative
demographic groups in the U.S. population. 34

33 In describing the prevalence of pre-dispute arbitration clauses in its December 2013 preliminary
results, the CFPB used measurements of account value for credit cards from public “call reports”
filed with regulators by banks and credit unions. See Preliminary Results at 20, available at:
http://files.consumerfinance.gov/f/201312_cfpb_arbitration-study-preliminary-results.pdf.

Call reports are available at the website for the Federal Financial Institutions Examination Council.
See www.ffiec.gov.
34 Mercator similarly weighted its credit card survey results against “the demographic structure of
U.S. population in the statistical analysis based on geographic location, gender, income, household
size, and age structure.” U.S. Credit Cardholders: Waiting for a Rebound, Mercator Advisory Group
(Sept. 2011) (Customer Monitor Survey Series, Insight summary Report Vol. 3, Report 1). Mercator
noted that the lowest economic strata of the U.S. population may be under-represented. “On the
other hand, the respondent base is certainly representative of the broad economic profile of
consumers in the U.S. who have banking relationships.”

20

This information collection will not include questions about sexual behavior and
attitudes, religious beliefs, disability, social security numbers, or other matters
generally considered private. The information in this collection is covered by the
Bureau’s Consumer Experience Research Privacy Impact Assessment (PIA)
published June 30, 2014. 35
12.

Estimated Burden of Information Collection

NonResponse
(Screening)
Telephone
Survey
13.

No. of
Respondents

Annual
Frequency
per
Response

Total
Annual
Responses

Hours per
Response

Total
Hours

9,000

1

9,000

0.05

450

1,500

1

1,500

0.130 36

195

Estimated Total Annual Cost Burden to Respondents or Recordkeepers

There are no annualized capital or start-up costs for the respondents to participate
in the survey.
14.

Estimated Cost to the Federal Government

There will be no annualized capital/start-up costs for the government to receive the
survey information. The testing is funded with non-appropriated funds. It is
anticipated that the contract to carry out the study will cost $186,500.
The Blanket Purchase Agreement, under which the survey will be conducted, was
awarded through a competitive bidding process.

35 Consumer Financial Protection Bureau, Consumer Experience Research Privacy Impact
Assessment, June 30, 2014, http://files.consumerfinance.gov/f/201406_cfpb_consumer-experienceresearch_pia.pdf.

The estimate for average burden per response was arrived at by testing the proposed survey
questions with multiple CFPB personnel that are unaffiliated with the proposed study.
36

21

15.

Program Changes or Adjustments

Because this is a new collection of information, all the burden is new and, as such,
considered to be a program change for the purpose of OMB’s Paperwork Reduction
Act inventory.
16.

Plans for Editing, Statistical Analysis, and Publication

The CFPB anticipates that the survey will be completed within eight to twelve
weeks of OMB approval. The contractor will develop the initial analysis and
summary of the study findings.
Skip patterns will be embedded in the computer assisted telephone interviewing
software, which should strongly mitigate erroneous data due to respondents being
asked inappropriate questions or questions that were inadvertently skipped. The
computer assisted telephone interviewing software will also contribute to data
quality by not allowing responses that fall outside of logical bounds. Further, the
CFPB and ICF International will review the data for logistical inconsistencies
before tabulation and, where appropriate, edit the collected data to mitigate or
correct detectible errors. Missing data that cannot be resolved through editing may
be imputed using accepted statistical techniques such as multiple imputation
through chained equations.
Other question combinations may not result in clearly erroneous answers, but will
still require that specific combinations of answers be noted. For example, some
questions are designed to provide rough estimates of respondents’ familiarity with
arbitration or litigation (e.g., “What does it mean to participate in arbitration?”). If a
respondent is unable to list any features of arbitration (or that arbitration
proceedings have binding results), their lack of background knowledge can be noted
when analyzing the consumer’s later responses.
Where appropriate, the contractor will convert answers provided in response to
open-ended questions to standardized codes, which will allow the CFPB and future
users of the data to better analyze the information. For example, Question 4 asks
consumers to list the most important reasons for choosing the credit card they use
most frequently. The survey itself will not provide potential answers. But
consumers’ open-ended responses will be converted to common codes, where
possible, so that all responses relating to low interest rates, for example, can be
grouped together; whereas other answers relating to specific rewards features
would be grouped into a different code. Some consumer answers, however, may not
be appropriate for grouping into codes. In such case, the individuals’ actual
responses will be noted or summarized. (Question 5, in contrast, includes a closedended set of potential responses.)

22

The report will provide tabulations at the national level. Once the data is tabulated,
it will be presented along with an executive summary and detailed findings about
the role that dispute resolution provisions play in consumer card acquisition
priorities and consumers’ default assumptions regarding dispute resolution rights.
The report will also provide an evaluation of the quality of the data, as well as areas
of potential improvement. The CFPB does not currently contemplate conducting
additional surveys on this subject, but the information may be useful for data
interpretation or designers of similar studies.
17.

Display of Expiration Date

The CFPB plans to display the OMB number and expiration date for OMB approval
on the pre-notification letter. Survey telephone administrators will also be provided
this information.
18.

Exceptions to the Certification Requirement

The Bureau certifies the this collection of information is consistent with the
requirements of 5 CFR 1320.9, and the related provisions of 5 CFR 1320.8(b)(3) and
is not seeking an exemption to these certification requirements.

23