Description of Study/Overview

Description of Study

OVERVIEW
The CFPB implementation team has been working with Kleimann Communication Group (the Contractor) to design an initial draft of a disclosure that will replace the current Good Faith Estimate and Truth in Lending disclosure statement required under RESPA and TILA, respectively. The Contractor will cognitive- and usability-test the disclosure form that is being developed using a qualitative methodology. After conclusion of the last round of qualitative testing, by which time the disclosure will have been revised several times, the draft disclosure will be evaluated using a quantitative methodology (which will be submitted in a separate ICR). For the qualitative portion, the CFPB implementation team plans to test at six sites in five rounds to allow for changes to the disclosure between rounds.

SAMPLE DESIGN AND DATA COLLECTION

At each of the six sites, the Contractor will recruit seven English-speaking consumer participants and five Spanish-speaking consumer participants who have diverse mortgage loan experiences (the Birmingham testing will involve only English-speaking participants). The Contractor will recruit a population that includes:

• Consumers who have bought or refinanced in the past five years, including at least one participant who has experienced a delinquency or requested a modification.

• Consumers who plan to buy or refinance in the next year, including at least one person who has no experience with buying, but intends to buy.

At each site, the Contractor will also recruit two lenders or brokers who:

• have been in business for more than four years

• across all twelve of the interviews, at least one works primarily with minority populations

• for lenders, at least three across all of the interviews are community bankers

For each round of testing, the Contractor contracts with research facilities in each specified location and ensures each facility locates and recruits the above consumer and lender/broker populations. In addition, the Contractor will make final decisions with the CFPB implementation team on the most important population demographics to capture. The two consumer populations will reflect the U.S. Census population demographics based on geographic location, metro/micropolitan city size, age, ethnic and racial diversity, education, income, gender, and marital status.

Sites were selected that provide diversity among small, medium, and large sized cities, using Census Bureau information to define city size based on the annual population metropolitan and micropolitan statistical areas from July 2009. Cities were also selected based on the geographic regions and divisions defined by the Census Map. As a result, the Contractor will test in Los Angeles, CA; Albuquerque, NM: Chicago, IL; Springfield, MA; Baltimore, MD; and Birmingham, AL.

To enable the Contractor to refine the integrated disclosure between tests, give the CFPB implementation team time to approve changes, and meet the January 2012 deadline, the Contractor will test in five rounds as shown below. Depending on how well the disclosure is working by Round 4, the Contractor tentatively has proposed testing in two sites concurrently. If, however, the disclosure still needs substantial refinement, the plan may be modified to test in two separate sites at two different times.

In most qualitative testing, a small sample size is typical. According to Virzi, 80% of usability problems are uncovered with five (5) participants and 90% with ten (10) participants. The Contractor will test seven (7) consumers per site per round (total of 42), as well as one (1) broker and one (1) lender per site per round.¹

Round	Location	Week of Testing	Number of Participants
			Consumers	Brokers	Lenders	Total
1	Baltimore, MD (south, medium)	May 16, 2011	7	1	1	9
	Spanish		5			5
2	Los Angeles, CA (west, large)	June 20, 2011	7	1	1	9
	Spanish		5			5
3	Springfield, MA (NE, small)	July 18, 2011	7	1	1	9
	Spanish		5			5
4	Chicago, IL (midwest, large)	August 22, 2011	7	1	1	9
	Spanish		5			5
4	Albuquerque, NM (west, medium)	August 22, 2011	7	1	1	9
	Spanish		5			5
5	Birmingham, AL (south, medium)	September 19, 2011	7	1	1	9
Totals	6 locations		67	6	6	79

TEST STRUCTURE AND DESIGN

The top level research questions for consumers include:

• Can consumers understand the transaction including costs and risks of this loan?

• Can consumers use the disclosure to compare the same types of loan?

• Can consumers use the disclosure to compare different types of loans?

• Can consumers use the disclosure to choose between loans and express a reasonable rationale for their choice?

The top level research questions for brokers and lenders include:

• Where do brokers and lenders have trouble understanding or explaining the information contained in the disclosure?

• Where do brokers and lenders expect consumers to have trouble understanding information in the disclosure?

• What suggestions do brokers and lenders have to improve the integrated disclosure for consumers?

• What suggestions do brokers and lenders have to ease implementation for industry?

The Contractor will use 90-minute interview sessions. This provides adequate time to collect the data the CFPB implementation team needs, provide the participant with a break, probe on anything unexpected, and not rush the interview. The Contractor will structure the testing sessions as follows:

• Introduction. The moderator explains the purpose of the testing, the ability to stop participating at any time, and confirms completion of the confidentiality forms and audio- and videotape waiver.

• Practice. The moderator explains the think-aloud methodology and allows the participant to practice the technique on a neutral document, such as a restaurant menu.

• Unstructured Think-Aloud Interview. In the think-aloud portion, the participant provides feedback based on what he or she notices, but without questions on the part of the moderator. The goal is to capture participants’ reactions before they can “learn” from the testing situation and from the interviewer asking them questions.

• Structured Interview. In the structured portion, the moderator follows up with probes to answer specific, predetermined research questions that will provide data about cognitive understanding of particular concepts and the implications of the overall disclosure. The goal in this part of the interview is to capture the participants’ behaviors and understanding to ensure revisions are guided by consumer performance.

Loan Products

The study will use 4 different loan products to verify that consumers can use and understand the loan disclosure. These are:

• A 30 year fixed rate loan;

• An adjustable rate loan (ARM) with a “teaser” intro rate (2/28);

• A 5 year interest only loan, amortizing over the remaining 25 years; and

• A negative amortization loan.

INTERNET INPUT STRUCTURE AND DESIGN

Input from the Internet may include several activities. Final drafts of the tools are still being finalized; however, to the extent that OMB Control Numbers are necessary, the tools will be submitted to OMB as soon as they are completed. For example, in conjunction with the first round of qualitative testing, the CFPB may structure the content and information collection as follows:

• Introduction. A simple landing page will present visitors with a greeting, and request, though not require, some non-personally-identifiable demographic information, such as geographic area and education level.

• Form Selection. A single screen will present the participant with two candidate forms or displays of information, and prompt them to pick the one they would like to comment on.

• Free Comment. Based on the participant’s previous selection, they will be given the opportunity to provide multiple free-form comments.

• Sharing. Having finished submitting feedback, the participant will be encouraged to share the feedback opportunity with others within their social graph via social media or e-mail. None of this requires any collection of additional information.

1 Virzi, R. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors 34, 457–486.

4