Paperwork Reduction Act Supporting Statement part 1 rev 0517 (Part B)

PART B

Data collection methods and procedures will vary based on the specific instrument; however, the primary purpose of this collection will be to inform the design of the integrated mortgage loan disclosure, while also providing information on facilitating compliance and easing implementation for lenders and other industry participants. It is anticipated that a report will be published discussing the methodology and results of the data collection to inform future disclosure design and testing.

The CFPB implementation team is using an iterative test process to develop effective disclosure forms. It will conduct one-on-one interviews with twelve consumers (7 English-language and 5 Spanish-language consumers) and two industry representatives in each round. Based on the data from these interviews, the team will work with the Contractor to revise the disclosure. The team will also consider information collected through the internet in revising the disclosure. The testing will involve five rounds in six locations (the fourth round will involve testing in two different locations), approximately one month apart from May through September 2011.

Expected Testing Period	Location	Expected changes to materials from prior round of testing
May 2011	Baltimore. MD	N/A
June 2011	Los Angeles, CA	Include additional comparisons in Task 4 (will have 4 products rather than only the fixed and adjustable rate loans in Round 1) Amend Task 5 comparison of two separate forms to comparison of two methods of displaying specific content using a single design
July 2011	Springfield, MA	Revise comprehension questions in Task 3 based on findings from prior testing rounds
August 2011	Albuquerque, NM/Chicago, IL	Revise comprehension questions in Task 3 based on findings from prior testing rounds
September 2011	Birmingham, AL	Revise comprehension questions in Task 3 based on findings from prior testing rounds

The participant selection methodology and the materials related to participant recruiting and screening will remain the same for each round of testing. The structure of the interview is expected to remain the same for each round, but the specific tasks may differ depending on the stage of development. In the first round of testing, when the focus is primarily on design, participants will be asked to compare two loan products. In that case, the participant is presented with two disclosures that are of the same design for the same type of loan product. The two disclosures will vary based on characteristics of the loan product – that is, one of these loans will be priced higher or have a potentially more expensive feature. The discussion is intended to elicit the participant’s logic in choosing one of the loans. Because personal and life characteristics influence the match between loan and participant, eliciting the participant’s logic is necessary to evaluate whether the participant has chosen the loan that best meets his or her personal criteria. This section also allows us to hear whether and to what extent the disclosure has made clear those aspects that influenced the participants’ decision-making or failed to convey key pieces of information. At the end of the interview, they will be presented with a disclosure using the alternate design and asked to discuss differences between the two designs.

It is likely that the first round of testing will identify the more effective design, in which case the second round of testing will begin to focus more on specific ways to communicate content. Thus, the interviews may involve questions about different ways to express a concept rather than on differences between designs. Again, the structure will remain the same: a think aloud, followed by a choice between two loans, then a probe to determine comprehension, then additional comparisons among loan products.

The internet tool will remain the same, offering a participant the same screens with the same activity; however, like the interviews, the specific task may differ. For example, the first round involves presenting two different designs and asking which one is more helpful in choosing a loan. The second round may present two different ways of expressing a specific concept and ask which is more effective.

Prior to each round of testing, the CFPB implementation team will provide OMB with a document identifying the revisions made to the disclosure, the accompanying revisions made to the Moderator’s Guides (Consumer and Lender/Broker), and any other elements of the collection that have changed. The team also will provide OMB with any changes to the internet tool, including screen shots of the revised tool.

1. UNIVERSE AND RESPONDENT SELECTION

The activities under this clearance will involve samples of self-selected customers, particularly through the internet channel. The qualitative testing will involve convenience samples recruited locally for each round of qualitative testing, with participants selected to include specific characteristics related to certain products. Results will not be used to make statements representative of the universe of study, to produce statistical descriptions (careful, repeatable measurements), or to generalize the data beyond the scope of the sample.

At five of the six sites, the Contractor will recruit seven English-speaking consumer participants and five Spanish-speaking consumer participants who have diverse mortgage loan experiences. The Birmingham testing will involve only English-speaking participants because Birmingham has a small Hispanic population. The Contractor will recruit a population that includes:

• consumers who have bought or refinanced in the past five years, including at least one participant who has experienced a delinquency or requested a modification.

• consumers who plan to buy or refinance in the next year, including at least one person who has no experience with buying, but intends to buy.

At each site, the Contractor will also recruit two lenders or brokers who:

• have been in business for more than four years

• across all twelve of the interviews, at least one works primarily with minority populations

• for lenders, at least three across all of the interviews are community bankers

For each round of testing, the Contractor contracts with research facilities in each specified location and ensures each facility locates and recruits the above consumer and lender/broker populations. In addition, the Contractor will make final decisions with the CFPB implementation team on the most important population demographics to capture. Among the 67 consumers that will participate, the two consumer populations (i.e., action in the last five years or intended action within the next year) will reflect the U.S. Census population demographics based on these criteria:

Demographic	Criteria Category
Geographic location	West, Midwest, Northeast, South
Metro/Micropolitan city size	Large population- 5 million and up, medium city population-800,000 to 4.99 million, and small city population under 800,000
Age	Under 30, 30-45, 46-60, and older than 60
Ethnic and racial diversity	Ethnicity: Non-Hispanic or Latino origin or Hispanic or Latino origin (Cuban, Mexican, Puerto Rican, South or Central American, or other Hispanic or Latino origin). Race: White, Black or African American, Asian, Native Hawaiian or Other Pacific Islander, American Indian or Alaska Native
Education	high school graduate/GED, some college, college grad or additional education
Income	less than $35,000, $35,001-70,000, $70,001-125,000, and over $125,000
Gender	male or female
Marital status	single, head of household, married

The Contractor is looking for a mix of participants and characteristics and does not expect to meet all of the criteria in any one site or round of testing. Patterns will be tracked across the six sites with later sites recruiting for characteristics that have not previously been filled.

The intended audience of the disclosure is consumers who have never bought a home, are moving from one house to another, or are refinancing their current loan. Participants who have experienced a delinquency or modification are not the primary audience of this disclosure. However, these participants bring a different perspective to viewing the disclosure and, thus, the Contractor is recruiting one such participant at each site. Across the sites, the Contractor will look for patterns, fully aware of how limited the size of this group of respondents will be.

In a similar way, respondents who have never bought a home will bring a different perspective to the content and design of the disclosures. However, the current housing market may make it difficult to find purchasers who have never bought a home and who are actively looking to purchase, so the recruiting goal is modest.

The site locations include geographic diversity within the West, Midwest, Northeast, and South.

Sites were selected that provide diversity among small, medium, and large sized cities. The CFPB implementation team used Census Bureau information to define city size based on the annual population of the metropolitan and micropolitan statistical areas from July 2009. The CFPB implementation team also selected cities based on the geographic regions and divisions defined by the Census Map. As a result, the CFPB implementation team will test in the following cities:

Census Region	Location	Size1
West	Los Angeles, Long Beach, Santa Ana, CA Albuquerque, NM	12,874,797 (large) 857,903 (medium)
Midwest	Chicago, Naperville, Joliet (IL, IN, WI)	9,580,567 (large)
Northeast	Springfield, MA	698,903 (small)
South	Baltimore, MD Birmingham-Hoover, AL	2,690,886 (medium) 1,131,070 (medium)

Qualitative surveys are tools used by program managers to change or improve programs, products, or services. The accuracy, reliability, and applicability of the results of these surveys are adequate for their purpose.

2. PROCEDURES FOR COLLECTING INFORMATION

Data collection methods and procedures will vary with the different activities being undertaken to develop, evaluate, and improve the disclosure form. The CFPB implementation team expects to use a variety of methodologies for these collections. The Contractor will use one-on-one cognitive interviewing to evaluate whether the form satisfies the goals of enabling consumers to understand the loan terms and compare loan products. Telephone scripts for screening, participant questionnaires for obtaining demographic information and confirming eligibility to participate in the interview, cognitive interviews with a moderator, and consent forms will be used.

Development of the integrated disclosure form is being conducted through an iterative process. The form will be evaluated through a qualitative testing process using one-on-one interviews. These data will be analyzed and used to revise the disclosure form and then evaluate it again with another set of interviews. Interviewing is planned at six sites over five rounds of testing, with each occurring at a different location with different participants. While structured interviewing with individual consumers and industry representatives is being conducted, the disclosures will be available to the public on the CFPB website. This will enable consumers, industry, and other stakeholders to track the CFPB’s progress and better understand the process. The CFPB implementation team has created a tool to collect feedback from the public on the disclosures.

The tool consists of two similar activities, one designed for industry (“Industry tool”) and the other designed for general audiences (“Consumer tool”). It is expected that, in addition to consumers, consumer advocates, housing counselors, academics, and others will choose the Consumer tool.

The tool will remain the same through each round of feedback corresponding to the rounds of interviewing, but the specific tasks will differ. For example, the initial task will be to select which design is more helpful in making a loan decision. For the second round of testing, the task may be to select between two features that convey the same information in different ways. The task at a later round may be to select between two loans using the disclosures. Each time, the process will be to present two options, allow the person to select one, provide some general comments in text boxes, and perhaps answer a few open-ended questions.

The main differences between the Industry tool and Consumer tool during the initial round of feedback are (1) the factor on which the choice between two designs is made (gives you the information you need OR you could most easily use to inform a consumer) and (2) the Industry tool asks about implementation and understanding of this form compared to the existing forms.

When visitors come to the CFPB website, they will see a large feature section on the top of the home page. If visitors click on the feature area “Know Before You Owe,” they will be taken to the Consumer tool. They will be given the option at that time to click instead on the Industry tool. Those who choose to participate in either the Consumer or Industry tool (“Participants”) will be presented with three screens, which are described in detail below.

Once a participant has completed the activity, he or she cannot return to the tool in either channel – Consumer or Industry – using the same computer. This is determined by using a cookie that does not collect any personally identifiable information, and so is a Tier 2 persistent cookie under OMB M-10-22. The cookie is designed to prevent a participant from engaging in the activities numerous times.

Screen 1:

The first screen will say “Welcome to the Know Before You Owe feedback tool. You are about to complete the feedback form designed for consumers. If you’re a member of the mortgage industry, use the separate industry tool form.” They then click “Continue to Consumer Tool” or “Switch to Industry Tool”. If they access the activity through a URL that was provided to an industry group, they will be told they are at the industry tool and given an opportunity to switch to the Consumer Tool.

Screen 2:

The screen will display two different designs of the disclosure: Option A and Option B, each of which consists of two pages. For the Consumer tool, the top of the screen will say: “Please select the disclosure that best gives you the information you need to make a decision about which loan to choose.” The participant can click on Choose Option A or Choose Option B.

For the Industry tool, the top of the screen will say: “Please select the disclosure that you could most easily use to inform a consumer about the loan they’ve requested.”

Screen 3:

Based on his or her choice, the participant will be presented with a form that is either the form the participant selected or the other one. Whether the participant receives the form he or she selected or the other one will be randomly determined.

If given the form they prefer, participants will be told: “This is the form that you selected. Please click on the parts of the form – up to four —you found helpful.” The participant can click on up to four areas of the form. When the participant clicks on an area of the form, a text box pops up that says “Why did you find this information helpful?” and gives the participant a text box in which to respond. This can be repeated up to four times. The participant can “unclick” and choose a different area to provide comments on.

If given the form they do not prefer, participants will be told: “This is the form that you did not select. Please click on the parts of the form – up to four —you found unhelpful.” Again, the participant can click on up to four areas of the form, including the option: “Important information was missing.” When the participant clicks on an area of the form, a text box pops up that says “Why did you find this information unhelpful?”

Screen 4:

This screen is different based on whether the participant is using the Consumer tool or the Industry tool.

In the Consumer tool, the participant is told:

“Thank you! You can be finished now, if you want. But we’d love for you to tell us more:”

Your zip code: ______

Which of the following best describes you:

• Consumer

• Real Estate

• Consumer Lending/Finance

• Insurance

• Design/Technology

• Consumer Advocate

• Research/Academic

• Housing Counselor

• Other:_______________

Participants are then asked: “Is there anything else you’d like to tell us about the redesigned forms?” and provided with a text box. They are also offered the opportunity to return to the tool to comment on the other design.

For the Industry tool, participants receive the same initial thank you, but are then offered two questions:

“Compared to the existing Truth in Lending and Good Faith Estimate forms, do you think the form that you thought was most helpful would be easier or more difficult to fill out and use, as a business matter?”

○Easier ○More difficult

“Compared to the existing Truth in Lending and Good Faith Estimate forms, do you think the form that you thought most helpful would be easier or more difficult for borrowers to understand?”

○Easier ○More difficult

They are also asked for their zip code and then given the following choices for affiliation:

• Lender

• Mortgage broker

• Appraiser

• Title insurance

• Other settlement service provider

• Insurance

• Investor

• Real estate broker/agent

• Other real estate

• Other:_______________

Industry tool participants will also be asked, “Is there anything else you’d like to tell us about the redesigned forms you saw today?”, provided with a text box, and given the opportunity to return to the tool to comment on the other design.

Data collection using the internet at different times during the development and evaluation of the disclosure form may include answering comprehension questions, identifying sections of the form on which visitors want to comment, prioritizing information on the form, or identifying missing information.

The CFPB implementation team expects to use technology to automate its collection and analysis of feedback.

3. METHODS TO MAXIMIZE RESPONSE

Information collected under this clearance will not yield generalizable quantitative findings; however, it can provide useful input into the design of the disclosure form. A quantitative test will be conducted to validate the disclosure form, for which a separate ICR will be submitted.

4. TESTING OF PROCEDURES

No formal testing is planned. Pretesting of the Moderator’s Guides will be conducted by the Contractor. The Contractor will conduct pretesting on 4 consumers and 2 industry representatives who are similar to the participants being recruited.

5. CONTACTS FOR STATISTICAL ASPECTS AND DATA COLLECTION

Because this data collection involves qualitative testing of a small sample, the CFPB implementation team does not expect to be using statistical analysis to analyze the data.

The qualitative testing design has been developed by Kleimann Communication Group. The key project staff with respect to design is:

Dr. Susan Kleimann, Kleimann Communication Group, 301-233-2844, [email protected]

1 Based on the Annual Estimates of the Population of Metropolitan and Micropolitan Statistical Areas: April 1, 2000 to July 1, 2009 (CBSA-EST2009-01) http://www.census.gov/popest/metro/CBSA-est2009-annual.html.