TILA-RESPA Quant SS (PART B) 3-26-13

CONSUMER FINANCIAL PROTECTION BUREAU
INFORMATION COLLECTION REQUEST –
SUPPORTING STATEMENT
QUANTITATIVE TESTING OF INTEGRATED
MORTGAGE LOAN DISCLOSURE FORMS
(OMB CONTROL NUMBER: 3170-XXXX)
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL
METHODS
1.

Respondent Universe and Selection Methods

As set out in Part A of this Supporting Statement, the data collected under this
quantitative research will be used to evaluate the results of qualitative testing conducted
by the Bureau using statistical data and assess the performance of the Bureau’s proposed
integrated disclosures in comparison to the current Federal mortgage loan disclosures.
This research will inform the Bureau’s evaluation of the proposed integrated disclosures
and finalization of the 2012 Proposal. 1
The study will involve a sample of 850 consumer respondents. With this sample size,
there is sufficient power (>.7) to detect medium-sized effects (15%-25%) across each of
the cells (assuming cell sizes of 50 consumer respondents) with 95% level of confidence.
The power increases substantially (>.9) if cells are collapsed so as to increase cell sizes to
100. The study will not regard the sample as a probability sample of the population
designed to be generally representative of the national population.
The main analysis will focus on comparisons in quality/cogency and accuracy of
responses (open-ended as well as closed-ended) for respondents exposed to the current
disclosures versus the proposed disclosures. Responses to individual questions as well as
aggregated responses (e.g., total number of correct responses) will be analyzed. Twotailed t-tests will be used to test for differences at the 95% confidence level. Additional
analysis will drill down to compare differences in responses to the current versus
proposed disclosures at different levels of one or more of the other factors (e.g.,
experience level, type of mortgage, and/or task difficulty). Overall, our analyses will be
similar to those conducted by FTC economists Lacko and Pappalardo in their 2007 study
on mortgage disclosures. 2

1

See the Bureau’s press release Consumer Financial Protection Bureau proposes “Know Before You Owe”
mortgage forms (July 9, 2012), available at http://www.consumerfinance.gov/pressreleases/consumerfinancial-protection-bureau-proposes-know-before-you-owe-mortgage-forms/; the Bureau’s blog post
Know Before You Owe: Introducing our proposed mortgage disclosure forms (July 9, 2012), available at
http://www.consumerfinance.gov/blog/know-before-you-owe-introducing-our-proposed-mortgagedisclosure-forms/. The 2012 Proposal was published in the Federal Register on August 23, 2012. See 77
FR 51116 (Aug. 23, 2012).
2
See Improving Consumer Mortgage Disclosures: An Empirical Assessment of Current and Prototype
Disclosure Forms, Federal Trade Commission Bureau of Economics Staff Report, by J. M. Lacko & J. K.

-1-

The quantitative study will exclude consumers who work in the mortgage loan or title
insurance industries, as well as persons who participated in the earlier qualitative
research. To ensure that the respondents are part of the targeted audience, the study will
ask potential respondents a scale of questions of whether, and when they intend to
finance the purchase of a home or refinance a mortgage loan in the future. The study will
exclude those who answer “never” because they would not be part of the intended target
audience. To refine their level of interest in obtaining or refinancing a mortgage loan in
the future, the study will ask them whether they intend to engage in such a transaction in
the next 2 years, 5 years, 10 years, 15 years or more, or never.
The contractor, Kleimann Communication Group, Inc. (the Contractor or Kleimann), will
use Knowledge Networks, now owned by GfK Research, as a quantitative data collection
partner. Study respondents will be recruited through GfK’s established KnowledgePanel,
as described in Part A to this Supporting Statement. Members of GfK’s KnowledgePanel
will be invited to participate via telephone calls that will follow the Recruiting Script
(attached as Appendix A). The Recruiting Scripts asks certain supplemental
demographic and residential mortgage market experience questions (as described in Part
of the Supporting Statement) to ensure the potential respondent qualifies for the study
and identify the variable group in which the respondent will be placed. The Recruiting
Script will invite the qualified panel members to a testing facility. The Recruiting Script
provides for a follow up letter and telephone call designed to remind the potential
respondent of the study date and time.
Demographics to be considered include geographic location, metro/micropolitan city
size, age, ethnic and racial diversity, education, income, gender, and marital status. For
example, as noted above, the testing facilities will be located in cities that cover the four
census regions and the census sub-regions. In addition, the facilities will be located to
ensure respondents are recruited from urban, suburban, and rural populations. The reason
for this is that real estate markets often have local characteristics, such as different
terminology or types of transactions. The study aims to evaluate the proposed integrated
disclosures in a variety of markets.
The Contractor typically looks for a mix of respondents and characteristics and does not
expect to meet all of the criteria in any one site. Patterns will be tracked across sites and
collections with later sites will recruit for characteristics that have not previously been
filled. Sites will be selected that provide diversity among small, medium, and large sized
cities, as well as geographic diversity.
The testing sessions will be collected in small group sessions (groups of 12 or less) in inperson testing facilities that are located in cities that cover the four census regions and the
census sub-regions. The testing sessions will be administered in person by the Contractor
using a trained researcher. To ensure that these locations are reasonably accessible to
Pappalardo (June 2007), available at:
http://www.ftc.gov/os/2007/06/P025505MortgageDisclosureReport.pdf.

-2-

respondents, we will recruit respondents who live within a 60-minute driving distance of
one of the locations.
However, since the study respondents do not represent a probability sample drawn from
the universe of interest (national population), specific results (e.g., x% people understand
concept y from the forms) are not statistically generalizable to the universe. Therefore,
the results will not be used to make statements about the universe or to generalize the
data beyond the scope of the sample.
2.

Information Collection Procedures

Data collection methods and procedures will include a Recruiting Script (attached as
Appendix A), a Respondent Questionnaire (attached as Appendix B), a Structured
Questionnaire (attached as Appendix C), and a Privacy Act Statement and Consent Form
(attached as Appendix D).
The data collected using the Structured Questionnaire will be used to evaluate the
disclosure forms being tested. The Structured Questionnaire used at the testing sessions
will be administered by a trained researcher who will act as a proctor. The trained
researcher will use a standard set of Proctor Instructions, which is attached as Appendix
F. The Proctor Instructions contain instructions for the trained researcher as well as
instructions for the respondents that the trained researcher will read verbally, to instruct
the respondents on the procedures of the testing session.
The testing materials that will be used during the testing session are eight sets of
application disclosures and eight sets of corresponding closing disclosures for both the
current disclosures and the integrated disclosures, for a total of sixteen sets of disclosures
for the current disclosures and sixteen sets of disclosures for the integrated disclosures.
The CFPB will use a structured questionnaire methodology for this collection that will
allow only questions that can be answered using the sets of disclosures.
Study Design
The design is a 2 by 2 by 2 by 2 between-subjects experimental design. The study factors
or independent variables include the following:
Form type (2 levels). The study includes a baseline test using the current
mortgage loan disclosures, to compare performance with the proposed integrated
disclosures. Therefore, respondents will be randomly assigned to either the
current disclosures or the proposed integrated disclosures. In this betweensubjects design, respondents will only use either the current or proposed
integrated disclosures, not both.
Loan type (2 levels). The mortgage market offers fixed-rate and adjustable-rate
mortgage loans. To capture this reality, respondents will be randomly assigned to

-3-

one of two conditions -- compare two fixed-rate loans or compare two adjustablerate loans.
Difficulty type (2 levels). The mortgage market offers mortgage loans with
different levels of complexity. For example, some loans may contain fixed
interest rates and payment and few other risk factors. In addition, consumers may
only face choices of loans among these less complex products. However, the
market also may present consumers with more complexity and risk factors. Such
loans could have a fixed interest rate, but contain an interest only period that will
cause the monthly payment to increase during the loan term. Or the loan may not
pay off all of the interest due each month causing the loan balance to increase. In
addition, in between the application and closing stages, consumers may be forced
to understand more complex changes in the loans, such as a change in the product
type or a significant increase in closing costs that may violate legal limits on
increases. To capture this reality, respondents will be randomly assigned to one
of two conditions -- compare two loans that are relatively easy or compare two
loans that are relatively complex and contain more risk factors. The respondent
will remain in this condition throughout the test (i.e., for both the application and
final disclosures). This approach ensures that data covers situations involving
both easy and more challenging loans and comparisons.
.
Consumer type (2 levels). As described above, we will include two groups of
consumers: (1) consumers who have no experience and (2) consumers with
experience in home purchase mortgage loans. We will exclude consumers who
work in the mortgage, real estate, and title insurance industries, who have
previously commented on the proposed forms, participated in the earlier
qualitative research project, or who respond they never intend to purchase a home
and /or finance with a mortgage.
The design, using an 850-respondent sample size as described above, may be depicted as
follows:
PROPOSED INTEGRATED
DISCLOSURES
Fixed
Adjustable
Rate
Rate
No
Experience
Experience
Totals

Easier
More Challenging
Easier
More Challenging

53
53
53
53
212

53
54
53
53
213
425

CURRENT
DISCLOSURES
Fixed
Adjustable
Rate
Rate
53
53
53
53
212

53
54
53
53
213
425

More detail regarding these variables and other aspects of the design is provided below.

-4-

Form Types
As noted above, this research will include a baseline test, to compare the performance of
the current mortgage loan disclosures with the proposed integrated disclosures. This
research will inform the Bureau’s evaluation of the proposed integrated disclosures.
However, primarily as a result of amendments to the statutory requirements made by the
Dodd-Frank Act, the proposed disclosures include more information than the current
disclosures. In addition, in an effort to reduce paperwork burden for both industry and
consumers, the Bureau’s proposed integrated disclosures include disclosures under other
statutory provisions that are currently provided as separate pages. For example, the
proposed Loan Estimate includes an appraisal notice required by the Equal Credit
Opportunity Act and a servicing disclosure required under section 6 of RESPA.
The current disclosures used in this study will not be altered to reflect the greater set of
information required by the Dodd-Frank Act or the other disclosures integrated into the
proposed integrated disclosures. There are no current regulations that implement the new
statutory disclosures, and thus, there is no “baseline” for them. In addition, lenders
currently present the disclosures required under other statutory provisions separately from
the mortgage disclosures and in varied formats. Thus, there is no “baseline” for mortgage
disclosures that include these other statutory requirements. Accordingly, the current
disclosures used in this study will only include the current Truth in Lending early and
final statements, the Good Faith Estimate, and the HUD-1 settlement statement mortgage
loan disclosures required by TILA and sections 4 and 5 of RESPA. 3
Loan Types
The study will involve both fixed rate and adjustable rate mortgage loans as two
conditions. As described below, these conditions will be further separated into relatively
easier and more challenging loans and comparisons. Each loan used by a respondent will
be identified by the name of the lender. There will be two sets of lender names to reduce
potential bias. A table with information regarding the loan types that will be used in this
research is attached as Appendix G to the Supporting Statement.
Although more prevalent in years prior to the recent financial crisis, loan features such as
interest-only payments, prepayment penalties, negative amortization, and optional
payments are still possible, and some are currently available to consumers. These
features present unique risks to consumers in the form of potentially increased payments,
principal balances, and costs of refinancing. In addition, new product features may be
developed that also contain such risks. The existence of such a risk in a loan is an
important aspect of the loan. Accordingly, this quantitative research will also include
several loans with such features to evaluate the performance of the proposed integrated
disclosures with respect to such risks.
Difficulty Types
The study will involve both fixed rate and adjustable rate mortgage loans as two
conditions. These conditions will be further separated into relatively easier and more
3

As implemented by current 12 CFR parts 1024 and 1026.

-5-

challenging loans and comparisons, denoted as the Difficulty Type. The relatively easier
loans and comparisons will involve simpler loan products and less variation in product
types, loan terms, and closing costs. The relatively challenging loans and comparisons
involve more complex products and more variations in the product type, loan terms, and
closing costs.
The difficult loans will include features such as interest-only payments, negative
amortization, and optional payments, as well as significant increases in closing costs,
including increases that violate legal limits on increases. These features present unique
risks to consumers in the form of potentially increased payments and principal balances,
for which the study should address the performance of the disclosures. In addition, the
use of two different Difficulty Types for each Loan Type mitigates any unintended
effects of a particular loan product that could be introduced if the study used only one set
of loans for each Loan Type.
Consumer Types
The study will include two categories of respondents: those with experience or no
experience with home purchase mortgage loans.


No Experience. This category will be defined as consumers who have
never purchased a home. To ensure that the respondents are a part of the
intended target audience of the disclosures, potential respondents would
also be asked a scale question of whether they intend in the future to
purchase a home and finance the purchase with a mortgage loan. To
further refine the scale of this interest, the question will ask whether the
potential respondent intends to do so in the next 2 years, 5 years, 10 years,
15 years or more, or never. Those who answer “never” will be excluded
from participation in the study because they would not be a part of the
intended target audience of the disclosures.



Experience. This category will be defined as consumers who have
obtained a purchase money mortgage loan or refinanced in the last five
years. To further refine their level of experience, the potential respondents
will be asked to identify how many times they have purchased a home
with a mortgage loan or refinanced a mortgage loan. To ensure that the
respondents are part of the intended target audience, potential respondents
will also be asked whether they intend to do so again in the future. To
further refine the scale of this interest, the potential respondents will be
asked whether they intend to do so in the next 2 years, 5 years, 10 years,
15 years or more, or never. Those who answer “never” will be excluded
from participation in the study because they would not be a part of the
intended target audience of the disclosures.

-6-

Both groups (No Experience and Experience) will be included in the study to increase
coverage of the study to the entire target audience for the disclosures and to allow
comparison of performance based on this experience variable.
Methodology
Recruitment
As described above and in Part A of the Supporting Statement, potential respondents will
be recruited from members of GfK’s KnowledgePanel. Participation is voluntary. If the
potential respondent possesses the desired characteristics and agrees to participate in the
study, a time for the testing is scheduled.
Test Session
Once a test session is scheduled with the potential respondent, it is the responsibility of
the potential respondent to travel to the testing facility where the session will be
conducted. When the potential respondent arrives, he or she will complete the
Respondent Questionnaire (attached as Appendix B) to confirm his or her information.
The respondent is also provided a Privacy Statement and Consent Form (attached as
Appendix B) to sign and a Privacy Act Statement (attached as Appendix E) in a form the
respondents can take with them. Both forms inform the respondent about how the
privacy of the respondent’s identity will be maintained.
As described above, the test sessions will be conducted in groups of 8-12 respondents.
The test sessions will be led by a trained researcher who will act as a proctor. The
trained researcher will read the Proctor Instructions (attached as Appendix F) to the
respondents and be available to answer procedural (not content) questions. Each
respondent will work individually to complete the structured questionnaire. The Form
Type and Loan Type will be randomized so that each respondent in a particular test
session will be in a different cell than the other respondents. No respondent will be able
to “copy” from another respondent and they will be told that each set of materials used by
each respondent in the session is different. Administering the structured questionnaires
in small groups, rather than individually, will enable the Bureau and the Contractor to
maximize time and resources available for the research. The testing sessions will last for
60 minutes. The Contractor’s testing personnel will keep track of the length of testing
sessions to ensure that they do not exceed 60 minutes.
Structured Questionnaire
The study will use the Structured Questionnaire (attached as Appendix C, specifically C1 for the proposed integrated forms and C-2 for the current disclosures, as described in
Part A of this Supporting Statement). The Structured Questionnaire will be divided into
five parts, with each part consisting of its own set of instructions, mortgage loan
disclosures, and questions. The mortgage loan disclosures will be provided to
respondents in different colored envelopes, and each color will correspond to its own part
of the questionnaire. At the beginning of the testing session, the respondent will be
informed that the session will take no more than 60 minutes. The proctors will take 5
minutes to introduce the study and give instructions for completing the questionnaire.

-7-

The first part of the Structured Questionnaire will last 10 minutes. The respondents will
be read a scenario in which the respondents have selected a home they are interested in
purchasing and have contacted two lenders about obtaining a mortgage loan.
Respondents will be asked to review two mortgage loans disclosures that would have
been provided after applying for a mortgage loan (application disclosure). Respondents
will be asked to select the loan they would be most interested in obtaining and to provide
a written explanation of their decision. The second part will also last 10 minutes, and ask
several multiple choice and open-ended questions about the two loans introduced in the
first section. These questions will ask respondents to identify and compare differences
between the two loans. The third part will last 15 minutes and ask multiple choice and
open-ended questions about one of the two loans.
Respondents will then begin the fourth part, when they will be provided with a scenario
that they have selected a loan and are preparing to go to settlement or closing. They are
told that they will receive the disclosure that they would receive around the time they
settle or close on the mortgage loan (final disclosure). The fourth part will last 10
minutes. Respondents will be asked to review one of the application disclosures from the
previous parts and a corresponding final disclosure. The respondent is asked to answer
multiple choice and open-ended questions that ask the respondent to compare the
estimated and final loan terms and costs. The fifth part will last five minutes, use the
same disclosures as the fourth part, and ask the respondents attitudinal questions about
the ease of use of the disclosures.
3. Methods to Maximize Response Rates and Address Issues of Non-Response
To reduce the number of no-shows, scheduled volunteers will be sent a reminder letter
giving the time of the testing session and directions to the testing facility. They will also
receive a reminder telephone call prior to their scheduled time. These are set forth in the
Recruiting Script attached as Appendix A to the Supporting Statement. Over-scheduling
will be used to ensure that an appropriate number of respondents are obtained for each
testing session. At least two additional respondents, as is determined necessary by the
Contractor, will be scheduled for each testing session.
4. Testing of Procedures or Methods
Pretesting of data collection materials is conducted by the Contractor, using volunteer or
Contractor employees as part of their routine job responsibilities.
5. Contact Information for Statistical Aspects of the Design
Questions regarding any statistical aspects employed, data collection procedures used, or
administrative questions regarding the CFPB use of this clearance should be directed to:
Richard Horn, Senior Counsel & Special Advisor, Office of Regulations
Bureau of Consumer Financial Protection

-8-

1700 G St. NW
Washington, DC 20552
Phone: 202-435-7406
Email: [email protected]

-9-