Part B (clean) Update

B. STATISTICAL METHODS

1. Universe and Respondent Selection

As of July 31, 2013 there have been 1,236,682 permanent modifications under HAMP. Of those, 325,491 (or roughly 26%) have re-defaulted. As demonstrated in Table 1 below, this population is comprised of loans which are owned or guaranteed by Fannie Mae or Freddie Mac (“GSE Loans”), and loans which are privately held (“Non-GSE Loans”).

Table 1: HAMP Re-Defaults As of July 31, 2013

	No. of Permanent HAMP Modifications	No. of Re-Defaults
Non-GSE Loans	639,878	175,708
GSE Loans	596,804	149,783
Total	1,236,682	325,491

While the Administration considers HAMP to be a single, comprehensive program, the management occurs separately for Non-GSE Loans and GSE Loans. Treasury sets program requirements for modifications of Non-GSE Loans, whereas Fannie Mae and Freddie Mac set program requirements for modifications of GSE Loans.

The target population for Treasury’s study is homeowners with Non-GSE Loans who have received permanent mortgage modifications under HAMP and subsequently missed three consecutive payments (“re-defaulted”). In order to have a large enough sample of homeowners to survey who have recent experience with the issue, the universe of potential respondents will be limited to those who missed their third consecutive payment on or after August 1, 2011.

The sample for the study will be stratified by two criteria: 1) Homeowner’s change in payment between their pre-modification payment and modified payment as a percentage (P&I Change), and 2) Homeowner’s delinquency status at the start of the HAMP modification’s trial period (Delinquency at Trial). Prior research has shown that these two variables significantly relate to subsequent HAMP performance. These criteria will be further broken down into six groups as described in the example provided in Table 2: Desired Response Broken Down by Re-Default Target Homeowners Sampling Categories.

Treasury’s goal is to collect a total of 2400 survey responses, with 400 responses in each group.

Table 2: Desired Response Broken Down by Re-Default Target Homeowners Sampling Categories

	P&I Change
Delinquency at Trial	<=24%	>24% - <=42%	>42%	Total
<= 6 months	400	400	400	1200
> 6 months	400	400	400	1200
Total	800	800	800	2400

The above subgroups are defined to mirror the total HAMP re-default population. For example, the ‘6-month” cut-off point will approximately evenly split the HAMP re-default population. This sampling stratification strategy will help ensure that the completed survey sample well represents the universe population with more precision. This will also help reduce sampling biases as certain subgroups might be less or more likely to respond to the survey than other subgroups.

In terms of sampling, the universe of potential respondents will be classified into the six groups and then, via random sampling, survey participants will be selected for each group to receive the survey.

The population of homeowners with Non-GSE Loans who re-defaulted on their HAMP modifications on or after August 1, 2011 is approximately 100,000 homeowners. Among these, about 40,000 homeowners re-defaulted over the past 12 months. In order to focus on homeowners with recent experiences the survey samples will be first drawn from the more recent re-default population.

At the 95% confidence level the margin of error is 1.94% and 1.98% respectively, when the total of 2400 completed sample is drawn from the most recent re-default population of 40,000 homeowners and the total universe of 100,000 homeowners, respectively.

Power Analysis

Assuming an effect size of 0.1 (or 10 percentage points difference on proportional data such as testing the difference between 50% saying Yes from Group A and 60% saying Yes from Group B), a sample size of 400 per group will allow us to statistically test this 0.1 difference (two-tailed test) at the 95% confidence level (alpha = 0.05), yielding a statistical power of 0.81. In addition, the statistical power will increase to nearly 100%, if we examine differences at the higher group level such as comparing the [<=6 months delinquency at trial] and [>6 months delinquency at trial] groups, with 1200 sample size per group.

Non-Response Bias

This study will pursue the following methods in reducing non-response biases:

1. We will compare the responses group and non-responses group on selected data from the HAMP system of record such as Loan-to- Value (LTV) ratio pre-modification and post-modification, Debt-to-Income (DTI) ratio pre-modification and post-modification, and credit score at modification to assess if borrowers with certain observable characteristics are more or less likely to respond.

2. Should significant differences be observed, we will create non-response adjustment weights and apply the weight to each bucket of “similar” loans, in order for the completed survey sample to represent the target universe.

2. Procedures for Collecting Information

Research Method

Names and addresses of homeowners who have received a HAMP modification are the only contact information we have available within the HAMP system of record. As a result, to optimize the response rate, a paper survey will be mailed to the target homeowners who have been randomly selected from the groups noted in Table 2, above. The survey will be mailed with a pre-paid return envelope and a cover letter inviting borrowers to participate. The survey is estimated to take approximately 10-15 minutes to complete. The cover letter will also provide a unique web address for an online version of the survey to allow respondents the option of completing the survey online instead of on paper. In order to lessen the likelihood of respondents completing multiple survey responses, each survey recipient will be assigned a unique identifying number which will be printed on the paper survey they return. If they elect instead to complete their response online, the interface will require them to enter their unique number in order to gain access. Checking the identifying numbers on responses received against the list of survey recipients will allow Treasury to identify and exclude any duplicate responses.

Survey participation will be completely voluntary. Respondents will not be asked for any personally identifying information, and will be instructed not to provide any extraneous text or responses on the survey form other than to select one of the multiple choice options present following each survey question. Analysis and reporting will be conducted at the aggregate level, with no personal information identified or reported.

The survey will be designed and constructed to meet the study’s research objectives of obtaining an understanding of the following: 1) Homeowner’s initial understanding of their modification and ability to pay the modified payments, 2) Homeowner’s change in circumstances impacting their ability to pay the modified amount, and 3) Homeowner’s perceptions of the MHA Program, process and outcomes.

Test Run followed by the Principal Mailing

In order to decrease the burden and minimize the number of homeowners that will need to be solicited the data collection will be conducted in two phases: 1) Test Run and 2) Principal Mailing.

A Test Run will be conducted to assess the likely homeowner response rate, and therefore inform the number of additional mailings needed for the Principal Mailing to achieve the target of 2,400 total responses. The Test Run will consist of mailing to 2,000 homeowners selected at random, a survey, a pre-paid return envelope, and a cover letter inviting them to complete and return the survey via paper response or use the provided web link to complete the survey online.

The results of the Test Run will be analyzed to determine the effectiveness of the survey materials to solicit a homeowner response, and to estimate the potential response rate that will result from the Principal Mailing. Based on this analysis, appropriate updates to the survey materials will be incorporated and approvals obtained, and a sample size of target homeowners to which the survey will be mailed for each of the six groups described in Table 2 will be established. This will maximize the likelihood of receiving the target 400 completed responses in each group, while minimizing the cost of the Principal Mailing.

Data Analysis/Reporting

In order to provide robust insights, data collected from the survey responses will be augmented with data points from the HAMP system of record. In addition to data points used from the HAMP system of record such as delinquency at trial and P&I change used to determine sampling categories, data points such as Loan-to-Value (LTV) pre-modification, LTV post-modification, Debt-to-Income (DTI) pre-modification, DTI post-modification, credit score, amortization term after modification, amount of principal forgiveness and/or forbearance may be used. None of the raw survey data, data points from the HAMP system of record, the analysis, or the report will contain personally identifiable information.

If the volume of responses received appears unlikely, in the Department’s judgment, to reach the desired 2,400 responses divided among the groups listed in Table 2 within a reasonable time, the Department may consider restructuring the analysis based on the actual number of responses received.

Research Ethics

Data collection and research protocols will be based on the CASRO Code of Standards and Ethics for Survey Research (http://www.casro.org/?page=TheCASROCode), developed by the Council for Survey and Research Organization (CASRO), which has long been the benchmark of the industry and provides a consistent, industry-accepted standard for implementing similar studies.

Research Timeline

Event	Target Dates
Study of MHA Program Performance
Complete development of survey materials (cover letter, survey questionnaire , FAQs)	Sept 20th
Receive Approval of survey materials and OMB Control Number	Sept 27th
Print survey materials for Test Run	Sept 30th
Mail the Test Run	Oct 4th
Review responses and print documents for Principal Mailing	Oct 21st
Mail the Principal Mailing	Nov 1st
Begin tabulation of responses	Nov 18th
Delivery of the Final Report	Dec 18th

Note that the foregoing timeline reflects a “best case” scenario. For example, if we cannot commence the Test Run on-time or must revise the survey materials based on the results of the Test Run, the Principal Mailing may be delayed until January 2014 due to concerns of low response rates during the holiday season.

3. Methods to Maximize Response

In order to achieve the desired target of 2400 respondents, the following methods will be used to maximize response rates:

• The cover letter and survey will be printed on Treasury letterhead, be signed by a Treasury official, and be sent in an envelope marked with the Treasury logo.

• The cover letter will be drafted in an engaging way that focuses on understanding the consumer mortgage process to help provide better assistance to all struggling homeowners.

• The survey will be drafted to minimize the likelihood of even inadvertent disclosure of personally-identifiable information.

• An official website of the MHA (“Making Home Affordable”) Program will refer to the ongoing survey activity so participants can confirm its legitimacy.

4. Testing of Procedures

A Test Run will be administered to 2,000 potential respondents before launching the Principal Mailing to a larger target homeowner population. This Test Run will allow the Treasury to get a better sense of the response rate, to inform the number of mailings needed to achieve the desired sample size. In addition, after the Test Run, the Treasury will review the results and have an opportunity to revise the cover letter and/or the survey for clarity, accuracy, appearance, and tone, in order to minimize cost and achieve success with the Principal Mailing.

5. Contacts for Statistical Aspects and Data Collection

• [email protected]