OMB Submission Part A Final (3.5.13)

The Department of Housing and Urban Development (HUD) seeks to understand the impact of pre-purchase homeownership counseling on a range of outcomes for low-, moderate-, and middle-income (LMMI) , first-time homebuyers. In light of the recent collapse of the U.S. housing market and the precarious nature of the ongoing recovery, such an understanding is critical to policymakers as well as to mortgage lenders and housing counseling organizations.

The Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation will examine the impact of different types of pre-purchase counseling and education on homeowner outcomes and loan performance for prospective first-time homebuyers with incomes below 120 percent of the area median. This evaluation will measure the impact of two different pre-purchase counseling and education interventions through a randomized experimental design.

There are many potential benefits of pre-purchase counseling and education.¹ Informed consumers are expected to choose more appropriate homes and mortgages, improve their overall financial situation, and (over the long term) experience lower levels of mortgage delinquency and exit from homeownership. Pre-purchase housing counseling may also play an important role for households who experienced a foreclosure and are preparing for a new home purchase.

Despite the widespread use of housing counseling, these benefits have been difficult to document. Because most housing counseling programs are small scale and offer a wide range of services to many different clients, robust comparisons are difficult. “Counseling” is also varied, as content and delivery mechanisms differ across programs and providers. Moreover, the direct effects of borrower education are often hard to distinguish from other factors such as borrower self-selection or sorting (i.e., more self-motivated homebuyers are likely to approach agencies for counseling services) or changes in lender behavior related to housing counseling (i.e., certain mortgage products are only available if counseling is completed).

The absence of conclusive empirical evidence regarding the effectiveness of pre-purchase housing counseling is detrimental to policy discussions about federal support for housing counseling and the value of counseling requirements. Lenders’ use of housing counseling requirements varies. About 15 years ago, many lenders—including Fannie Mae and Freddie Mac—made the receipt of housing counseling a requirement for loan products targeted to lower-income, lower-wealth borrowers. This decision reflected a belief that pre-purchase housing counseling created value for the lender and a more stable housing situation for the borrower.

The current study represents an important effort to measure the impact of pre-purchase housing counseling using experimental evaluation methods. Whereas many of the previous studies have been limited by small sample sizes and non-experimental designs, this evaluation will recruit a large sample for an experimental impact evaluation in which individuals are randomly assigned to receive housing counseling or not. This design will produce impact estimates with strong internal validity with respect to each of the following modes of pre-purchase housing counseling:

Remote Counseling: Online education + telephone counseling
In-Person Counseling: Classroom-based education + in-person counseling

These modes of housing counseling reflect the most common types of pre-purchase housing education and counseling currently available. The curriculum used for each housing counseling mode will be consistent with the National Industry Standards for Homeownership Education and Counseling.²

The data collection activities for which OMB approval is requested in this submission consist of data to be collected from lenders, study participants, and HUD-approved housing counseling agencies during the enrollment and implementation phases of the study as described in A.2.A second OMB approval request, under a separate contract between HUD and the study team, will cover the follow-up participant interviews that will collect the primary outcome measures for impact analysis.

How and by Whom the Data will be Collected and Used
1. Project Overview

The study design will be a randomized experiment. We will recruit up to 7,000 LMMI, first-time homebuyers in 28 jurisdictions (cities) nationally. Enrolled study participants will be assigned to one of three groups: remote housing counseling (online education + telephone counseling), in-person housing counseling (classroom-based education + in-person counseling), or a control group that does not have access to these housing counseling services. Implementation of this study in the field requires that the study team coordinate with both lenders and HUD-approved housing counseling agencies:

Lenders: The study team has partnered with three national lenders—Bank of America, Citibank, and Wells Fargo—for the implementation of this study. The participating lenders were chosen purposively, based on their volume of loan originations and willingness to partner for this study. Study participants will be recruited through the lenders’ centralized, telephone-based customer service teams. The telephone customer service teams with each lender will introduce the study to customers, referring interested customers to the study team.

HUD-Approved Housing Counseling Agencies: The study team will refer study participants in each treatment group to HUD-approved housing counseling agencies to receive housing counseling services. The housing counseling agencies include one national remote housing counseling provider and local affiliates of a national in-person housing counseling intermediary. The study team selected CredAbility as the provider of remote housing counseling for the evaluation. For in-person housing counseling, the study team is working with NeighborWorks America and will recruit local NeighborWorks’ affiliate organizations to provide in-person housing counseling in each jurisdiction. However, depending on NeighborWorks’ coverage, the study team may explore other housing counseling agencies that fit the study’s needs in each jurisdiction. The housing counseling agencies will document all services provided to study participants.

The study team is responsible for all other study activities—as well as for monitoring and coordinating the activities of the lenders and housing counseling agencies.

The recruitment and enrollment process includes several steps. Following lender intake and referral, the study team will conduct the eligibility assessment, consent process, baseline survey, and random assignment. While lenders will apply screening criteria to determine whether to offer participation in the study, the eligibility assessment will confirm the customer meets all study eligibility requirements. The study team estimates that 40 percent of customers referred by participating lenders will be contacted by the study team, meet all study eligibility requirements, and agree to study participation. Therefore, we expect to screen approximately 17,500 customers for participation in this study.

The eligibility criteria are:

Customer has made contact with one of three participating lenders in one of the 28 selected study jurisdictions about a purchase mortgage;
Customer seeks a purchase mortgage for an owner-occupied home;
Customer has not owned a home in the previous three years;
Customer is low-, moderate-, or middle-income: total income of (co)borrowers is less than 120 percent of Area Median Income (AMI);
Customer has not previously completed pre-purchase housing counseling;
Customer does not already have a housing counseling requirement from a down payment assistance provider or any other source;
Customer has regular access to a computer and the internet;
Customer has reliable access to transportation; and
Customer has English or Spanish competency.

Customers who meet these eligibility requirements will complete the consent process and baseline survey with the study team. Once the baseline survey is completed, the study participant will be randomly assigned to receive one of the two housing counseling interventions or to the control group. The study team will work with housing counseling agencies to monitor participants’ initiation and completion of the housing counseling intervention. HUD-approved housing counseling agencies will provide information on the services provided to study participants for 12 months following the enrollment of the final study participant. This service tracking data will permit the study team to monitor participants’ completion of the intervention and will provide a key source of information for the analyses of housing counseling impacts.

The study team seeks to recruit up to 7,000 study participants. The study team’s estimated conversion rates for each step of the recruitment process imply that this sample size requires lender outreach to approximately 87,500 customers. The waterfall below shows the study’s estimated conversion rates for each step of the recruitment process, which are based on the projected conversion rates developed by the survey managers responsible for each step. These figures are estimates, and the study team will stop recruitment if 7,000 study participants complete enrollment before the end of the enrollment period. The estimated recruitment volumes and recruitment rates include:

Lender Outreach: Lenders’ identify approximately 87,500 customers as potential study participants and initiate contact.
Lender Referrals to the Study Team: Lenders successfully contact and obtain consent to share contact information for 17,500 customers (~20% conversion rate).
Enrolled Study Participants: The study team successfully contacts the referred customer and completes the eligibility assessment, consent process, and baseline survey with 7,000 study participants (~40% conversion rate).

The study seeks OMB clearance for data collection for 7,000 study participants. The study has the resources and funding available to cover this number of study participants. However, given the number of first-time homebuyer applications that participating lenders have seen over the past 18 months, the study is more likely to recruit 6,000 study participants. Therefore, the submitted study materials that are attached to this OMB package note the study having 6,000 study participants. The study’s statistical power under each sample size is described in Part B.

The study team will also conduct an implementation study to document the enrollment and implementation experience of the evaluation. The implementation study will draw upon data collected from site visit interviews and observations that will take place during the first 16 weeks of study participant enrollment. The implementation study will also make use of information from the study team’s monitoring activities that will occur throughout the enrollment period. The enrollment period will last up to 24 months. Additionally, the implementation study will include telephone interviews with a small subset of study participants that will collect information on their experience with the enrollment process and reasons for completing (or not completing) housing counseling. The study team will only select for the interviews study participants who have either completed the housing counseling intervention or have not initiated contact with their assigned housing counseling agency. No study participants will be selected for the interview that are in the midst of completing housing counseling.

The study will also include large-scale follow-up surveys with study participants to document outcomes after enrollment and completion of housing counseling. These follow-up surveys will be covered by a later OMB submission.

Data collection activities for which approval is requested in this submission consist of data to be collected from study participants, lenders, and housing counseling agencies during the enrollment and implementation phases of the study. The data to be collected from each party are summarized in Exhibit A-1and described below.

Exhibit A-1. Data Collected from Lenders, Study Participants, and Counseling Agencies

Respondent Type	Data to be Collected
Lenders	Lender intake data Lender interviews and observations Loan origination and performance data
Participants	Eligibility assessment Consent agreement Baseline survey Co-borrower informed consent agreement Implementation study telephone interview Study participant tracking
Housing Counseling Agencies	Service tracking data Housing counseling agency interviews

Lender Intake Data: The evaluation will rely on lenders providing data (names and contact information) on customers who meet the eligibility criteria for participation in this study. Lenders will screen for customers who are LMMI, first-time homebuyers who are in the early stages of the borrowing process. Lenders will provide the study team this information on potential study participants either by data extracts or by entering information into the study’s data system.

Study Participant Eligibility Assessment and Consent Agreement: Prior to administering the study’s participation consent agreement to eligible customers who are interested in participating in the study, the study team (Abt Associates’ survey subsidiary group) will administer an eligibility assessment to verify that the customer meets the definitions of LMMI and first-time homebuyer. The eligibility assessment will also be used to determine whether the customer meets the study’s eligibility criteria in four other areas:

Prior housing counseling. Because enrolled participants may be randomly assigned to the control group, we will exclude any recruited customers who have completed activities associated with a pre-purchase education or counseling program or who face a housing counseling requirement.
Internet and transportation access. Because participants may be randomly assigned to receive on-line counseling or in-person counseling, we also will exclude those who do not have access to the internet or to transportation.
Language. We will exclude those who would not be able to respond to a survey or read curriculum materials in either English or Spanish.

Once customers' eligibility has been determined, the study team will read them a statement that describes the study and their role in the study. If they are still interested in participating in the study, we ask them for a verbal consent and record that consent authorization at the end of the eligibility assessment.

While capturing verbal consent from the study participant, the study team will ask the participant to provide their social security number. Collection of social security numbers is necessary to track and assemble origination and servicing data from the Federal Housing Administration, lenders, and credit bureaus over the life of the study. The last four digits of the SSN also allow the study team to authenticate the identity of the respondent during follow-up surveys.

Study Participant Baseline Survey: After the eligibility assessment and consent have been completed, the baseline survey will be administered. The baseline survey is the primary source of information on participant characteristics prior to their participation in housing counseling and their purchase of a home. It has two purposes:

1. Document the preparedness for home purchase of participants at the time of study entry. The baseline survey collects information that reflects the respondent’s stage in the home purchase process and preparedness for home purchase. For example, the extent of respondents’ home search prior entering the study, their intended down payment amounts, and their expected timelines for home purchase will provide information about the timing of random assignment and housing counseling relative to other steps in the purchase process.

2. Collect baseline characteristics of participants. The baseline survey collects baseline measures for each of the time-varying outcomes. For example, financial literacy, non-housing savings/debts, and budgeting activity all reflect outcomes for which a baseline measure prior to housing counseling will help to evaluate the impact of counseling. The baseline survey also collects the set of baseline participant attributes that may be used as covariates in the eventual impact analyses. The attributes collected must include any attributes used to segment the sample into subgroups for analysis.

Study Participant Co-Borrower Informed Consent Agreement: If a study participant secures a home loan with a co-borrower from a study participating lender, then the study team needs to obtain consent from the co-borrower to collect information on that home loan’s origination and performance data. The study team will collect co-borrower consent either at the time the study participant is providing consent for study participation or during a follow-up telephone call.

Service Tracking Data from HUD-Approved Housing Counseling Agencies: To provide the study team with essential information on the pre-purchase education and counseling services completed by study participants, housing counseling agencies will input or upload data into the study’s data system on housing counseling participation and intensity. This will provide the primary source of information on treatment take-up, intensity, and completion. Housing counseling activities will continue throughout the contract period (and into the subsequent study period, to be funded under a separate contract). The study’s data system will document these activities throughout the study period, allowing the data to be extracted and analyzed at any point.

Staff Interviews with Housing Counseling Agencies: Staff interviews at housing counseling agencies will document the implementation procedures and activities conducted at each organization and gather staff members’ input about the issues involved in enrollment and implementation. These interviews will occur within the first 16 weeks of study participant enrollment.

On-site Observations and Interviews with the Lenders’ Customer Service³ Teams: Observations with the lenders’ customer service teams will document how they have been trained to conduct study recruitment calls with customers. Interviews with the customer service teams will document their feedback on the recruitment calls, noting how the study is perceived by customers and if customers are interested in learning more about the study. The study team will also ask each lender’s customer service team to document the outcomes of the recruitment calls and provide us with a weekly report.

Study Experience Interviews with a Small Subset of Study Participants: During the final weeks of the enrollment period, we will conduct 200 study participant telephone interviews across the 28 study sites. Interviews will explore participants’ progress towards home purchase and their interaction with the participating lenders and housing counseling agencies. Selected study participants will be asked about the processes of study recruitment, enrollment, and participation in housing counseling services. Selected study participants will include participants who have and who have not completed housing counseling activities to explore participants’ decisions to complete housing counseling activities. These study participant interviews will aid in our understanding of the study’s implementation.

Loan Origination and Performance Data: Study participants’ loan information and performance data will be requested at six month intervals throughout the study period from participating lenders and the Federal Housing Administration. This data will provide the study team detailed information on the timing of a home purchase, underwriting characteristics, monthly housing costs, and mortgage terms and conditions for any study participant who purchases a home through a study participating lender.

Credit Bureau Data: Collection of credit information during the enrollment period is necessary both to monitor the credit score distribution of enrolled participants and as a key baseline measure of participant credit quality. To accommodate this, the study team will request Equifax to provide credit report data on study participants. Specifically, the study team will purchase credit report data at six different points during study participant enrollment.

Purpose of Data Collection

We are requesting OMB approval for 11data collection activities that will occur with study participants as well as study participating counseling agencies and lenders:

Collection of lender intake data on potential study participants;
Collection of customers’ information from the study’s eligibility assessment and participation consent agreement;
Collection of study participants’ co-borrower consent;
Collection of study participants’ baseline survey;
Collection of data on the intervention and on implementation of the study through interviews with study participants;
Collection of study participants’ housing counseling service data from HUD-approved housing counseling agencies;
Collection of data on the intervention and on implementation of the study through interviews with staff from housing counseling agencies;
Collection of data from observations, interviews, and reports from the lenders’ customer service teams;
Collection of contact data from study participants;
Collection of study participants’ credit scores and credit report data through Equifax; and
Collection of data on study participants’ loan origination and mortgage performance.

The purpose of each data collection activity is described below.

Collection of lender intake data on potential study participants:

The first step in the enrollment process is for lenders’ staff to determine whether a customer meets the initial criteria to be offered the study. The key criteria such as income will be collected through each lender’s normal protocol for early-stage customers. To supplement this information, the lender may need to ask questions to verify that the customer is a first-time homebuyer, seeks a mortgage loan to purchase a home to occupy as a primary residence, and is interested in being referred to the study interviewers for the official eligibility assessment and study enrollment. The lender intake form is provided in Appendix A.

Collection of customers’ information from the eligibility assessment and participant consent agreement:

The information collected from the eligibility assessment will be used to verify that a customer meets the definitions of a LMMI and first-time homebuyer. Additionally, the eligibility assessment will determine whether the customer has received any pre-purchase counseling prior to study enrollment or currently faces a counseling requirement. This assessment will also support screening out a customer based on lack of English or Spanish language competency, transportation availability, and accessibility of a computer connected to the internet. The eligibility assessment form is provided in Appendix B.

Once eligibility is determined, the consent agreement will be read to the customer, and for those who provide a verbal consent to participate in the study, the consent will be collected. The contact and identifying information (social security number and date of birth) collected in the study’s participation consent agreement will be used to contact and track study participants throughout the life of the study. The study participants’ social security numbers will be collected to ensure that study participants can be accurately identified in origination, servicing, and credit bureau records. The consent form is provided in Appendix C.

Collection of study participants’ co-borrower consent:

For study participants who originate a loan with one or more co-borrowers, the co-borrowers’ consent will be obtained and provided to lenders to collect home loan origination and loan performance data. In order to receive this information from lenders participating in this study, we need consent from each person who is listed on a study participant’s home loan. The co-borrower’s consent form is provided in Appendix D.

Collection of study participants’ baseline survey:

The baseline survey will be used to collect baseline information on study participants that will be useful for several parts of the study. The impact analyses will rely on the baseline survey for information on covariates, subgroups, and baseline measures of outcomes. Additionally, the baseline survey includes the measures necessary to describe the study sample and examine the random assignment of study participants across groups. The baseline survey is provided in Appendix E.

Collection of data on the intervention and on implementation of the study through interviews with study participants:

Data collected through interviews with study participants will explore participants’ progress towards home purchase and their interaction with the participating lenders and counseling agencies. Interview participants will be asked about the processes of applying to a lender, study recruitment, enrollment, and participation in counseling services. The participant interview guide is provided in Appendix F.

Collection of study participants’ housing counseling service data from HUD-approved housing counseling agencies:

The data collected from housing counseling agencies will serve as the primary source of information on housing counseling take-up, intensity, and completion. The housing counseling agency service tracking data collection module is provided in Appendix G.

Collection of data on the intervention and on implementation of the study through interviews with staff from housing counseling agencies:

Data collected through the housing counseling agencies’ staff interviews will document the implementation procedures and activities conducted at each organization and gather staff members’ input about the issues involved in implementation. This information will also document feedback from housing counseling agencies on whether there are problems with the implementation of the study design and the extent to which the study is implemented as designed. The housing counseling agency staff interview guide is provided in Appendix H.

(8) Collection of data from observations, interviews, and reports from the lenders’ customer service teams:

Data collected through observations and interviews with the participating lenders’ customer service teams will document how staff have been trained to conduct study recruitment telephone calls with customers and how they are implemented. Additionally, follow-up interview questions will allow the study team to document the customer service teams experience making recruitment telephone calls, specifically how the study is perceived by customers and if customers are interested in learning more about the study. The lender, observation and interview guide, and lender recruitment outcome report are presented in Appendices J, and K.

Collection of contact information from study participants.

Every four months, the study team will mail study participants a letter asking for updated contact information. If their contact information has changed since they agreed to participate in the study, study participants simply need to update the form and mail it back. The study participant tracking letter is provided in Appendix L.

Collection of study participants’ credit bureau data from Equifax:

Credit score data obtained from Equifax will provide the study team the ability to monitor the credit score distribution of enrolled study participants and a key baseline measure of study participant credit quality.

Collection of data on study participant’s loan origination and mortgage performance:

Data collected from lenders and the Federal Housing Administration on study participants loan origination and mortgage performance will document detailed information on the timing of a home purchase, purchase price and down payment, monthly housing costs, and mortgage terms and conditions. The loan origination and performance data collected from lender are presented in Appendix M.

Additionally, there are 7 appendices that will be included in this submission that are not data collection instruments:

Appendix N contains the Bank of America concierge team’s study participant recruitment script.
Appendix O contains the Bank of America concierge team’s study participant voicemail script.
Appendix P contains the Wells Fargo/Maritz team’s study participant recruitment script.
Appendix Q contains the Citibank study participant recruitment script.
Appendix R contains the study recruitment letter to local HUD-approved housing counseling agencies.
Appendix S contains samples of the random assignment letter to study participants.
Appendix T contains the samples of the outreach letter to non-responsive study participants.
Appendix U contains language for a study overview document that will be sent to potential study participants.
Appendix V contains language for a letter sent to potential study participants.
Appendix W contains a sample brochure that will be sent to potential study participants and used by lenders at their branch locations and community events.

Who Will Use this Information

The primary beneficiary of the planned data collection will be HUD, which will use the information from the study to understand the impact of pre-purchase homeownership education and counseling on a range of outcomes for LMMI , first-time homebuyers. The findings of the eventual impact analyses will be published by HUD and are relevant to a broader set of policymakers, financial institutions, counseling agencies, and researchers.

Instrument Item-by-Item Justification Chart

Exhibit A-2 describes the target respondents, content, and reason for inclusion for each data collection activity that involves study participants and staff from participating lenders and counseling agencies. This chart only lists items that are data collection instruments. Copies of the data collection instruments are provided as Appendices.

Exhibit A-2. Item-by-Item Justification of Data Collection Instruments

Data Collection Activity	Data Collection Instrument(s)	Respondents, Content, and Reason for Inclusion
Intake Information on Potential Study Participants	Lender Intake Form (Appendix A)	Respondents: Lenders’ staff at 3 national study-participating lenders. Content: Customer’s first and last name; Customer’s address; Customer’s telephone number and email address; Customer’s date of birth; Date customer first contacted lender about home loan; Date customer gave consent for contact information to be sent to the study team;; and, Unique customer identifier Language preference Reason: The collection of these data is necessary to identify potential study participants that meet eligibility criteria. This information is also necessary to link each enrolled study participant to lenders’ administrative data on prequalification/application, origination, and loan performance.
Study Participant Eligibility Determination and Consent to Participate	Eligibility Assessment(Appendix B) and Participant Consent Agreement (Appendix C)	Respondents: Eligibility Assessment: 17,500 (anticipated); Consent Form: 7,000 LMMI , first-time homebuyers who are interested in pursuing a loan for home purchase through study participating lenders. Content: Customer’s first and last name; Customer’s contact information (address, email, telephone); Customer’s social security number; Customer’s date of birth; Customer intends to use loan to purchase an owner-occupied home; Customer meets income threshold; Customer is first-time homebuyer; Customer has regular access to a computer and the internet; Customer has reliable access to transportation; Customer has English or Spanish competency; Customer has not previously completed pre-purchase homeownership counseling and/or an education course; and, Customer is not subject to a housing counseling requirement Reason: The collection of this information is necessary to determine if customers are eligible to participate in this evaluation. The collection of this information is linked to a consent agreement that documents that customers have been informed about the study and have agreed to participate in the study.
Co-borrower’s Consent to Share Data	Co-borrower’s Consent (Appendix D)	Respondents: Any person who is a co-borrower on a study participant’s home loan. (The number of respondents for this activity is undetermined at this time. However, the study team estimates the number of co-borrowers to be approximately 30 percent of the study’s sample or 2,100 respondents.) Content: Co-borrower’s first and last name;Reason :In order for the study team to receive information on study participants’ loan originations and performance data, lenders need consent from all borrowers on the home loan to share this information.
Study Participant Baseline Survey	Baseline Survey (Appendix E)	Respondents: 7,000 study participants across 28jurisdictions Content: Demographic characteristics (e.g., race/ethnicity, gender, marital status) and related preferences and tendencies (e.g. financial decision-making, self-efficacy, learning styles, follow-through); The stage in the home purchase process and preparedness for home purchase (e.g., the extent of respondents’ home search prior to pre-qualification, intended down payment amount, expected timeline for home purchase); and Baseline measures of time-varying study outcomes (e.g., income sources and total family income, financial literacy, non-housing savings/debts, budgeting activities, and employment and earnings). Reason: The baseline survey is the primary source of information of participant characteristics prior to their participation in housing counseling and their purchase of a home. It has two purposes: To document the preparedness of participants at the time of entering the study; and, To collect baseline characteristics of participants that will be used as covariates, subgroups, and baseline outcome measures in the impact analysis
Study Participant Experience Interviews	Participant Experience Interviews on Implementation (Appendix F)	Respondents: 200 study participants across 28 study jurisdictions Content: Participant’s experience with the study’s recruitment and enrollment processes; Random assignment of housing counseling services and whether or not study participants sought and received them; Opinions on the pre-purchase housing counseling and education services study participants may have received; and Participant’s stage in home purchase process. Reason: Interviews will explore participants’ interaction with participating lenders and housing counseling agencies, including participation in housing counseling services. Respondents will include study participants assigned to a treatment group who have and who have not completed housing counseling activities to explore their decisions to complete housing counseling activities. Interviewees will also include study participants who have and have not purchased a home to understand how housing counseling played into the home purchase decision. These interviews are necessary for our understanding of study implementation, as well as why some participants decided to complete or not complete pre-purchase housing counseling and education.
Housing Counseling Agency Service Data	Service Tracking Data Collection Module (Appendix G)	Respondents: Housing counseling agency staff who record and enter data on services received by study participants. (The number of housing counseling agency staff will be determined once all housing counseling agencies have been selected and recruited into the study.) Content: Name and contact information for participant; Date of birth for participant; First contact date with counseling agency; Number of times study participant signs into web-based educational module (online education only); Type of session; Date of session; Number of participants in session; Length of session time; Topics covered; Counseling agency staff member who taught/led session; Action steps before, during, or after session; Percent of educational curriculum completed; Score of online module testing; Outcome(s) of session; and Recommendation(s) to study participant. Reason: This service tracking data will create a detailed record of the housing counseling and education services provided to study participants. The compiled data will provide the basis for description of the housing counseling interventions in the final report. A detailed understanding of the housing counseling services received by treatment group members is central to interpreting the evaluation outcomes and impact estimates.
Interviews with Housing Counseling Agency Staff	Housing Counseling Agency Interview Guide (Appendix H)	Respondents: Front line and management staff at participating HUD-approved housing counseling agencies (The number of housing counseling agency staff will be determined once all housing counseling agencies have been selected and recruited into the study.) Content: Information on housing counseling agency mission, populations served, services provided, and funding; Feedback on training provided by study team on study protocols, random assignment interventions, and service tracking data collection; Experience with contacting study participants; and Description of pre-purchase education and housing counseling provided to study participants. Reason: The collection of these data is necessary to document feedback from housing counseling agencies on whether there are problems with the implementation of the study design and the extent to which the study is implemented as designed. Additionally, these interviews will gather information on pre-purchase housing counseling and education services offered to study participants.
Observations and Interviews with Customer Service Teams with Lenders	Lenders’ Customer Service Team Observation Guide (Appendix J)	Respondents: Customer service team members with participating lenders Content: Observations made by the study team during training; Observations made by the study team during study recruitment calls to customers; and, Documenting responses on the experiences of the customer service team members while making recruitment calls Reason: The collection of this information is necessary to document the training and implementation of study recruitment calls to customers, as well as documenting the experiences of conducting this study activity.
Outcomes of Lenders’ Customer Service Teams Study Recruitment Telephone Calls	Lenders’ Recruitment Call Outcome Report (Appendix K)	Respondents: Study participating lenders’ customer service teams Content: Weekly outcomes of study recruitment telephone calls; and, Total disposition of study recruitment telephone calls Reason: The collection of this information is necessary to document the outcomes of the study recruitment telephone calls made by the lenders’ customer service teams. The information in this report will provide the study team with data on the number of customers who were interested in learning more about the study, the number of customers who were not interested in learning more about the study, and the number of other dispositions that the customer service team received.
Tracking of Study Participants	Study Participant Tracking Letters (Appendix L)	Respondents: 7,000 study participants across 28 jurisdictions Content: Updated contact information for the study participant and a person outside of the participant’s household who would know how to locate the participant Reason: The collection of this information is necessary to confirm that the study team’s contact information is accurate and we will be able to reach him or her for the study’s follow-up data collection activities.
Collection of loan origination and mortgage performance data from participating lenders	Loan Origination and Performance Data from Lenders (Appendix M)	Respondents: Lenders’ staff that provide the study team data on loan originations and mortgage performance for study participants’ home loans. Content: Loan number; Date of closing; Purchase price; Monthly income; Back end ratio; Front end ratio; Down payment; Initial LTV; Mortgage term; Mortgage type; Interest rate; Closing costs and fees; Ever 30 days delinquent; Ever 90 days delinquent; Prepayment; Foreclosure; Bankruptcy; and Updated mortgage balance Reasons: Data collected from lenders and the Federal Housing Administration on study participants loan origination and mortgage performance will document detailed information on the timing of a home purchase, purchase price and down payment, monthly housing costs, and mortgage terms and conditions.

Use of Improved Technologies

Improved information technology will be used in this evaluation in three distinct ways:

To maintain data in a single location on all eligibility assessments, random assignments, loan pre-qualifications and housing counseling services received;
To assist the ongoing sample tracking and locating efforts; and
To facilitate collection of the survey data in standardized and accurate ways that also ensures the protection of the confidentiality of sensitive data.

The two main technologies that will be used are the Random Assignment and Service Tracking (RAST) system created for the study and CATI interviewing technology.

Random Assignment and Service Tracking System (RAST)

The study will generate a substantial amount of data, including eligibility assessments of potential study participants, random assignment records, and information provided by lenders and housing counseling agencies. A random assignment and service tracking system (RAST) will be developed to manage the various sources of data. The database will house information from lenders, housing counseling agencies, and study participants throughout the life of the study.

A key function of the RAST system is to support the efforts of the study team to monitor housing counseling take-up and track study participants over the course of the study period. Housing counseling agencies will upload or enter servicing tracking data directly into the RAST system, allowing study team members to ensure that study participants are completing assigned housing counseling activities. This will allow the study team to actively track study participants’ completion of housing counseling activities and to promptly conduct outreach to study participants who have not completed housing counseling activities. For such participants, site liaisons will contact the participant by phone, email, and mail regarding completion of housing counseling service.

A second function of the RAST system is to support the tracking of study participants over the course of the study to ensure a high response rate for future surveys. Throughout the study, the study team will passively track study participants through the U.S. Postal Service Change of Address database. Also, every four months, the study team will mail study participants a letter asking for updated contact information. If their contact information has changed since they agreed to participate in the study, study participants simply need to update the form and mail it back.

The RAST system will store all tracking updates we collect, and link them to the participants. Through active and passive tracking, the study team will be able to monitor both counseling service completion and change in location of study participants.

Computer-Assisted Telephone Interviewing (CATI)

The study team will use CATI technology to administer the baseline surveys. The use of CATI reduces respondent burden, as interviewers can proceed more quickly and accurately through the survey instruments, minimizing the interview length and the need for subsequent call-back. Computer programs enable respondents to avoid inappropriate and non-applicable questions. For example, respondents who were not offered housing counseling will be routed past questions only relevant to those who were offered housing counseling. CATI also improves data quality through more uniform administration of the survey questions, more accurate implementation of the skip patterns, and immediate application of range checks, edit checks, and consistency checks of item-by-item responses.

Efforts to Avoid Duplication

Staff at lenders and housing counseling agencies collect and enter a variety of customer information in their own data systems. To avoid duplicate data entry, the study team will develop a data upload feature within the study’s RAST system for staff at lenders and housing counseling agencies to use. Participating lenders and housing counseling agencies will be able to create extracts of data from their own systems and upload them directly to the RAST system. This feature is intended to decrease the burden on housing counseling agency and lender staff associated with providing the data and to increase data completeness and accuracy. During development of data collection instruments, the study team has been reviewing sample data from lenders and housing counseling agencies to understand the data that are already being collected. The study team will continue to work with housing counseling agencies and lenders to determine whether the existing data they collect on study participants will be sufficient or whether additional information will be needed.

Involvement of Small Entities

The research team may work with small housing counseling agencies within study jurisdictions to deliver in-person pre-purchase education and housing counseling. We will work with NeighborWorks to ensure minimal burden of participating in this study. Participation in this study is voluntary for all housing counseling agencies.

Consequences of Less Frequent Data Collection

The collection of data through the intake form, eligibility assessment, baseline survey, and study participant tracking efforts are essential to conducting the analysis of the impacts of pre-purchase education and counseling. The intake form, eligibility assessment, and baseline survey are conducted only once. Participant tracking will occur every four months over the course of the next three years. Future tracking activities that extend past that date will be presented in future OMB packages.

The collection of service tracking data by housing counseling agencies is triggered by study participants’ use of the housing counseling services. The study team is requesting that housing counseling agencies upload service tracking data that they already collect directly into the study’s data collection system (RAST) and, when needed, input supplemental data into this system. To ensure data completeness of the study’s required data collection fields, housing counseling agencies will need to provide information on both the homeownership education and the individualized counseling that study participants receive.

Special Circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6

(Controlling Paperwork Burden on the Public, General Information Collection Guidelines). There are no circumstances that require deviation from these guidelines.

Consultations Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and Urban Development (HUD) published a notice in the Federal Register on October 19, 2011. The docket number is FR-5486-N-24. The Federal Register Notice appeared on pages 64367 and 64368.

The Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation is being implemented in partnership with Abt Associates Inc. (the prime contractor), Sage Computing, the Center for Community Capital at the University of North Carolina, Doug Dylla LLC, and Professor Stephanie Moulton of Ohio State University.

In addition, the study has an advisory panel of approximately eight academic and professional experts in the homeownership counseling field. The advisory panel includes Peter Zorn (Freddie Mac), Janis Bowdler (NCLR), J. Michael Collins (University of Wisconsin-Madison), Christopher Herbert (Harvard University), Kenneth Breevort (Consumer Financial Protection Bureau), Jennifer Turnham (Abt Associates Inc.), Kristopher Rengert (FDIC), and Carolina Reid (UC Berkeley). Additionally, Dr. Don Dillman an expert in survey and response rate methodology at the Washington State University is a consultant on this project. The advisory panel was engaged to review two documents. First, October 2012, the advisory panel reviewed the research design, data collection and analysis plan for the study and provided feedback during an Expert Panel Meeting convened at HUD on October 5, 2012. Next, at the conclusion of the participant enrollment period, approximately September 2014, the advisory panel will review a baseline report on participant characteristics and the early implementation of the study. HUD and the Abt team will convene meetings of the advisory panel at that time so that the panel can discuss their input and guidance as a group. The purpose of this consultation is to ensure the technical soundness of the research design and implementation plan.

Payment to Respondents

For the evaluation to be most successful, we have determined that payments should be provided both to study participants and to HUD-approved housing counseling agencies to compensate them for the time associated with study participation. Each type of compensation is discussed below.

Compensation for Study Participants
1. The Use of Pre-Incentives

Prior to study enrollment, all customers that are referred to the study team by participating lenders will receive a mailing that contains information about the study. Specifically, this mailing will contain a study brochure, a study overview, a copy of the study participant consent form, and a $2 cash pre-incentive. This mailing will go to all customers who express interest in learning more about the study. The written documents are being provided for two reasons. First, these documents will further explain the study components including study activities, random assignment, and study incentives. By having a copy of the study consent form, customers will be able to read along when the study team calls them for the study enrollment call, which will occur a few days later. Second, these documents will provide legitimacy to the study. Therefore, when the study team contacts them for the study enrollment call, they are more likely to recognize the reason for the call and not think it is a scam.

The $2 cash is designed to provide an incentive for study participation. Survey research literature has consistently shown that offering incentives can significantly increase response rates in telephone and mail survey. A study conducted in the early 1990s found that adding a $5 pre-paid cash incentive to the first mailing of the questionnaire significantly increased the response rate (49%) compared to a one-dollar bill (41%) and no incentive (21%).⁴

As discussed in Dillman, Smyth, and Christian’s 2009 book Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, providing a small financial incentive to potential study participants can be extremely beneficial to a research study.

“One of the largest contributors to improved response rates, second only to multiple contacts (to be discussed later), is the appropriate use of prepaid token financial incentives. The token incentive serves two functions. It brings social exchange into play and encourages respondents to reciprocate by completing the survey. Equally important, though, it is a novel and unexpected gesture that brings additional attention to the request so that respondents read and contemplate it rather than just tossing it.”⁵

Incentives for Study Activities

All study participants will be provided a token of appreciation for completing the baseline survey and for participating in activities related to study enrollment. . Participants will receive a $30 check for completing the baseline survey, which is the last stage in the enrollment process. The study team estimates that the entire enrollment process will last between 35 and 40 minutes, including the completion of the eligibility assessment, the informed consent, and baseline survey. This incentive is commensurate with the amount of time that study participants invest in the enrollment process.

Throughout the study period, study participants will also receive a $5 check for returning tracking letters to confirm their current contact information (this letter is mailed approximately five months after they enroll in the study). Every four months, the study team will mail study participants a letter asking for updated contact information. Study participants will receive a $5 check for each tracking letter that is mailed back to the study team. Following random assignment, participants who are assigned to the treatment groups will receive$150to as a thank you for the time required to participate in housing counseling. The payment will be made in two parts. First, participants will receive a $50 check as a token of appreciation for their study participation when they initiate housing counseling with the assigned housing counseling provider. Second, they will receive a $100 check upon completing housing counseling in recognition of the time and other expenses they may incur in order to participate in housing counseling. Study participants, who are assigned to the treatment groups, are expected to spend between 7 and 11 hours completing pre-purchase education and counseling through a study participating counseling agency. Additionally, these study participants may also incur expenses for childcare, transportation, parking, internet service, and phone charges. The incentives provided to the treatment group members are intended help offset some of these charges and are commensurate with the amount of effort spent on study activities. In addition, the housing counseling services will be provided free of charge to participants in the two treatment groups. Participants who are assigned to the control group will receive a $50 check after random assignment.

We believe that these incentive payments are necessary to encourage participation in the study and completion of the counseling interventions. Past studies have encountered issues with both low enrollment and low participant take-up rates for counseling, which affected the study’s ability to effectively measure the impact of counseling. Survey research literature indicates that higher incentives produce higher response rates.⁶ In a recent incentive experiment, participants in one wave of a longitudinal study were offered $20, $30, or $50.⁷ Offering the highest incentive of $50 showed the greatest improvement in response rates and also had a positive impact on response rates for the next four waves. We feel that this compensation structure will help ensure that we reach high take-up rates for counseling in this study..

In addition to the compensations discussed above, a subset of 200 study participants will be recruited to participate in telephone interviews. These participants will receive a $20 check as a token of appreciation for their time for the telephone interview.

Exhibit A-3 summarizes the compensations that will be provided to study participants:

Exhibit A-3: Compensation for Study Participants

Participant Type	Payment Trigger	Payment Amount
All referred customers	Pre-incentive	$2
All Participants	Completion of baseline survey	$30
All Participants	Return of tracking letter	$5 (each tracking activity; approximately 3 tracking letters for 3 years; a total of $45.)
Participants assigned to the control group	Assignment to control group	$50
Participants assigned to the treatment groups	Initiation of housing counseling	$50
Participants assigned to the treatment groups	Completion of housing counseling	$100
Treatment group members selected for implementation telephone interviews	Completion of interview	$20

Compensation for HUD-Approved Housing Counseling Agencies

The compensation structure proposed for HUD-approved housing counseling agencies is a two part structure that recognizes (1) the burden of responding to interview questions and (2) the burden of participating in the service tracking data collection and reporting, which will vary substantially across housing counseling agencies depending on how many study participants the housing counseling agency serves. The compensation is important for ensuring a high rate of participation in the study and full housing counseling agency cooperation in the data collection effort. This study will rely heavily on the data provided by housing counseling agencies. Our experience with missing/unreliable data from housing counseling agencies participating in the Housing Counseling Outcome Study underscores the importance in incentivizing the housing counseling agencies to ensure that they take seriously the responsibility for capturing and recording the data necessary for the study.

For the online education plus telephone housing counseling intervention, we have recruited one national housing counseling provider to serve treatment group participants across all study jurisdictions. We have budgeted for a $20,000 payment to compensate this organization for the costs associated with conducting outreach to study participants to ensure a high completion rate for online-education and telephone counseling, documenting online education and telephone housing counseling activities, and uploading the information regularly into the study’s data collection system. The study will be assigning over 1,700 study participants to this agency.

Due to the number of study participants being referred to this agency and the burden of additional data collection and reporting tasks that we are asking them to complete, we feel that a $20,000 payment is appropriate for their effort in the study.

The classroom education plus in-person housing counseling intervention requires a more complex incentive structure, because it is unlikely that we will find one organization that is able to serve treatment group participants in all jurisdictions. Instead, we have recruited NeighborWorks America, a national organization that has local affiliates in each of the selected study jurisdictions. NeighborWorks America will facilitate the recruitment of local housing counseling agencies and support the study team’s monitoring and data reporting activities. We have budgeted for the national organization to receive a $10,000 incentive. Having their involvement in the study will provide important motivation for the local NeighborWorks America’s affiliate agencies.) given the relationships between the intermediaries and their sub-grantees. We have also budgeted for each of the local NeighborWorks America’s agencies serving study participants to receive a $2,000 incentive, plus $100 per study participant served. Depending on NeighborWorks America’s coverage, the study team may explore other housing counseling agencies that fit the study’s needs in each jurisdiction. Additional housing counseling agencies (outside of NeighborWorks America’s network) will also receive a $2,000 incentive, plus $100 per study participant for data reporting.

These agency incentives compensate the organizations for the staff burden associated with conducting outreach to study participants, collecting service tracking data on all study participants’ education and counseling activities, and reporting that data to the study team on a biweekly basis, activities that are outside of their usual housing counseling agency requirements.. Any existing arrangements between the housing counseling agencies and lenders or other funding sources will not be affected by the study. However, the study incentives will not directly compensate the organization for providing housing counseling to a study participant when no existing funding source is available to compensate the organization for housing counseling. The national provider of online education and telephone counseling has confirmed their willingness to provide housing counseling to all study participants in the assigned treatment group without charging a fee. Our conversations with NeighborWorks suggest that a sufficient number of local NeighborWorks affiliate agencies will also be willing to participate in the study without compensation for the provision of housing counseling services because they are a HUD housing counseling grantee.

The purpose of these incentive payments is to help offset the burden of study participation for housing counseling agencies and their staff. It will also help ensure study compliance, and accurate and consistent data reporting. The compensation described will be provided to the agency and not to individual agency staff. Abt Associates will sign a grant agreement with each agency as a condition of providing the payment. The grant agreement will state that “the compensation is expected to be used to defray personnel costs due directly to this information collection.”

Arrangement and Assurances Regarding Confidentiality

Because of the nature of the information collected from and about study participants, strict confidentiality procedures will be followed for this evaluation. The information requested under this collection is protected and held confidential in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974). As required by 5 U.S.C. 552a (Privacy Act of 1974), HUD has published a Systems of Record Notice (SORN) in the Federal Register.

The study protocols and consent process are currently being reviewed in detail by Abt Associates’ Institutional Review Board (IRB). The IRB review process will review all study protocols, including the consent process and confidentiality protections. Detailed plans for informed consent and data security procedures are described below.

Informed Consent

Following the eligibility assessment, Abt Associates’ survey subsidiary will administer the study participation consent agreement. (The consent agreement is provided in Appendix C.) The study team member will read the participant consent agreement to the prospective study participant. The agreement briefly describes the study, the random assignment process, the incentive structure, the risks associated with study participation, what is being done to mitigate these risks, and what will be expected if the customer agrees to participate in the study. Those who agree to participate in the study will provide verbal consent, as the participation agreement will be administered by telephone. Participant consent will be collected verbally because it reduces the burden on lenders to collect this information and mail the completed agreement forms back to the study team. This consent agreement will be recorded to provide assurance to study participating lenders that their customer has consented to all study activities.

If a study participant secures a home loan from a participating lender with a co-borrower, then the study team also needs to obtain consent from the co-borrower to collect information on that loan origination and performance data. (The co-borrower consent agreement is provided in Appendix D.) Like the participant agreement, the co-borrower consent agreement will be read over the telephone and recorded to provide assurance to study participating lenders that their customer has consented to have the lender provide loan origination and performance data to the research team.

Data Confidentiality Protections

The research team has established rigorous data security and confidentiality provisions that are documented as part of the study’s Data Security Plan and in the Department’s SORN and Privacy Impact Assessment. First, all data users will be aware of and trained on their responsibilities to protect participants’ personal information, including the limitations on uses and disclosures of data. The study’s RAST data system will be designed to limit access to data to authorized users with levels of access commensurate with each person’s role on the project. The web server hosting the data system will be maintained in a secure facility with power back up, network redundancy, and system monitoring. In addition, daily back up of the server will be maintained at the data center and an off-site location. The RAST will be password protected, and access will be provided after user authentication. In the case of a forgotten password, the system will email the password to the registered user's email address.

An assurance of confidentiality is included in the study participation consent agreement through which participants will provide informed consent (see Appendix C). An assurance of confidentiality also will be made to all respondents as part of the introduction to each of the surveys. For both survey data and corresponding administrative data on sample members, computer security will be maintained by passwords known only to a limited number of project staff members who need access to these files.

The following safeguards shall be used to secure data in storage, retrieval, during access, and disposal.

All personal data will be maintained on a secure workstation or server that is protected by a firewall and complex passwords, in a directory that can only be accessed by the network administrators and the analysts actively working on the data.
Data files used for analysis will be stored in a separate location from files with identifying information to minimize the risk that an unauthorized user could use the unique identification number to link de-identified files with the identifiers.
Access rights to the data are granted to limited researchers on a need-to-know basis, and the level of access provided to each researcher is based on the minimal level required that individual to fulfill his research role.
Abt Associates will back-up the data on a regular basis and use offsite storage to reduce the effects of system failures or disasters. Backup media will be encrypted. Data will never be stored on a laptop or on a movable media such as CDs, diskettes, or USB flash drives without encryption.
If an authorized researcher leaves employment or is no longer working on this project, their user ID and access will be terminated within one day. These steps will be documented as part of termination process.
Study team interviewers will securely store any hard copy documents with personal protected information, such as tracking letters.

Sensitive Questions

The lender intake form collects information on the customer’s name and contact information, co-borrower’s name and contact information, date of birth, income, and first-time homebuyer status. This information will be treated as sensitive, confidential information to be used only for the purposes of this study.

The eligibility assessment collects information on the customer’s first-time homebuyer status (validation check for lender intake form), access to transportation and a computer, previous experience with pre-purchase education and/or housing counseling, and language competency. The participation consent agreement collects the customer’s name, social security number, and date of birth. These questions are necessary to determine consent status and to confirm the identity of study participants, and, therefore, all questions must be answered by customers interested in participating in the study. This information will be treated as sensitive, confidential information to be used only for the purposes of this study.

The baseline survey includes study participant’s contact information; demographic characteristics (e.g., race/ethnicity, gender, marital status); the stage in the home purchase process and preparedness for home purchase (e.g., the extent of respondents’ home search prior to pre-qualification, intended down payment amount, expected timeline for home purchase); and baseline measures of time-varying study outcomes (e.g., income sources and total family income, financial literacy, non-housing savings/debts, budgeting activities, and employment and earnings).This information will be treated as sensitive, confidential information to be used only for the purposes of this study. Prior to the baseline survey, respondents will be reminded that they can refuse to answer any questions with no repercussion.

The co-borrower’s consent agreement collects and records the co-borrower’s name and social security number. This information is necessary to collect loan origination and performance data from study participating lenders.

The implementation study telephone survey includes the participants’ experiences with the study’s recruitment and enrollment process; participants’ assignment to housing counseling services and completion of housing counseling; and participants’ opinions on the pre-purchase housing counseling they received. This information will be treated as sensitive, confidential information to be used only for the purposes of this study. Respondents will also be reminded that they can refuse to answer any questions with no repercussion.

The interview and observation guides developed for housing counseling agencies and lenders do not include any questions of a sensitive nature.

Estimate of Burden of Hours

Estimation of the total number of hours needed to prepare the information collection including number of respondents, frequency of response, and hours of response:

The average time per customer for the 87,500 customers to receive a study recruitment call from participating lenders is 3 minutes. This number estimates a 20 percent take-up rate for customers expressing interest in learning more about participating in the study. The study team anticipates that 20 percent is a very conservative take-up rate for customers, and we do not expect that lenders will have to contact 87,500 customers. The average time per customer for the 17,500 potential study participants to complete the eligibility assessment is 3 minutes and 30 seconds. The average time for pre-test participants to complete the eligibility assessment was 3 minutes and 12 seconds.
The average time per study participant for the 7,000 potential study participants to complete the consent form is 11 minutes. The average time for pre-test participants to complete the consent process was 11 minutes and 10 seconds.
The average time per study participant for the 7,000 enrolled study participants to complete the baseline survey is 25 minutes. The average time for pre-test participants to complete the baseline survey was 23 minutes and 18 seconds.
The average time per study participant for the 7,000 enrolled study participants to complete each tracking letter is 5 minutes.
The average time per study participant for the 200 study participants completing the implementation study telephone interview is 30 minutes.
The average time for each study participant’s co-borrower to complete the co-borrower consent agreement is 3 minutes.
The average time per housing counseling agency to spend on pre-study enrollment activities and the site visit is 84 hours. These activities include training staff, planning for a site visit, preparing an outreach plan and data transfer protocol, and participating in a site visit.
The average time per study participant for housing counseling agencies to document the services provided to study participants is 10 minutes. Each housing counseling intervention contains both homeownership education and individualized housing counseling. Housing counseling agencies will create a separate entry for each education and counseling service received. The estimated total burden is 20 minutes per participant.The average time for housing counseling agencies to spend conducting outreach to study participants is 2 hours per participant.
The average time for housing counseling agencies to communicate with the study team is estimated to be 100 minutes per study participant. This includes discussions about outreach attempts, completion of pre-purchase education and counseling activities, and service tracking data.
The average time per customer for lender staff to contact customers and input their intake information is 3 minutes, with contact and intake conducted with up to 87,500 potential study participants.
The average time for the lender’s customer service team to complete recruitment call outcome report is 30 minutes. These reports will be provided to the study team weekly through the enrollment period. The enrollment period is estimated to last 52 weeks for each lender.
The average time for the lender’s customer service team to participate in post-observation interview questions is 60 minutes. Up to 12 staff will be interviewed at each participating lender..
The average time for lenders to prepare study participants’ loan origination and performance data for the study team is 60 minutes. The study team will ask for this data at 6 points during the next 3 years.

The total burden for the study is 42,805 hours: 14,946 hours for study participants and potential study participants, 105 hours for study participants’ co-borrowers, 6,972 hours for one-time activities for the counseling agencies, 16,238 hours for counseling agencies’ study-related activities with study participants, and 4,544 hours for lenders.

Exhibit A.12.1 Burden of Hours

Respondents	Number of Respondents	Responses per Entity	Average Time to Complete in Minutes (min, max)	Burden (hours)
Intake: Outreach to potential study participants	87,500	1	3 min. (2,4)	4,375*
Eligibility Assessment	17,500	1	3 min 30 seconds (2, 5)	1,021*
Consent Agreement	7,000	1	11 min (9,12)	1,283*
Baseline survey: study participants	7,000	1	25 min. (20,30)	2,917*
Follow-up interviews: study participants	200	1	30 min. (20,40)	100**
Tracking Letters	7,000	9	5 min (3, 10)	5,250***
Subtotal: Study (and potential) participants	126,200	14	77.5	14,946
Co-borrower consent	2,100	1	3 min (2, 4)	105*
Subtotal: Study participant co-borrowers	2,100	1	3	105
Counseling agencies’ one-time activities
Counseling agency planning (training counseling agency staff)	84	3	480 min/ 8 hrs. (7,9)	2,016
Preparing for outreach and data transfer protocols	84	3	780 min./13 hrs. (10,16)	3,276
Site Visit Preparation and on-site monitoring activities	84	2	360 min./6 hrs.(4,8)	1,008
Interviews: Housing counseling agency staff	84	8	60 min./1 hr (45,75)	672
Subtotal: Housing Counseling agencies one-time activities	336	16	1,680 min/28 hrs.	6,972
Counseling agencies’ study-related activities per study participant
Service tracking information collection	4,026	2	10 min. (0,20)	1,342****
Outreach to study participants	4,026	1	120 min/2hrs. (1,3)	8,052****
Ongoing communication with study team	4,026	1	100 min/1.7 hrs (1,2)	6,844****
Subtotal: Housing counseling agencies per participant	12,078	4	230 min.	16,238
Intake: Lender staff (concierge teams)	36	2,431	3 min (2,4)	4,376*
Interviews: Lender staff	3	12	60 min. (45,75)	36
Recruitment Outcome Report	3	52	30 (15, 45)	78*
Loan origination and performance data	9	6	60 min (45, 75)	54***
Subtotal: Lenders	51	2,501	153	4,544
Total	140,765	2,536	2,143.5	42,805

*This activity will span the duration of the enrollment period. The enrollment period may last up to 24 months depending on the rate of recruiting the study sample.

** This activity will occur during the final weeks of the enrollment period.

***This activity will span three years.

**** This activity will occur until 12 months after the end of enrollment.

Estimated Record Keeping and Reporting Cost Burden on Respondents

This data collection effort involves no recordkeeping or reporting costs for respondents other than the time burden to respond to questions on the data collection instruments as described in item A.12 above. There is no known cost burden to the respondents.

Estimated Cost to Federal Government

The total contract amount for this evaluation effort is $3,953,503.The budget for data collection is estimated to be approximately $3,402,086.

The research team is currently seeking a modification because of changes to the study design. This modification would increase the total contract amount for the evaluation.

Reasons for Changes in Burden

This submission to OMB is a new request for approval; there is no change in burden. However, there is a change in burden from the 60-day OMB submission due to a contract modification. In particular, the study sample increased from 4,000 to 7,000 study participants, resulting in an overall increased burden.

This submission to OMB is an updated request from the 30-day OMB submission due to a participating lender altering their recruitment process. As a result burden hours have decreased for lenders.

Additionally, in October 2012 the study team convened an expert panel that made some suggestions that altered the baseline survey and recruitment strategy. As a result, the baseline survey has been modified and the recruitment scripts and study participant outreach materials have been updated.

Tabulation Plan, Statistical Analysis and Study Schedule
1. Tabulation Plan and Statistical Analysis

The data collection activities covered in this OMB submission are the enrollment and implementation activities for the Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation. Following completion of the enrollment period, these activities will be used to produce baseline analyses that describe the baseline characteristics of study participants, evaluate the random assignment of participants to treatment and control groups, and document the enrollment and implementation process. Additionally, a small number of initial impact analyses may be performed with respect to outcomes measured through credit report and lender data. A second package, under a separate contract between HUD and the study team, will cover the follow-up participant interviews that will collect the primary outcome measures for impact analysis.

The specific use of these data collection activities within each evaluation component is described below.

Implementation Analyses

Implementation analyses will document the implementation experience of each lender and housing counseling agency partner in each jurisdiction. Due to the number of staff at both participating lenders and housing counseling agencies that the study team will be working with, it is extremely important to document procedures and protocols that sites implement. The implementation analyses will integrate the information collected through the lender customer service teams’ observations, recruitment call outcome reports, and housing counseling agency staff interviews with the site liaisons’ documentation of each partner’s implementation achievements.

The implementation analyses will describe lenders’ fidelity to the study design and enrollment targets for each jurisdiction. We will examine the steps that each lender used for early stage customers and compare them across lenders and jurisdictions. In particular, we will document the extent to which lenders introduced the study to all customers who might qualify for the study. The implementation analyses will identify the key operational challenges that emerged as recruitment was implemented, and we will examine how, and how well, those challenges were addressed.

The study team will examine three components of the housing counseling interventions assigned to study participants; (1) housing counseling agencies’ fidelity to the specific interventions; (2) study participants’ experiences with services received at their assigned counseling agency; and (3) housing counseling completion rates within each intervention. This information will be collected through the study’s data collection system, site monitoring, staff interviews with housing counseling agencies, and interviews with a subset of study participants.

The information collected through the study’s data collection system will be used to document the timing and nature of the enrollment and housing counseling activities that occur during the study period. These data will be coupled with remote site monitoring, on-site observations, staff interviews, and study participant interviews to observe and learn about the experiences that study participants had at the housing counseling agencies. The study participant interviews will also examine why some study participants did not pursue or complete their assigned housing counseling service.

Evaluation of the Random Assignment of Study Participants

Data from the lender intake form, eligibility assessment, and baseline survey will be used most importantly for the impact analysis as well as to describe the study population, but also to examine the extent to which the randomized groups are balanced across relevant participant characteristics.

Baseline Analyses

Data from the enrollment process—the lender intake form, eligibility assessment, and baseline survey—will be used to accomplish four objectives:

We will present detailed descriptive information about the sample of recruited borrowers, including initial measures of financial literacy, savings, debts, credit quality, and other characteristics.
We will document the stage of the home purchase process and the extent of previous search activities conducted by enrolled participants.
We will compare the sample of study participants to lender data and to secondary data sources to examine the extent to which the sample differs from two groups. First, we will compare the study sample to the universe of customers at each lender that meet the basic screening criteria to be offered participation (defined in Section B.1.1). This comparison provides insight into the extent to which refusal of study participation (or selective offer of study participation by loan officers) is more likely for certain types of lender customers. Second, we will compare the study sample to the population of LMMI first-time homebuyers and to the population of FHA homebuyers using secondary data and FHA administrative data, respectively. These comparisons will examine the extent to which the sample of study participants differs from these broader populations. In both cases, the analysis will be descriptive and cannot be used to formally extrapolate from the sample to either population. Instead, these comparisons are intended to provide context for how the study sample compares to broader populations of interest.
Finally, as is common in experimental evaluations of policy initiatives, we will use the baseline data to affirm that treatment and controls groups differ only by chance.

Impact Analyses

The primary impact analyses for the evaluation will be conducted using outcomes collected in follow-up surveys, which will be covered in a separate OMB package. These analyses will rely on the data collection activities covered in this submission for baseline measures of the time-varying study outcomes and to define covariates and subgroups for the impact analyses. These covariates serve the important purpose of increasing the precision of impact estimates, thereby reducing the size of an impact that we can detect, if it exists, given sample size constraints. In addition, subgroups may be defined with respect to borrower personal characteristics (from the baseline survey); financial characteristics (from the lender intake form and baseline survey); and neighborhood characteristics (which requires knowledge of the borrower’s address from the baseline survey). We also anticipate examining endogenously-defined subgroups (for example, housing counseling intensity levels (from the housing counseling service data)), which provide justification for our inclusion of various baseline measures on participants’ financial decision-making, self-efficacy, learning styles, and follow-through, for example.

A small number of preliminary impact analyses may be performed at the time of the baseline analyses using outcome measures defined from administrative lender data. These analyses are similar to the impact analyses with respect to outcomes measured in the follow-up interviews. The data collection activities covered in this submission would be used as baseline measures for time-varying outcomes, to define covariates needed to increase precision and subgroups of interest, and to define the initial outcome measures.

Study Schedule

Under the current schedule, study participant recruitment and enrollment will be conducted over a 1 month pilot period followed by an enrollment period that may last up to 24 months. The study’s current design has study participant recruitment and enrollment conducted over a 12 month period. However, the recruitment and enrollment period may extend up to 24 months in order to reach the minimum sample size of 5,000 study participants. Assuming OMB approval in February 2013, data collection will take place during the pilot month in April 2013 then during the enrollment period from mid-May 2013 until mid-May 2015 with a draft baseline report submitted in September 2015.

Expiration Date Display Exemption

All data collection instruments will prominently display the expiration date for OMB approval.

Exceptions to Certification

This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 CRF 1320.9).

1Henceforth in this document we use the term pre-purchase “counseling” to refer to both individual counseling services and group homebuyer education.

2The National Industry Standards can be found at http://www.homeownershipstandards.org/standards.

3All study participating lenders have designated groups (either staff within the bank or outside of the bank) to be responsible for study recruitment calls to customers. Bank of America has assigned this responsibility to its concierge team. Wells Fargo has contracted this responsibility to Maritz Research. Citibank will either use an internal customer service team or contract a vendor to conduct the recruitment calls. To standardize the name of these teams across lenders in the OMB submission, the study team has identified them as ‘customer service teams’.

4James, Jeannine M. and Richard Bolstein. 1992. “Large Monetary Incentives and Their Effect on Mail SurveyResponse Rates.” Public Opinion Quarterly 56: 442-453.

5Don A. Dillman, Jolene D. Smyth, Leah Melani Christian. Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method (Hoboken, NJ: John Wiley & Sons, Inc., 2009), 238.

6Goldenberg, Karen L., David McGrath, and Lucilla Tan. 2009. “The Effects of Incentives on the Consumer Expenditure Interview Survey.” Proceedings of the Survey Research Methods Section, American Statistical Association (ASA). Accessed via http://www.amstat.org/sections/srms/proceedings/allyearsf.html

7Rodgers, Willard. 2011. “Effects of Increasing the Incentive Size in a Longitudinal Study.” Journal of Official Statistics 27 (2): 279-299.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Abt Single-Sided Body Template
Author	Nichole Fiore
File Modified	0000-00-00
File Created	2021-01-30

OMB Submission Part A Final (3.5.13)

OMB Submission Part A Final (3.5.13).docx

Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation

OMB: 2528-0293

Justification

Circumstances that Make the Collection of Information Necessary

How and by Whom the Data will be Collected and Used

Project Overview

Purpose of Data Collection

Who Will Use this Information

Instrument Item-by-Item Justification Chart

Use of Improved Technologies

Random Assignment and Service Tracking System (RAST)

Computer-Assisted Telephone Interviewing (CATI)

Efforts to Avoid Duplication

Involvement of Small Entities

Consequences of Less Frequent Data Collection

Special Circumstances

Consultations Outside the Agency

Payment to Respondents

Compensation for Study Participants

The Use of Pre-Incentives

Incentives for Study Activities

In addition to the compensations discussed above, a subset of 200 study participants will be recruited to participate in telephone interviews. These participants will receive a $20 check as a token of appreciation for their time for the telephone interview.

Compensation for HUD-Approved Housing Counseling Agencies

Arrangement and Assurances Regarding Confidentiality

Informed Consent

Data Confidentiality Protections

Sensitive Questions

Estimate of Burden of Hours

Estimated Record Keeping and Reporting Cost Burden on Respondents

Estimated Cost to Federal Government

Reasons for Changes in Burden

Tabulation Plan, Statistical Analysis and Study Schedule

Tabulation Plan and Statistical Analysis

Implementation Analyses

Evaluation of the Random Assignment of Study Participants

Baseline Analyses

Impact Analyses

Study Schedule

Expiration Date Display Exemption

Exceptions to Certification