Supporting Statement for Paperwork Reduction Act Submission
Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation
Contract # R-CHI-01108
Part A
Supporting Statement for Paperwork Reduction Act Submission
Table of Contents
A.1 Circumstances that Make the Collection of Information Necessary 1
A.2 How and by Whom the Data will be Collected and Used 3
A.3 Use of Improved Technologies 12
A.4 Efforts to Avoid Duplication 13
A.5 Involvement of Small Entities 13
A.6 Consequences of Less Frequent Data Collection 13
A.8 Consultations Outside the Agency 14
A.10 Arrangement and Assurances Regarding Confidentiality 15
A.12 Estimate of Annualized Burden of Hours 16
A.13 Estimated Record Keeping and Reporting Cost Burden on Respondents 18
A.14 Estimated Cost to Federal Government 18
A.15 Reasons for Changes in Burden 18
A.16 Tabulation Plan, Statistical Analysis and Study Schedule 18
The Department of Housing and Urban Development (HUD) seeks to understand the impact of pre-purchase homeownership counseling on a range of outcomes for low-, moderate-, and middle-income first-time homebuyers. The Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation will examine the impact of different types of pre-purchase counseling and education on homeowner outcomes and loan performance for prospective first-time homebuyers with incomes below 120 percent of the area median. This evaluation will measure the impact of two different pre-purchase counseling and education interventions through a randomized experimental design.
The current study represents an important effort to measure the impact of pre-purchase housing counseling using experimental evaluation methods. Whereas many of the previous studies have been limited by small sample sizes and non-experimental designs, this evaluation is recruiting a large sample for an experimental impact evaluation in which individuals are randomly assigned to receive housing counseling or not.
The study’s goal is to enroll 6,000 study participants across 28 study sites during a 20 month enrollment period. The study began by implementing a pilot period in three sites (Los Angeles, Miami, and Phoenix) for eight weeks in fall 2013. In January 2014, full study enrollment began across all 28 sites. As of the end of September 2014, the study had enrolled 2006 study participants.
The full set of materials for the enrollment stage of the study was reviewed under a previous submission (Control #2528-0293). This submission contains updates and renewals for the continuation of existing baseline data collection activities and a new data collection activity for the first stage of follow-up. Each of these activities is described below.
1. Updates to the Initial Baseline Data Collection Activities
Modifications to the approach used to collect consent from co-borrowers of study participants: For study participants who originate a loan with one or more co-borrowers, the collection of consent from study participants’ co-borrowers’ is necessary to allow the study to collect data related to the origination and servicing attributes of their mortgage loans. The origination and servicing data is the primary source of data for several outcome measures.
Over the course of the study, the study team will use four activities to collect co-borrower consent:
The study team will mail an advance letter prior to the start of telephone calls for the first follow-up telephone survey, which will occur 12 months after enrollment. The advance letter will include a copy of the co-borrower consent form and a postage-paid return envelope for each co-borrower identified during the baseline survey that did not provide consent previously.
The first follow-up telephone survey occurs 12 months after enrollment and will ask the study participant to confirm whether they purchased a home with one or more co-borrowers. If any co-borrower has not provided consent through step 1, the interviewer will ask the study participant whether the co-borrower is available, and attempt to collect co-borrower consent verbally or arrange a time to callback to collect the consent.
The study team will conduct outbound calls to any co-borrower identified during the first follow-up telephone survey who did not provide consent through any of the previous steps.
For those who mention a co-borrower in the interim survey but we are not able to consent over the telephone, we will mail a copy of the co-borrower consent form along with a paid postage envelop in the study participant tracking letters that are mailed every four months during the study period.
2. Replacement of study participant experience telephone interviews with focus groups to collect qualitative data: The information collected through the focus groups is a primary data source for the study’s implementation analysis. The implementation analysis will provide useful contextual information to aid in interpreting the impact of counseling interventions. The goal of the analysis is to identify and document implementation, including challenges or other factors that affect study participants’ behaviors or the quality of the collected data. In general an implementation study examines what happened, the extent to which it happened as planned and reasons for variance from the plan. This research will be able to describe what the interventions are about, how participants experienced them and any unexpected wrinkles that would have implications for interpreting results from the impact analysis.
The focus groups replace the Study Participant Experience Interviews in the previous submission. This change is the result of a change to the study’s research design to replace one-on-one interviews with focus groups. The study team believes that the focus groups will allow us to probe these questions in greater depth and improve the quality of the data collected.
3. Continuation of Data Collection Activities from the Baseline Period
Loan Origination and Performance Data: Study participants’ loan information and performance data will be requested at six month intervals throughout the study period from participating lenders and the Federal Housing Administration. This data will provide the study team detailed information on the timing of a home purchase, underwriting characteristics, monthly housing costs, and mortgage terms and conditions for any study participant who purchases a home through a study participating lender.
Credit Bureau Data: The credit bureau data contain study participants’ credit scores and selected credit file attributes. The credit bureau data collected during the follow-up period will contain the same measures that the study team is currently collecting in the enrollment period requests. This data includes information about housing outcomes (e.g. housing debt, monthly housing debt payment), loan performance (e.g. mortgage delinquency, bankruptcy, foreclosure), and other key outcomes for the study’s impact analyses.
Service Tracking Data from HUD-Approved Housing Counseling Agencies: The study’s collection of service tracking data from participating counseling agencies will continue under the Impact Evaluation during the 12 months following the end of the study’s enrollment period. Because study participants are eligible to complete counseling services for up to one year following random assignment, the study team will continue to collect service tracking data in order to monitor counseling completion and conduct outreach to study participants who have not completed counseling services. This will provide the primary source of information on treatment take-up, intensity, and completion of housing counseling activities.
Updated Tracking Information for Study Participants: During the baseline data collection period, the Abt team developed a robust tracking system that utilizes both passive and active measures and involves reaching out to study participants regularly. The tracking plan builds upon the study’s existing data system and supplements it with data obtained through passive searches every four months of proprietary databases—including the National Change of Address and Social Security Death Index databases. These sources are supplemented by mailed requests for the study participants to review and return a form that includes contact information for the participant as well as up to 3 people who always know how to reach the participant. This robust tracking plan helps the study team keep the participant contact information accurate. These efforts began during the study enrollment period and will continue until a study participant completes the Final Survey. Panel tracking for the Interim Survey should begin seamlessly following the end of the panel tracking efforts for study enrollment.
3. Follow-up Data Collection
Interim Survey: The Interim Survey will be administered approximately 12 months following random assignment by interviewers using a telephone with field follow-up methodology.
Sample will be released on a rolling basis each month according to when the study participant completed the baseline survey. Two weeks prior to the release of new sample, Abt SRBI will send all sample members an advance letter. This letter describes the study, alerts the participants that interviewers will attempt to reach them, encourages their participation, and provides a toll-free study number for them to call with questions. The advance letters include a $2 pre-incentive. The advance letters will also include a copy of the co-borrower consent form. Abt SRBI interviewers will first attempt to reach each study participant by telephone. Cases will be worked by telephone until all leads are exhausted. Cases not completed in the phone center will be sent to the field interviewers and attempted in-person.
The study design is a randomized experiment. Enrolled study participants are assigned to one of three groups: remote housing counseling (online education + telephone counseling), a choice group with the option of completing services remotely or in-person (online education + telephone counseling or classroom-based education + in-person counseling), or a control group that does not have access to these housing counseling services. Implementation of this study in the field requires that the study team coordinate with both lenders and HUD-approved housing counseling agencies:
Lenders: The study team has partnered with three national lenders for the implementation of this study. The participating lenders were chosen purposively, based on their volume of loan originations and willingness to partner for this study. Study participants are recruited through the lenders’ centralized, telephone-based customer service teams. The telephone customer service teams with each lender introduce the study to customers, referring interested customers to the study team.
HUD-Approved Housing Counseling Agencies: The study team refers study participants in each treatment group to HUD-approved housing counseling agencies to receive housing counseling services. The housing counseling agencies include one national remote housing counseling provider and local affiliates of two national in-person housing counseling intermediaries. The study team selected eHome America and CredAbility as the provider of remote housing counseling for the evaluation. For in-person housing counseling, the study team worked with NeighborWorks America and the National Council of La Raza (NCLR) to recruit local affiliate organizations to provide in-person housing counseling in each jurisdiction. However, based on NeighborWorks and NCLR’s coverage, the study team also recruited other HUD-approved housing counseling agencies that fit the study’s needs in each jurisdiction.
The study team is responsible for all other study activities—as well as for monitoring and coordinating the activities of the lenders and housing counseling agencies.
The recruitment and enrollment process includes several steps. Following lender intake and referral, the study team conducts the eligibility assessment, consent process, baseline survey, and random assignment. While lenders apply screening criteria to determine whether to offer participation in the study, the eligibility assessment confirms that the customer meets all study eligibility requirements.
This submission requests OMB approval for seven data collection activities that reflect updates to the study’s previous OMB package.
Collection of co-borrower consent starting with the advance letter prior to the start of the 12 month follow-up survey;
Focus groups that collect information for the implementation analysis and qualitative analysis of study participants’ experiences with homeownership education and counseling;
Collection of data on study participants’ loan origination and mortgage performance;
Collection of study participants’ credit scores and credit report data;
Collection of study participants’ housing counseling service data from HUD-approved housing counseling agencies;
Collection of contact data from study participants; and,
Collection of study participants’ responses to an interim survey.
The purpose of each data collection activity is described below.
Collection of Co-borrower Consent: The collection of co-borrower consent is necessary to collect information on the origination and servicing attributes of study participants’ loans. Consent from all co-borrowers is required for the study team to receive data from any of the study’s participating lenders.
The study’s projections suggest that approximately 30 percent of study participants will use a co-borrower to purchase their home. For each participant, the use of a co-borrower and who that may be can change over time. The baseline survey asks study participants to identify any co-borrowers that they expect to have.
The collection of co-borrower consent through the advance letter prior to the interim survey will be conducted by Abt Associates and Abt SRBI. The advance letter will contain a hard copy of the co-borrower consent form for each co-borrower identified during the baseline survey. The letter will also include an envelope with a return mailing address and postage for the consent forms to be returned to the study team. Abt SRBI will mail the letters and Abt Associates will store the returned consent forms, as well as documentation of which study participants have co-borrowers and which co-borrowers have provided consent to the study team. The co-borrower consent form is included as Appendix A.
Focus Groups: The focus groups are necessary for our understanding of study implementation, the experiences with each intervention, and, in particular, why some participants decided to complete or not complete pre-purchase housing counseling and education. During the final four months of the enrollment period, study team members from Abt Associates and the University of North Carolina will conduct 12 focus groups across three study sites. The focus groups will allow the study team to collect more in-depth data to address questions in three areas:
What was the experience of study participants in study recruitment, enrollment, and with the education and counseling to which they were assigned? What challenges and obstacles did they encounter completing study activities?
Were there any issues with the study design or implementation that may affect the outcomes of interest or our interpretation of the analyses?
What was the experience of study participants as first time homebuyers and as consumers of each type of pre-purchase education (in-person vs. online) and housing counseling services (in-person vs. telephone)?
In each of the three selected sites, we will conduct a focus group with the four sub-groups outlined below.
In-person Non-completers: Study participants offered in-person pre-purchase education and counseling services through a housing counseling agency who either a) chose not to take up the pre-purchase education and counseling services; or b) started the pre-purchase education and counseling services but did not complete them;
Remote Non-completers: Study participants offered on-line pre-purchase education and telephone counseling services who either a) chose not to take up the pre-purchase education and counseling services; or b) started the pre-purchase education and counseling services but did not complete them;
In-Person Completers: Study participants who completed the pre-purchase education and counseling services through a housing counseling agency; and
Remote Completers: Study participants who completed the pre-purchase on-line education and telephone counseling services.
Additionally, in each group we will try to include study participants from each of the three lenders and all of the counseling agencies participating in a site. We will also include study participants that enrolled at various points of time during the home purchase process. For example, those who had purchased a home at enrollment, since enrollment, are still looking, and are no longer interested in purchasing a home.
Exhibit A-1: Participant Experience Focus Groups
Focus Group Characteristics |
City 1 Northeast |
City 2 Southeast |
City 3 |
1. Offered the in-person intervention but did not take up or did not complete the intervention |
|
|
|
2. Offered the remote intervention but did take up or did not complete the intervention |
|
|
|
3. Completed the in-person intervention |
|
|
|
4. Completed the remote intervention |
|
|
|
These focus groups are necessary for our understanding of study implementation and, in particular, why some participants decided to complete or not complete pre-purchase housing education and counseling.
The process for completing the focus group data collection relies on the 5 steps described below.
Step 1: Collection of Study Participant Contact Information
Using the study participant database, the study team will develop a list of study participants that includes the following fields:
Location (study site)
Type of pre-purchase education and counseling offered: a) in-person; or b) remote
In-person housing counseling agency (if more than one in the study location)
Whether pre-purchase education and counseling was: a) completed; b) started but not completed; or c) not started
Step 2: Contacting Study Participants
Once the lists of potential recruits for focus group participants are generated, a lead letter will be sent to notify them of the opportunity to participate in the focus group and to explain its purpose. The study team will allow four days for the letter to arrive before calling the study participant to assess his or her interest and eligibility for participating in a focus group. A study participant can also call the Study Hotline to initiate the process of focus group enrollment.
Step 3: Determining Eligibility
Once the study team reaches the study participant and confirms that he or she is interested in participating in the focus group, the study team will administer a brief eligibility assessment. The telephone script for this assessment is included as Appendix B. The purpose of the eligibility screening is to ensure that we have adequate numbers of participants for the four types of focus groups outlined above.
Through the eligibility assessment, the study team will collect information necessary to determine if study participants are eligible to participate in the focus group as well as the information needed to assign them to a particular focus group.
Step 4: Focus Group Assignment
After establishing that a study participant is eligible and willing to participate in a focus group, we will ask an additional set of questions in order to ascertain to which of the four types of focus groups he or she should be assigned. We will use the following focus group assignment criteria:
Intervention Status: The type of housing education and counseling to which the study participant was assigned: a) in-person; b) remote; c) choice-remote preference; d) choice- in-person preference.
Completion Status: Whether housing education and counseling was: a) completed; b) started but not completed; or c) not started.
Homeownership Status: Whether the participant has a) purchased a home; b) still in the process of purchasing a home; or c) no longer interested in purchasing a home.
After collecting this information, the study team contact will assign the focus group participants to the applicable focus group and give them the date, time and location for their assigned focus group. A follow-up letter with the same information will be mailed to focus group participants and they will also receive reminder telephone calls and, if appropriate, an email reminder. At the close of the eligibility and focus group assignment call, the study team member will confirm the participant’s mailing address and, if available, an email address where he or she can be reached.
Step 5: Focus Group Implementation
Members of the study team from Abt Associates and the University of North Carolina will conduct the focus groups, which will be recorded for later transcription. The study team anticipates that some eligible study participants may decline to participate in the focus groups and that we may also have some fall-off between the time that participants confirm their interest in participating and the actual time of the focus group. Therefore, we will recruit more study participants than necessary from the population of study participants that meet our sampling criteria. We will recruit approximately 12 participants for each group to ensure that we have at least 8 of the focus group. The instruments for the focus groups are provided in Appendix C. There is a focus group discussion guide and a short self-administered survey for each type of group (four versions). The short survey will be given to the study participants to complete prior to the start of the group discussion.
Collection of loan origination and loan performance data
Data collected from lenders and the Federal Housing Administration on study participants’ loan origination and mortgage performance will document detailed information on the timing of a home purchase, purchase price and down payment, monthly housing costs, and mortgage terms and conditions. The loan origination and performance data collected from lenders were part of in the previous submission (Control #2528-0293).
Collection of study participants’ credit bureau data from a credit bureau
The credit bureau data contain study participants’ credit scores and selected credit file attributes. This data includes information about housing outcomes (e.g. housing debt, monthly housing debt payment), loan performance (e.g. mortgage delinquency, bankruptcy, foreclosure), and other key outcomes for the study’s impact analyses. Abt has established a purchase agreement with the credit bureau to cover data pulls months 24, 30, 36, 42, 48, 54, and 60. These pulls are timed to match the collection of lender/FHA data, as well as to ensure timely collection of data for the interim and final analyses.
Collection of study participants’ housing counseling service data from HUD-approved housing counseling agencies:
The data collected from housing counseling agencies will serve as the primary source of information on housing counseling take-up, intensity, and completion. The housing counseling agency service tracking data collection module was submitted in the previous submission (Control #2528-0293).
Collection of contact information from study participants:
Every four months, the study team will mail study participants a letter asking for updated contact information. If their contact information has changed since they agreed to participate in the study, study participants simply need to update the form and mail it back. The study participant tracking letter was part of in the previous submission (Control #2528-0293).
Collection of study participants’ interim survey responses:
The interim survey will be used to collect follow-up information on study participants that will be useful for several parts of the study. The impact analyses will rely on the interim survey for information on covariates, subgroups, and measures of outcomes. The interim survey is provided in Appendix D. The advance letter that will be provided two weeks prior to administering the interim survey is provided in Appendix E.
The primary beneficiary of the planned data collection will be HUD, which will use the information from the study to understand the impact of pre-purchase homeownership education and counseling on a range of outcomes for low-, moderate-, and middle-income, first-time homebuyers. The findings of the eventual impact analyses will be published by HUD and are relevant to a broader set of policymakers, financial institutions, counseling agencies, and researchers.
Exhibit A-2 describes the target respondents, content, and reason for inclusion for each data collection activity. Copies of the data collection instruments are provided as Appendices.
Exhibit A-2. Item-by-Item Justification of Data Collection Instruments
Data Collection Activity |
Data Collection Instrument(s) |
Respondents, Content, and Reason for Inclusion |
Co-borrower Consent |
Mailed Co-borrower Consent Form (Appendix A) |
Respondents: Any person who is a co-borrower on a study participant’s home loan. (The number of respondents for this activity is undetermined at this time. However, the study team estimates the number of co-borrowers to be approximately 1,800 respondents.) Content:
Reason: In order for the study team to receive information on study participants’ loan originations and performance data, lenders need consent from all borrowers on the home loan. |
Focus Groups Eligibility Assessment |
Focus Group Eligibility Assessment (Appendix B) |
Respondents: 720 study participants across three study sites Content:
Reason: The focus group eligibility assessment is necessary to collect the information necessary to determine whether the study participant is eligible to participate in a focus group and to determine which of the four focus groups is appropriate to his or her situation. |
Focus Groups |
Focus Group Instruments: Discussion Guide and Self-Administered Survey (Appendix C) |
Respondents: 120 study participants across 3 study sites Content:
Reason: Focus groups will explore participants’ interaction with participating lenders and housing counseling agencies, including participation in housing counseling services. Respondents will include study participants assigned to a treatment group who have and who have not completed housing education and counseling activities to explore their decisions to complete these activities. Respondents will also include study participants who have and have not purchased a home to understand how housing counseling played into the home purchase decision. These short surveys and group discussions are necessary for our understanding of study implementation, as well as why some participants decided to complete or not complete pre-purchase housing counseling and education. Focus group discussions will allow us to probe more deeply into the experiences of the study participants. |
Collection of loan origination and mortgage performance data from participating lenders |
Loan Origination and Performance Data from Lenders (Submitted in prior submission) |
Respondents: Lenders’ staff that provide the study team data on loan originations and mortgage performance for study participants’ home loans. Content:
Reasons: Data collected from lenders and the Federal Housing Administration on study participants loan origination and mortgage performance will document detailed information on the timing of a home purchase, purchase price and down payment, monthly housing costs, and mortgage terms and conditions. |
Housing Counseling Agency Service Data
|
Service Tracking Data Collection Module (Submitted in prior submission) |
Respondents: Housing counseling agency staff who record and enter data on services received by study participants. Content:
Reason: This service tracking data will create a detailed record of the housing counseling and education services provided to study participants. The compiled data will provide the basis for description of the housing counseling interventions in the final report. A detailed understanding of the housing counseling services received by treatment group members is central to interpreting the evaluation outcomes and impact estimates. |
Tracking of Study Participants |
Study Participant Tracking Letters (Submitted in prior submission) |
Respondents: 6,000 study participants across 28 jurisdictions Content:
Reason: The collection of this information is necessary to confirm that the study team’s contact information is accurate and we will be able to reach him or her for the study’s follow-up data collection activities. |
Study Participant Responses to Interim Survey |
Interim Survey (Appendix D) |
Respondents: 6,000 study participants across 28 jurisdictions Content:
Reason: The interim survey will be used to collect follow-up information on study participants that will be useful for several parts of the study. The impact analyses will rely on the interim survey for information on covariates, subgroups, and measures of outcomes. |
Improved information technology will be used in this evaluation in three distinct ways:
To maintain data in a single location on all eligibility assessments, random assignments, loan pre-qualifications and housing counseling services received;
To assist the ongoing sample tracking and locating efforts; and
To facilitate collection of the survey data in standardized and accurate ways that also ensures the protection of the confidentiality of sensitive data.
The two main technologies that will be used are the Random Assignment and Service Tracking (RAST) system created for the study and CATI interviewing technology.
The study will generate a substantial amount of data, including eligibility assessments of potential study participants, random assignment records, and information provided by lenders and housing counseling agencies. A random assignment and service tracking system (RAST) will be developed to manage the various sources of data. The database will house information from lenders, housing counseling agencies, and study participants throughout the life of the study.
A key function of the RAST system is to support the efforts of the study team to monitor housing counseling take-up and track study participants over the course of the study period. Housing counseling agencies will upload or enter servicing tracking data directly into the RAST system, allowing study team members to ensure that study participants are completing assigned housing counseling activities. This will allow the study team to actively track study participants’ completion of housing counseling activities and to promptly conduct outreach to study participants who have not completed housing counseling activities. For such participants, site liaisons will contact the participant by phone, email, and mail regarding completion of housing counseling service.
A second function of the RAST system is to support the tracking of study participants over the course of the study to ensure a high response rate for future surveys. Throughout the study, the study team will passively track study participants through the U.S. Postal Service Change of Address database. Also, every four months, the study team will mail study participants a letter asking for updated contact information. If their contact information has changed since they agreed to participate in the study, study participants simply need to update the form and mail it back.
The RAST system will store all tracking updates we collect, and link them to the participants. Through active and passive tracking, the study team will be able to monitor both counseling service completion and change in location of study participants.
The study team will use CATI technology to administer the interim survey. The use of CATI reduces respondent burden, as interviewers can proceed more quickly and accurately through the survey instruments, minimizing the interview length and the need for subsequent call-back. Computer programs enable respondents to avoid inappropriate and non-applicable questions. For example, respondents who were not offered housing counseling will be routed past questions only relevant to those who were offered housing counseling. CATI also improves data quality through more uniform administration of the survey questions, more accurate implementation of the skip patterns, and immediate application of range checks, edit checks, and consistency checks of item-by-item responses.
During the study’s design phase, the study team worked closely with the study’s participating lenders and counseling agencies to identify the specific attributes needed for the study and to eliminate duplication. Each lender requires documentation of the consent of each co-borrower on a mortgage before data on a loan’s origination and servicing attributes can be shared. The study team must therefore keep detailed records of which study participants have co-borrowers and which co-borrowers have provided consent. The co-borrower consent collection activities will only be conducted when the study data suggests both that the study participant has a co-borrower and that the co-borrower has not yet provided consent.
The research team is working with local counseling agencies in each of the study sites to deliver in-person pre-purchase education and counseling services. The focus groups may be hosted at one of these agencies if the local agency has a suitable space that is available for the study team to use. Alternatively, the study team will find a location for the focus groups at a different organization or community facility. The use of local organizations’ space to conduct the focus groups is the option of the agency and not a requirement of participation in the study. The previous submission describes the study role of local counseling agencies in greater detail.
The co-borrower consent form will be completed by each co-borrower only once and is necessary to collect administrative data from the lenders.
The focus groups will be conducted only once and are necessary to document the participant experiences for the implementation analysis.
Participant tracking will occur every four months over the course of the next three years. Future tracking activities that extend past that date will be presented in future OMB packages.
The collection of service tracking data by housing counseling agencies is triggered by study participants’ use of the housing counseling services. The study team is requesting that housing counseling agencies upload service tracking data that they already collect directly into the study’s data collection system (RAST) and, when needed, input supplemental data into this system. To ensure data completeness of the study’s required data collection fields, housing counseling agencies will need to provide information on both the homeownership education and the individualized counseling that study participants receive.
The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6
(Controlling Paperwork Burden on the Public, General Information Collection Guidelines). There are no circumstances that require deviation from these guidelines.
In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and Urban Development (HUD) published a notice in the Federal Register on March 10, 2014. The docket number is FR-5758-N-03. The Federal Register Notice appeared on pages 13326 and 13327.
The Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation is being implemented in partnership with Abt Associates Inc. (the prime contractor), Abt SRBI, the Center for Community Capital at the University of North Carolina, Doug Dylla LLC, and Stephanie Moulton at Ohio State University.
In addition, the study has an advisory panel of approximately six academic and professional experts in the homeownership counseling field. The advisory panel includes Peter Zorn (Freddie Mac), J. Michael Collins (University of Wisconsin-Madison), Christopher Herbert (Harvard University), Jennifer Turnham (Abt Associates Inc.), Kristopher Rengert (FDIC), and Carolina Reid (UC Berkeley). Additionally, Dr. Don Dillman an expert in survey and response rate methodology at the Washington State University is a consultant on this project.
Abt Associates and Abt SRBI will conduct all activities associated with the collection of co-borrower consent. Abt Associates and the Center for Community Capital will conduct all activities associated with the focus groups
The previous submission describes in detail the incentive payments that will be provided to study participants for participation in the study.
Following random assignment, participants who are assigned to the treatment groups receive $150 as a thank you for the time required to participate in housing counseling. The payment is made in two parts. First, participants receive a $50 check as a token of appreciation for their study participation when they initiate housing counseling with the assigned housing counseling provider. Second, they receive a $100 check upon completing housing counseling in recognition of the time and other expenses they may incur in order to participate in housing counseling. Study participants who are assigned to the treatment groups are expected to spend between 7 and 11 hours completing pre-purchase education and counseling through a study participating counseling agency. Additionally, these study participants may also incur expenses for childcare, transportation, parking, internet service, and phone charges. The incentives provided to the treatment group members are intended help offset some of these charges and are commensurate with the amount of effort spent on study activities. In addition, the housing counseling services are provided free of charge to participants in the two treatment groups. Participants who are assigned to the control group receive a $50 check after random assignment.
Throughout the study period, study participants will also receive a $5 check for returning tracking letters to confirm their current contact information. Every four months, the study team will mail study participants a letter asking for updated contact information.
The study team will not provide any additional incentive to co-borrowers who complete and return the co-borrower consent form.
A subset of 120 study participants will be recruited to participate in the focus groups. Focus group participants will receive an incentive of $60 to compensate them for their time associated with focus group participation.
Study participants who complete the interim survey will receive $35 for their time associated with completing the survey. An advance letter will be mailed to study participants two weeks prior to administering the interim survey. The advance letter will contain a $2 pre-incentive.
Because of the nature of the information collected from and about study participants, strict confidentiality procedures will be followed for this evaluation. The information requested under this collection is protected and held confidential in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974). As required by 5 U.S.C. 552a (Privacy Act of 1974), HUD has published a Systems of Record Notice (SORN) in the Federal Register.
Prior to beginning the data collections included in this submission, the study protocols and materials will be reviewed in detail by Abt Associates’ Institutional Review Board (IRB).
All study participants complete the informed consent process at the time of study enrollment. The study also has procedures that allow study participants to withdraw consent at any time using procedures that have been communicated to study participants during the consent process and that can be found in the consent form or by calling the study hotline. Before administering the interim survey, study participants will be reminded of their consent to participant and options to withdrawal from the study at any time.
The mailed co-borrower consent form contains the consent information provided to co-borrowers. This information has been approved by the Abt Associates IRB. Co-borrowers are also able to withdraw their consent at any time by calling the study hotline.
The focus group instruments in Appendix C contain the additional informed consent information that will be read to study participants related to their participation in the focus groups. Focus group participation is voluntary and participants can leave at any time.
The study has established rigorous data security and confidentiality provisions that are described in the previous submission. Additionally, the study’s data system and data security protocol have since been reviewed by HUD and each of the study’s participating lenders.
An assurance of confidentiality is included in the study participation consent agreement through which participants provide informed consent. An assurance of confidentiality also will be made to all respondents as part of the introduction to each of the surveys.
The co-borrower consent form contains the confidentiality promise made to co-borrowers. The focus group instruments in Appendix C contain the confidentiality promise associated with participation in the focus groups.
The focus groups include the participants’ experiences with the study’s recruitment and enrollment process; participants’ assignment to counseling services and completion of counseling; and participants’ opinions on the pre-purchase counseling they received. This information will be treated as sensitive, confidential information to be used only for the purposes of this study. Respondents will also be reminded that they can refuse to answer any questions with no repercussion.
The co-borrower consent form collects the co-borrower’s name and date of birth. The study’s data security plan details how this information will be stored and protected.
The interim survey includes study participant’s contact information; the stage in the home purchase process and preparedness for home purchase (e.g., the extent of respondents’ home search prior to pre-qualification, intended down payment amount, expected timeline for home purchase); and interim measures of time-varying study outcomes (e.g., income sources and total family income, financial literacy, non-housing savings/debts, budgeting activities, and employment and earnings); and mortgage characteristics and features (e.g. down payment, monthly mortgage payment, total loan amount, and loan type).This information will be treated as sensitive, confidential information to be used only for the purposes of this study. Prior to the interim survey, respondents will be reminded that they can refuse to answer any questions with no repercussion.
The collection of co-borrower consent involves including the co-borrower consent form in the study’s regular tracking letters, along with a request for the co-borrower to review, sign, and return the written consent form. For co-borrowers who do not return the written form, the study will also be able to collect consent verbally at the time of the interim survey and during phone calls with the co-borrower. The study estimates that approximately 1,800 study participants will have co-borrowers. The co-borrowers’ review of the co-borrower consent information and completion of the consent process is estimated to require approximately 5 minutes per co-borrower.
A total of up to 144 study participants will participate in 12 focus groups of 8-12 participants each. Each focus group will take approximately 120 minutes. Prior to the focus groups, the study team will contact eligible study participants in three study sites to request participation in a focus group and confirm eligibility information. The recruitment calls are estimated to take approximately 5 minutes per person. The study’s enrollment projections suggest that the three largest study sites will include approximately 720 eligible study participants, although recruitment will end sooner if the available focus group slots are filled.
The average time per study participant (up to 6,000 study participants) to complete the interim survey is 35 minutes.
As noted in the previous submission, the average time per client for counseling agencies to document the services provided to study participants is 10 minutes, with responses required for both the educational component and for the counseling services. The average time for lenders to prepare study participants’ loan origination and performance data for the study team is 60 minutes. The study team will ask for this data at 6 points during the next 3 years from each lender. The study mails study tracking letters 2-3 times per year. The average time for study participants’ review of the letters and return of the tracking form is 5 minutes
Information Collection |
Number of Respondents |
Frequency of Response |
Responses Per Annum |
Burden Hours Per Response |
Total Burden Hours |
Annual Burden Hours |
Hourly Cost Per Response |
Annual Cost |
Interim Survey |
6000 |
1 |
1 |
35 min |
3,500 hours |
3,500 hours |
$35 |
$210,000 |
Focus group recruitment calls |
720 |
1 |
1 |
5 mins |
60 hours |
60 hours |
$25/hour |
$1,500 |
Focus groups |
144 |
1 |
1 |
120 mins |
288 hours |
288 hours |
$25/hour |
$7,200 |
Co-borrower consent form |
1800 |
1 |
1 |
5 mins |
150 hours |
150 hours |
$25/hour |
$3,750 |
Loan origination and performance data: Lenders |
3 |
6 |
2 |
60 min |
18 hours |
6 hours |
$35/hour |
$630 |
Service tracking information: Housing counseling agencies |
3450 |
2 |
1 |
10 min |
1,150 hours |
1,150 hours |
$25/hour |
$28,750 |
Tracking Letter |
6000 |
9 |
3 |
5 min |
4,500 hours |
1,500 hours |
$5 |
$270,000 |
Total |
18,117 |
21 |
10 |
|
9,666 hours |
6,654 hours |
|
$521,830 |
This data collection effort involves no recordkeeping or reporting costs for respondents other than the time burden to respond to questions on the data collection instruments as described in item A.12 above. There is no known cost burden to the respondents.
Data collection of the focus groups is $94,770. This data collection activity is accounted for in the cost associated with our prior submission (Control #2528-0293).
The total costs to the government of all other activities described by this submission is $4,382,604 if all contract options associated with these data collection activities are executed.
The focus groups replace the study participant interviews in the previous submission. This change is necessary to improve the quality of the data collected for the implementation analysis. The focus group recruitment calls and focus groups together require 348 total burden hours.
The revisions made to the collection of co-borrower consent increases the total burden hours to 150. This is an increase of 45 hours from the previous submission.
This section describes the tabulation plan and analyses that will be conducted by the study. The implementation analyses and impact analyses rely on data collected under both the prior submission and this submission. The focus group analyses rely on the data collected by the focus groups described in this submission. The baseline analyses and balance testing will be conducted using data from the prior submission.
Implementation analyses will document the implementation experience of each lender and housing counseling agency partner in each jurisdiction. Given the number of staff at both participating lenders and housing counseling agencies that the study team is working with, it is extremely important to document procedures and protocols that sites implement. The implementation analyses will integrate the information collected through the lender customer service teams’ observations, recruitment call outcome reports, and housing counseling agency staff interviews with the site liaisons.
The implementation analyses will describe lenders’ fidelity to the study design and enrollment targets for each jurisdiction. We will examine the steps that each lender used for early stage customers and compare them across lenders and jurisdictions. In particular, we will document the extent to which lenders introduced the study to all customers who might qualify for the study. The implementation analyses will identify the key operational challenges that emerged as recruitment was implemented, and we will examine how, and how well, those challenges were addressed.
The study team will examine three components of the housing counseling interventions assigned to study participants; (1) housing counseling agencies’ fidelity to the specific interventions; (2) study participants’ experiences with services received at their assigned counseling agency; and (3) housing counseling completion rates within each intervention. This information will be collected through the study’s data collection system, site monitoring, staff interviews with housing counseling agencies, and focus groups with study participants.
The information collected through the study’s data collection system will be used to document the timing and nature of the enrollment and housing counseling activities that occur during the study period. These data will be coupled with remote site monitoring, on-site observations, staff interviews, and focus groups to observe and learn about the experiences that study participants had at the housing counseling agencies. The focus groups will also examine why some study participants did not pursue or complete their assigned housing counseling service.
The focus groups contribute to the study’s Implementation Analysis. The focus groups will be recorded for later transcription by a professional transcription service. Transcripts will be reviewed by the study team to identify an initial set of themes that correspond to the issues addressed by the implementation analysis. These themes will then be translated into code and entered in to NVivo or a comparable qualitative analysis software. The use of such software allows us to identify and catalogue each time a theme is mentioned in the transcripts. The output allows the study team to flag issues that arise consistently, refining themes through an iterative process if necessary. This approach to the analysis produces rich qualitative information for the implementation analysis.
Data from the lender intake form, eligibility assessment, and baseline survey will be used both to describe the study population and to examine the extent to which the randomized groups are balanced across relevant participant characteristics.
As a reminder, the baseline information collected (under the prior submission) during the enrollment process—the lender intake form, eligibility assessment, and baseline survey—will be used in the baseline analyses to accomplish four objectives:
We will present detailed descriptive information about the sample of recruited borrowers, including initial measures of financial literacy, savings, debts, credit quality, and other characteristics.
We will document the stage of the home purchase process and the extent of previous search activities conducted by enrolled participants.
We will compare the sample of study participants to lender data and to secondary data sources to examine the extent to which the sample differs from two groups. First, we will compare the study sample to the universe of customers at each lender that meet the basic screening criteria to be offered participation (defined in Section B.1.1). This comparison provides insight into the extent to which refusal of study participation (or selective offer of study participation by loan officers) is more likely for certain types of lender customers. Second, we will compare the study sample to the population of LMMI first-time homebuyers and to the population of FHA homebuyers using secondary data and FHA administrative data, respectively. These comparisons will examine the extent to which the sample of study participants differs from these broader populations. In both cases, the analysis will be descriptive and cannot be used to formally extrapolate from the sample to either population. Instead, these comparisons are intended to provide context for how the study sample compares to broader populations of interest.
Finally, as is common in experimental evaluations of policy initiatives, we will use the baseline data to affirm that treatment and controls groups differ only by chance.
Data from the Interim Survey and administrative data will supplement the information collected during the initial phase of the study to conduct the impact analyses. These analyses will rely on the data collection activities covered by the prior submission for baseline measures of the time-varying study outcomes and to define covariates and subgroups for the impact analyses. These covariates serve the important purpose of increasing the precision of impact estimates, thereby reducing the size of an impact that we can detect, if it exists, given sample size constraints. In addition, subgroups may be defined with respect to borrower personal characteristics (from the baseline survey); financial characteristics (from the lender intake form and baseline survey); and neighborhood characteristics (which requires knowledge of the borrower’s address from the baseline survey). We also anticipate examining endogenously-defined subgroups (for example, housing counseling intensity levels (from the housing counseling service data)), using the baseline measures of participants’ financial decision-making, self-efficacy, learning styles, and follow-through, for example. The counseling agencies’ service tracking data collected under both this submission and the prior submission will provide the measures of treatment intensity for the impact analyses.
The administrative data and Interim Survey provide the information used to construct the outcomes for the Impact analyses. The experimental evaluation design balances the pre-existing characteristics of the study participants assigned to each of the treatment groups and to the control group. Therefore, the impact of being offered a treatment can be estimated by simply comparing mean outcomes of those offered each treatment, relative to those in the control group. Part B describes the study’s statistical approach for conducting the treatment-control comparisons, subgroup analyses, and dosage analyses in more detail.
To supplement the quantitative impact analyses, the focus groups will be used to conduct a limited amount of qualitative analysis of study participants’ experiences with pre-purchase homeownership education and counseling services—in addition to their contribution to the implementation analyses described above. The focus groups will examine study participants’ experiences with the two types of pre-purchase education and counseling offered. While the study’s design does not contain a robust qualitative study, analysis of the focus group transcripts using NVivo software will be used to supplement the findings of the quantitative impact analysis by extracting qualitative information about how the pre-purchase homeownership education and counseling services may have affected study participants’ behaviors.
Under the current schedule, study participant recruitment and enrollment will be conducted over a 20 month period. The interim survey will be administered to study participants 12 months after study enrollment. The collection of co-borrower consent will correspond to the start of the administering the interim survey to study participants and continue throughout the tracking period.
The study team will continue to collect loan origination and performance data once every six months throughout the study period. We will also continue to collect service tracking data from participating housing counseling agencies until 12 months after the end of study enrollment.
All data collection instruments will prominently display the expiration date for OMB approval.
This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 CRF 1320.9).
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Abt Single-Sided Body Template |
| Author | Nichole Fiore |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |