OMB Submission Part B 8.20.12

The Pre-Purchase Homeownership Counseling Demonstration and Impact Evaluation has three potential response universes: (1) study participants; (2) participating lenders; (3) participating housing counseling agencies.

B.1.1 Study Participants

The study design will be a randomized experiment. We will recruit up to 7,000 low-to-moderate income, first-time homebuyers from three national lenders across 20 jurisdictions nationally. Lenders’ staff will conduct an initial assessment of each customer that they work with during the enrollment period, offering study participation to any customer that meets the basic criteria for participation. Customers interested in the study will be referred to the study team for further eligibility screening.

The universe of potential study participants includes all customers who work with a participating study lender in one of the 20 study jurisdictions, meet each of the eligibility criteria, and are offered the opportunity to participate in the study. This universe of potential study participants is not designed to be representative of the population of low- to moderate-income, potential first-time homebuyers throughout the nation or in the selected MSAs. For this study, recruitment of a representative sample of potential homebuyers is not possible, because of the logistics involved in participant recruitment and provision of counseling. The core feature of this study is instead the random assignment of study participants, which provides the strongest basis for interpreting the difference between treatment and control groups as the causal impact of counseling.

The logistics of study enrollment also limit our ability to identify precisely the universe of potential study participants in lenders’ administrative data or the other data sources collected for this study. While lenders’ staff will screen for eligibility criteria prior to introducing the study to potential participants, additional eligibility criteria will not be observed until the eligibility assessment—after some customers have refused the offer to participate. As a result, some customers who refuse the initial offer of participation will not meet all of the study eligibility criteria and therefore be outside the universe of potential participants.

B.1.2. Study Participating Lenders

The study team will be working with three national lenders to implement this evaluation. Each lender will identify its branch offices, call centers, loan officers, and customer service teams that will be recruiting customers within the study jurisdictions. Branch offices will be limited to those that assist customers who seek a loan for a home purchase. The loan officers, sales representatives, call-center staff, and managers selected for interviews and observations will come from the universe of lender staff who recruit customers through the participating branch locations and call centers.

B.1.3. Study Participating Housing Counseling Agencies

The study team selected CredAbility as the provider of remote housing counseling for the evaluation. For in-person housing counseling, the study team is working with NeighborWorks America and will recruit its local affiliate organizations to provide in-person housing counseling in each jurisdiction. However depending on NeighborWorks coverage, the study team may explore other housing counseling agencies that fit the study’s needs in each jurisdiction. The housing counselors and agency staff selected to participate in the housing counseling agency staff interviews will be drawn from the universe of housing counselors and agency staff who are involved with the study at each housing counseling provider.

B.2 Statistical Methods

B.2.1 Sampling Plan

The evaluation requires sampling of jurisdictions, housing counseling agencies, lenders, study participants, implementation study participant interviewees, and lender and housing counseling staff interviewees. Below we discuss the sampling plan for each of these groups. It should be noted that the sample is not designed to produce a representative sample of housing counseling recipients, housing counseling agencies, or lenders. This limitation is common among randomized experiments where the logistics of implementing the experiment do not allow for a representative sample to be drawn. For this study, the strong internal validity provided by random assignment is the core feature of the study design.

Jurisdictions

Participant enrollment and study implementation will occur in 20 jurisdictions. Because current volumes of loan applications and originations are relatively low, the primary criterion for selecting jurisdictions is the expected volume of eligible study participants. However, the process of selecting the sample of jurisdictions will also consider the ability of local in-person housing counseling agencies to provide services to enrolled participants throughout the jurisdiction. The process of matching in-person housing counseling agencies with study jurisdictions is currently underway and will be completed shortly, with the selected jurisdictions identified in the Research Design, Data Collection, and Analysis Plan.

The specific steps for determining the selected jurisdictions are as follows:

Collect loan application data from each lender. Use these data to rank jurisdictions (MSAs) based on the number of applications that meet the study eligibility criteria that are observable in the lenders' data. Then, identify the 20 to 30 jurisdictions with the highest number of eligible applications. Exhibit B.2.1 shows the top 30 jurisdictions, separating the draft sample of 20 jurisdictions from the next 10 ‘replacement’ jurisdictions.
Obtain locations and coverage information for local NeighborWorks affiliates in each jurisdiction. Exhibit B.2.1 is currently being reviewed by NeighborWorks America to identify any jurisdictions where coverage is low. Once this review is completed, we will eliminate any jurisdictions where housing counseling coverage is low.
Select the jurisdictions with the highest number of applications that also have acceptable coverage of housing counseling agencies. The evaluation will include 20 jurisdictions, depending on the number required to enroll 7,000 study participants. The final set of jurisdictions will be identified in the Research Design, Data Collection, and Analysis Plan.
Replace any jurisdiction where local housing counseling agencies cannot be recruited. The study team will first try to identify and recruit local NeighborWorks’ agencies. However, depending on NeighborWorks’ coverage, the study team may explore other housing counseling agencies that fit the study’s needs in each jurisdiction. It is possible that the recruitment of local housing counseling agencies may not succeed in all jurisdictions. If a local housing counseling agency refuses to participate, the study team will explore the extent of any gap in housing counseling coverage and the potential to recruit a replacement organization. If adequate coverage of a jurisdiction is not possible, it may be necessary to replace that jurisdiction during the housing counseling agency recruitment process in Fall 2012. Potential replacements will be considered in the order they appear in Exhibit B.2.1.

Exhibit B.2.1 Top 20 MSAs by 12 Month Application Volume

	Draft Sample of 20 MSA/Jurisdictions
	Atlanta-Sandy Springs-Marietta, GA
	Chicago-Joliet-Naperville, IL-IN-WI
	Dallas-Fort Worth-Arlington, TX
	Detroit-Warren-Livonia, MI
	Houston-Sugar Land-Baytown, TX
	Los Angeles-Long Beach-Santa Ana, CA
	Miami-Fort Lauderdale-Pompano Beach, FL
	Minneapolis-St. Paul-Bloomington, MN-WI
	New York-Northern New Jersey-Long Island, NY-NJ-PA
	Orlando-Kissimmee-Sanford, FL
	Philadelphia-Camden-Wilmington, PA-NJ-DE-MD
	Phoenix-Mesa-Glendale, AZ
	Riverside-San Bernardino-Ontario, CA
	Sacramento-Arden-Arcade-Roseville, CA
	San Diego-Carlsbad-San Marcos, CA
	San Francisco-Oakland-Fremont, CA
	San Jose-Sunnyvale-Santa Clara, CA
	Seattle-Tacoma-Bellevue, WA
	Tampa-St. Petersburg-Clearwater, FL
	Virginia Beach-Norfolk-Newport News, VA-NC
	‘Replacement’ MSA/Jurisdictions
1	Stockton, CA
2	Portland-Vancouver-Hillsboro, OR-WA
3	Raleigh-Cary, NC
4	St. Louis, MO-IL
5	Boston-Cambridge-Quincy, MA-NH
6	San Antonio-New Braunfels, TX
7	Richmond, VA
8	Jacksonville, FL
9	Fresno, CA
10	Pittsburgh, PA

Note: The initial 20 jurisdictions are ordered alphabetically. The next 10 ‘replacement’ jurisdictions are listed in the order that they will be considered by the study team.

Lenders

The study team has partnered with three national lenders—Bank of America, Citibank, and Wells Fargo—for the implementation of this study. The participating lenders were chosen purposively, based on their volume of loan originations and willingness to partner for this study.

HUD-Approved Housing Counseling Agencies

The study team has partnered with CredAbility and NeighborWorks America to provide housing counseling services to study participants. Each housing counseling agency was selected purposively, based on the adherence of the housing counseling curricula and delivery mechanisms to the study’s defined interventions: remote and in-person counseling services.

Remote Housing Counseling (Online Education + Telephone Counseling): Study participants assigned to the remote housing counseling treatment group will receive services from CredAbility, a national organization that provides pre-purchase housing counseling services to potential homeowners. CredAbility offers an online education module that covers topics such as personal finances, initial home search, and the home purchase process. CredAbility also offers pre-purchase telephone housing counseling to customers. Telephone housing counseling sessions are individualized based on the client’s needs, but can cover topics such as credit repair, household budgeting, mortgage options and financial assistance. Because CredAbility offers its services remotely, that organization will be able to provide pre-purchase housing counseling and education to study participants within any study jurisdiction.

In-person Housing Counseling (Group Education + One-on-One Counseling): Since this evaluation will be implemented in 20 jurisdictions nationally, study participants who are assigned to the in-person housing counseling treatment group will need a referral to a local housing counseling agency. To facilitate this process, the study team has partnered with NeighborWorks America. NeighborWorks America is a national organization that oversees hundreds of community-based non-profit organizations, many of which provide pre-purchase housing counseling and education services. However, depending on NeighborWorks coverage, the study team may explore other housing counseling agencies that fit the study’s needs in each jurisdiction.

Study Participants

The universe of potential study participants is defined in Section B.1.1 to include all eligible customers who communicate with a study participating lender during the enrollment period. The sample of study participants is a 100 percent sample of the universe, excluding only those customers who cannot be contacted, refuse the initial offer of study participation, do not meet the study’s eligibility requirements, or do not provide consent.

The complicating factor is that the universe of study participants cannot be directly observed in the data collected for the study. Ideally, all of the eligibility criteria would be applied to customers before the initial offer of study participation, allowing the universe to be directly observed in the data. However, the logistics of the enrollment process require that the initial offer of study participation (and some customer refusals) occurs before the full eligibility assessment. This section therefore clarifies the steps through which the sample of study participants is determined. Individuals who are screened for not meeting one of the eligibility requirements are not in the universe of potential study participants. Instead, the steps in the enrollment process simultaneously screen out customers who are not in the universe (i.e. do not meet one of the eligibility criteria) and produce the sample of study participants from the universe (i.e. remove any customer in the universe who is not interested or who does not consent).

The steps in the enrollment process are as follows:

Early-Stage Borrower Seeks a Home Loan: Low-to-moderate income homebuyers contact one of three lenders participating in the study about a mortgage purchase for an owner-occupied. Lenders’ staff screen households that meet the definition of a first-time homebuyer, the income criteria, and the criterion that the customer must be in the early stages of seeking a home loan.
Lender Recruits Potential Study Participant: If customers meet the above criteria, lenders’ staff will discuss the opportunity to participate in a HUD-sponsored study examining the effects of pre-purchase counseling on potential home buyers. If the customer expresses interest in study participation, lenders’ staff direct the customer to the study team.
Study Team Conducts Eligibility Assessment and Administers the Participation Consent Agreement and Baseline Survey: The eligibility assessment will verify that the customer meets the definitions of low-to-moderate income and first-time home buyer and will determine whether the customer has received any pre-purchase housing counseling prior to enrollment. The study team will also screen out any customers who are only just beginning to think about homeownership and who do not intend to purchase a home in the subsequent year. This assessment will also ask questions about a customer’s ability to participate in housing counseling services and complete the study's interviews such as language preference, transportation options, and accessibility of a computer connected to the internet. Based on responses to these questions, some customers will be unable to qualify for the study. Those that do qualify for the study will be asked for their informed consent to participate. However, some customers may be unwilling to agree to participate, despite their earlier expression of interest.

Study Participant Implementation Process Interviewees

Within the last eight weeks of study enrollment a small subset of study participants that were randomly assigned to receive either housing counseling intervention (i.e., assigned to one of the two treatment groups) will be selected and contacted for a 30 minute telephone interview. Specifically, 200 study participants from all study jurisdictions will be randomly selected to participate in the interview. The number of study participant interviews in each jurisdiction will be drawn in proportion to the number of study participants in each jurisdiction. Thus, jurisdictions with more enrolled study participants will have a larger number of study participant interviews than jurisdictions with fewer enrolled study participants.

Within each jurisdiction, the interview will include participants assigned to each of the two treatment groups, with an equal number of participants drawn from each treatment group. No participants will be drawn from the control group. We will only select study participants who have either completed the housing counseling intervention or have not initiated contact with their assigned housing counseling agency. No study participants will be selected for the interview that are in the midst of completing counseling.

The study team is targeting a 75 percent housing counseling take-up rate among treatment group members overall. We will also apply that percentage to the allocation of interviewed participants, interviewing three participants who have completed housing counseling for every one participant who decided not to initiate housing counseling. This allocation reflects the substantive content of the interviews, which seek to collect information on the take-up decision but also place emphasis on study participants’ interactions with counseling providers.

The study team anticipates that some study participants may decline to participate in these interviews. Due to this factor, we expect to have approximately an 80 percent response rate and therefore we will plan to oversample from the population of study participants that meet our sampling criteria.

Once the sample is drawn, the study team will inspect the allocation of interviews across in-person housing counseling agencies to ensure that it includes coverage of any specific agencies of interest (e.g., any non-NeighborWorks affiliate agencies).

It is possible, though not likely, that the experience of participating in the interview could affect a study participant’s decision to purchase a home or lead the participant to initiate housing counseling (if he/she has not done so already). However, the study team feels that the information gleaned from these interviews is valuable and necessary. Because the number of study participants being interviewed is relatively small, any effect the interviews have on home purchase or housing counseling take-up rates would likely be too small to be detected.

Lender and Housing Counseling Staff Interviewees

The study also includes interviews with lender and housing counseling agency staff in each jurisdiction during a site visit toward the beginning of study enrollment. These interviews will occur within the first 16 weeks of study participant enrollment and review the implementation activities, understanding of study processes, and obstacles for lenders and housing counseling agencies. Because each jurisdiction is likely to include multiple lender branches and housing counseling agencies, site visits will not be able to include interviews with each participating lender and housing counseling agency. Additionally, the study team will work with lenders to identify staff from call-centers who been instrumental in the process of a centralized recruitment effort. The selection of specific lender and housing counseling agency staff for interviews is purposive. In each jurisdiction, we will interview the point person for study activities at selected housing counseling agencies and selected lenders, along with any managerial staff that played major roles in implementation. Additionally, we will select a subset of lenders’ staff and housing counseling agency staff for interviews. We will prioritize high-volume agencies and lender operations—as well as organizations that encountered obstacles to implementation—and the selection of staff will depend in part on staff availability. This selection approach ensures that the interviews will capture variation in key dimensions of interest to understand the early implementation experience in each jurisdiction.

B.2.2 Analysis Plan

The data collection activities included in this request will be used to conduct three main types of analyses:

Implementation analyses;
Analysis and documentation of random assignment; and
Baseline analyses.

Each of these analyses is discussed in this section, followed by a discussion of the impact analyses that will primarily be conducted in the next phase of the study using the follow-up data that will be collected. (We will prepare a separate OMB request for the collection of follow-up data.)

Implementation Analyses

Implementation analyses will document site operations at lenders and housing counseling agencies and examine housing counseling completion rates with study participants within each intervention type. Because of the number of lending branches and centralized telephone operations and housing counseling agencies that the study team will be working with, it is extremely important to document the implementation experience in each jurisdiction.

The analysis of site monitoring activities will focus on how implementation procedures and study protocols were followed at each site and how sites responded to challenges, obstacles, and successes with study activities. Site monitoring activities will be complemented by observations and staff interviews during each jurisdiction’s site visit. The analysis will seek to identify common themes across housing counseling agencies and lenders in all study jurisdictions.

Study participant interviews will aid in our understanding of study implementation. The study participant interviews will examine participants’ experience with the study, including enrollment, monitoring, and any interactions with housing counseling agencies. In particular, the study participant interviews will examine why study participants in each treatment group decided to complete or not completed pre-purchase housing counseling and education. Interviews will be analyzed using NVivo software to examine common experiences and opinions of study participants.

Analysis and Documentation of Random Assignment

Data from the baseline survey and lender origination file is required to document the random assignment of study participants and assess the extent to which it was successful. We will test for whether baseline characteristics are similar across the treatment groups and control group. The initial analysis will examine the extent to which the randomized groups are balanced across a variety of customer attributes including demographic (race, gender, marital status, education) and financial (credit score, savings, income) characteristics.

Baseline Analyses

Data from the lender intake form, eligibility assessment, and baseline survey—along with administrative data provided by each lender—will be used to accomplish three objectives.

First, using data from the enrollment process, we will document the stage of the home purchase process and the extent of previous search activities conducted by enrolled participants.

Second, we will use the baseline survey to analyze the financial literacy, savings, debts, credit quality, and other characteristics of study participants at the time of enrollment. Using this information, the baseline report will present detailed descriptive information about the sample of recruited borrowers.

Third, we will compare the sample of study participants to lender data and to secondary data sources to examine the extent to which the sample differs from:

The universe of customers at each lender that meet the basic screening criteria to be offered participation (defined in Section B.1.1); and
The population of low- to moderate-income first-time homebuyers and the population of FHA homebuyers.

The comparison of the study sample to the universe of customers at each lender that meet the screening criteria provides insight into the extent to which refusal of study participation (or selective offer of study participation by loan officers) is more likely for certain types of lender customers. The results on this analysis will help to inform the extent to which some types of customers may be less likely to meet the study’s eligibility requirements or to provide consent for study participation. The comparison of the study sample to the population of low- to moderate-income first-time homebuyers and FHA homebuyers will be conducted using FHA administrative data and secondary data such as the American Housing Survey. These comparisons will examine the extent to which the sample of study participants differs from these broader populations. In both cases, the analysis will be descriptive and cannot be used to formally extrapolate from the sample to either population. Instead, these comparisons are intended to provide context for how the study sample compares to broader populations of interest.

Impact Analyses

The primary impact analyses for the evaluation will be conducted using outcomes collected in follow-up surveys, which will be covered in a separate OMB package. These analyses will rely on the data collection activities covered in this submission for baseline measures of time-varying outcomes and to define covariates and subgroups for the impact analyses. For example, subgroups may be defined with respect to borrower personal characteristics (from the baseline survey); financial characteristics (from the lender intake form and baseline survey); neighborhood characteristics (which requires knowledge of the borrower’s address from the baseline survey); and dosage levels (from the counseling services data).

A small number of preliminary impact analyses may be performed at the time of the baseline analyses using outcome measures defined from administrative data from lenders. In these preliminary analyses, the data collection activities covered in this submission would play the same role discussed above for the primary impact analysis: establishing baseline measures of time-varying outcomes and defining covariates, and potential subgroups.

The experimental evaluation design balances the pre-existing characteristics of the study participants assigned to each of the treatment groups and to the control group. Therefore, the impact of being offered a treatment can be estimated by simply comparing mean outcomes of those offered each treatment, relative to those in the control group. We will estimate this difference in means with multivariate regressions, as controlling for additional study participant characteristics will increase precision. This same approach can be used for each outcome of interest. For example, if we find that study participants who receive Online Education + Telephone Housing Counseling have higher homeownership rates than control group members, we can interpret the difference in homeownership rates as the causal impact of receiving Online Education + Telephone Housing Counseling.

The randomization of participants to treatment group and control groups produces an estimate that reflects the impact of being offered housing counseling— referred to as the Intent to Treat (ITT) estimate. Commonly, adjustment of the ITT allows estimation of the impact of housing counseling on those who actually complete the prescribed housing counseling—referred to as the Treatment on the Treated (TOT) estimate. The two will differ because of incomplete take-up (i.e., some people who are offered a treatment will choose not to take it up). We will compute TOT estimates by two-stage least squares regression.

As an extension to the analyses described above, we will perform subgroup analyses based on post-intervention borrower characteristics. For instance, we will compare outcomes for participants who fully complete the housing counseling intervention, relative to those who complete only part, relative to those who do not initiate housing counseling at all. To capture the effect of varying “dosage,” we will use a propensity score matching approach which allows creation of subgroups that remain internally valid. Identifying these subgroups on the basis of their pre-existing traits allows comparison of subsets of the treatment and control groups that creates an unbiased impact on that subgroup.

B.2.3 Justification of Level of Accuracy

This section presents minimum detectable effects for the impact analyses discussed in the previous section. Minimum detectable effects (MDEs) are the smallest true effects of an intervention that researchers can expect to detect as statistically significant when analyzing samples of a given size. The calculation of MDEs assumes that the recruitment process successfully enrolls 7,000 study participants and that the response rate to the final follow-up survey reaches 75 percent of the initially recruited sample. We propose to allocate the sample as follows: 2,975 (43 percent) control group; 2,013 (29 percent) remote counseling; and 2,013 (29 percent) in-person counseling.

Exhibit B.2.3 presents the MDE values for the proposed allocation. The MDEs in Exhibit B.2.3 are calculated using standard assumptions regarding power, significance level, R-squared values, and crossover. These assumptions are:

80 percent power;
A two-tailed test at the .10 significance level; and,
An R-squared value of .10.

The MDE and Impact Size figures in Exhibit B.2.3 are shown for dichotomous outcomes—variables like foreclosure that take on a value of 0 or 1. The initial column of estimates presents the MDE and Impact Size figures when the mean value of the outcome is .1 (i.e., 10 percent of control group members experience foreclosure). This value also applies if the mean value is .9. The final two columns present similar values when the mean value of the outcome is .3 (or .7) and when it is .5.

Panel A presents MDE values for comparing each of the two types of counseling to the control group. For a dichotomous outcome variable with a mean control group value of .5, the MDE estimate when there is a 25 percent rate of no shows and crossovers (our target rate) is 5 percentage points. This value implies that the average value for the treatment group would have to increase to 55 percent for the impact of counseling to be significant. The required average value is 54 and 58 percent if 10 percent and 50 percent of sample respondents are no shows.

The actual impact on treated participants must be proportionally larger than the ITT estimate if some treatment group participants do not receive treatment. The row labeled ‘Impact Size’ shows the necessary size of the actual impact on the set of participants who take up counseling. The impact size increases non-linearly as no shows increase. For an outcome variable with a control group mean of .5, the actual impact of housing counseling when 25 percent of treatment group members do not complete housing counseling is 7 percentage points—compared to 5 percentage points for the MDE. If 10 percent of treatment group members do not complete housing counseling, the actual impact of housing counseling on counseled individuals must be 5 percentage points (i.e., it must increase the likelihood of home purchase by 5 percentage points). If 50 percent of treatment group members do not complete housing counseling, the necessary impact size is 16 percentage points.

Comparing Types of Counseling: In addition to comparing the treatment groups to the control group, it may be informative to compare the relative impact of different counseling types by comparing across treatment groups. Panel 2 of Exhibit B2.3 shows the MDE estimates for comparing treatment groups. For a dichotomous outcome variable with a mean control group value of .5, the MDE estimate when there is a 25 percent rate of no shows and crossovers is 6 percentage points. This value implies that the average value for the treatment group would have to increase to 56 percent for the impact of counseling to be significant. The required average value is 55 and 59 percent if 10 percent and 50 percent of sample respondents are no shows.

Exhibit B.2.3. MDE Estimates for Impact Analysis of Dichotomous Follow-Up Outcomes: Two Treatment Groups and One Control Group

	Mean Outcome for Control Group
	.10	.30	.50
Panel A: Treatment/Control Comparisons
10% No Show & Crossover:
MDE	0.026	0.040	0.044
Impact size	0.029	0.044	0.049
25% No Show & Crossover:
MDE	0.031	0.048	0.052
Impact size	0.042	0.064	0.070
50% No Show & Crossover:
MDE	0.047	0.072	0.079
Impact size	0.094	0.144	0.157
Panel B: Treatment/Treatment Comparisons
10% No Show & Crossover:
MDE	0.029	0.044	0.048
Impact size	0.032	0.049	0.053
25% No Show & Crossover:
MDE	0.034	0.052	0.057
Impact size	0.046	0.070	0.076
50% No Show & Crossover:
MDE	0.052	0.079	0.086
Impact size	0.103	0.157	0.172

α = 0.10; baseline N = 7000; baseline treatment group members = 2013; follow-up treatment group members = 1509; baseline control group members= 2975; follow-up control group members = 2231.

B.2.4 Unusual Problems Requiring Specialized Sampling Procedures

There are no unusual problems associated with this sample.

B.2.5 Any Use of Periodic (less frequent than annual) Data Collection Cycles to Reduce Burden

Not applicable to this study.

B.3 Maximizing Response Rates

For this evaluation, encouraging study participants to complete housing counseling is a central challenge. Ensuring a high rate of housing counseling completion requires both that a large share of treatment group participants initiate housing counseling activities (‘take-ups’) and that these participants complete the full housing counseling curriculum (‘completions’). The study is designed on the assumption that 75 percent of assigned participants will complete housing counseling. To achieve these completion rates, the study team and participating housing counseling agencies will need to work together to monitor take-up and to encourage both housing counseling initiation and housing counseling completion.

The first step in maximizing housing counseling take-up and completion is to use the eligibility screening to weed out potential participants that are unlikely to complete housing counseling. The screening criteria will require that participants indicate that they (a) have regular access to the internet to complete a web-based curriculum and (b) are able to travel to complete in-person housing education and counseling sessions if assigned to the in-person housing counseling treatment group. Enforcing these screening criteria may create a conflict with the need to recruit 7,000 study participants. However, the enrollment process should err on the side of prioritizing high housing counseling take-up. The reason is that the options for enrolling additional study participants are much better than the options for encouraging housing counseling take-up after the fact.

The second step in encouraging housing counseling take-up and completion is to communicate clearly the results of the random assignment and to facilitate the initial contact with the housing counseling agency. The random assignment results will be communicated via email and a hard-copy letter, both of which will clearly state the assignment group, the contact information for the closest housing counseling agencies or information for online access to education (as applicable), and the steps for completing housing counseling and receiving the incentive within two days of study enrollment. These communications will also emphasize the availability of the toll-free Support Desk to answer any questions. This Support Desk will be staffed by liaisons from the study team.

The final step is to conduct additional outreach to study participants who do not immediately initiate contact with the housing counseling agency. The site liaisons from the study team and the housing counseling agencies will have the list of assigned study participants through the service tracking system. Housing counseling agencies will be encouraged to contact newly assigned treatment group members proactively to initiate the housing counseling process. Additionally, the housing counseling agency and site liaisons will review the list of assigned participants on a weekly basis to identify participants who require additional outreach.

Treatment group participants who have not initiated housing counseling or whose housing counseling activities have stalled will be targeted for supplemental outreach. The study team’s site liaisons will send regular communications to these participants through email or hard-copy letters. The site liaisons communications will emphasize the benefits of housing counseling participation, highlight the incentive for housing counseling completion, and provide easy-to-follow steps for initiating counseling (e.g., a hyperlink to the online education platform or the housing counseling agency website). The communications will also remind participants of the participation agreement in which they indicated their willingness to participate in the study and complete the assigned housing counseling activities. The outreach activities will be repeated at regular intervals for participants who do not initiate/restart housing counseling activities.

B.3.1 Tests of Procedures or Methods

Drafts of the data collection instruments will be reviewed by HUD personnel, Abt Associates staff and other project team members, and the study’s advisory panel to ensure that the instruments are clear, flow well and are as concise as possible. HUD personnel, Abt Associates staff, and other project team members have reviewed these instruments prior to OMB submission. The study’s advisory panel will review the data collection instruments in September 2012 when they review the Research Design Data Collection and Analysis Plan.

The enrollment process includes a pilot period in which the recruitment and enrollment process will be implemented in a small number of jurisdictions. This process will allow the study team to identify any issues or obstacles, allowing time for adjustments to be made before enrollment begins in all jurisdictions. It will also allow the study team to test the random assignment, referral, and monitoring processes to explore alternative methods for encouraging treatment group participants to initiate and complete counseling.

To support the enrollment and monitoring processes, the study is developing the Random Assignment Study Tracking (RAST) data system. The RAST system will provide a secure system for managing the data collected through the study. The system will allow the data from lenders and housing counseling agencies to be uploaded in a standardized format, integrating the data collected from alternative sources while limiting access to the key study team members. The data system will be developed and tested prior to the start of enrollment. The pilot period offers a further opportunity to ensure that the system is operating as planned before the enrollment begins in all jurisdictions.

B.3.2 Statistical Consultation and Information Collection Agents

HUD’s Office of Policy Development and Research will work with the contractor, Abt Associates, to conduct the proposed data collection. Marina L. Myhre, Ph.D., a Social Science Analyst in HUD’s Office of Policy Development and Research, Program Evaluation Division, serves as Government Technical Representative (GTR). Her supervisor is Ms. Carol Star. Dr. Myhre and Ms. Star can be contacted at (202) 402-5705 and (202) 402-6139, respectively. The study’s Principal Investigators are Dr. Laura Peck from Abt Associates and Dr. Roberto Quercia from the Center for Community Capital at the University of North Carolina at Chapel Hill. Dr. Peck can be reached at (301) 347-5537. Dr. Quercia can be reached at (919) 843-2493. The study’s Project Director is Dr. Jonathan Spader from Abt Associates. Dr. Spader can be reached at (301) 347-5789.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Abt Single-Sided Body Template
Author	Nichole Fiore
File Modified	0000-00-00
File Created	2021-01-30