SUPPORTING STATEMENT
FAMILY SELF SUFFICIENCY EVALUATION
OMB 2528-0296
A1. Circumstances Necessitating Data Collection
This request for OMB Clearance covers the next phase of data collection for the Family Self-Sufficiency (FSS) Evaluation—the 36-month follow-up survey. This study was authorized under the Consolidated Appropriations Act, 2010, Pub. Law 111-117, 123 Stat. 3034, which was approved on Dec. 16, 2009. This evaluation is being conducted by MDRC and its subcontractors, Branch Associates and M. Davis. on behalf of HUD.
This Supporting Statement provides information for the proposed 36-month survey effort to further determine the effectiveness of the Family Self-Sufficiency program (FSS). It builds upon the baseline data already collected and on ongoing collection of administrative records on employment, earnings, and housing status authorized under OMB control number 2528-0296, Expiration Date 07/31/2016.
The primary goal of the Family Self-Sufficiency evaluation is to increase our knowledge about the effectiveness of FSS, which is aimed at helping housing-assisted populations secure and maintain employment and gain independence from public support programs. The survey will cover a range of outcomes that are hypothesized in the FSS Model that cannot be measured using administrative records alone. These include, for example, material hardship, financial well-being (savings/debt, credit history), and employment characteristics and educational attainment.
A.1.1 Background and Policy Context
The linkage of housing assistance to economic opportunities has long been an important component of US housing policy. Indeed, for many years, policymakers and analysts have argued that low-income, working-age people who receive a rent subsidy from the government should strive for some degree of economic self-sufficiency, and that housing subsidy systems should play a more active role in supporting these efforts, such as through the Family Self-Sufficiency (FSS) program.
FSS goals, features, and participation
The FSS program, created in 1990 by Section 554 of the National Affordable Housing Act,1 is administered by state and local public housing agencies. HUD provides funding for FSS case coordinators to manage the program and ensure that participants are linked with appropriate services, and it also funds the escrow accounts. HUD does not fund services; PHAs must rely on their own or other resources available in the community. In March 2011, HUD announced that a total of $54 million would be made available for FSS in the Housing Choice Voucher (HCV) program. The funds were allocated to roughly 700 PHAs for an average grant of approximately $90,000.
Tenants’ participation in FSS is voluntary. While PHAs may screen participants for interest or motivation, they cannot screen for education, job history, marital status, or credit rating. Tenants who elect to participate execute a 5-year self-sufficiency contract that specifies their obligations and the services that will be provided over the course of the contract (US Department of Housing and Urban Development, 2011).
Central to the FSS program model has been the inclusion of a rent escrow provision. Rent increases that, under HUD rent rules, would normally follow a rise in earned income are instead diverted into an interest-bearing, PHA-administered escrow savings account, the proceeds of which the family can access after completing their contract, and if no family members are receiving cash public assistance (such as TANF). After families successfully complete their contracts, they receive the full value of their accumulated savings without restriction on how the money can be used. However, participants may qualify for interim disbursements to cover expenses that can help them meet the terms of their contracts, such as for paying for education or training courses. FSS introduced this feature both as an incentive for households to increase work and earnings, and as a long-term savings vehicle to help families build their financial assets. These asset-building provisions appeared in early research on FSS, to play an important role in motivating families’ continued participation in FSS (Rohe and Kleit, 1999: 358).
The escrow feature is believed to be critical to the potential success of FSS because it helps to address the HUD rule requiring families to pay 30 percent of their adjusted income in rent. Some experts view this rent rule as imposing an implicit tax that may severely “dampen a parent’s enthusiasm for work” (Newman and Harkness, 2002:24).
The second component of the FSS program is case management. HUD funds the PHAs to hire coordinators, who are expected to work with program participants to connect them to services meant to address their barriers to employment. Past research2 suggests the low-income individuals face difficulties in securing and advancing employment including lack of diploma or certification, poor mental and physical health, low English language fluency, drug addiction, criminal records, and lack of “soft skills” including interview protocol and effective communication with superiors. MDRC’s experiences in the New York City Work Rewards demonstration, (discussed below) suggest, additionally, that a high proportion of single mothers (over 80 percent of the sample in the Work Rewards) enrolled in FSS.3 This population faces additional barriers to employment that include access to high-quality childcare and transportation options. FSS coordinators are expected to tailor referral services provided to the particular needs of the client. The assumptions made about and understanding of these barriers to employment on the part of FSS program staff as well as the overall approach the PHA takes toward the self-sufficiency contract may be crucial factors in determining whether FSS is effective in helping participants advance and achieve self-sufficiency.
Around the time the study was launched, roughly 77,000 households participated in FSS nationwide, with the majority (over 80 percent according to HUD) being Housing Choice Voucher (HCV) recipients rather than public housing residents. FSS households make up a very small share of the families receiving various forms of HUD assistance.4 Recent HUD funded studies of FSS participation reveal that about 5 percent of voucher program users were enrolled in FSS (de Silva et al., 2011: 13; Ficke and Piesse, 2004: xiii).
A1.2 Overview of the FSS Evaluation
MDRC is conducting a comprehensive study of the FSS program, structured around three research components: an impact analysis, an implementation and participation analysis, and a benefit-cost analysis5. More fully described in the initial OMB submission package,
the evaluation design is structured around three research components: an impact analysis, an implementation and participation analysis, and a benefit-cost analysis. The overarching goal of the evaluation is to test the effectiveness of the FSS program, the main self-sufficiency intervention that program participants will be exposed to.
The study is designed as a two-group randomized control trial (RCT) involving 2,551 households across 18 public housing agencies (PHAs or sites). Participants were randomly assigned to one of two groups:
FSS group. These individuals have access to the core elements of the FSS program – case management as well as rent escrow provisions.
Control group. These individuals will not be enrolled in FSS and will not have access to FSS case management or escrow.
When properly implemented, random assignment helps eliminate systematic differences between the program and control groups prior to the start of the program and any subsequent differences in outcomes – for example, differences in employment or earnings and differences in family income and poverty—can be attributed to the program with confidence.
The research is positioned to speak to the following types of research questions:
Does the FSS program improve self-sufficiency outcomes for program participants?
Do the effects vary across types of people and places?
How does variation in the implementation of the interventions affect participants’ experiences and the interventions’ success?
Does the intervention produce positive benefit-cost results?
The following section describes how the outcome measures included on the 36-month survey will be used as part of the impact analysis.
Contribution of the 36-month follow-up survey
The 36-month follow-up survey is critical to the impact analysis, which will assess the overall and independent effects of the FSS program by comparing the key outcomes of the treatment group to the outcomes of the control group. The overall FSS study will track both the program and the control groups for a number of years using administrative and survey data to measures outcomes.
The impact analysis will examine the program’s effects on a wide range of outcomes, some of which can only be determined through use of a survey. Key clusters of outcomes for this study are included below.
Education and Work: MDRC will use both Unemployment Insurance (UI) wage records, obtained from the U.S. Department of Health and Human Services’ National Directory of New Hires database, and the survey to collect data on employment, earnings, job characteristics, and work search behaviors. Discussions with PHAs have revealed that some programs take a human capital development approach to self-sufficiency and thus emphasize degree, diploma and certification achievement. MDRC will track educational attainment among study participants through 36-month survey data.
Income, assets, finances, and rent burden: If FSS affects participants’ disposable income, it may help them accumulate assets. With survey data, MDRC will assess the effects of the program on household finances and financial behaviors (such as savings, access to credit, and debt reduction, outcomes which several FSS program focus on). Data on income, combined with housing authority and survey data on tenant rent and utilities payments, would be used to construct measures of rent burden.
Material hardship, and family well-being: Increases in disposable income, may produce reductions in material hardships, including housing-related hardships such as disconnection of phone and utilities, and reductions in food insufficiency. MDRC observed such effects on poverty and hardship in its study of NYC’s conditional cash transfer program, which included a significant housing-assisted population. The 36-month follow-up survey will allow investigation in these potential impacts.
A2. How, by Whom, and for What Purpose Are Data to be Used
How the Information will be used?
The findings from the study will be used to inform the Federal government, Public Housing Authorities and other stake-holders about the effectiveness of FSS in helping HCV holder secure and maintain employment and achieve self-sufficiency.
As a study commissioned by HUD, the Department will use the information from the follow-up survey, in conjunction with information collected on the baseline information form, and through administrative records to assess the impacts of the FSS program. As the first national evaluation to assess the effects of FSS, these data will be important for HUD to begin answering questions about whether FSS makes a difference and helps achieve HUD’s self-sufficiency goals. The comprehensive study will speak to the program’s impacts in multiple domains – for example, housing assistance receipt and exits; self-sufficiency; material hardship, and financial wellbeing.
In addition, PHAs will also use the data to understand how their programs work and to target resources in effective ways. Over 700 PHAs operate FSS, and the findings from the evaluation will be critical to their own implementation decisions. Ultimately, these data will benefit the public and social policy community, researchers, policy analysts, and policy makers who are interested in developing policy initiatives to promote self-sufficiency and reduce poverty among housing voucher recipients in a wide range of program areas. This project offers the first opportunity to obtain reliable measures of the effects of the FSS program at a national scale. The long-term indirect benefits of this research are therefore likely to be substantial
Who Will Collect the Information?
M. Davis and Company (MDAC), the subcontractor to MDRC for the national evaluation, will administer and field the 36-month follow-up survey. The survey will be administered to the full research sample via phone (including Computer-assisted Survey Interviews (CATI), see below) and should take approximately 40 minutes to complete.
Item-by-Item Justification
This section provides an overview of the items submitted for clearance: the 36-month follow-up survey. The instrument and necessary item-by-item justification are presented in the appendices. Appendix A contains the 36-month adult follow-up survey instrument; Appendix B contains the item-by-item justification for the 36-month adult follow-up survey instrument.
In developing the 36-month follow-up survey instrument, we attempted to balance the need to capture all of the required data against placing undue burden on the respondents, excluding items that—while potentially interesting—are not critical to the measurement of outcomes needed to analyze the impacts of the FSS intervention. Another goal was to keep the time needed for survey administration to a reasonable duration, thereby limiting respondent burden. This section provides a brief overview of the content of the follow-up survey.
The survey will include questions concerning respondents’ current situation at the time of their interview and other questions that focus on the time period between study enrollment and the point of the 36-month follow-up survey.
As shown in Appendix A, the following broad topics will be included in the 36-month adult follow-up survey:
Education, Training and Service Receipt during Follow-up Period
Degrees, licenses and certifications received
Courses and programs attended
Support services (e.g. housing, childcare) received
Self Sufficiency
Employment during the follow-up period
Earnings at the time of the follow-up survey
Income sources; total family income
Food security
Economic stressors: material hardship, income instability, emergency expenses
Savings, Debt, and Financial Inclusion
Types and amounts of debt
Use of mainstream banking services (e.g. checking accounts, credit cards)
Use of alternative financial service providers (e.g. check cashers, payday lending)
Savings and savings instruments (e.g. IDA)
FSS Program Participation (program group only)
Understanding / awareness of program rules
Escrow accumulation
Contacts and case management experiences
Referrals and service receipt
Program satisfaction
Informed Consent
Study participants completed a participation agreement when they enrolled in the study, providing their informed consent to participate in the research study. The original participation agreement allows for the collection and retention of data on study participants for up to 10 years after random assignment. A copy of the Informed Consent Form administered at baseline can be found in Appendix C of this submission.
Wherever possible, advanced technology will be used in data collection efforts to reduce burden on study participants and on site staff. The following methods will be used:
1) CATI helps to reduce respondent burden, as interviewers can proceed more quickly and accurately through the survey instruments, minimizing the interview length and the need for subsequent call backs. Computer programs enable respondents to avoid inappropriate or non-applicable questions. Key data will be pre-loaded on the survey based on response to previous survey questions (in the same survey) and where applicable, at baseline). CATI also improves data quality through more uniform administration of the survey questions, more accurate implementation of the skip patterns, and immediate application of range checks, edit checks, and consistency checks of item-by-item responses.6
2) We will utilize survey tracking information. MDRC collects contact information semi-annually from participating housing authorities and quarterly from federal HUD database. MDRC is forwarding the information to M. Davis and Company (MDAC). Address changes come from mailings to the participants and passively tracking respondents through the U.S. Postal Service Change of Address database (this approach provides an inexpensive method for being able to collect more recent contact information for respondents). Further, MDAC will be sending a welcome packet prior to the tracking survey, a tracking letter midway between tracking and 36-month survey, and an advance or pre-notification letter just prior to the 36-month survey. All of them will ask the sample member to update address and phone.
3) Integration of other data sources. When relevant person-level data has been identified as available through an accessible centralized, computerized source, the information has generally been excluded from the proposed data collection package. For example, UI data will be obtained through administrative records and data on voucher exits will be obtained from housing records. While implementation data collection relies on evaluation staff efforts on-site, we have sought wherever possible to minimize overlap between questions we include in implementation questionnaires and protocols and questions that will be asked through computer-assisted surveys.
The information collection will not duplicate information that is already available. Where possible, the evaluation uses available data sources, such as UI wage records and housing data. The survey will collect data on various other outcomes not available routinely or systematically in program records.
A central topic of these evaluations involves the tracking of employment, service participation, and well-being of participants over time. Thus, the evaluation uses a core set of consistently worded questions in order to track how the impacts of the program change over time on a common set of measures (for example, some questions on the 18-month mini-survey are repeated on the 36-month follow-up survey, to track change over time).
A.5 Burden on Small Business
We do not anticipate that this study will burden small businesses.
This evaluation represents an important opportunity for the Federal Government to add to the body of knowledge about the impacts of a key employment-oriented program for HCV recipients. This is consistent with the Administration’s strong focus on evidence-based policymaking. With the exception of the Work Rewards demonstration, New York City’s FSS study, there is no rigorous evaluation of the FSS program.
If the survey is not administered and the data not collected, analyzed, reported, and disseminated, Federal program or policy activities will not be informed by high quality evidence on a variety of outcomes central to the FSS intervention. Limiting analysis to only those outcomes available through administrative records will complicate critical decisions regarding future investments in self-sufficiency programs for voucher households.
The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320 (Controlling Paperwork Burdens on the Public). There are no special circumstances that require deviation from these guidelines.
A8. Form 5 CFR 1320.8(d) and Consultations Prior to OMB Submission
Please see Appendix X for a copy of the HUD’s notice in the Federal Register, required by 5 CFR 1320.8(d), soliciting comments on the information collection prior to submission to OMB.
The 60-day notice appeared in Vol. 81, No. 88 Friday May 6, 2016. HUD received one comment and was addressed via email on July 28, 2016.
b. Consultations Outside of the Agency
All data collection instruments included in this package have gone through extensive review by expert consultants, HUD staff, and members of the research team. During study design, we have sought the input of nationally recognized experts on public housing and Section 8 housing assistance, including John Goering and Ingrid Gould-Ellen. We continue to consult with outside experts as we move toward project launch.
During this stage of the evaluation one type of payment to respondents is planned: payment will be distributed upon survey completion. Justification for this payment is provided below.
Payment upon completion of the survey. Payment upon survey completion is intended as a token of appreciation. As documented in the literature, this token of appreciation is likely to improve response rates by decreasing the number of refusals, enhancing respondent retention, and providing a gesture of goodwill to acknowledge respondent burdens. This technique is proposed in addition to many of the techniques suggested by OMB to improve response rates that have been incorporated into our data collection effort and are described in Section B3 (see Supporting Statement B), because our experience has shown that small monetary amounts are useful when fielding data collection instruments with low-income and/or hard-to-reach populations as part of a complex study design. In a seminal meta-analysis, Singer, et al. (1999) found that incentives in face-to-face and telephone surveys were effective at increasing response rates, with a one-dollar increase in incentive resulting in approximately a one-third of a percentage point increase in response rate, on average. They found some evidence that incentives were useful in boosting response rates among underrepresented demographic groups, such as low-income and non-white individuals.7 This is a significant consideration for this study. Another important consideration is the burden posed by this data collection, which will take on average 45 minutes of the participant’s time for the follow-up survey.
The survey firm will also include a small token (e.g. a $5 gift card) for completing a mini-survey, approved by OMB, during an initial tracking call conducted by MDAC.
The survey instrument that will be used to collect 36-month follow-up data from sample members has unique aspects that make administration difficult and potentially threaten response rates. We are therefore requesting approval to offer a small monetary amount ($30) to all sample members who complete the survey. Aspects of the data collection effort that also make it more difficult for surveys of low-income households to obtain high completion rates are:
The surveys include questions that could be perceived as intrusive and therefore could make respondents uncomfortable (i.e., questions about their finances and health which are topics discussed with FSS case managers).
Participants may have negative feelings about the other services received that are of interest, such as welfare, Medicaid, job training, etc.
Educationally and economically disadvantaged groups have been found to be more difficult than the general population to convince to participate in surveys.
We are aiming to achieve an 80 percent survey completion rate for the 36-month follow-up survey. Even with the best data collection practices, it would be very difficult, if not impossible, to obtain such a high completion rate without providing a token of appreciation to participants.
Every effort will be made to maintain the privacy of respondents, to the extent permitted by law. Please see the informed consent form used during baseline data collection (and approved as part of the previous OMB submission) in Appendix C. All respondents included in the study were informed that information they provide will be used only for the purpose of this research. Individuals will not be cited as sources of information in prepared reports. All research staff working on the project have been trained to protect private information and have signed a pledge stating that they will keep all information gathered private to the extent permissible by law. All papers that contain participant names or other identifying information will be kept in locked areas and any computer documents containing identifying information will be protected with a password. The Research Design and Data Collection and Analysis Plan prepared for this study provides additional information on how evaluation data will be protected. It is available on request.
Many of the questions envisioned for the survey are potentially sensitive for respondents, but they include questions covering information generally collected from program participants at the time of enrollment in FSS or in in follow-up discussions with case managers. Respondents are asked about topics about potential barriers to employment (for example, poor credit, childcare difficulties, debt, and limited educational attainment) and, depending on the FSS program, these topics are addressed directly by program case managers, in service referrals, and in participants’ “independence plans.” Respondents will be informed by survey staff prior to the start of the survey that their answers are confidential, that they may refuse to answer any question, that results will only be reported in the aggregate, and that their responses will not have any effect on any services or benefits they or their family members receive.
The hour burden for the data collection for participants is outlined in Table 1 below. The estimates included below are based on experience with previous random assignment studies involving similar populations and data collection instruments.
Table 1
Hours and Burden
Members of the affected public:
Families receiving subsidized housing and enrolled in the FSS program (treatment group): |
1,281
|
Families receiving subsidized housing and not enrolled in the FSS program (control group): |
1,270
|
Estimation of the total number of hours needed to prepare the information collection including number of respondents, frequency of response, and hours of response:
Instrument |
Number of Respondents |
Number responses per respondent |
Average burden/ response (in hours) |
Total burden hours |
36-Month Survey |
2,551 |
1 |
45 minutes (.75) |
1,913 |
A.13. Other Cost Burden to Respondents
The proposed data collection will not require the respondents to purchase equipment or services. Therefore, there are no additional costs to respondents.
The total cost to the federal government for this phase of the study, including but not limited to the data collection activities discussed in this submission is $3,093,372 over a 36-month period. This includes $901,413 for Option 1, Extended Data Collection and $2,188,959 for Option 2, 36-month survey task. Included are costs associated with background research, evaluation design, development of data collection instruments and data collection activities, analysis and reporting.
The period of performance is as follows:
Option 1: quarter 1 2017 through quarter 3 2018
Option 2: quarter 4 2015 through quarter 3 2018 (36-months)
This submission is a revision of a currently approved collection.
As detailed in the previous OMB submission, to determine the effectiveness of the targeted programs, MDRC will collect four categories of data: 1) baseline data, 2) program implementation, services, and process data (some of which will support the cost effectiveness study), 3) administrative records, and 4) surveys of study sample members.
As shown in Table 2 below, the evaluation data will be analyzed from 2014 to 2018 and MDRC intends to produce three formal deliverable drawing on the various data sources.
Table 2:
Analysis and Publication Schedule
Activity |
Schedule |
Analysis |
01/01/2016- 07/31/2018 |
1st Report (internal memo)
Draft
Final |
04/01/2015
05/01/2015 |
2nd Report
Draft
Final |
11/15/2016
01/15/2017 |
3rd Report
Draft
Final |
09/01/2017
11/01/2017 |
Final Report |
08/15/2018 |
Data Files and Documentation |
08/15/2018 |
A.17. Reasons for Not Displaying OMB Approval Expiration Date
The expiration date for OMB approval will be displayed on all forms completed as part of the data collection.
A.18. Exceptions to Certification Statement
No exceptions are necessary for this information collection.
APPENDIXES
APPENDIX A (36-MONTH FOLLOW-UP SURVEY INSTRUMENT)
APPENDIX B (36-MONTH FOLLOW-UP SURVEY ITEM-BY-ITEM JUSTIFICATION)
APPENDIX C (INFORMED CONSENT FORM)
1 A brief description of HUD’s prior efforts at FSS-like programs is found in Rohe and Kleit (1999).
2 See for example MDRC’s Chicago Employment Retention and Advancement Project (http://www.mdrc.org/publications/441/full.pdf)
3 Verma, Tessler, Miller, Riccio, Rucks, and Yang (2012), Nunez, Verma and Yang (2015)
4 HUD data indicate that there are currently over 3.3 million units of public and assisted housing (Schwartz, 2010; US Department of Housing and Urban Development, 2008).
5 The original OMB submission includes more details on the each of the study components.
6 Another benefit of CATI is that interviewers can focus on the respondent rather than management of the survey instrument, creating a more pleasant experience for the respondent. The technology ensures that scheduled appointments are honored with respondents. CATI also ensures adherence to dialing protocols, maintaining the integrity of the study without unduly burdening sample members and eliminates many human errors, such as accidental calling of resolved sample records. Finally, CATI provides translated scripts for crisp script delivery to non-English speakers.
7 Berlin, M., L. Mohadjer and J. Waksberg (1992). An experiment in monetary incentives. Proceedings of the Survey Research Section of the American Statistical Association, 393-398; de Heer, W. and E. de Leeuw. “Trends in household survey non-response: A longitudinal and international comparison.” In Survey Non-response, edited by R. M. Groves, D. A. Dillman, J. L. Eltinge, and R. J. A. Little. New York: John Wiley, 2002, pp.41-54; Singer, E. and Kulka, R. Studies of Welfare Populations: Data Collection and Research Issues, Panel on Data and Methods for Measuring the Effects of Changes in Social Welfare Programs. Ploeg, Robert A.Moffitt, and Constance F.Citro, Editors. National Academies Press, Washington, DC, 2000, pp. 105-128.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | nunez |
| File Modified | 0000-00-00 |
| File Created | 2021-01-23 |