Family Options Study_Part A_36-Month Follow-up_FINAL

The US Department of Housing and Urban Development (HUD) is undertaking an evaluation of the Impacts of Housing and Services Interventions for Homeless Families to provide research evidence to help federal policymakers, community planners, and local practitioners make sound decisions about the best ways to address homelessness among families. In the remainder of this document, we refer to the study as the Family Options Study.¹ As shown in Exhibit A-1 below, the study has randomized 2,282 homeless families (with 5,397 children) across 12 sites to one of four interventions distinguished by the type and duration of housing assistance and supportive services.²

Exhibit A-1

	All RA as of 1/31/2012
	Alameda County	Atlanta	Baltimore	Boston	CT	Denver	Honolulu	Kansas City	Louisville	Minneapolis	Phoenix	Salt Lake City	Total*
CBRR	56	73	20	53	73	8	44	30	18	52	62	80	569
PBTH	49	41	17	0	18	23	66	42	24	4	65	19	368
SUB	76	0	0	64	47	76	42	54	32	62	71	75	599
UC	77	75	21	64	76	65	65	50	35	63	81	74	746
Total	258	189	58	181	214	172	217	176	109	181	279	248	2282

*Total was adjusted to remove the 25 families that were ineligible at time of enrollment.

Current data collection ends 18 months after random assignment, a time too early to judge the effectiveness of interventions that may still be ongoing at that point. HUD has chosen to continue measuring outcomes for 36 months after study enrollment, to maximize learning from this ambitious experiment. This request for OMB Clearance covers the next phase of data collection for the Family Options Study—the 36 month follow-up data collection. It builds upon the baseline data already collected under OMB approval number 2528-0259, Expiration Date 5/31/2013, and the 18-month data collection approved under 2528-0259, Expiration Date of 3/31/2015. The 18-month data collection funded by the Department, and approved by OMB covered the 18-month follow-up adult interview, and parental report on one focal child. It also covered collection of program cost data from local providers.

In 2010, Vanderbilt University and Abt Associates, Inc. received a grant from the Eunice Kennedy Shriver National Institutes of Child Health and Human Development (NICHD) to conduct a child outcomes study that capitalizes on the experimental design of the Family Options Study. This study substantially augments HUD’s 18-month follow-up study data collection on, and analysis of, the impact of the interventions on child outcomes. Specifically, the NICHD grant was used to collect information at the 18-month follow-up survey from the parent on a second focal child (vs. the one focal child funded by HUD) and to interview and assess up to two children per household.

The18-month follow-up child data collection was completed in November 2013. It examines both children’s developmental outcomes and potential environmental precursors that are plausible consequences of the interventions. For all focal children, parents report on children’s health and access to health care, pro-social behaviors, and emotional and behavioral problems with the Strengths and Difficulties Questionnaire (Goodman, 1997), and sleep disruptions, which are associated with a variety of emotional and behavioral disorders (Dahl & Harvey, 2007). Additional instruments are tuned to children’s developmental stage.

Children 12 to 41 months are assessed with the Ages and Stages Questionnaire (ASQ-3)family of questionnaires which assesses gross and fine motor skills, social development, and communication and problem solving, as observed by parents (Squires et al., 2009).

Children from 3 years 6 months to 7 years 11 months are assessed with the Woodcock Johnson III letter word identification and applied problems scales (Woodcock et al., 2001), which are early indicators of verbal and quantitative/analytic skills. The Head Toes Knees Shoulders task assesses self-regulation, where children must remember rules and inhibit incorrect responses (for example, by following instructions to touch their head when the interviewer says “touch your toes”).

In interviews, youth from 8 years to 17 years 11 months report on anxiety (the trait anxiety scale for youth; Kirisci et al., 1997), fears (Ramirez et al., 1991), and substance use (CDC Surveillance measure, 2011). This array of measures, along with parental report, captures the most likely mental health consequences of homelessness and behavioral responses. Parents report on school grades and attendance and youth report on attendance, school effort, attendance, and disciplinary problems, to capture functioning in the key developmental domain of school. Youth also complete the Hope scale (Snyder et al., 1997), a measure of self-efficacy.

The child survey, ASQ-3 and child assessment tools (Woodcock Johnson and HTKS) were used in the 18-month data collection effort but were not subject to OMB approval because they were funded by the NICHD grant.

HUD will fund a 36-month follow-up effort, focusing on both adult and child sample members. Under this request for clearance the we seek OMB approval for:

The36-month follow-up adult survey, to be administered to all adult respondents previously enrolled in the Family Options Study. The adult interview includes the parent report about developmental and behavioral outcomes for up to two focal children 20 months to 17 years and 11 months of age. This includes the collection of Ages and Stages Questionnaire (ASQ-3) data on focal children 20-66 months of age. For focal children between ages of 18 and 20 we ask limited questions on household and family formation, employment status, post-secondary education and training.
The child assessment protocols to be administered to selected focal children 3 years 6 months to 7 years 11 months of age. The child assessment protocols are the Woodcock Johnson Applied Problems and Letter Word Identification modules and the Head Toes Knees Shoulders protocol;
The child survey to be administered to focal children 8 to 17 years 11 months of age; and
The battery of participation agreements. This includes the renewed informed consent that provides for continued participation of the adult respondents in current and future data collection efforts. It also includes the parental permission, authorizing the team to collect data from focal children three years 6 months to 17 years 11 months of age via direct child assessments or surveys. It includes the child assent form which is administered to child 8-17 years of age before the child survey. Finally, it includes aconsent for release of information form. This form was developed to allow the researchers to seek permission to release the personal identifiers to HUD at the end of the study.

The Family Options study will compare four combinations of housing and service interventions for homeless families in a rigorous, multi-site experiment, to determine what interventions work best to promote family stability and well-being and, within the limits of statistical power, what sorts of families benefit most from each intervention. The interventions are: 1) a permanent housing subsidy without services (Subsidy Only); 2) Community-Based Rapid Re-housing (CBRR), consisting of temporary housing subsidy provided in conventional housing with limited supportive services; 3) temporary housing subsidy provided in facility-based housing with intensive services but no guarantee of a permanent subsidy (Project-Based Transitional Housing-PBTH); and 4) shelter, with whatever services the shelter ordinarily provides to its residents and any other assistance available in the community (Usual Care). This study will also exploit to the extent possible naturally occurring variation in program features within these categories and across sites to explore, non-experimentally, what features of programs seem most responsible for success.

The U.S. Department of Housing and Urban Development has invested considerable resources in strategies to address family homelessness. The results of this evaluation will provide evidence to inform policy makers how best to set priorities for those funds and to design eligible activities. Similarly, in response to HUD Continuum of Care funding requirements and in an effort to maximize the effectiveness of limited resources, communities systematically examine their homeless assistance systems to decide which housing and service interventions should be funded. Some cities often expend their own appropriations on interventions for homeless individuals and families. Unfortunately, past research is inadequate to guide federal policy and local practice. While there is a significant amount of research on the characteristics and needs of homeless families and an emergent body of descriptive research on intervention programs and outcomes for families who use them, there is almost no information about the relative effectiveness or cost-effectiveness of different interventions. Senate Report 109-109 for The FY2006 Transportation, Treasury, Judiciary, HUD, and Related Agencies Appropriations bill directed the Department to focus its energies on homeless families and to “undertake research to ascertain the impact of various service and housing interventions in ending homelessness for families.” This study is intended to respond to this mandate.³

Although the 18-month follow-up survey will provide valuable data on early outcomes of the experiment, it will leave important questions unanswered. At 18-months following random assignment, families could still be living in project-based transitional housing (PBTH) (which allows families to stay for up to 24 months),^⁴ or could just be completing participation in rapid-rehousing (where assistance can last up to 18-months). How families fare after these interventions end is an important concern. The critical question of whether the same interventions that are effective in the short term are also effective in the longer term cannot be answered fully with data collection that ends at 18-months. Repeating the 18-month data collection protocols at 36-months, a year after the statutory maximum stay in PBTH allows families to stabilize in new situations or, alternatively, for new arrangements to deteriorate.

For children as for adults, the central question to be examined at 36 months is whether the same interventions that promote children’s welfare at 18 months continue to have long-term benefits. PBTH offers supports for parenting, which might have enduring benefits for children, but also disrupts family routines and rituals (Mayberry et al., in press) and requires a potentially stressful move to another housing arrangement at or before the end of two years. Both SUB and CBRR allow families to obtain their own residence, but if families cannot maintain that residence after the CBRR subsidy ends, they may make additional moves or even double up with others. The ongoing financial support associated with SUB might reduce parental stress and allow families to invest more in learning supports for their children, with benefits that could increase over time. The little research on long-term outcomes of homelessness for children offers inconsistent findings: Some studies suggest that homelessness and residential or school instability, particularly in the early years, set children on less favorable trajectories and hence have enduring effects (Herbers et al., 2012; Voight et al., 2012) although additional moves have additional short-term effects (Voight et al. 2012). Other studies suggest that child outcomes rebound after an episode of homelessness and are more sensitive to current environments than to past experiences (Rafferty et al., 2004; Shinn et al., 2008). Most prior studies of child outcomes of homelessness have been observational, rather than experimental, and have been confined to a particular city. Thus the 36-month follow-up of children in Family Options families will supply important new information. We collect data directly from children as well as parental reports about children, which are valuable but limited sources of data.

The Department seeks approval for the 36-month adult and child follow-up surveys. These survey instruments measure important study outcomes that are not available in administrative records. For the primary outcome of housing stability, administrative data from Homeless Management Information Systems (HMIS) data will allow for tracking returns to shelter, but only survey data can assess street homelessness, doubling up, frequent moves or associated outcomes such as housing quality, crowding, and cost burden. For self-sufficiency, survey data will be used to assess other sources of income, total household income, educational attainment, food security, and economic hardship. For family preservation, administrative data from child welfare records can be used to assess formal foster care placement, but survey data are needed to understand informal arrangements which may be more common, and separation of adult household members for reasons associated with housing. For adult and child well-being, survey data capture health, mental health and substance use problems, and positive indicators of well-being and potential mediators of these outcomes, which are not available from administrative sources.

A.1.2 Homeless Assistance System Background

This section provides background information on current homeless assistance program models and other sources of housing assistance that may be available to families experiencing homelessness. The research team reviewed these program models to develop the interventions tested in this study. The most widely adopted typology of homeless assistance programs is defined in terms of the residential components of the Continuum of Care (CoC): emergency shelter, transitional housing, and permanent supportive housing. Among and within each of these three program types there is significant variation in quality, housing structure and location, privacy and independence for participants, tenure, average and expected lengths of stay, services provided, rules, and expected outcomes (Locke et al., 2007). Emergency and transitional housing are time-limited programs, which rely on families moving on to subsidized or unsubsidized permanent housing. Permanent supportive housing programs offer permanent housing subsidies coupled with services and are available only to families in which at least one parent has a qualifying disability. Permanent supportive housing programs are not examined in this study.

Emergency shelters typically serve as the first response to homelessness. This makes shelters a good place to draw a research sample when studying the impacts of different housing and services interventions. Frequently shelters are 24-hour congregate settings, though each family may be provided an individual room or even an apartment. Services vary from basic shelter services (e.g., meals, showers, clothing, and transportation) to minimal case management and referrals to intensive case management augmented by specialized services, such as employment and/or drug or medical treatment. There were 34,031 emergency shelter units (corresponding to 229,206 beds) for homeless families throughout the country in 2012 (2012 Volume 1, Annual Homeless Assessment Report (AHAR), 2012). Many families have short lengths of stay, leading to high turnover within emergency shelter programs.

Transitional housing offers homeless families housing or rental assistance with supportive services for longer periods, generally six to 24 months. Often families are referred to transitional housing from emergency shelter if shelter workers determine they need more intensive assistance and meet eligibility criteria. Transitional programs follow several models: some are offered in facility-based settings with shared or private rooms or apartments, others are independent units in clustered or scattered site locations where the program maintains the lease and program participants must leave upon completion of the program, while still others are in scattered site community locations where families rent their own apartments with temporary financial assistance from the program. There are approximately 34,952 transitional housing units for homeless families (corresponding to 197,192 beds) (AHAR, 2012), though there is no national data on the composition of these units across the different models. Stays in transitional housing are longer than those in emergency shelter; data from Volume II of the 2012 Annual Homeless Assessment Report (AHAR 2013) indicates that the median length of stay is 124 nights, as compared with 22 nights in emergency shelter. However, those stays are based on a one-year reporting period. Since transitional housing offers housing assistance that generally lasts six to 24 months, the AHAR length of stay is likely truncated. As noted in the Interim Report (Gubits et.al.),PBTH providers participating in the Family Options Study reported the median length of stay in their programs for families assigned to receive project based transitional housing was 17 months.

As with emergency shelters, services provided through transitional housing vary substantially from one program to another, though the nature of services is typically more intensive than in shelters. Transitional program services may include childcare, case management and referrals, benefit acquisition and retention, family reunification, education and employment training, mental health and substance abuse treatment, and children’s services. Most transitional housing programs aim to place participants in permanent housing at program completion and may help to broker access to mainstream subsidized housing, but the homelessness system itself rarely funds housing subsidies beyond the temporary rental assistance provided as part of a transitional housing program. Burt (2006) offers a thorough description of the range of transitional housing models.

Families experiencing homelessness may also potentially use other federally funded housing assistance such as permanent subsidies or temporary subsidies to promote rapid exit from shelter (Fisher et al., 2013 (in press). Permanent housing subsidies without specialized services are provided by public housing agencies in facilities they own and operate; via project-based rental subsidies for particular properties; and through the Housing Choice Voucher (HCV) program that provides tenant-based rental subsidies that families can use to rent market housing in the community. Permanent housing subsidies are renewable, as long as the family remains eligible, and reduce families’ monthly costs for rent and utilities to approximately 30 percent of income.

Temporary rental assistance has also been available to assist families experiencing homelessness. This type of assistance is referred to as community-based rapid re-housing and provides short-term subsidies (to a maximum of 18 months with quarterly recertification of eligibility), with services focused on housing and self-sufficiency. The goal is to provide each family with the minimum level of assistance needed until they can pay market rent, so subsidies are individually structured and may be shallow as well as short-term. This type of assistance most recently was funded under the Homelessness Prevention and Rapid Re-housing Program (HPRP) as part of the American Reinvestment and Recovery Act of 2009, but was based on earlier models implemented by some localities (Burt, The Urban Institute, Pearson, Montgomery, & Walter R. McDonald & Associates, 2005).

The current study has defined four distinct interventions for assisting homeless families that will be tested using an experimental research design: permanent subsidy; project-based transitional housing; community-based rapid rehousing; and usual care. Families entering emergency shelter that remained for at least seven days were randomly assigned to one of the designated interventions. The study design relies upon random assignment to existing programs that meet the definitions of the experimental interventions. It is important to recognize that, although practitioners and researchers use shorthand terms such as "transitional housing” or “supportive housing,” these labels do not necessarily reflect uniform approaches. In reality, as Rog and Randolph (2002) note, even when programs of a particular "type" are specifically chosen for study, their characteristics can overlap considerably with other programs that nominally use a "different" model. To circumvent this ambiguity in models, the research team selected programs for study based, not on the nominal models they use or claim to use, but rather on direct assessments of the characteristics of the programs ascertained as part of site selection.

A.2 Information Users

A.2.1 How will the Information Collected be Used?

The information collected for this study will be used by policy makers and local homeless assistance program operators to decide which types of assistance (combinations of housing and services) have the greatest potential to improve the well-being of homeless families in their jurisdictions. A cost-effectiveness analysis conducted for the study also will assess the relative costs of each intervention in relation to their impacts to provide policymakers with information about how the impacts of the interventions compare to their costs. Data were collected at baseline using a previously approved data collection instrument. Baseline data were collected from study participants to describe the population of families seeking assistance from emergency shelter and to verify that random assignment is successful, resulting in well-matched groups without significant differences in demographic characteristics. The information will also be used to define subgroups for analysis and for improving the precision of impact estimates with covariates constructed from baseline variables. Contact information for each sample member gathered at baseline and updated at each tracking interview, and the 18-month survey will be used to maintain contact with each sample member to facilitate the follow-up survey at 36-months.

The 36-month follow-up instruments for the adult survey, as well as the 36-month follow-up child survey, are largely the same as the instruments used in the 18-month follow-up survey. The instruments used in the 18-month data collection measure outcomes in the research domains of interest for the study. Repeating these items at the 36-month data collection allows the research team to measure these important outcomes again over a longer follow up period. Collecting the same measures again for the 36-month follow up survey also allows longitudinal analyses of these critical outcomes and family characteristics.

The 36-month adult follow-up survey instrument was modified only to add a small number of questions to the section of the survey that captures parental report on focal children. These new questions are intended to collect information about focal children who age into adulthood. For children who age into adulthood, we added five questions to the parent on child module to determine if the focal children 18 and older have graduated high school and their postsecondary education participation; whether or not they have formed their own households/families; and whether or not they are employed. Other minor changes to the adult and child surveys were made to adjust the reference period and age ranges as needed. There are no items collected in the 18-month instrument that require dropping from the 36-month interview.

The direct child assessment protocols to be administered at 36-months are identical to the ones implemented in the 18-month follow-up. Like the 36-month adult survey, the 36-month child survey and assessment data will be used to answer the central question of whether the same interventions that promote children’s welfare at 18 months continue to have long-term benefits, as measured at 36-months.

A.2.2 Purpose of the Data Collection

This request for clearance covers the instrument for the 36-month follow-up survey interview to be administered to adult heads of household who enrolled in the study. The adult survey was designed to measure outcomes of the interventions on five key domains: housing stability; self-sufficiency; adult well-being; child well-being; and family preservation. This request also covers the 36-month follow-up child interviews, direct child assessments, and collection of parental report on child development. The above child data collection protocols were originally funded by the Eunice Kennedy Shriver National Institutes of Child Health and Human Development (NICHD), in an effort to enhance the child outcomes measured at the time of the 18-month follow-up under the HUD study.

The 36-month child data collection will enhance the data available to measure child outcomes and determine whether the same intervention effects found in the short-term follow-up continue to hold true on the longer-term child outcomes. Data about children, as well as aspects of the home environment that foster their development, can be collected from parents, and children can be interviewed and assessed directly to provide information about child well-being unavailable from other data sources. Specialists in child development believe that multiple reporters are essential for credible assessment of child outcomes, since parental reports may be biased or affected by parents’ own mental state.

This evaluation will offer new evidence concerning the effects of various housing and services interventions for homeless families that will allow policymakers to make informed choices about optimal investment in homelessness assistance. The experimental design will generate data to draw inferences about the effects of the housing subsidies and services for families, independent of all other factors affecting the lives of study participants—both adults and children. Random assignment serves to ensure that the different intervention groups are well-matched to one another on both observed and unobserved characteristics at the time of their entry into the study. It thus establishes the strongest possible foundation for understanding which of the interventions tested can lead to improved housing stability, self-sufficiency, adult well-being, child well-being, and family preservation.

A.2.3 Who Will Use the Information

The primary beneficiary of the planned survey data collection will be HUD. HUD will use the information from the follow-up survey, in conjunction with information collected on the baseline, tracking, and the 18-month follow-up survey, to assess the impacts of the four types of assistance packages for homeless families. These data will begin to answer HUD’s questions about impacts of housing assistance and services in all study domains: housing stability; self-sufficiency; adult well-being; child well-being; and family preservation.

Secondary beneficiaries of this data collection will be those in the public policy and social science research community who are interested in developing policy initiatives to address homelessness among families. Local service providers and decision makers will also use the data to understand how their programs work and to target resources in effective ways. Local program providers will be able to use the study findings on the impacts and cost-effectiveness of the alternative approaches to make decisions about how to focus local resources in the most effective ways. Ultimately, these data will benefit researchers, policy analysts, and policy makers in a wide range of program areas. This project offers the first opportunity to obtain reliable measures of the effects of various housing and services interventions for homeless families. The long-term indirect benefits of this research are therefore likely to be substantial.

Data collected to date were used in the Interim Report published in March 2012 (Daniel Gubits et al., Interim Report, Family Options Study (Washington, DC: U.S. Department of Housing and Urban Development, March 2013). http://www.huduser.org/portal/publications/homeless/hud_503_FOS_interim_report.html). The data have also been used to develop a publication on housing decisions for families in the study (Fisher et al., in press).

A.2.4 Item-by-Item Justification

This section provides an overview of the contents of each of the items submitted for clearance: the 36-month follow-up adult survey, the child assessment protocols, the child survey, and the participation agreements used to obtain informed consent from respondents to the follow-up survey. Instruments and necessary item-by-item justifications are presented in the appendices. Appendices A1-A4 contains the participation agreements/informed consent documents for the 36-month data collection efforts; Appendix B contains the 36-month adult follow-up survey instrument; Appendix C contains the item-by-item justification for the 36-month adult follow-up survey instrument; Appendix D contains the 36-month follow-up child survey; Appendix E contains the item by item justification for the child survey.

In developing the 36-month adult follow-up survey instrument and the child survey instrument and assessments, we attempted to balance the need to capture all of the required data against placing undue burden on the respondents, excluding items that—while potentially interesting—are not critical to the measurement of outcomes needed to analyze the impacts of the four housing and services interventions. Another goal was to keep the time needed for survey administration to a reasonable duration, thereby limiting respondent burden. This section provides a brief overview of the content of the follow-up survey.

36-Month Adult Follow-up Interview. The follow-up interview for adults will be used to measure participant outcomes over the longer follow-up period. It will facilitate a comparison of the degree of impact of the four interventions included in the study, as well as a comparison of impact for subgroups. The 36-month data will provide a clear comparison between the effects of time-limited housing assistance like PBTH and CBRR, which may still have been ongoing or have recently ended at 18 months, and those of the permanent subsidy offered in the SUB treatment arm. Relative intervention effects may change over time for other reasons as well.

To measure outcomes of the interventions on child well-being, the parent will be asked detailed questions about up to two focal children. Focal children are those who were selected at the time of the 18-month follow-up. In families where the 18-month follow-up was not completed, focal children will be selected at the time of the 36-month interview, should the adult respondent agree to participate.

The adult survey will collect information on homelessness and housing stability between the 18-month and 36-month interview dates; employment outcomes, including employment rates, and wages of family heads who work during the same period; family preservation, adult well-being, and child well-being. These topics will be included in the 36-month adult follow-up survey:

Housing Stability

Homelessness during follow-up period

Residential moves during the follow-up period

Housing satisfaction, affordability, quality

Receipt of housing assistance; participation in housing assistance programs

Self Sufficiency

Employment during the follow-up period

Earnings at the time of the follow-up survey

Education and training

Income sources; total family income

Food security

Economic stressors

Family Preservation

Child separations

Placements in foster care and informal placements

Reunifications of children with parents

Whether housing contributed to other family separations

Child Well-being measured from Parent reports about focal child(ren)

School attendance

Grade completion

Health status and access to health care Behavioral strengths and challenges

Pre-school/Head Start

Home Environment

Child development (through Ages and Stages Questionnaire)

Sleep disruptions

Transition to Adulthood

Services Received during Follow-up Period

Services received

Relationship with service provider

Adult Well-being measured for Custodial parent

Physical health

Substance use

Behavioral health symptoms

Depression

Trauma symptoms

Parenting

Focal Child Data Collection

In addition to the data collected on focal children through the parent report on child module of the adult survey, this request for clearance seeks approval of direct child data collection.

Direct child assessments are completed with focal children from 3 years 6 months to 7 years 11 months. The study uses two child assessment protocols:

Woodcock Johnson III letter work identification and applied problems scales (Woodcock et al., 2001). These assessments are early indicators of verbal and quantitative/analytic skills.
The Head Toes Knees Shoulders (Ponitz et al., 2008). This task assesses self-regulation. In this assessment, children must remember rules and inhibit incorrect responses (for example, by following instructions to touch their head when the interviewer says “touch your toes”).

36-month child follow-up survey. This survey is administered to youth from 8 years to 17 years 11 months of age to capture data on the following outcomes:

Mental health

Anxiety⁵
Fears
Life events
Substance use

School

Grades
Behaviors

Self-efficacy

Hope scale

Involved Vigilant Parenting

Informed Consent. Study participants completed a participation agreement when they enrolled in the study, providing their informed consent to participate in the research study. The original participation agreement did not specify a time limit for consent, and we believe the consent remains in force throughout the study period. At the time of the 18-month data collection, adult respondents renewed their consent to participate in the study. Adult respondents were also asked to provide their permission for the contractor to collect data from the selected focal child(ren) in the family. Similarly, focal child sample members aged 8 and up were asked to give their assent to participate in the child survey. The research team will again renew consent from study participants at the time of the 36-month follow-up survey. Obtaining renewed consent may facilitate data collection from administrative data sources. Renewed consent is not necessary to pursue administrative data collection because both the baseline and 18-month follow up consent provided permission for this type of data collection. However, our experience is that some providers of administrative data may require more recent participant consent than what was provided at the time of enrollment, between September 2010 and January 2012 and at the 18-month follow up survey from 2012- 2013. The contractor will also renew parental permission forms and child assent at the time of the 36-month data collection. The prior forms did not seek explicit consent to release the personal identifiers of the sample members to HUD. At the end of the adult data collection period, the contractor will also obtain consent to release personally identifiable data collected through the study to HUD. The full set of revised informed consent documents are presented in Appendix A.

A.3 Improved Information Technology

Improved information technology will be used in this evaluation in the following ways:

to maintain all demonstration data in a single location; and
to facilitate collection of the survey data and child assessments in standardized and accurate ways that also accommodates the confidential collection of sensitive data.

The Family Options Study will continue to generate a substantial amount of data on both adults and selected focal children. For adults, data collection includes interviews with enrolled families at baseline, random assignment records, tracking interviews, and the 18 and 36-month follow-up surveys. For focal children (up to two per enrolled family), data collection includes a survey at 18 and 36-month follow-up for focal children 8-17 years of age; direct child assessments of children 3 years 6 months to 7 years 11 months of age (at both 18 and 36-month follow-up); and parental report on child development for children 12-41 months of age (at 18 month follow-up) and 20-66 months of age at the 36-month follow-up. A study data base will be developed to manage the various sources of data. The database will house information from administrative data from HUD and other agencies that may provide data, baseline, tracking, and follow-up survey data, random assignment output, and data about each program included in each sites’ study interventions.

A.3.1 Information Technology and Survey Administration

Information technology assists in the survey data collection in three ways:

Design and management of the sample;
Survey administration; and
Survey data management.

Each of the study surveys will be administered using computer-assisted personal interviewing (CAPI) technology. The CAPI technology ensures that the survey data is of high quality. Data quality is enhanced in three key ways. First, CAPI technology controls the flow of the interview, ensuring that skip patterns are followed properly. It also allows the interviewer to both confirm responses (to minimize data entry errors) and check the logic of some responses by establishing allowable range of values. CAPI technology also allows interviewers to easily record verbatim responses to open-ended questions. Further, it records the current status of each case to facilitate monitoring of response rates and prompt resolution of problems if necessary.

A.4 Duplication of Similar Information

The purpose of the 36-follow-up survey for the Family Options Study is to obtain information about the status and well-being of families who enrolled in the study over the 36-month follow-up period. The sample for this evaluation is all families that enrolled in the study between 2010 and 2012. Information about the study participant’s experiences with respect to homelessness history, housing, employment status, participation in education and training, family composition, family preservation, income and income sources, physical and mental health, substance use, and other characteristics is not available through any other source. These data are essential for measuring long-term outcomes of the study interventions needed to conduct the impact analysis.

Duplication will be avoided in this study by use of the centrally maintained database, which will link all data collected in the follow-up survey to information collected from study participants previously on the baseline and tracking surveys, as well as with any administrative data collected for study participants. This reduces the need to ask about personal characteristics and background factors on the follow-up survey that have already been collected for the family. In addition, information collected on the tracking or 18-month interviews has been stored in the study database and will be referenced during the follow-up interview. In this way, the follow-up survey will involve verification and correction of previously-provided information on family composition and contact information rather than collecting each item anew. Information collected in the 18-month follow-up data collection will be used to establish the recall period ensuring that families that completed the 18-month data collection will report only on their experiences since they completed that interview. Families that did not complete the 18-month interview will report on experiences between enrollment and the 36-month follow-up.

A.5 Small Businesses (Involvement of Small Entities)

Respondents for this data collection include families who enrolled in the study, not business entities. There is no expected burden on small businesses.

A.6 Less Frequent Data Collection

The 36-month follow up survey data collection effort is essential to conducting the analysis of the impacts of the housing and services interventions. Less frequent data collection would jeopardize HUD’s ability to conduct the impact analysis.

Although the 18-month follow-up survey will provide valuable data on early outcomes of the experiment, it will leave important questions unanswered. At 18-months following random assignment, families could still be living in project-based transitional housing (PBTH) (which allows families to stay for up to 24 months),^⁶ or could just be completing participation in rapid-rehousing (where assistance can last up to 18-months). How families fare after these interventions end is an important concern. The critical question of whether the same interventions that are effective in the short term are also effective in the longer term cannot be answered fully with data collection that ends at 18-months.

The interventions under study are rooted in different understandings of the course of homelessness, and how interventions may affect it. The rationale for CBRR is that homelessness is a temporary crisis. Evidence for this perspective includes the fact that most families who use shelter do so only once for relatively brief periods and do not return.⁷ Knowing how families fare after assistance ends is critical to evaluate this approach. Collecting data at 36-months, a year after the longest intervention would end, allows us to determine if the short-term impacts remain effective in the long-term.

A.7 Special Circumstances

The proposed data collection activities are consistent with the guidelines set forth in 5 CFR 1320.6 (Controlling Paperwork Burden on the Public, General Information Collection Guidelines). There are no circumstances that require deviation from these guidelines.

A.8 Federal Register Notice/ Consultation Outside the Agency

In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and Urban Development (HUD) published a notice in the Federal Register on September 16, 2013. The docket number was FR-5689-N-08 and the document number is FR Doc. 2013-22456 Filed 9-13-13. The Federal Register notice appeared on pages 56910 -56911. The authority is Section 3506 of the Paperwork Reduction Act of 1995, 44 U.S.C, Chapter 35, as amended. The notice provided a 60-day period for public comments, and comments were due by November 15, 2013. A copy of the notice is shown in Appendix F.

The Family Options Study design was developed and is being implemented with the assistance of Abt Associates Inc., the prime contractor. Several subcontractors and consultants have collaborated with the Abt team to develop the study design. Key members of the Abt team include Dr. Stephen Bell, Dr. Jill Khadduri, Mr. Jacob Klerman, Ms. Michelle Wood, Ms. Brooke Spellman, and Ms. Mary Joel Holin. Dr. Marybeth Shinn (Vanderbilt University), Dr. Dennis Culhane (University of Pennsylvania), Dr. Martha Burt (MBR Consulting), Dr. Ellen Bassuk (Center for Social Innovation). Dr. Beth Weitzman (New York University) and Dr. Larry Orr also worked with Abt Associates to develop the study design.

The child outcomes study was initially developed as an additional research effort funded by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD). The grant was awarded to Vanderbilt University and Abt Associates. The study was designed to expand the research platform by including a child outcome study. The study design and data collection protocols selected for that study were led by Dr. Marybeth Shinn, Lindsay Mayberry, Dr. Velma McBride-Murray at Vanderbilt University; Michelle Wood, Dr. Jill Khadduri, Jacob Klerman and Debi McInnis of Abt Associates and the study’s Technical Advisory Group members: Dr. Ann Masten, University of Minnesota; Dr. Rashmita Mistry & Kirby Chow, UCLA; Dr. Cybele Raver, NYU; and Dr. Hiro Yoshikawa, Harvard.

HUD has collaborated on the design of the evaluation with the Abt Associates team throughout all phases of the study to date. The purpose of such consultation is to ensure the technical soundness and usefulness of the data collection instruments in carrying out the aims of the evaluation.

A.9 Payments to Respondents

Incentive payments are a powerful tool for maintaining low attrition rates in longitudinal studies. Adult respondents completing the 36-month follow-up survey will receive $50 for their time. Child respondents that complete the 36-month child follow-up survey or the child assessments will receive $25 in appreciation of their time. Incentive payments for child participation will be made to the parent on behalf of the child. The use of incentive payments for the Family Options Study can help ensure a high response rate, which is necessary to ensure unbiased impact estimates. Low response rates increase the danger of differential response rates between the four intervention groups, leading to non-comparability between the two groups and potentially biased impact estimates.

Three factors helped to determine the incentive amounts for each survey:

Respondent burden, both at the time of the interview and over the life of the study;
Costs associated with participating in the interview at that time; and
Other studies of comparable populations and burden.

The Family Options Study panel is small (2,282 families; roughly 3,010 focal children) and avoiding attrition is essential to the success of the study. This population size will permit detection of impacts in the likely size range only if panel attrition is kept very low (precision of the impact estimates is discussed in Section B.2). Even with no attrition, only fairly large effects can be detected. Therefore, we believe it is absolutely necessary to take every possible step to minimize panel attrition over the study follow-up period. This minimal attrition rate is the core justification for an incentive system for the follow-up interview. The need to maintain the panel is further complicated by the housing instability likely in this study population.

Our experience with this study population shows that it is more likely to respond positively to incentive payments. Previous research has shown that sample members with certain socio-economic characteristics are significantly more likely to become survey respondents when incentive payments are offered. In particular, sample members with low incomes and/or low educational attainment have proven responsive to incentives, as have minority group members. These characteristics are expected to be heavily represented in this study panel (Duffer et al. 1994); Educational Testing Service (1991).

Based upon these considerations and prior research experience, we believe that that the use of incentives will improve substantially the probability of panel retention and the viability and power of this experimental research study. The Family Options Study represents the first experimental research project HUD has initiated to test the effects of various interventions to assist homeless families; the Department thus places a high level of importance on ensuring that the study panel remains of sufficient size so that the intended statistical measures can be used to draw firm policy conclusions. As described further in Part B.3, the contractor will monitor response rates by site and intervention groups. Although not part of the currently funded design, should there be disparity in the response rates for particular sites or intervention groups, we will work with the evaluation contractor to assess the feasibility of offering differential incentives. Should that be necessary—and determined to be feasible—we would increase the incentives offered to hard-to-complete adult respondents. Such an approach was implemented under the MTO final evaluation (Gebler, et. al., 2012).

A.10 Confidentiality

The subjects of this information collection and the nature of the information to be collected require strict confidentiality procedures. The information requested under this collection is protected and held confidential in accordance with 42 U.S.C. 1306, 20 CFR 401 and 402, 5 U.S.C.552 (Freedom of Information Act), 5 U.S.C. 552a (Privacy Act of 1974) and OMB Circular No. A-130. A Privacy Impact Assessment is on file with the Department, and a System of Records Notice (SORN) was published in the Federal Register in October 2010 (FR-5386-N-10). This data collection effort is further protected through a certificate of confidentiality CC-HD-12-32, issued June 13, 2012 by the Department of Health and Human Services, NICHD. Detailed procedures used to obtain informed consent are discussed below along with data security procedures.

A.10.1 Informed Consent

Procedures used to enroll adult participants into the study were designed to ensure that participants were able to make a genuinely informed decision about study participation. Vigorous outreach with a clear message and strong supporting materials were used to ensure that those assigned to the interventions tested through the study understand the opportunities available and were likely to take advantage of the intervention’s benefits.

Those families who participate actually face little risk by agreeing to be part of the evaluation. The outreach effort emphasized this fact. Families with at least one child age 15 or under, residing in an emergency shelter for at least seven days were invited to enroll in the evaluation. The informed consent of each sample member was obtained through a signed consent form, the “Participation Agreement,” which described the evaluation, the process of random assignment, and the information requirements of the evaluation. As part of the 18-month follow-up data collection, the research team updated the participants’ consent to participate in the study. The researchers will renew consent again at the time of the 36-month follow-up. The revised form is shown in Appendix A of this submission.

A local site interviewer hired and trained by the research team conducted intake and random assignment in each study site. The site interviewer described the other implications of participating in the study, which related mostly to data collection. Study participants were required to complete the baseline interview and also agreed to be contacted in the future for tracking and for the follow-up interviews. Families who agreed to enroll in the study also granted the researchers permission to access information about them from other administrative records systems, like HUD’s Homeless Management Information System (HMIS), and HUD’s Public and Indian Housing’s Information Center (PIC), and from other providers of subsidized housing programs in order to monitor receipt of housing assistance. Permission to access data from other sources is necessary to support the collection of other types of administrative data. Sources to be collected for the 36-month follow-up data collection include child welfare data and unemployment insurance wage data, in addition to the HUD PIC/TRACS data and HMIS data already collected regularly from the local sites

Prior consent forms did not include a request for permission to release study participants’ personal identifiers to HUD. Since families will be contacted again for the 36 month follow up survey, it provides an opportunity to obtain permission for the Abt research team to share PII with HUD. The consent to release information form is included as Appendix A4.

A.10.2 Data Confidentiality Protections

The data collected in the adult and child surveys or direct child assessments for the Family Options Study as well as any administrative data collected from HMIS, HUD’s Public Housing Information Center (PIC) data system (this system provides records on the receipt of housing assistance through the public housing or Housing Choice Voucher programs), child welfare, unemployment insurance or any other source will be used only for the purposes of evaluating the housing and services interventions tested in the evaluation. All communications with study participants include assurances that participation is voluntary, that all information will be kept confidential, and that the respondents' answers will be reported only in aggregate form. An assurance of confidentiality is included in the Participation Agreement (see Appendices A1-A3). Contractor staff must sign a pledge of confidentiality as a condition of employment. Separate data files will be maintained for questionnaire responses and identifying information; linking will be possible by a common identification number. For both survey data and corresponding administrative data on sample members, computer security will be maintained by passwords known only to project staff members that require access to these files. Further access to data is restricted only to project staff members that require direct access to the raw data files for analysis purposes; analysts and authors will have access only to aggregate data.

In addition, all design documents, random assignment protocols, and analysis files must be protected. The study’s data collection plan, this OMB statement, and the proposed survey instrument are also subject to the review and approval of the contractor’s Institutional Review Board (IRB). The contractor’s IRB previously approved the study design, baseline, tracking survey and all adult and child data collection efforts for the 18-month follow-up data collection procedures.

A.10.3 Data Storage and Handling of Survey Data

To ensure data security and enhance data quality, the survey data collection will be done using Computer Assisted Personal Interviewing (CAPI) technology. Survey data will be collected using the Confirmit CAPI System. The Confirmit CAPI System has the following security features:

Data on the CAPI console is encrypted with Rijndael algorithm (256 bit key).
CAPI data transfers use Web Services Enhancements (WSE 3.0) for security. The messages sent and received from the console are encrypted. WSE 3.0 provides AES128 + RSA 1.5 as default algorithms for symmetric encryption and key-wrap. The contractor has also implemented Secure Conversation with an X509 certificate (which uses 1024 bit key).

In addition to the standard security features offered through the CAPI software, the contractor has implemented the following enhancements:

Use of PGP whole disk encryption on all CAPI laptops, and
The file transfers are made to servers running SSL.

Once the surveys are completed, data will be transferred from the CAPI system to the study’s database. Transfer to the database will be done in a secure manner, using a FIPS-certified encryption algorithm.

A.11 Sensitive Questions

Like the 18-month follow-up survey currently in use, the 36-month follow-up adult survey includes questions about history of homelessness, household income and other financial circumstances. The interviews also include questions about physical and emotional health, substance use, and questions about the focal child’s health, educational attainment, and behavior—items that can be considered sensitive. The child survey also asks questions about fears, experiences and anxiety, which can be considered sensitive. These items are necessary to evaluate the impacts of the housing and services interventions being tested. As with all information collected on the interviews, responses will be kept confidential and will be used only for the purposes of evaluating the housing and services interventions tested in the evaluation. To encourage candid responses, respondents will be reminded during the interviews that their responses will be kept confidential. Respondents will also be reminded that they can refuse to answer any question.

The contractor has not experienced any difficulty administering these questions under the 18-month follow-up data collection. The interviewers have not reported any negative feedback from respondents on these questions, either.

A.12 Burden Estimates (Total Hours and Wages)

The 36-month follow-up survey data collection for the Family Options Study will be implemented beginning in March 2014 and continue through March 2015. Interviewing is expected to continue over a 12-month period, in order to attempt interviews with study participants approximately 36 months after random assignment.

Exhibit A-2 shows the estimated respondent burden for the 36-month adult and child follow-up survey interviews, and the direct child assessments. It shows the average time, in hours, estimated to be spent by study participants who complete each data collection component.

Exhibit A-2 also shows estimated burden for state and local administrators of child welfare data systems and Unemployment Insurance wage records, to provide data use agreements and to provide administrative records data to the research team. The contractor will negotiate with each state or local agency to develop an agreement for matching identifiers of study sample to these administrative records systems. The administrators will then provide records showing quarterly wage amounts; and child welfare records regarding foster care placements to the contractor. We estimate that burden hours for establishing the data use agreements, conducting the match, and providing the data will total 40 hours per site for each administrative system.

Exhibit A-2. Estimated Respondent Burden Hours and Costs

Instrument	Total # of Respondents	Total # of Responses per Respondent	Avg. Burden Hours per Response (in minutes)	Total Burden Hours	Avg. Annual Burden Hours	Avg. Hourly Wage	Total Annual Cost
Appendix B: 36-month Adult Follow-up Survey (N=2,282)⁸; Appendix A1: Adult respondent consent form (N=2,282); Appendix A2: Parental Permission Form (N=2,282); Appendix A4: Release of Information Form	1,712	1	75 (60 minutes for survey plus 15 minutes for consent forms assuming 5 minutes per each of the IC forms)	2,140	2,140	$10.15	$21,720
Direct Child Assessments⁹ (N=1,330)	811¹⁰	1	50	676	676	N/A
Appendix D: 36-month Child Follow-up Survey (N=1,380); Appendix A3: Child Assent Form (N=1,380)	883¹¹	1	35 (30 minutes for the survey and 5 minutes for informed consent)	515	515	N/A
#4 Administrative Data: UI Wage Records	12	2	1,200	480	480	$45.08	$21,638
#5 Administrative Data: Child Welfare Records	12	2	1,200	480	480	$45.08	$21,638
Total Burden Hours and Costs				4,291	4,291		$64,996

*Response burden for the adult sample is based on a 75 percent response rate. Ages and Stages response rates are based on an overall 69 percent completion rate. Child assessment response burden is based on an overall assessment completion rate of 61 percent (see footnote 9). The response burden for the child survey was based on an overall child survey completion rate of 64 percent (see footnote 10).

The table also shows the annualized cost burden for respondents that participate in the data collection effort. The cost burden was calculated by determining an average hourly wage with benefits. The average hourly wage was calculated for each adult respondent group based on information from the Bureau of Labor Statistics¹² or the federal minimum wage.

The average hourly rate¹³ for each respondent group was calculated as follows:

Study participant: the minimum hourly wage ($7.25) plus a 40 percent adjustment to account for benefits, or $10.15 per hour.
Annualized cost estimates were not calculated for the child sample. The child sample eligible to participate in the 36-month follow-up data collection will be under the age of 18. Most, if not all, will be enrolled in school and working part-time at the most. Thus, we did not calculate an hourly wage for the child sample.
Annualized cost estimates for the administrative data collection were based on the mean hourly wage rates for state government $32.20plus a 40 percent adjustment to account for benefits, or $45.08 per hour.

Using the average times, the total burden of the Impacts of Housing and Service Interventions for Homeless Families data collection from survey respondents is 4,149 hours. The total cost burden is estimated at $63,550.

A.13 Capital Costs (Maintenance of Capital Costs)

This data collection effort involves no recordkeeping or reporting costs for respondents other than the time burden to respond to questions on the data collection instruments as described in item A.12 above. There is no known cost burden to the respondents.

A.14 Costs to the Federal Government

The estimated cost to the federal government of the planned 36-month follow-up survey data collection and the non-HUD administrative data collection activities for the Family Options Study is $2,882,387. These are subtotals of the total cost of the final phase of the evaluation, which equals $4,298,981. The final phase of the evaluation includes costs associated with 36-month adult and child follow-up survey data collection, direct child assessments, collection of administrative data, analysis of intervention impacts and costs, and preparation of final impact report. The final phase does not include costs associated with research design, site recruitment, participant enrollment, baseline and 18-month follow-up data collection.

Exhibit A-3 shows the costs to the federal government of the planned follow-up survey data collection activities. HUD’s current evaluation contractor, Abt Associates, prepared these estimates. These costs are entirely federal costs.

Exhibit A-3. Estimated Costs to the Federal Government

Data Collection Activity	Cost to the Federal Government
Participant Surveys
Adult Follow-up Survey	$1,837,741
Child Surveys and Direct Child Assessments	$803,807
Administrative Data Collection (Non-HUD)	$240,389
Total Data Collection Costs	$2,882,387

A.15 Program or Burden Changes

This request for clearance does not involve a change in burden due to any program changes or adjustments. It concerns a revision to an existing collection (2528-0259). The revision covers a repeat of the 18-month follow-up adult interview, 36-months after random assignment. It also includes the request for clearance for the child interview and direct assessments, originally funded under NICHD grant 1R01HD066082-01. Thus, the information collection included in this supporting statement will increase the public reporting burden.

A.16 Publication and Tabulation Dates

The data collected for the Family Options Study will be analyzed, tabulated, and reported to HUD by the evaluation contractor, Abt Associates Inc, and Abt’s team of subcontractors and consultants.

A.16.1 Time Schedule for Analysis and Reporting

Collection of baseline survey data from study participants began in September 2010 and ended in January 2012. The baseline survey data set was cleaned, appended to the study database, and used to produce the Interim Report (Gubits et al., 2012). The 18-month follow-up adult and child survey data collection and child assessments began in July 2012 and will conclude in November 2013. Analysis began in October 2013 and will continue through May 2014, with a report due in December 2014.

Baseline Data Collection September 2010 through January 2012

Participant Tracking February 2011 through September 2014

Program-level Data Collection June – December 2012

Baseline Data Analysis March 2012 through June 2012

Interim Report Draft June 2012; Final September 2012

18-month Follow-up Survey Data Collection June 2012 – November 2013

Impact Analysis October 2013 – May 2014

Draft Final Report Sept 2014

Final Report December 2014

36-month Follow-up Data Collection March 2014-March 2015

Impact Analysis September 2014 – September 2015

Draft Final Report February 2016

Final Report July 2016

Data collection for the 36-month follow-up is expected to start in March 2014 and conclude in March 2015. Analysis of the data collected from the prior approved information collections, administrative data sources, program providers and the data collected under this information collection request, will run from September 2014 to September 2015. A final report is due in July 2016.

Impact findings will be reported in tables that show estimated impacts on outcomes measured with survey data.

A.16.2 Analytic Techniques, Tabulations, and Reporting

The ultimate goal of the study is to estimate the impact of each of the housing and services interventions compared to Usual Care and to the other interventions. The baseline data collection and participant tracking for the study and the 18-month follow-up data collection were approved in a previous submission for OMB clearance. This submission seeks approval for the collection of the 36-month follow-up survey data and the child data collection components. Exhibit A-4 displays a template for presenting information on intervention impacts using data from the follow-up survey.

Exhibit A-4. Impacts on [Key Outcome 1] and [Key Outcome 2] at Follow-Up

Outcome/Impact Comparison	SUB Mean	CBRR Mean	PBTH Mean	UC Mean	Impact Estimate	Standard Error
At least one night spent homeless or doubled up during past six months
SUB vs. UC	0.nn			0.nn	0.nn	0.nn
CBRR vs. UC		0.nn		0.nn	0.nn	0.nn
PBTH vs. UC			0.nn	0.nn	0.nn	0.nn
SUB vs. CBRR	0.nn	0.nn			0.nn	0.nn
SUB vs. PBTH	0.nn		0.nn		0.nn	0.nn
CBRR vs. PBTH		0.nn	0.nn		0.nn	0.nn
Any return to emergency shelter in previous 18 months
SUB vs. UC	0.nn			0.nn	0.nn	0.nn
CBRR vs. UC		0.nn		0.nn	0.nn	0.nn
PBTH vs. UC			0.nn	0.nn	0.nn	0.nn
SUB vs. CBRR	0.nn	0.nn			0.nn	0.nn
SUB vs. PBTH	0.nn		0.nn		0.nn	0.nn
CBRR vs. PBTH		0.nn	0.nn		0.nn	0.nn

Notes: SUB = Subsidy Only; CBRR = Community-Based Rapid Re-housing; PBTH = Project-Based Transitional Housing; UC = Usual Care.

*** = p<.01, ** = p<.05, * = p<.10 in two-tailed t-test.

Impact estimates are regression-adjusted.

A.17 Expiration Date

All data collection instruments created for the Impact of Housing and Services Interventions for Homeless Families evaluation, or Family Options Study will display prominently the expiration date for OMB approval.

A.18 Certification Statement

This submission describing data collection requests no exceptions to the Certificate for Paperwork Reduction Act (5 CFR 1320.9).

References

Allison, Paul D., (2002). Missing Data. Thousand Oaks, CA: Sage University Paper No. 136

Angrist, J. D., & Pischke, J.-S. (2008). Mostly harmless econometrics: An empiricist’s companion. Princeton, NJ: Princeton University Press.

Burt, M.R. (2006). Characteristics of transitional housing for homeless families: Final report. Prepared for the US Department of Housing and Urban Development.

Burt, M. R., The Urban Institute, Pearson, C. L., Montgomery, A. E., & Walter R. McDonald & Associates, Inc. (2005). Strategies for preventing homelessness. Washington, D.C and Rockville, MD: Urban Institute and Walter R. McDonald & Associates, Inc

CDC Youth Risk Behavior Surveillance substance use questions. (2011). http://www.cdc.gov/healthyyouth/yrbs/pdf/questionnaire/2011_hs_questionnaire.pdf

Cowal, K., Shinn, M., Weitzman, B.C., Stojanovic, D., & Labay, L. (2002). Mother-child separations among homeless and housed families receiving public assistance in New York City. American Journal of Community Psychology, 30, 711-730.

Culhane, D., Metraux, S., & Byrne, T. (2011). A prevention-centered approach to homelessness assistance: A paradigm shift? Housing Policy Debate, 21, 295-315

Culhane, D. P., Metraux, S., Park, J.M., Schretzman, M. & Valente, J. (2007). Testing a typology of family homelessness based on patterns of public shelter utilization in four U.S. jurisdictions: Implications for policy and program planning. Housing Policy Debate, 18(1), 1-28.

Culhane, D. P. (1992). The quandaries of shelter reform: An appraisal of efforts to “manage” homelessness. Social Service Review, 66, 428–440.

Dahl, R.E. & Harvey, A. G. (2007). Sleep in children and adolescents with behavioral and emotional disorders. Sleep Medicine Clinics, 2, 501-511.

Duffer, Allen P. et al., "Effects of Incentive Payments on Response Rates and Field Costs in a Pretest of a National CAPI Survey" (Research Triangle Institute, May 1994).

Fisher, B. W., Shinn, M., Mayberry, L., & Khadduri, J. (in press). Leaving homelessness behind: Housing decisions among families exiting shelter. Housing Policy Debate.

Gebler, Nancy and Margaret L. Hudson, Matthew Sciandra, Lisa A. Gennetian and Barbara Ward. (2012). Achieving MTO's High Effective Response Rates: Strategies and Tradeoffs. Cityscape, 14(2), 57-86.

Goodman R (1997). The Strengths and Difficulties Questionnaire: A Research Note. Journal of Child Psychology and Psychiatry, 38, 581-586.

Greene, W. H. (2003). Econometric analysis: Fifth edition. Upper Saddle River, NJ: Prentice-Hall, Inc.

Gubits, Daniel et al., Interim Report, Family Options Study, Washington, DC: U.S. Department of Housing and Urban Development, March 2013. http://www.huduser.org/portal/publications/homeless/hud_503_FOS_interim_report.html

Herbers, J. E., Cutuli, J. J., Supkoff, L. M., Heistad, D., Chan, C.-K., Hinz, E., & Masten, A. S. (2012). Early reading skills and academic achievement trajectories of students facing poverty, homelessness and high residential mobility. Educational Researcher, 41(9), 366-374.

Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability. Lucien M. Le Cam & Jerzy Neyman & Elizabeth L. Scott (editors). Berkeley, CA: University of California Press, Vol. 1, 221-233.

Locke, G., Khadduri, J. & O’Hara, A. (2007). Housing models (Draft).

Masten, A. S., Neemann, J., & Andenas, S. (1994). Life events and adjustment in adolescents: The significance of event independence desirability, and chronicity. Journal of Research on Adolescence, 4(1), 71-97.

Mayberry, L. S., Shinn, M., Benton, J. G., & Wise, J. (in press). Families experiencing housing instability: The effects of housing programs on family routines and rituals. American Journal of Orthopsychiatry.

Mills, G., Gubits, D., Orr, L., Long, D., Feins, J., Kaul, B., Wood, M., Amy Jones and Associates, Cloudburst Consulting, & the QED Group. (2006). Effects of Housing Vouchers on Welfare Families: Final Report. Prepared for the U.S. Department of Housing and Urban Development, Office of Policy Development and Research. Cambridge, MA: Abt Associates Inc.

Mistry RS, Lowe E, Benner AD, Chien N. (2008). Expanding the family economic stress model: Insights from a mixed methods approach. Journal of Marriage and Family, 70,196–209.

Orr, L.L., Feins, J., Jacob, R., Beecroft, E., Sanbonmatsu, L., Katz, L., Liebman, J. & Kling, J. (2003). Moving to Opportunity interim impacts evaluation: Final report. Cambridge, MA: Abt Associates Inc. and National Bureau of Economic Research.

Ponitz, C. C., McClelland, M. M., Jewkes, A. M., Connor, C. M., Farris, C. L., & Morrison, F. J.

(2008). Touch your toes! Developing a direct measure of behavioral regulation in early childhood. Early Childhood Research Quarterly, 23, 141–158. doi: 10.1016/j.ecresq.2007.01.004.

Puma, Michael, J., Robert B. Olsen, Stephen H. Bell, and Cristofer Price (2009). What to Do When Data Are Missing in Group Randomized Controlled Trials (NCEE 2009-0049). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Rafferty, Y., Shinn, M., & Weitzman, B. C. (2004). Academic achievement among formerly homeless adolescents and their continuously housed peers. Journal of School Psychology, 42(3), 179-199. doi: 10.1016/j.jsp.2004.02.002

Ramirez, M., Masten, A., Samsa, D. (1991, April). Fears in homeless children. Paper presented at the biennial meeting for the Society for Research in Child Development, Seattle, WA.

Rog, D.J. & Randolph, F.L. (2002). A multisite evaluation of supported housing: Lessons learned from cross-site collaboration. New Directions for Evaluation, 94, 61-72.

"National Adult Literacy Survey Addendum to Clearance Package, Volume II: Analyses of the NALS Field Test" (Educational Testing Service, September 1991), pp. 2-3.

Shinn, M., Weitzman, B. C., Stojanovic, D., Knickman, J. R., Jiminez, L., Duchon, L., James, S., & Krantz, D.H. (1998). Predictors of homelessness from shelter request to housing stability among families in New York City. American Journal of Public Health, 88 (10), 1651-1657.

Shinn, M., Schteingart, J. S., Williams, N. P., Carlin-Mathis, J., Bialo-Karagis, N., Becker-Klein, R., & Weitzman, B. C. (2008). Long-term associations of homelessness with children’s well-being. American Behavioral Scientist, 51, 789-810.

Snyder, C. R., Hoza, B., Pelham, W. E., Rapoff, M., Ware, L., Danovsky, M., . . . Stahl, K. J. (1997). The development and validation of the Children's Hope Scale. Journal of Pediatric Psychology, 22(3), 399.

Squires, J., Bricker, D. D., Potter, L., & Twombly, E. (2009). Ages & stages questionnaires : a parent-completed child monitoring system (3rd ed.). Baltimore: Paul H. Brookes Pub. Co.

Stretch, J. J. & Kreuger, L. W. (1993). A social-epidemiological five year cohort study of homeless families: A public/private venture policy analysis using applied computer technology. Computers in Human Services, 9(3-4), 209-230.

U.S. Department of Housing and Urban Development (2012). Volume I of the 2012 Annual Homeless Assessment Report (AHAR). https://www.onecpd.info/resource/2753/2012-pit-estimates-of-homelessness-volume-1-2012-ahar

U.S. Department of Housing and Urban Development (2013). Volume II of the 2012 Annual Homeless Assessment Report (AHAR).

Voight, A., Shinn, M., & Nation, M. (2012). The longitudinal effects of residential mobility on the academic achievement of urban elementary and middle school students. Educational Researcher, 41, 385-392.

White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48, 817–830.

White, H. (1984). Asymptotic theory for econometricians. Orlando, FL: Academic Press.

Woodcock, R. W., McGrew, K. S., Mather, N., & Schrank, F. A. (2001). Woodcock-Johnson III tests of cognitive abilities. Itasca, IL: Riverside Pub

1 Initially, the study was to be referred to as the Impacts of Housing and Services Interventions for Homeless Families or Homeless Families Impact Study. However, as implementation began in the study sites in September 2010, the research team sought an alternative name to refer to the study locally. To avoid potential stigma for participating families, we refer to the study locally, and throughout the remainder of this document, as the Family Options Study.

2 The final enrollment count for the study was 2,307 families. However, upon reviewing baseline data collected, the research team determined that 25 families had been enrolled in error and did not satisfy the family eligibility requirement of having at least one child age 15 or younger. These 25 families, scattered at random among the four random assignment arms in the study design, have been removed from the research sample without skewing the statistical equivalence of the arms, leaving 2,282 families.

3 Senate Report 109-109 to accompany HR 3058. July 26, 2005 (page 176). The report is available at http://frwebgate.access.gpo.gov/cgi-bin/getdoc.cgi?dbname=109_cong_reports&docid=f:sr109.pdf, accessed on June 11, 2009.

4 Previous research has shown that actual stays in transitional housing average 6-12 months. The 2012 Volume II Annual Homeless Assessment Report (AHAR 2013)) reports a median transitional housing stay of 124 nights. However, those stays are based on a one-year reporting period. Since transitional housing offers housing that generally lasts six to 24 months, the AHAR length of stay is likely truncated. Within the Family Options Study, the median length of stay for families assigned to receive project based transitional housing was 17 months(Gubits et al,, 2012)

5 The child survey is presented in Appendix D. The TRAITS module, which measures anxiety, is a proprietary instrument. Appendix D therefore shows only illustrative items, not the full battery.

6 Previous research has shown that actual stays in transitional housing average 6-12 months. Within the Family Options Study, the median length of stay for families assigned to receive project based transitional housing was 17 months(Gubits et al,, 2012)

7 Culhane, Metraux, Park, Schretzman, & Valente, 2007.

8 There are 200 focal children who will be eligible for parental report using the Ages and Stages Questionnaire (ASQ-3) and will also complete the direct child assessments. When describing the total focal child sample (see for example Exhibit B-3), we only count these 200 focal children once, in the sample count for direct child assessments. For calculating burden estimates, these 200 focal children must be included in the burden estimates for each data collection effort. As a result the ASQ-3burden estimates are based on 500 focal children (300 who will receive only the ASQ-3 and 200 who receive both the ASQ-3 and direct assessments

9 The Direct Child Assessments will be administered to focal children between 3.5 and 7.11 years and includes Woodcock Johnson Applied Problems and Letter Word Identification modules and the Head Toes Knees Shoulders protocol, which are proprietary instruments, and thus not included.

10 The child assessment data collection at 18 months had an overall completion rate of 66 percent. This rate was the product of the 81 percent response rate of the family heads and the 81 percent completion rate for focal children identified in interviews with family heads. At 36 months, we assume a 75 percent response rate for family heads and the same 81 percent completion rate for identified focal children in the child assessment age range. The product of these two assumptions is an overall child assessment completion rate of 61 percent.

11 The child survey data collection at 18 months had an overall completion rate of 69 percent. This rate was the product of the 81 percent response rate of the family heads and the 85 percent completion rate for focal children identified in interviews with family heads. At 36 months, we assume a 75 percent response rate for family heads and the same 85 percent completion rate for identified focal children in the child survey age range. The product of these two assumptions is an overall child survey completion rate of 64 percent.

12 http://www.bls.gov/oes/current/oes_nat.htm

13 Assuming 2080 FTE hours worked.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Anne Fletcher
File Modified	0000-00-00
File Created	2021-01-28