MTOFinalEval_Part A rev 2

Part A: Justification
A.1 Circumstances Making Information Collection Necessary
This request is for the clearance of survey instruments, youth achievement testing, adult
biomarker data collection and interviewer observations of respondent’s neighborhoods for
the Final Evaluation of the Moving to Opportunity for Fair Housing (MTO) demonstration
program. MTO is a unique experimental research demonstration designed to answer the
question of whether moving from a high-poverty neighborhood to a lower-poverty
community improves the social and economic prospects of low-income families.
Originally authorized by Congress in Section 152 of the Housing and Community
Development Act of 1992, MTO made use of Section 8 rental assistance, in combination
with intensive housing search and counseling services, to assist low-income families with
children living in public or Section 8 project-based housing to move from some of our
nation’s most distressed urban neighborhoods to lower-poverty communities. The program
was run in five cities: Baltimore, Boston, Chicago, Los Angeles, and New York.
The authorizing Congressional legislation of 1992 also required that HUD undertake a
long-term evaluation of MTO’s impacts on families, which is the subject of this request.
HUD recognized that more families would wish to participate in the program than HUD
had Section 8 rental assistance. This excess demand for Section 8 rental assistance allowed
program participation to be determined by a random lottery, which in turn has been central
to HUD’s ability to support rigorous research efforts to understand the MTO program’s
causal impact on participating families. The enrollment and randomization phase of MTO
ended in February 1999, but MTO families continue to receive the housing vouchers they
were offered under the program and there have also been several phases of HUD-supported
research to capture MTO impacts over time.
The MTO demonstration has two broad research goals. The first short term goal was to
compare the costs and services of the MTO program with the routine implementation of
the Section 8 tenant-based rental assistance program. HUD reported to Congress in 1996
on these findings regarding the progress and effectiveness of the demonstration. The
second longer term goal is to assess the impact of the demonstration on family and
children’s well-being including their housing conditions, mental and physical health,
employment and earnings, receipt of social program assistance and income, education, and
delinquent or risky behavior of children.
To meet these goals, HUD took advantage of the excess demand for Section 8 rental
subsidies and randomly assigned through a lottery-like process all families who signed
up for MTO into one of three groups:
¾ the MTO EXPERIMENTAL GROUP, which received Section 8 certificates or vouchers
usable only in low-poverty areas (areas with less than 10 percent of the population
below the poverty line in 1990), along with counseling and assistance in finding a
private unit to lease;
1
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

¾ the SECTION 8 COMPARISON GROUP, which received regular Section 8 certificates or
vouchers (geographically unrestricted) and ordinary briefings and assistance from the
Public Housing Authorities (PHA); and
¾ the IN-PLACE CONTROL GROUP, which received no certificates or vouchers but
remained eligible for public or project-based housing and other social programs that
families would otherwise have been entitled to
The participants assigned to these three research groups have been tracked and surveyed 23 years after random assignment (the “short term” MTO evaluation) and 4-7 years after
random assignment (the “interim” MTO evaluation) to understand the effects of the
demonstration on participating families. The experience of families assigned to the
Experimental group receiving the special MTO assistance can be compared with that of
families who receive the "regular" Section 8 treatment. The in-place control group is
essential to correctly estimate the impacts of Section 8 rental assistance separate from the
impacts of MTO assistance with counseling, providing a benchmark against which the
outcomes of the two other groups can be measured.
The MTO final evaluation (the subject of this request) will examine many aspects of family
life measured on average 10 to 12 years after enrollment in the demonstration program, with
a focus on those outcome domains that may have been affected by MTO participation. The
final evaluation represents the first attempt since the interim evaluation to interview sample
members in depth, using common survey instruments and data collection techniques across
all sites.
A total of 4,608 families enrolled in the MTO demonstration and were randomly assigned to
one of the three research groups between September 1994 and August 1998. All enrolled
families completed a baseline information form under a clearance granted by OMB in 1994.1
The MTO research team maintained contact with MTO enrolled families to update
information about addresses, and changes in family status, employment and receipt of
program services through brief canvasses conducted in 1997 and 2000, under clearances
previously granted by OMB.2 The interim MTO evaluation of program impacts after 4 to 7
years was completed in 2002. Abt Associates has tracked addresses and changes in family
status of MTO families since the interim evaluation and will continue to do so, under contract
with HUD, through September 2007.3
This submission is a request for clearance of the following instruments, and data
collection, to provide HUD with the information needed to determine the long-term
effects of the MTO demonstration on the lives of participating low-income families
and children:
1

Clearance No. 2528-0161, initially expiring June 1997, finally expiring November 30, 2000.
Clearance of the MTO canvass data collection was originally granted by OMB under clearance number 2528-0189,
expiration date January 1999, extended to April 1999 (see Notice of Short Term Extension from Donald R. Arbuckle, OMB,
dated 1/19/1999). This clearance was subsequently extended through June 30, 2002 (see Notice of Office of Management
and Budget Action from Donald R. Arbuckle, OMB, dated June 24, 1999).
3
Clearance of the MTO Interim evaluation was granted by OMB under clearance number 2528-0218, expiration date
October 31, 2004. Clearance of additional tracking activities was granted by OMB under clearance number 2528-0233,
expiration date November 30, 2007.
2

2
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

¾ An adult respondent survey, designed to gather data on outcomes and mediating
factors concerning the respondent and other members of the household;
¾ A youth survey, designed to gather information on outcomes and mediating factors
for youth who are ages 10 to 20 at the end of 2007 (just prior to the start of our
survey data collection period) and who were residing with MTO families at time of
enrollment;
¾ Educational achievement data through administration of portions of the math and
reading assessments of the U.S. Department of Education’s Early Childhood
Longitudinal Study of Kindergartners in 1998 (ECLS), with supplementation of items
from the reading and math assessments of the U.S. Department of Education’s
National Educational Longitudinal Survey of 8th graders in 1988 (NELS). Further
details regarding collection of achievement data are provided in section A.2.1 below.
¾ Biomarker data collection including dried blood spots, blood pressure, and height and
weight readings among adults, and height and weight readings among youth;
¾ Language assessments of adults and youth through audio-taping of open-ended
survey questions and a reading passage;
¾ Interviewer observational data on the characteristics of the respondent’s residence and
the immediate neighborhood through trained interviewer neighborhood “walkarounds.”

A.2 Purpose and Use of Information
HUD selected the National Bureau of Economic Research (NBER) to perform the final
impact evaluation (contract #C-CHI-00808). The data collected during the final evaluation
survey interview will be used by NBER and its team of researchers to measure and assess
MTO's impacts in seven primary domains:
1. Housing assistance and mobility
2. Neighborhoods and social outlook
3. Employment and earnings
4. Household income, public assistance and savings
5. Mental and physical health
6. Delinquency and risky behavior
7. Academic achievement and educational attainment
The hypothesis underlying the MTO evaluation is that relocation of families to low-poverty
neighborhoods will lead to improved well-being for adults and children in these seven
domains. Exhibit 1 summarizes a general model outlining the different pathways through
3
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

which relocation to low-poverty neighborhoods leads to improved outcomes for families,
based in part on the theoretical framework presented by Jencks and Mayer (1990).

At the most general level MTO might affect the outcomes and well-being of participants by
changing either the institutional / physical environment of the neighborhoods in which
families reside, or else by changing the social environment of these neighborhoods. These
two general characteristics of the neighborhood environment may in turn affect the specific
daily experiences of MTO families including their attitudes towards or information about
work, schooling or different health-risk behaviors, the frequency or nature of their social
interactions, their language patterns, basic features of their decision making, or the quality of
the household environments that MTO children experience through changes in parenting
This general distinction between the role of the neighborhood institutional / physical
environment versus the social environment is important because in principle policymakers
can directly modify the institutional or physical characteristics of neighborhoods without
moving families across areas. On the other hand, changing the social environments that lowincome families experience will require either a very detailed understanding of precisely how
4
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

local social interactions affect behavior in order to change the social functioning of
neighborhoods, or else policy interventions that change the composition of communities by
providing people with subsidies to move into different types of neighborhoods.
Institutional models of “neighborhood effects” on parents and children emphasize the role
played by local public and private institutions, including schools (Rivkin et al., 2005),
policing services (Sherman, 2003), access to high-quality health care services, or proximity
to suitable job opportunities or adequate public transportation options (Kain, 1968). The
physical environments of neighborhoods might matter for family behavior as well, since for
example exposure to environmental toxins may vary across areas and influence through
obvious pathways physical health and mental health as well (for example if exposure to noise
from highways or train tracks contributes to anxiety or depression).
The social environment of neighborhoods could be at least as important in understanding the
ways in which MTO might affect the behavior and well-being of participating families.
Epidemic models emphasize the power of peers to spread behaviors through learning, pure
preference externalities (individuals enjoy imitating their peers), stigma effects (the negative
signal from anti-social behaviors declines when more people do them), and physical
externalities (e.g., higher rates of crime reduce the chances of getting arrested) (see Cook
and Goss, 1996, Glaeser and Scheinkman, 1999, Manski, 2000, and Moffitt, 2001). Some
epidemic models predict peer influences on behavior that varies with the prevalence of the
behavior within a community, which can lead to non-linearities in peer effects or “tipping
points” (Jencks and Mayer, 1990, Crane, 1991). In this case the overall volume of antisocial behavior could be reduced by ensuring that no community has a prevalence of highrisk families above the relevant threshold. Collective socialization models concentrate on
the way adults in a neighborhood influence young people who are not their children, for
example by acting as role models (Wilson, 1987) or by enforcing shared values as in the
“collective efficacy” model of Sampson, Raudenbush and Earls (1997); see also Coleman
(1988). Neighbors can also be a source of information, such as about job opportunities, or
about the consequences of engaging in risky or illegal behavior.
In addition new research in behavioral economics implies that moving to a less distressed
area could improve youth outcomes by influencing future orientation, altruism and other
aspects of decision making (Loewenstein and O’Donoghue, 2004).
Ultimately, these changes in family- and person-level mediators lead to the outcomes
specified in the model: improvement in the family economic situation, improved health for
adults, youth and children in the family, improved social well-being for youth and children,
and improved educational achievement for youth and children. However the theoretical
social-science literature raises the possibility that MTO moves to less distressed
communities could also have adverse effects on adult or youth outcomes. For example
competition models emphasize the competition between neighbors for scarce resources like
grades or jobs, and relative deprivation models focus on the psychological impact on
individuals or self-evaluation based on relative standing in the community (Luttmer, 2005).
These suggest the importance of also measuring future aspirations, peer and social
networks, self-esteem and outlook, and discrimination.
5
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

It is important for the evaluation to collect information on these mediating factors as well as
on outcomes, in order to make MTO as informative as possible for policy design. We wish
to structure the impact analysis to shed light not only on the ultimate impacts of moving out
of public housing but also on the causal mechanisms through which those effects occur.
Therefore, in each domain we not only specify the outcomes of interest but also describe
alternative pathways through which impacts on those outcomes might occur and the
mediating factors along those pathways. Estimation of impacts on those mediating factors,
as well as on final outcomes, can help to distinguish the causal mechanisms responsible for
the estimated impacts.
A.2.1 Evaluation Overview

The MTO Demonstration

The Moving to Opportunity (MTO) demonstration was originally authorized in Section 152
of the Housing and Community Development Act of 1992. HUD is working closely with the
National Bureau of Economic Research (NBER) to fulfill the mandates of this legislation in
evaluating MTO’s long-term effects. The demonstration combines Section 8 rental
assistance with intensive housing search and counseling services that are intended to ease
families' relocation to low-poverty communities and help them become self-sufficient. The
legislation set the basic parameters of the demonstration as follows:
Family eligibility: Eligibility for the voluntary MTO program was limited to lowincome families with children living in public housing or Section 8 project-based housing
that were located in census tracts with poverty rates of 40% or more. Families were recruited
by Public Housing Authorities (PHAs) in each city through fliers, tenant associations and
other means. Everyone who was interested was given the chance to apply for the program.
Local PHAs in each of the demonstration’s five sites placed interested households on the
MTO waiting list and provided group orientation sessions where they learned about the
demonstration, its experimental nature, and the fact that they would be randomly assigned to
one of three groups. Households that remained interested after the briefing were asked to
both complete the extensive Baseline Survey and sign an Enrollment Agreement. Before
being formally accepted into the program, families were screened for Section 8 eligibility.
Almost all of the 4,608 households that signed up for MTO were headed by a female, nearly
two-thirds of whom were African-American and most of the rest were Hispanic. Threequarters of household heads were on welfare at baseline, fewer than half had graduated from
high school, and on average these households had three children. While we do not know the
immigration status of MTO participants, most speak English.
Site eligibility: The demonstration was restricted to no more than six very large
cities with populations of at least 400,000 in metropolitan areas of at least 1.5 million people.
Of the 21 cities eligible to participate in MTO, five cities were selected by a competitive
process for the demonstration. They are Baltimore, Boston, Chicago, Los Angeles, and
New York;
Demonstration operations: Local programs were created via grant agreements
between the Secretary of HUD and nonprofit organizations (NPOs) to provide counseling
6
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

and services in connection with the demonstration, and public housing agencies (PHAs) to
administer the rental assistance. The NPOs were funded to help pay for the costs associated
with counseling participating families, assisting them in finding appropriate units, and
working with landlords to encourage their participation in the MTO program. Local
programs had to match federal counseling funds with funds from state or local public or
private sources. PHAs received administrative funds for the increased number of Section 8
certificates or vouchers made available through the MTO program.
MTO research offers a unique platform to study the impact of government mobility
interventions like MTO and, in turn, the effects of large changes in neighborhood context, on
low-income families and children. Building on valuable lessons learned from the MTO
interim evaluation, the final evaluation design has several features that will contribute to
policy discussions as well as to research science. Most importantly, prior non-experimental
studies of other mobility programs have been unable to determine conclusively whether
observed outcomes were attributable to the impact of the program or simply reflected the
characteristics of the families who chose to enter the program. The MTO demonstration is
carefully designed to answer crucial questions about the impact of neighborhood on social
and economic opportunity for very low-income families.
The mechanism that HUD has chosen to address these questions is an experimental research
design involving the three-way random assignment of participants to:
¾ The MTO experimental group, which received certificates or vouchers usable only
in low-poverty areas, along with counseling and assistance in finding a private unit to
lease;
¾ The Section 8 comparison group, which received regular Section 8 certificates or
vouchers (geographically unrestricted) and ordinary briefings and assistance from the
PHA; or
¾ The in-place control group, which received no certificates or vouchers and could
continue to receive project-based assistance.
Interim findings from MTO.4 HUD previously sponsored the MTO “interim”
evaluation that was designed to measure MTO’s impacts on outcomes of participating
families measured 4 to 7 years after enrollment in the MTO demonstration. The key findings

4

See, Orr, L., J.D. Feins, R. Jacob, E. Beecroft, L. Sanbonmatsu, L. F. Katz, J.B. Liebman, and J.R. Kling
(2003). Moving to Opportunity Interim Impacts Evaluation. Washington, DC: U.S. Department of
Housing and Urban Development. Also, see Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz
(2007) “Experimental Analysis of Neighborhood Effects.” Econometrica. 75(1): 83-119; Kling, J. R., J.
Ludwig, and L. F. Katz (2005) “Neighborhood Effects on Crime for Female and Male Youth: Evidence
from a Randomized Housing Voucher Experiment.” Quarterly Journal of Economics. 120(1):87-130;
Ludwig, J. and J. R. Kling (2007) Journal of Law and Economics; Sanbonmatsu, L., J. R. Kling, G. J.
Duncan and J. Brooks-Gunn (2006) “Neighborhoods and Academic Achievement: Results from the
Moving to Opportunity Experiment.” Journal of Human Resources.

7
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

from the interim evaluation are summarized here as they set the stage for measures to be
collected in the long-term.
Among the households assigned to the Experimental group, 47 percent used a MTO voucher
to relocate to a low-poverty Census tract, while 62 percent of those assigned to the Section 8
group relocated through MTO. The explicit goal of MTO was to help move families into less
economically distressed communities, and by this measure MTO was successful. One year
after random assignment families in the two MTO treatment groups live in Census tracts with
average poverty rates 11-13 percentage points (25-30%) below those of the Control group.
The gap declines somewhat over time in part because of subsequent mobility among all
groups. But even 6 years after random assignment, the treatment-control differences in tract
poverty equal 7-8 percentage points (20% of the control mean), while the differences in
cumulative exposure to neighborhood poverty (duration weighted averages) are 9-10
percentage points (20-25% of the control mean).
The interim MTO evaluation found that assignment to either of the MTO mobility groups led
participating adults to feel safer and more satisfied with their housing and neighborhoods
(Orr et al., 2003, and Kling, Liebman and Katz, 2007). The MTO intervention had no
detectable effect on the labor market outcomes or social program participation of adults, but
did improve adults’ mental health as well as several important aspects of physical health such
as obesity and health-risk behaviors including diet and exercise.
The effects of MTO on youth outcomes in the interim evaluation were different for males
versus females, a difference that we aim to understand better with the long-term data
collection being requested for clearance here. For a wide range of measures of risky and
delinquent behaviors as well as school engagement, the interim MTO evaluation found that
MTO improved outcomes for female youth but on balance had deleterious impacts on male
youth. MTO had no detectable impacts for either boys or girls on academic achievement, as
measured by Woodcock-Johnson tests in reading and math, or school dropout (Sanbonmatsu,
Kling, Duncan, and Brooks-Gunn, 2006).
The MTO interim evaluation data has helped eliminate some hypothesized theoretical
mechanisms through which neighborhood environments or mobility programs more
generally might affect adults and youth. The disruption of moving per se does not appear to
explain the gender differences in MTO effects for youth, as suggested by the fact that MTO’s
deleterious impacts on male youth do not show up until a few years after random assignment
(Kling, Liebman and Katz, 2007; Kling, Ludwig and Katz, 2005). The gender difference in
impacts is also not due to families with boys versus girls moving to different types of
neighborhoods, since moves are generally similar across families with boys and girls.
Brothers and sisters within the same families also appear to respond differently to the MTO
intervention. One important implication of this last finding is that most existing theories of
“neighborhood effects” on behavior cannot explain the gender difference in MTO impacts,
since none of these theories in their standard form predict such sharp differences in how boys
versus girls will respond to neighborhood environments.
Qualitative interviews conducted after the time of the MTO interim evaluation suggest that
the nature of how boys and girls interact socially with peers may enable girls to more
8
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

successfully adapt to life in low-poverty areas. Girls were more likely to visit with friends on
their porches or inside their homes, in part because some parents may place girls on a
“shorter leash” than they do boys. Boys, on the other hand, often “hang out” in public
spaces, which puts them at elevated risk for conflict with neighbors and police, and increases
their exposure to delinquent peer groups as well as opportunities to engage in delinquent
activities themselves (Clampet-Lundquist, Edin, Kling, Duncan, 2005).5
The final impact evaluation provides an opportunity to examine the same set of outcomes
studied at the interim evaluation point in 2002, to learn more about how MTO impacts on
participating families change over time. There are plausible reasons to believe that the
effects of MTO moves on participants’ behaviors may increase over time as families become
more socially integrated into their new lower-poverty communities, and learn more about
how to take advantage of new schooling or work opportunities available to them in these
areas. The MTO final evaluation also provides a chance to collect additional, more detailed
information compared to the interim MTO evaluation on certain outcome domains that are
important for public policy and the interim MTO findings suggest could have been affected
by the intervention, such as mental health or crime victimization.
The final impact evaluation is designed to exploit the randomized experimental design of the
MTO demonstration to best address the key questions described below:
1. What are the long-term effects of MTO on participating families, and how do these
impacts evolve over time?
If differences in average neighborhood characteristics across MTO groups persist over
time, MTO’s effects on well-being and behavior may increase over time as ties to old
social networks diminish and families become more socially integrated into their new
communities and learn more about how to take advantage of new neighborhood
institutional resources. Social integration itself might require families to learn new
modes of dress, language or interactions to “fit in.”
2. What are the long-term effects of MTO on those who were young children at
baseline?
A growing body of research suggests the malleability of behavior may decline with age
(Becker and Murphy, 2000; Shonkoff and Phillips, 2000; Carniero and Heckman, 2003;
Knudsen et al., 2006). While MTO children who were very young at randomization were
too young to provide meaningful measures for outcomes like dropout and risky behavior
in the interim study, enough time will have elapsed for our long-term study to measure
5

This collaborative project between Jeffrey Kling, Greg Duncan and Kathy Edin started in 2003 to conduct
semi-structured open-ended interviews of 233 households in Baltimore and Chicago. One of the main purposes
of this effort was to generate new hypotheses about the mechanisms through which neighborhoods affect youth,
and particularly about drivers of differences in effects by gender. Interviews were conducted with the mothers
of youth, and with youth themselves; another component of the study included classroom observations and
teacher interviews for younger children. The households were a random sample from the two sites including
members from the experimental, Section 8, and control groups. Interviews were completed with 83% of
households. At the time of this proposal re-submission analysis of the transcripts from these interviews is still in
progress.

9
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

such impacts. This is important because these behaviors may be more susceptible to
social policy influences than are outcomes such as achievement test scores.6
3. Do neighborhood effects vary non-linearly with neighborhood characteristics over
the long run?
This hypothesis implies the existence of “tipping points” in neighborhood composition,
whereby a critical mass of compositional factors is needed to achieve positive
neighborhood effects on the well-being of neighborhood residents (Jencks and Mayer
1990). Evidence for tipping points would be important because it would mean that
policies to resort disadvantaged families across neighborhoods could be fine-tuned to
control the number of such families that are introduced into targeted neighborhoods of
specified composition in order to affect not only the distribution of outcomes across
neighborhoods, but also to minimize the prevalence of negative outcomes in the
population as a whole. Note that evidence on the existence of tipping points is also
directly relevant to determining the likely impacts of the MTO intervention on the
outcomes of other people who live in the neighborhoods into which MTO families are
moving, which for a variety of reasons cannot be estimated directly in this intervention.7
4. What are the mechanisms through which MTO affects long-term outcomes?
The long term evaluation is an opportunity to learn more about the mechanisms through
which MTO affects the behavior and life chances of participants. One particular focus
will be to measure the same mediating mechanisms hypothesized by the interim MTO
evaluation to affect behavior, to determine whether these theories are better able to
explain the behavioral patterns of MTO families over the long term as they become
more socially integrated into their new communities. We also seek to better understand
differences in MTO program impacts on male versus female youth, as this may inform
gender differences in the educational and economic progress made over time in the U.S.
as a whole among African-American males and females. Our key hypotheses for the
source of these gender differences in MTO impacts include the following:
H1: Males more than females retain social ties to old neighborhoods and peer groups.
H2: MTO reduces domestic violence victimization and sexual abuse, or changes sex
roles and relationships, which is more beneficial for female than male youth.
6

Outcomes like dropout rely mostly on the behavioral decisions of MTO youth. In contrast, cognitive
achievement test scores are the result of a more elaborate “production function” that includes other inputs such
as parent behavior, school quality and academic track assignments. Moreover, economists usually conceive of
test scores as a function of the child’s entire history of school and out-of-school inputs, so pre-randomization
exposure to less developmentally productive inputs may affect the degree to which differences in postrandomization inputs modify test scores.
7

One problem is that MTO families choose, rather than are randomly assigned, into their new neighborhoods,
which makes causal inference complicated – their neighborhoods could have different trends in outcomes
compared to other areas for reasons entirely unrelated to the in-migration of MTO families. In addition by
design MTO avoided clustering new families in particular areas, and so any MTO impact on destination areas
would be too small to be detected with available data sources. Finally, confidentiality concerns rule out the
collection of more targeted information on people who live very close to MTO families.

10
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

H3: Neighborhood institutions and schools in particular, are better able to deal with the
problems of disadvantaged female than male youth.
H4: Parental investments of time, money, and attention in youth in response to mobility
will favor females relative to males.8
H5: Effects of neighborhood mobility on basic decision making processes, if whatever
makes females better able than males to delay gratification more generally (for
example Silverman, 2003) also makes decision-making processes by females more
responsive to environmental influences.
H6: Females more than males adapt better to changes in their relative social position or
competencies and tend to sort themselves into more pro-social peer networks.
H7: Females more than males respond to positive role models that may be present in
their new neighborhoods and schools.
The Final Impact Evaluation

Background. While the number of people living in concentrated poverty9 neighborhoods
declined somewhat during the 1990s, it is still the case that in 2000 there were nearly 8
million people in America living in high-poverty urban neighborhoods, more than the
number living in such areas three decades earlier in 1970 (Jargowsky, 2003). A wide range
of government policies contribute to the concentration of poor people in high-poverty areas,
including decisions about whether to enable suburban municipalities to zone out low-cost
housing, the vigor with which government agencies enforce fair-housing or antidiscrimination laws, and the ways in which government agencies provide housing assistance
to low-income families. For instance some of the nation’s most notorious public housing
projects such as the Robert Taylor Homes or Cabrini-Green in Chicago contributed to
dramatic concentrations of poverty by constructing high-density housing for poor families,
which in turn became so dangerous few working- or middle-class families were willing to
live nearby.
It is well known that rates of employment, school dropout or criminal involvement vary
dramatically across neighborhoods in the U.S., which raises the possibility that neighborhood
environments – and the government policies that contribute to the concentration of poverty –
may affect the well-being of poor families. However determining whether and how
neighborhood context exert causal influences on the long-term life chances of poor families
remains difficult. A large non-experimental literature finds that neighborhood characteristics
do appear to predict adult and child development outcomes, even after conditioning on
8

For example data from the New Hope anti-poverty experiment revealed that mothers preferred to use
resources to help sons over daughters to in part compensate for the greater threats posed to boys from living in
disadvantaged neighborhoods (Romich 2000). In this case improvements in neighborhood conditions may
cause parents to reduce the resources that they allocate to boys. Other reallocations of resources are possible
depending on how neighborhood mobility affects the marginal productivity of parental investments in sons and
daughters, and how parents value equity versus efficiency in allocating resources across children; see Becker
(1991).
9
We follow convention in the social science literature and define “concentrated poverty neighborhoods” as
census tracts with poverty rates of 40 percent or more.

11
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

observable individual or family attributes (Sampson et al., 2002; Kawachi and Berkman,
2003; Leventhal and Brooks-Gunn, 2000; Ellen and Turner, 2003). However these studies
essentially compare behavior and life outcomes of low-income people who have chosen to
live, or been able to secure residence in, low-poverty neighborhoods with those of other lowincome families who are living in higher-poverty areas. Such comparisons potentially
confused the effects of neighborhood with the effects of the characteristics of families who
lived in those two types of residential areas. Put differently, these studies may confound the
effects of neighborhoods per se with the effects of unmeasured or difficult-to-observe
individual- or family-level attributes that are associated with residential selection and
location.
Motivated by the encouraging findings from the Gautreaux mobility program in Chicago
(Rubinowitz and Rosenbaum, 2001), HUD launched the Moving to Opportunity (MTO)
demonstration to support direct analysis of neighborhood impacts by employing an
experimental design (random assignment) to measure the effects of neighborhood without
these confounding factors. To fulfill Congress’ mandate in the original authorization of
MTO, HUD commissioned the interim MTO evaluation that 4-7 years after random
assignment measured outcomes across a wide range of domains as well as candidate
mediating mechanisms. This interim MTO evaluation serves as the starting point for the
long-term MTO evaluation proposed here, which will measure outcomes 10-12 years after
random assignment.
Under contract with HUD to carry out the long-term MTO evaluation, NBER has proposed
an evaluation plan that: focuses on the same key outcome domains as in the interim MTO
study, in order to learn more about how MTO impacts change over time; expands
measurement of policy-relevant outcome domains that were not intensively measured in the
interim study but on the basis of both social science theory and other interim MTO findings
we have reason to believe might be affected over the long term (such as mental health or
biomarkers indicating risk of long-term disease); and includes items designed to measure
new hypotheses about mediating mechanisms suggested by the interim MTO findings (such
as those relating to the gender difference among youth in MTO impacts). The sample frame
for the final impact analysis consists of one adult from each of the 4,604 households in the
MTO experiment and up to 6,311 youth ages 10-20 (up to three per MTO household) who
resided with MTO families at time of enrollment.
To carry out the survey data collection, NBER is partnering with the Survey Research Center
(SRC) of the Institute for Social Research at the University of Michigan, one of the nation’s
leading social science research firms with extensive experience carrying out high-quality
large-scale household surveys with high response rates.10 SRC fieldwork features rigorous
10

SRC consistently achieves high response rates, even for longitudinal studies that focus on low-income urban
populations similar to the MTO study group. For example for the African-American Health study SRC
achieved re-interview response rates of 90-95% for each follow-up survey from 2001 to 2004 for a sample of
nearly 1,000 African Americans drawn from two areas in St. Louis (inner city and nearby suburbs). For the
Women’s Employment Study SRC achieved a 92% follow-up response rate among a sample of 753 welfare
recipients in Flint, Michigan. For the Panel Study of Income Dynamics, an ongoing longitudinal study that has
collected information on more than 65,000 individuals spanning more than 36 years of their lives, SRC has

12
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

interviewer training, extensive field quality control monitoring, and careful data postprocessing, all of which will be implemented in the MTO long-term follow-up survey.11 The
impacts of both the MTO experimental treatment and regular Section 8 assistance will be
estimated for a wide range of outcomes in the domains specified above. Data for this
analysis will come from a combination of sources, including interviews with heads of
household and with youth ages 10-20, achievement tests administered to youth by the
University of Michigan’s Institute for Social Research, and extraction of data from
administrative records of earnings, welfare benefits, housing assistance, student tests, and
involvement with the criminal justice system.
Policy Context. While a majority of Americans agree that the government should provide
housing assistance to low-income families,12 what form should this assistance take? One
possibility is for the government to directly provide housing units to low-income families, a
strategy that began in earnest with the U.S. Housing Act of 1937 and led to a system of
public housing that peaked in size in the mid-1990s at around 1.4 million units (Olsen,
2003). An alternative approach is for the government to subsidize low-income households
to rent housing in the private market – a type of “tenant-based” subsidy that over time has
accounted for a growing share of all new federal commitments for low-income housing
(Quigley, 2000).13
In addition to their impacts on the quality of housing consumed by low-income families,
housing policy decisions may have important implications for their neighborhood
environments as well. By providing families with more choice over where they live,
recipients of tenant-based subsidies on average wind up living in lower-poverty tracts than do
families with project-based subsidies (Newman and Schnare, 1997; Khadduri et al., 1998;
Devine et al. 2003; Olsen, 2003).14 Other housing policy decisions that affect residential
outcomes include choices about where to locate project-based units, and how to fund and
operate tenant-based subsidy programs. Of particular concern to the public has been the subset of project-based housing in very high-poverty areas (Kotlowitz, 1992, Jones and
Newman, 1998). In fact the 1998 Housing Act requires local public housing authorities
(PHAs) to convert “distressed” public housing to vouchers, and makes it easier for PHAs to
convert other projects to vouchers as well. If neighborhood context exerts an independent
achieved response rates of 95% or higher for each wave. And in addition to carrying out the NCS, SRC carried
out the NCS follow-up, which re-interviewed original respondents a decade after the original interviews. A
response rate of 88% was obtained in that follow-up survey despite the fact that active household tracking was
not used to monitor the movement of NCS respondents over the decade between the two surveys.
11
SRC maintains an active national field staff of more than 600 interviewers, including a bilingual interviewing
staff. Interviewers for MTO would receive up to 6 days of training in Ann Arbor that would introduce the study
and cover refusal aversion / conversion procedures; confidentiality; consent; administration of the
questionnaires; and sample management and reporting time and progress.
12
A 2001 survey by NPR, Kaiser Family Foundation and Harvard found 75% of respondents support more
spending “for housing for poor people” by government.
www.npr.org/programs/specials/polls/poverty/staticresults3.html
13
There is a third possibility: government subsidies to private providers of specific housing units, although
from the perspective of MTO these impose locational constraints on recipients as do other project-based
subsidies.
14
In addition, a number of studies provide suggestive evidence that per-unit costs may be lower for tenantcompared to project-based subsidy programs (Olsen, 2000; Shroder and Reiger, 2000; GAO, 2001).

13
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

effect on family behavior, housing programs could have cascading influences on other
aspects of family’s lives such as earnings and health, as well as the development of their
children. This is important because the goal of federal housing policy since at least the
Housing Act of 1949 has been to improve family well being, broadly defined.
The MTO final evaluation will contribute to a variety of questions facing policy makers in
the arenas of poverty, housing assistance, and education. The basic experimental contrast
between project-based assistance and tenant-based vouchers addresses a fundamental policy
choice that first arose in the 1970s and has not been fully resolved in the intervening
decades. Over that period, there has been increasing concern that the high concentration of
poverty associated with public housing projects may adversely affect resident families.
Partly for that reason, a large part of the expansion of housing assistance since 1980 has
taken the form of certificates and vouchers that provide subsidies to obtain housing in the
private market.15 Absent compelling evidence of adverse effects, however, we have
continued to maintain the existing stock of project units. This evaluation will cast new light
on the desirability of replacing some of these units with rental assistance in the private
market.
Within this broad policy issue, there is a question as to whether it is sufficient to move
families out of projects into the surrounding community or whether it is necessary to change
their environment substantially. Left to their own devices, public housing tenants who
receive vouchers will tend to move to areas that still have relatively high rates of poverty. It
is not clear whether such moves are stable or sufficient to overcome any deleterious effects
associated with project-based assistance. Long term effects examined through the
experimental contrast between the effects of regular Section 8 vouchers, which place no
restriction on where the recipient moves, and those of the MTO experimental vouchers,
which require that the recipient move to a low-poverty area, speak to this issue.
The experiment is not, however, simply a test of two specific assistance programs. More
fundamentally, it seeks to measure the effects of neighborhood on the lives of low-income
families with children and, by extension, the potential benefits of policies designed to
disperse those families into low-poverty areas. What we learn about the effects of
neighborhood on the lives of low-income families may also speak to the desirability of
policies that seek to change the neighborhoods in which these families currently live. If the
truly comprehensive changes induced by MTO have little or no effect on outcomes, then the
more modest changes that can be made in their existing neighborhoods seem unlikely to have
the potential for meaningful effects. Alternatively, large estimates of neighborhood effects
may indicate that important changes in individual outcomes can be brought about by
community influences. Specific mechanisms may also be identified that will help target
issues that can be directly addressed in today's high-poverty communities, such as the
physical safety of areas in which children play or the availability of after-school or summer
programs to encourage constructive activities over risky behaviors.

15

Between 1980 and 1997, over 40 percent of the net growth in the number of assisted families resulted from increases in
household-based assistance in existing housing (U.S. House of Representatives (1998), Section 15, Table 15-26).

14
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Participant Data Collection for the Final Impact Evaluation. Clearance is being requested for:

¾ a survey instrument for female adult respondents
¾ a survey instrument for youth ages 10 to 20,16
¾ youth achievement testing using the Early Childhood Longitudinal Study math and
reading assessments supplemented with questions from the NELS
¾ adult biomarker data collection of height, weight, blood pressure and dried blood
spots, and collection of youth height and weight
¾ audio-taping to allow language assessments of adults and youth, and
¾ interviewer observations of the respondent’s residence and neighborhood.
Exhibit 2 summarizes these and other components of the MTO long-term evaluation data
collection effort.
Exhibit 2 Participant Data to be Collected for the MTO Long-Term Evaluation
Adult

Youth
Up to three youth per household ages
10 to 20 as of December 2007
Up to 6,311 youth total
Neighborhood & social network
Education and schooling
Employment and earnings
Risky behavior & behavior problems
Mental and physical health
Decision making
Youth reports on parenting

Direct measurement

One adult per household, female
caregiver
Up to 4,604 adults
Household roster
Housing and mobility
Neighborhoods, networks
Education
Employment & earnings
Income, public assistance
Savings and assets
Mental & physical health
Decision making
Relationships & parenting
Reports on household outcomes
Height, weight, blood pressure

Test assessments
Biomarkers

None
Dried Blood Spots

Survey sampling plan

Survey content

Height & weight
ECLS math and reading assessment

16

We exclude children under age 10 from the survey sample frame because most of these children will have
been born after randomization into the MTO study. The possibility that MTO treatment assignment may affect
fertility behavior means that the average background characteristics for children born after randomization may
not be balanced across MTO groups, which would introduce the possibility of bias to our estimates.

15
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Language assessments

Interviewer observations
Administrative data

Neighborhood indicators

Adult

Youth

Audio taping of open-ended
questions and reading passage
for pre-selected random
subsample
Neighborhood walk-around
Addresses, housing program
participation, earnings, social
program participation, arrests,
post-secondary school
enrollment, mortality
Census

Audio taping of open-ended question
and reading passage for pre-selected
random subsample

Elementary and secondary school
records, post-secondary school
enrollment, arrests, earnings, social
program participation, mortality
Census

17

The survey questionnaires are provided in Appendix A (Final Survey of Adult
Respondents) and Appendix B (Final Survey of Youth Respondents).
The Adult Respondent Survey. The final survey of adult respondents consists of a 75minute interview with one adult per core MTO household. This adult is in most cases the
female head of the MTO core family, as defined by the applicant during the Section 8
eligibility determination process. The respondents will be asked questions about his/her
mobility, housing and neighborhood conditions, employment status and history, educational
attainment, physical and mental health, and household composition.
In developing the final survey of households, we have drawn heavily on the MTO Interim
Evaluation instrument and other existing studies and instruments including the National
Health Interview Survey, the Panel Study of Income Dynamics, the Current Population
Survey and surveys from the Project on Human Development in Chicago Neighborhoods.
All of the items on mental health disorders, victimization and substance abuse have been
drawn from the National Comorbidity Surveys (NCS) under the close guidance of Ronald
Kessler, principal investigator of the NCS, Professor of Health Care Policy at the Harvard
Medical School, and co-investigator of NBER’s long-term evaluation of MTO. By
drawing whenever possible on questions from the interim MTO study and national sources
we will be able to compare impacts from the MTO interim to final evaluation, to use
measures that have been extensively pre- and field-tested and proven significant in other
research, and to have national data with which to compare the MTO results.
The adult survey also includes a series of questions on basic features of decision making.
These questions are additions to items collected at the interim MTO survey, motivated by
recent research in behavioral economics suggesting the possibility that MTO moves to lowercrime, lower-poverty neighborhoods could change future orientation, responses to risk, and
even altruism by reducing exposure to neighborhood stressors and temptations (Loewenstein
and O’Donoghue, 2004). In addition to these questions, a subsample of MTO respondents
17

Licensing agreements or request for permissions to use proprietary items are in progress for the Child
Behavior Checklist (from Achenback System of Empirically Based Assessments
http://www.aseba.org/products/cbcl6-18.html; Achenback and Edelbrock, 1981), the SF-36 questions on
chronic health conditions (from Quality Metric Incorporated in Lincoln, Rhode Island), and the self-reported
physical health thermometer (from the European Health Survey System:
http://ec.europa.eu/health/ph_information/dissemination/reporting/ehss_en.htm).

16
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

will participate in a simple decision making exercise. The details of this nested decisionmaking exercise and resulting respondent payment is described in Section A.9. Developed in
the experimental economics literature, this decision-making exercise is designed to elicit time
preferences under conditions where respondents face real incentives to choose according to
their underlying preferences. Our choice exercise is original to the MTO evaluation, but
designed to be a simplified version of those employed in this literature.18
The Youth Survey. The final survey of youth will be administered to sample children
between the ages of 10 and 20 as of December 2007. The youth survey will be 60 minutes in
length for youth aged 13 to 20, and approximately 30 minutes for youth aged 10 to 12. The
length of our proposed MTO youth surveys is comparable to several other large scale youth
surveys including the National Longitudinal Survey of Youth 1997, National Study of
Adolescent Health, The Project on Human Development in Chicago Neighborhoods, the
National Co-Morbidity Survey, and the fifth grade follow-up of the Early Head Start
evaluation. Our MTO youth surveys will cover education and schooling, employment
behavior, social networks, ties to the neighborhood, mental and physical health, decision
making, and risky behavior. As with the Adult Respondent survey, we have taken great
care to select questions from the MTO Interim Evaluation and existing surveys whenever
possible, ensuring that the questions are developmentally appropriate, have been
successfully administered to similar populations, and for which national distributions are
available.
Educational Achievement Testing for Youth. Sampled children ages 10 to 20 will be
asked to complete an educational math and reading achievement test, as described in
Appendix C.19 The achievement assessment will be 45 minutes in length (approximately 20
minutes allocated to an assessment of math and 25 minutes allocated to an assessment of
reading). Our primary measures of educational achievement will be derived from these
reading and math tests administered directly to youth from MTO families. We will not
conduct any testing of individual aptitude (such as IQ) as distinct from academic
achievement.
For all children age 10 to 20 we will administer the reading and math assessments designed
for the 5th and 8th grade follow-up waves of the U.S. Department of Education’s ECLS study.
We have chosen not to replicate administration of the Woodcock-Johnson-Revised (WJ-R)
tests that were used in the interim MTO study because the test exhibited several limitations
including evidence of important “interviewer effects” (test scores for MTO children varied
systematically by who administered the test) and little variation in average test scores by age
across MTO youth ages 13 to 18 (Sanbonmatsu et al. 2006). Donald Rock of the Education
18

See Holt and Laury (2002) who infer risk aversion coefficients from choices over lotteries. Harrison, Lau, and
Williams (2002) and Coller, Harrison, and Rutstrom (2003) infer individual discount rates from choices over
alternative streams of payments. Eckel, Johnson, and Montmarquette (2004) follow similar procedures in
offering risk and time choices to low- and middle-income subjects.

19

The ECLS-K direct child assessments were adapted from several copyrighted assessment batteries. Therefore,
individual items from the direct child assessments are not available for review and are not submitted as part of
this OMB request. A detailed description of the direct child assessment is available in the ECLS-K User's
Manuals and Psychometric Reports (http://nces.ed.gov/ecls/kinderinstruments.asp).

17
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Testing Service (ETS), a leading expert in the development of achievement assessments, is a
consultant on our team and has been assisting us with the selection of appropriate new
achievement assessments. Dr. Rock’s recommendation was to employ the ECLS
achievement tests for the long-term MTO youth data collection for the following reasons:
First, the ECLS tests are designed to more closely measure what children learn in school
compared to other tests such as the Woodcock-Johnson, which measure achievement or
aptitude more generally. Thus, the ECLS tests may provide a more sensitive measure of the
degree to which MTO moves children into improved schooling and learning environments.
Second, the ECLS assessments, up to 5th grade, have had extensive pre-testing and piloting.
This latter feature is important as it will decrease interviewer coding variation, which can
contribute to interviewer effects. Pollack, Najarian, Rock, Atkins-Burnett and Hausken
(2005) show that the tests adequately capture the content of school learning and have enough
range in difficulty of items so that students distribute relatively evenly across the three
second stage forms. The 5th grade test also has high internal consistency, which is measured
by the correlation of achievement test scores of same children who take the test at multiple
points in time. These reliability ratios range from 0.72 (on the lowest form) to 0.82 (on the
highest form) for reading and 0.58 to 0.78, respectively, for math. .
Third, the 5th and 8th grade ECLS tests together will be appropriate for the wide range of ages
covered by the youth survey sample for our long-term MTO evaluation (youth who will be
ages 10 to 20 at the time). Because the ECLS tests are intended to be used with a nationally
representative sample, there will be low risk of floor effects from administering the ECLS 5th
grade tests to MTO 10 year olds. The risks of ceiling effects from administering the ECLS
8th grade test to MTO children ages 13 to 20 should be low as well in part because given their
baseline socio-economic characteristics and previous research on the strong association
between socio-economic background and student achievement, we expect the low-income
MTO youth sample to be performing below grade level. In any case to insure against
potential ceiling effects, the ECLS 8th grade test will be supplemented with a small set of
math and reading items from the U.S. Department of Education’s National Educational
Longitudinal Survey 10th and 12th grade test. Donald Rock is assisting the NBER team with
choosing the items that will best meet the needs of the MTO long-term evaluation.20
Fourth, the ELCS assessments are adaptive -- respondents first take a short “routing test” that
then directs them to test forms of different difficultly levels – in order to reduce the time

20

There are two reasons why the ECLS 8th grade test will be supplemented with items from the NELS-88 10th
and 12th grade tests rather than more recent testing through the Education Longitudinal Study. First, ETS
constructed all the items for NELS as part of a contract with the National Center for Education Statistics so they
may be released by NCES without infringing on copyright issues. Copyright issues may be further exacerbated
because many of the items in ELS were also taken from other sources including the Program for International
Student Assessment (PISA). Second, several of the ELS reading items that might have been appropriate for
supplementing the MTO assessments are PISA reading items that tend to be based on long passages which
would present a problem with MTO time constraints.

18
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

required to accurately measure a subject's academic achievement level.21 Many other tests
are good at determining whether or not a student is at a given grade level, but they are not
good at distinguishing how far below or above grade level a student may be; the ECLS has
good discriminating power across a wide range of ability levels. We hypothesize that many
MTO children may be below grade level, and we want to have a test that can measure their
specific achievement levels with some precision. If for instance children in the experimental
group were performing at a level that was 2 years below their grade of school enrollment but
control children were 4 years below grade, this would represent an important benefit from the
MTO intervention – even though all children in both groups are still performing below grade
level – because previous research documents that improvements in academic skill directly
improves future life outcomes such as earnings (Murnane et al., 1995).
Fifth, the ECLS is administered to national samples, which provides a nationally
representative sample against which we can benchmark the MTO results.
The ECLS 5th and 8th grade tests are also among the few existing achievement assessments
that can allow us to measure youth achievement in both the reading and math domains within
the time we have available (45 minutes) to assess MTO youth. The ECLS 5th grade reading
and math achievement tests together on average take around 45 minutes, and are designed
with a “discontinue” rule so that children are not asked to continue when they are not able to
proceed through a progressively more difficult cluster of items. That is, the ECLS tests asks
questions that get increasingly difficult as the test progresses, and as soon as children answer
incorrectly to some specified number of questions in a row then they stop taking the test.
The ECLS 8th grade reading and math tests do not have the same “discontinue” rule, and so
preliminary data suggest that on average they have taken more than 45 minutes to administer
to ECLS study participants. However our team is currently working with Donald Rock to
develop a “discontinue” rule for ECLS 8th grade tests similar to those employed by the ECLS
5th grade test in order to keep the ECLS 8th grade assessments within 45 minutes for an
average administration time.
Direct Measurement of Blood Pressure, Height, Weight

In part because the MTO interim evaluation found some changes in diet and exercise among
some MTO adults and youth, as well as reductions in stress and improvements in mental
health, we will complement the survey data collection described above with direct
measurements of blood pressure, height and weight for adult respondents and of height and
weight for youth respondents. These measurements will be taken in the home, using
appropriate and up-to-date equipment, by interviewers well-trained in these procedures. We
estimate that these measurements will take approximately 10 minutes to complete.

21

The ECLS 5th and 8th grade tests are two stage self-administered paper and pencil tests. The first stage is a
short routing test so that the combination of age, grade and performance help determine the difficulty of the test
(as compared to age or grade alone). This routing test is scored on-site by an assessor and is then used to direct
children to the appropriate level for the second stage. The second stage 5th grade test is available in three levels
and the second stage 8th grade test is available in two levels, where items in the highest level of the 5th grade
test, for example, overlap with some items in the lowest level of the 8th grade test.

19
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Measurement of adult blood pressure is motivated in part by the fact that this is an
important health indicator. Elevated blood pressure (hypertension) is well known to be
associated with increased risk of cardiovascular disease, and is particularly prevalent among
African-Americans (two-thirds of the MTO program population is African-American). We
believe MTO might affect blood pressure at the time of our long-term evaluation data
collection given interim MTO evaluation findings of program effects on stress, weight, and
activity patterns such as exercise and diet. Interviewers will use automated
sphygmomanometers approved by the American Association for the Advancement of
Medical Instrumentation Standard, accepted by the FDA as the national standard.
Measurement of height and weight is also motivated by findings of changes in obesity,
diet and exercise for adults and youth in the interim MTO evaluation, together with public
health concerns associated with the social costs of escalating rates of obesity. Obesity even
in young children has been found to be predictive of later health problems. Moves to
lower-poverty neighborhoods may reduce obesity through several mechanisms including
lower incidence of depression and stress, behavioral changes such as increased physical
activity, or different social norms about eating habits. Interviewers will bring portable
equipment into the home to carry out these measurements.
Direct Measurement of Blood The interim MTO results suggest that around 4-7 years after

random assignment the intervention reduced stress and obesity, and improved mental health,
diet, and exercise in ways that may lead to long-term improvements in important health
outcomes, including reductions in long-term disease conditions that impose great costs on
society. Yet many of these key health gains generated by MTO may not yet be in evidence
in the form of noticeable health symptoms, even 10-12 years after random assignment. For
example if MTO improves diet and exercise, as the interim MTO evaluation suggests, the
result may be reductions in the lifetime risk of cardiovascular disease. Because the average
MTO adult will be about 45 years of age at the time of the long-term survey interview, it may
not be possible to detect the prevalence of actual symptoms or adverse health experiences
associated with cardiovascular health, such as heart attacks, through survey self-reports.
As a result direct measurement of blood samples from MTO participants will be important
for assessing the interventions effects on the precursors to important long-term health
outcomes such as diabetes, cardiovascular disease (CVD), or metabolic disease that may
result from MTO-induced changes in health behaviors (diet, exercise), obesity, or stress. We
will use dried blood spots (DBS) to measure a number of “biomarkers” that have been
demonstrated to be highly predictive of these long-term health problems, specifically we will
seek to measure total cholesterol (TC), high density lipoprotein (HDP), glycosylated
hemoglobin (HbA1c), C-reactive protein (CRP), cortisol, and Epstein-Barr virus (EBV)
antibody levels. These biomarkers are affected by health-risk factors that have been shown
to be affected by MTO in earlier evaluation rounds, including environmental toxicity, dietary
quality, levels of physical activity, and psychosocial stress. In addition each of these
biomarkers affect long-term health outcomes though well-known physiological pathways,
and so the biomarkers representing these pathways provide reliable and valid measures of
exposure to important health risks that are causally proximate to important health outcomes.
20
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Because the long-term social and public costs of poor health has emerged as an important
policy issue, collection of biomarker data of this type is becoming increasingly common in
large national studies including for example, the National Study of Adolescent Health, the
Health and Retirement Study, and several studies sponsored by the National Institute of
Health. An additional benefit to MTO participants is that some of the results from blood
measurement will be shared with them – such as total cholesterol levels.22 Since MTO
respondents will receive important health information as part of this biomarker measurement,
the costs and benefits to participating in biomarker collection from the perspective of the
MTO respondents themselves are similar to those of a regular visit to a primary care
physician for a routine check-up.
Concentrations of TC and HDL are important clinical predictors of cardiovascular disease
(CVD) risk. Higher total cholesterol—typically due to poor diet and/or low level of physical
activity—is associated with greater risk for CVD. The interim MTO study’s findings of
experimental group impacts on diet and exercise would have important implications for longterm health outcomes if they translated into lower levels of TC, given that a 10% decrease in
total cholesterol reduces the risk of heart disease by as much at 30%. HDL (“good”
cholesterol) is protective, with higher concentrations independently associated with reduced
risk of CVD. A 10% increase in HDL is associated with a 20-30% decrease in risk of CVD.
The assessment of glycosylated hemoglobin (HbA1c) provides an integrated measure of
average blood glucose levels over the past two to four weeks and is relevant for
understanding health problems associated with diabetes, a disease of glucose metabolism (we
refer here to type II, non-insulin diabetes, which comprises approximately 95% of all cases).
Higher blood glucose levels increase the proportion of HbA1c, and several clinical trials have
demonstrated that HbA1c is the single best predictor of diabetes complications. HbA1c is
also highly predictive of diabetes onset: In non-diabetic individuals with HbA1c of <5.6% at
baseline, the annual incidence of diabetes is 0.8%. Diabetes incidence is 2.5% for
individuals with HbA1c between 5.6 and 6.0%, and 7.8% for HbA1c greater than 6.0%
(Edelman et al. 2004). The value of this measure for understanding MTO impacts on the
long-term risk of diabetes is highlighted by previous research finding that HbA1c is a better
predictor than is obesity of the onset of diabetes. Diabetes disproportionately affects
minority populations, with the risk of disease for African-American men and women nearly
twice that of European-Americans. Diet, physical activity, and obesity – all factors found to
have been affected by MTO in the interim study – are major determinants of diabetes risk.
Complications from diabetes include blindness, nerve damage, kidney failure, amputation,
and increased CVD risk.
CRP is a biomarker of inflammation that is an important predictor of CVD and metabolic
disease and is now being incorporated into clinical practice. Smoking, overweight, obesity,
and stress – each of which may be affected by MTO -- are all associated with increases in
CRP. Concentrations of CRP <1.0 mg/L are considered low risk, while concentrations >3.0
mg/L are considered elevated risk. Small increases in CRP are predictive of disease:
individuals in the 2nd quartile of the CRP distribution are 2.7 times more likely to experience
22

Results on total cholesterol and HDL will be shared at the time of the interview by using instant measurement
devices. Other results will be mailed back to the participant.

21
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

stroke or myocardial infarction (i.e., heart attack), and 2.8 times more likely to develop
metabolic disease, than those in the lowest quartile (Ridker et al. 2003). EBV antibody
levels, an indirect measure of cell-mediated immune function, has been shown to be among
the strongest and most consistent immunological correlates of recent chronic stress.
Conversely, stress management interventions and disclosure of previously repressed trauma
have been associated with declines in EBV antibody levels. Concentrations of EBV can be
assayed in dried blood spots.
We also plan to measure levels of the stress hormone cortisol for the long-term MTO
evaluation given that concerns about crime was far and away the most important reason
families signed up for the MTO program. The interim MTO evaluation did indeed find that
the intervention reduced rates of crime victimization, which presumably is one important
contributing factor to program impacts in the interim evaluation on adult self-reports of
feeling stressed. Physiological stress systems in the body are designed to translate
psychosocial and emotional stress into “physiological action,” such as flight or fight
reactions. In the long term, however, too frequent or chronic physiological stress activation
negatively affect the functioning of stress systems and a wide variety of related physiological
processes relevant to mental and physical health (Chrousos and Gold, 1992). Cortisol levels
provide an important indicator of physiological stress system functioning, and predict the
development of disease processes such as depression and obesity as well as other long-term
health outcomes (Goodyer, Herbert and Tamplin, 2003; Rosmond, 2005; Chrousos and Gold,
1992).
TC, HDL, HbA1c, CRP, cortisol, and EBV can all be quantified in dried blood spot (DBS)
samples. We are optimistic about our chances of obtaining a high compliance rate for MTO
given the experience of prior research with compliance rates ranging from 75 to 98%, the
experience of our senior research team that obtained DBS compliance rates as high as 99%,
and given our survey subcontractor’s (ISR) experience with biomarker collection.23 We will
also share with MTO respondents the results of their biomarker analyses, especially TC and
HDL, which should further enhance compliance rates.
To collect a blood sample using the DBS method, the participant’s finger is cleaned with an
alcohol preparation, and then pricked with a sterile, disposable commercial lancet of the type
commonly used in home pregnancy or HIV testing or by diabetics to monitor blood glucose.
Up to five drops of whole blood are applied to standardized filter paper, where the sample
dries and becomes stabilized. An additional drop of blood from the same finger stick can be
placed into a portable “point-of-care” device for instant measurement, which can be used to
report instant measures of selected health outcomes to MTO participants and thereby increase
the benefits to MTO study members themselves. Dried blood spot samples are stacked and
stored in re-sealable plastic bags, and then shipped to the laboratory for storage and analysis.
23

The Great Smoky Mountains Study samples around 1,000 youth ages 9-15 and obtained a DBS compliance
rate of 75.2-80.7%. The Chicago Health, Aging and Social Relations Study obtained a DBS compliance rate of
97.9% for 200 respondents 50-67 years of age, while compliance rates for DBS equaled 80.1% for the National
Social Life, Health and Aging Study (2,000 respondents 57-84 years of age, for which Thomas McDade, senior
consultant on this project, was a co-investigator), and the Illinois Family Study pilot achieved a 99.5%
compliance rate among 205 respondents 20-40 years of age (with which McDade and Duncan are involved).
ISR has collected DBS and saliva samples for a number of studies, including the Health and Retirement Study.

22
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Most biomarkers are stable at normal room temperature for at least two weeks. Interviewers
are protected by wearing gloves during the procedure. Finger prick blood sampling provides
a logistically feasible, minimally-invasive means for collecting physiological information in
community-based settings.
Detailed protocols for collection of these biological samples are described further in
Appendix D.
Audio-taping for language assessments. The interim MTO survey asked a variety of
questions about discrimination by race, in order to learn more about whether or how racial
discrimination influences the ability of MTO participants to take advantage of new job,
schooling or other opportunities in their new communities. However address tracking data
revealed that MTO moves to lower-poverty areas wound up generating surprisingly little
change in neighborhood racial composition. As a result for the long-term MTO evaluation
we are interested in learning more about MTO participants’ experiences with discrimination
along social class lines, as indicated for instance by styles of dress or manner or language.
As part of our effort to learn more about social class discrimination in MTO, we plan to
analyze MTO’s long-term impacts on the language patterns of participants by taking
advantage of the fact that our survey subcontractor, ISR, is already planning to digitally
audiotape the MTO long-term survey interviews for their own quality control purposes.
Specifically, in order to assess MTO’s effects on language we will ask a random sub-sample
of MTO adults and youths to verbally respond to two language measurement tasks at the end
of their survey interviews, which will include either open-ended questions or a passage to
read out loud. These will be recorded via audio tapes, transcribed and then analyzed for
specific characteristics with respect to vocabulary, grammatical structure or pronunciation.
Appendix E details data collection for language assessments and the power to detect effects
on proposed language outcomes.
There is good reason to believe that MTO moves to lower-poverty areas may affect language
patterns among participating adults and youth given that language is socially constructed and
serves as a salient indicator of the speaker’s socio-economic background (Labov et al 1968;
Wolfram 1969). Previous research has found substantial variation in language patterns such
as grammatical structure, phonological features and pronunciation across neighborhoods in
New York, Philadelphia and other areas, particularly among African-American and Latino
samples. MTO participants may have experienced strong social pressure in their baseline
neighborhoods, related in part to issues about social identity in high-poverty areas.24 MTO
families who moved to lower-poverty areas may have experienced additional pressure on
their language patterns because they may have moved to areas where mainstream English is
more common. This type of pressure on MTO movers could lead to what Labov (1972)
terms “linguistic insecurity” or even “linguistic self-hatred.” MTO moves might even lead to
“hypercorrection” (for example: “Whom did you say was calling?”), or to the phenomenon of
24

In 1873 Gabriel Tarde argued: “It appears to me almost beyond dispute that language is a phenomenon of
imitation … the acquisition of foreign words by fashion and their assimilation by custom, the contagion of
accent, the tyranny of usage in itself, suffices to show at one glance its imitative character” (cited in Labov
2001, p. 23).

23
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

“code switching” when individuals alter their language patterns to respond to specific social
contexts.25
Previous research suggests differences in language patterns across people serves as the basis
for discrimination in both housing and labor markets. For example in a telephone audit study
of landlords in Philadelphia in 1999, the likelihood of speaking with an agent and being told
a rental property was available was affected by the language patterns of the caller. White
males speaking White Middle-Class English (WME) were more likely to speak with an agent
and be told about a property than were people speaking Black Accented English (BAE,
defined as “standard English with a black pronunciation of certain words”) and especially
those speaking Black English Vernacular (BEV, defined as “the combination of nonstandard
grammar with a black accent, [which] signals lower-class origins” (Massey and Lundy, 2001;
Fischer and Massey, 2004).
MTO impacts on language may also help us better understand the gender differences in MTO
impacts on youth that were documented in the interim MTO evaluation. Prior sociolinguistic research has documented gender differences in language patterns, and in particular
that females are generally more likely than males to use “mainstream” language. This could
help explain why female youth in MTO seem to adapt better to their new, lower-poverty
neighborhoods compared to males, consistent with findings from the interim MTO
qualitative research that “Non-dominant cultural capital skills (e.g. use of language) that male
youth learned in their high-poverty neighborhoods may have isolated them through police
harassment or through fear and misunderstanding and may have led to maladaptive behavior
from the mainstream when they moved to lower-poverty contexts” More generally those
MTO participants who were very young at the time of random assignment, and who serve as
the key youth survey sample for the long-term evaluation discussed here, may be particularly
susceptible to changes in cognitive outcomes including language skills. Language and
speech patterns have been shown in previous research to be highly predictive of subsequent
achievement in school.
Interviewer observations of residence and neighborhood. During the interim MTO study,
interviewers documented observations of the interior of the respondent’s housing unit (noise
level, carpeting, peeling paint, etc.) and exterior of the respondent’s home (type of building,
physical condition, bars on windows, etc.) These interviewer observations of the residence
will be repeated for the long-term survey. In addition to the observations of the respondent’s
residence, SRC interviewers will conduct short “walk-arounds” in the MTO respondent’s
neighborhood before conducting the long-term survey interviews. The interviewers will be
asked to complete checklists for various indications of neighborhood physical and social
conditions. The checklist is a subset of items that were developed for walk-arounds by
interviewers as part of the Project on Human Development in Chicago Neighborhoods
(PHDCN). The protocols developed for PHDCN were also conducted by SRC.

25

An adult respondent in one study described his own experiences with adapting his language to new social
settings: “Ultimately I somehow learned to be polylingual and to become sensitive linguistically in the way that
animals are able to sense the danger of bad weather” (Lippi-Green, 1997, p. 192).

24
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

For the PHDCN, interviewers walked around the 4 streets surrounding the PHDCN
respondent’s housing unit before the survey interview and completed a check-list of 102
survey questions about the social and physical characteristics of the neighborhood. These
questions include for instance whether people on the street reacted to the interviewer in a
friendly way or with suspicion, whether there are signs of gang graffiti or litter in the streets,
and the presence or absence of different types of institutions such as liquor stores or
commercial banks. On average these PHDCN walk-arounds took 78 minutes to complete,
and the results of the PHDCN walk-around neighborhood observations wind up being highly
predict of how neighborhood residents themselves report important features of the
neighborhood social environment such as “collective efficacy” that has been demonstrated to
be important in explaining variation in neighborhood crime rates and other outcomes
(Sampson et al., 1997). Communities with high collective efficacy, defined as mutual trust
among residents and a willingness to intervene for the collective good (Sampson et al., 1997,
Sampson, 2003, Browning and Cagney, 2003), may be better able to monitor and deter antisocial behavior by residents (Sampson et al., 1997), and address neighborhood environmental
risk factors such as abandoned buildings and graffiti (Browning and Cagney, 2003). Put
differently, these neighborhood walk-arounds have the potential to provide key information
about how other neighborhood residents who are not MTO program participants experience
the neighborhood, which will serve as an important complement to the MTO participant selfreports about how they experience the neighborhood. Differences in how MTO families and
other families experience these areas would be informative about, among other things, the
degree of social integration of MTO participants into their communities.
Because of budget and time constraints we will not be able to have SRC interviewers devote
a full 78 minutes to conducting walk-around check-lists of community social and physical
characteristics. As a result we have conducted our own original re-analysis of the PHDCN
walk-around data and find that having interviewers report on just a sub-set of the full
PHDCN check-list (around 80 of the 102 items) for just a single street (rather than 4 streets)
around the respondent’s address is nearly as informative as the full-scale walk-around data
collection conducted by the PHDCN (Bader, 2007). This scaled back version of the walkaround can be conducted in 13-15 minutes.
Participant Data Collection Procedures. NBER’s contractor for survey data collection, the
University of Michigan’s Institute for Social Research, has designed data collection
procedures to coordinate the various parts of this effort. The Adult respondent survey will
be administered either in person by trained interviewers, using the Blaise Computer-Assisted
Personal Interviewing (CAPI) system on a laptop computer, or over the phone also using
computer-assisted interviewing technology. The survey will be administered in the
respondent's home or over the phone to maximize survey response rates for a fixed budget,
with the interview session scheduled at the respondent's convenience. As described in
Section A.11 below, the in-person interview technology will permit the respondent to listen
to the questions using a headset and self-administer sensitive questions.
The Youth survey will be conducted in conjunction with the administration of achievement
tests. Youth will be interviewed and tested in their homes, coordinated with the household
adult survey whenever possible. Interviewing and testing in households where both adults
25
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

and youth are in our sample frame will be conducted by sending a team of two trained
interviewer/testers to the home together. The purpose of coordinating the youth and adult
data collection is three-fold: to reduce the degree of intrusion and time burden; to ensure that
the parent is home at the time of the youth data collection (for reassurance); and to occupy
parent and youth separately so that their interview and testing sessions can be separately
completed (so that the parent does not influence the youth’s answers or performance).
A.2.2 Purpose of the Data Collection

As discussed above, prior studies of mobility programs have been unable to demonstrate
whether observed outcomes were the result of program impacts, or instead due to the
characteristics of the families who chose to enroll in the program. The MTO study has been
carefully designed to allow comparison of well-matched groups of families in three different
locations: public housing in high-poverty areas; private housing in moderate-to-high poverty
areas; and private housing in low-poverty areas. The purpose of the final MTO evaluation
data collection is to support and extend the interim research on MTO families. While the
interim MTO evaluation measured program impacts 4-7 years after random assignment, with
the final evaluation we seek to determine the impacts of moving out of public housing in
high-poverty areas over the long term, which is ultimately of the greatest policy importance.
The proposed data collection activities will provide reliable measures of a broad range of
outcomes; impacts on these outcomes will be estimated for both the MTO experimental
group, who moved to low-poverty areas, and the Section 8 comparison group, who were free
to move to any area (but who primarily moved to moderate-to-high poverty areas). In both
cases, the impact of moving will be measured relative to the outcomes of the in-place control
group, who initially remained in public housing.
A.2.3 Who Will Use the Information

The primary beneficiary of the final evaluation data collection will be HUD, which will
use the information to assess longer-term effects of MTO for families who have been in
the demonstration between ten and twelve years. These data will begin to answer HUD's
questions about impacts in the domains of housing, employment and earnings, cash
assistance, educational achievement, health, and delinquency and risky behavior, for the
families assisted under the demonstration program. Evaluation contractor NBER will
produce a Final Report for the final MTO evaluation in October 2009.
Secondary beneficiaries of this data collection will be those in the social science and policy
research community who have expressed interest in the MTO demonstration and in working
with the MTO data. The survey data collected for the interim MTO evaluation have been
made available to several research teams for secondary analysis through HUD subject to
standard confidentiality provisions. Specifically these data have been analyzed by
researchers at Harvard, MIT, Princeton, the University of Pennsylvania, Northwestern
University, the Urban Institute, the City University of New York, the University of Illinois,
the University of South Carolina, Indiana University, and Rutgers University. The NBER
team is also currently working with HUD to discuss ways of creating a public use file (PUF)
version of the interim MTO survey data that would be constructed in a way to make
identification of individual MTO respondents impossible, which could then be disseminated
26
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

more broadly by reducing the time burden on HUD to review and monitor applications for
access to the MTO data.
Findings from previous waves of MTO research have also been presented at the annual
research meetings of almost all of the major social science disciplinary organizations
(economics, sociology, public policy, demography, epidemiology, and child development),
as well as to policy audiences across the country at conferences and presentations sponsored
by the Brookings Institution, the Administration on Children and Families at the U.S.
Department of Health and Human Services, the National Academy of Sciences, the House of
Representatives Ways and Means Committee, and the Joint Center for Poverty Research
(among others). Additionally, previous MTO findings have been discussed in news stories in
major media outlets such as the New York Times, Washington Post, Wall Street Journal, and
television programs such as NBC Dateline.
Ultimately, these data will benefit researchers and policy analysts in a wide range of areas.
The effect of neighborhood context on the well-being of low-income families is likely to
manifest itself in numerous ways, and may be relevant to a broad array of public programs.
This project offers the first opportunity to obtain reliable measures of these effects. The
long-term indirect benefits of this research are therefore likely to be substantial.
A.2.4 Instruments, Item by Item Justification

In this section, we present our justification of these instruments and their contents. Two
survey instruments have been developed for the final evaluation, because of the interest in
measuring impacts on adults and youth. The household (adult) instrument has several
sections concerning the respondent (head) and the household, including housing consumption
and mobility, neighborhoods, education, employment, income and assistance, savings and
assets, decision making, mental and physical health modules, parenting and decision making.
It also contains a Parent-on-Youth module (to be administered if the parent has a child ages
10-20 who is in our MTO long-term youth survey sample frame), and a module designed to
obtain updated information about core MTO household members. Finally, the household
instrument contains a member roster and an update on secondary contact information.
The youth instrument contains sections on education, employment and earnings,
delinquency and risky behavior, mental and physical health, neighborhood, decision
making, and family interactions. The youth instrument has been designed to focus on the
topics best reported by the children and youth themselves. The youth survey for the
youngest of this group, youth aged 10 to 12 will exclude modules on risky behavior,
conduct disorder, and several of the modules on other mental health disorders. The
contents of the Parent-on-Youth module have been coordinated with the youth instrument
so that they focus on topics better reported by the parent, on topics where a uniform report
is needed across all ages of sampled children, or on a topic where more can be learned by
obtaining parent as well as youth reports. For example, parent reports of monitoring (such
as reporting on whether she knows her child’s close friends) can often be confounded by
youth willingness to communicate or disclose information with the parent. Obtaining
youth reports of the same measure (does your mother know who your close friends are?)
and comparing parent with youth reports can help reveal the extent to which youth
27
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

disclosure is affecting the measure of parental monitoring and also itself provides
information about the nature of the parent-youth relationship.
Because of their length, the item-by-item justifications for these instruments have been
provided in appendixes rather than within this section’s text. Appendix F provides item-byitem justifications of the questions in the Adult Respondent Survey. It shows not only the
content and reason for inclusion but also the source of the survey question. Appendix G
provides the same item-by-item information for the Youth Survey.
The largest substantive revision to the long-term MTO adult and youth survey is the addition
of several modules designed to measure clinically significant mental health disorders, i.e.
those that meet the criteria for clinical significance as outlined by the APA’s Diagnostic
Standards Manual, version IV (DSM-IV). These survey modules were developed and
validated from the National Co-Morbidity Replication Study, directed by Ronald Kessler,
Professor of Health Care Policy at Harvard Medical School and one of the Co-Investigators
of the MTO long-term evaluation NBER team.26 The interim MTO evaluation found some
impact on an indicator of general psychological distress. However the personal and social
costs associated with mental health problems depend crucially on both the specific type and
severity of the mental health disorder that people experience. So for public policy purposes
there is great value in obtaining more detail in the long-term MTO evaluation about the
degree to which MTO participants meet the clinical criteria for different specific mental
health disorders.
The adult respondent survey includes modules designed to measure a number of clinical
mental health disorders including major depressive disorder, general anxiety disorder, panic
disorder, mania, post-traumatic stress disorder, intermittent explosive disorder, bipolar I
disorder and bipolar II disorder. For youth aged 13 to 20, the survey includes modules
designed to measure the same mental health disorders as for adults as well as conduct
disorder, attention deficit disorder and oppositional defiant disorder. Several of these
disorders are important in their own right as described in the justification column of these
tables. Measuring each specific disorder is also important for deriving an overall assessment
of experiencing “any disorder.” This cluster of modules includes one module that serves as a
screener for the disorders. The items in the screening section determine the extent to which a
respondent appropriately skips out of the remaining modules. The NCS mental health items
26

These assessments will be identical to those used to assess the same disorders in the recently completed
National Comorbidity Survey Replication (NCS-R; Kessler & Merikangas, 2004), thus providing a nationally
representative benchmark to the MTO results. The diagnostic instrument used will be the version of the World
Health Organization’s (WHO) Composite International Diagnostic Interview (CIDI) that was expanded and
updated for the WHO World Mental Health Survey Initiative (Demyttenaere et al., 2004). This instrument,
which revised the original CIDI to make diagnoses according to the definitions and criteria of the DSM-IV
(American Psychiatric Association, 1994), was recently approved by WHO as the official version of CIDI to be
used throughout the world until the publication of ICD-11 in the year 2011. Kessler and Ustun (2004) present a
detailed discussion of the CIDI, and Kessler et al. (2005) discuss analyses of onset, prevalence and comorbidity
of DSM-IV disorders in the National Comorbidity Survey Replication. It is important to recognize that the CIDI
is a fully structured diagnostic interview. This means that it is designed for use by trained lay interviewers
rather than by clinicians and that clinical judgments are not required in scoring. However, methodological
research has documented good concordance between diagnoses made by the CIDI and independent diagnoses
made by clinical interviewers.

28
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

add a great number of questions to the MTO survey, but, importantly, because many
respondents will skip the mental health modules altogether on the basis of negative answers
to a short set of NCS screener questions, or else will answer just enough questions in the
NCS mental health modules themselves to determine that they do not have the disorder,
average time of completion for these new mental health modules is substantially less than
what would be expected if every respondent answered each item.
NBER has devoted great care to accurately estimate the amount of time that the mental
health modules will require of MTO respondents. From previous waves of the NCS study
itself, we know the exact fraction of respondents in the nationally representative NCS sample
who answer each of the mental health questions that we ask (versus skip over the question on
the basis of previous survey responses). Using data from the interim MTO study, we
estimate how the expected prevalence rates for mental health disorders for MTO adults and
youth should compare to national samples. We then use these estimates to calculate the
average number of NCS mental health questions that MTO respondents would complete in
our mental health modules given the expected mental health disorder rates for our MTO
sample.
A.3 Use of Improved Information Technology

Improved information technology will be used in this evaluation in three distinct ways:
¾ to assist the sample tracking and locating efforts;
¾ to measure certain outcomes through data abstracted from administrative records; and
¾ to facilitate collection of the survey data in standardized and accurate ways that also
accommodate the confidential collection of sensitive data.
The administrative data collection will significantly reduce the burden on respondents to the
household and youth surveys, as will the linkage of final evaluation data with data collected
at interim and baseline for MTO families.
A.3.1 Information Technology and Sample Tracking

Tracking the MTO population prior to survey data collection will use several electronic
databases as part of the passive locating effort, in order to minimize respondent burden.
The searches of electronic data files include:
¾ Periodic comparisons of administrative databases; and
¾ Quarterly searches of electronic data maintained by outside vendors.
These methods do not involve direct contact with the MTO families; they are not intrusive
and are effective ways to maintain current information on the MTO families. Each strategy
is described briefly below.
Routine Checks of Administrative Databases. The Institute for Social Research (ISR)
and the NBER research team will collect periodic extracts of tenant characteristics and
29
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

certification data (HUD Form 50058 data) for MTO families, from the Multifamily
Tenant Characteristics System (MTCS) at HUD.
Searches of Other Electronic Databases. Passive tracking for the MTO sample also
involves use of electronic databases. ISR and NBER will routinely check the National
Change of Address Database (which catalogs U.S. Postal Service change-of-address notices).
ISR and NBER will also check national consumer credit databases which list address
information provided by creditors based on credit applications and ongoing account
maintenance to obtain updated address information prior to and during the survey data
collection period.
A.3.2 Information Technology and Administrative Data Collection for the Evaluation

The second way in which improved information technology will benefit the MTO final
evaluation is through the collection of administrative data on selected outcomes and
mediating factors. By accessing administrative information at the state, national, and local
level, the evaluation contractor has been able to reduce the scope and burden of the survey
instruments. Exhibit 3 shows the sources for collecting administrative data on employment
and earnings, delinquency and risky behavior, social programs, health and housing
assistance. The administrative data collection strategy seeks to provide a representative
picture of the outcomes of everyone in all three of the MTO groups without collecting
administrative data from the hundreds of jurisdictions in which MTO families have lived.
The sampling strategy has been designed to balance the objectives of minimizing the
number of data-sharing agreements that need to be established with local or state
government agencies while at the same time minimizing bias and sampling variance for the
final impact estimates. After determining the combinations of jurisdictions in which MTO
participants have lived, these combinations will be stratified by the per participant costs of
obtaining data. Strata with lower per person costs of obtaining data (i.e., jurisdictions with
relatively more MTO participants) will be sampled at higher rates. For analysis, the data
will be weighted by the inverse probability of selecting the jurisdictions.
Exhibit 3 Administrative Data Sources for Outcomes27
Domain

Outcomes

Data Sources

Employment and
Earnings

Quarterly employment and
earnings

State Unemployment Insurance (UI)
wage records
Census Bureau's Longitudinal
Employer-Household Dynamics
(LEHD)

27

HUD’s access to these data sources depends on the data sharing agreements that govern them. In some cases,
the state or local agency with which NBER makes an agreement will require that HUD co-sign the agreement,
in which case HUD will have access to the data. In other cases, the agreement will be between only NBER and
the state or local agency (some agreements expressly prohibit NBER from sharing the data), in which case HUD
will not have access to the data. At this time, NBER cannot specify the data sources to which HUD will have
access because all of the data sharing agreements have not been reached.

30
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Domain

Outcomes

Data Sources

Delinquency and Risky
Behavior

Arrests and court dispositions

State and local agencies that
maintain data on adult and juvenile
criminal records

Welfare and other
Social Programs

Monthly TANF benefits, monthly
Food stamp benefits, exits from
cash assistance, date of TANF
time limit, and TANF sanctions

State welfare agency records

Education

Primary and secondary school test
scores, graduation, absences,
grade retention, and disciplinary
actions
College enrollment

School district student records

National Student Clearinghouse

Health

Mortality and cause of death

National Center for Health Statistics'
National Death Index

Housing Assistance

Receipt of housing assistance
Amount of housing assistance

Multifamily Tenant Characteristics
System (HUD) Tenant Rental
Assistance Certification System
(HUD)

Administrative Data Sources for Mediating Factors
Domain

Mediating Factors

Administrative Data Sources

Education

school quality

US Dept. of Education Common Core
of Data on schools and Private
School Survey

school resources
crime rates for local area
unemployment rate
school attendance, grade
completion, test scores

FBI, local police departments,
Census 2000, Bureau of Labor
Statistics (BLS)
Local school district web sites,
National School-Level State
Assessment Database, and
published data

Employment and
Earnings
Delinquency and Risky
Behavior

crime rates for local area

FBI, local police departments

unemployment rate

BLS

crime rates for local area

FBI, local police departments

school resources

US Dept. of Education Common Core
of Data and Private School Survey

school quality

31
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Domain

Outcomes

Data Sources

student SES level

Welfare and other
Transfer Programs

unemployment rate

BLS

receipt of public assistance in the
local area

Census 2000

crime rate in the local area
FBI, local Police Departments
Housing Assistance

Fair Market Rents (FMRs) for local
area, by housing unit size

HUD

A.3.3 Information Technology and Survey Administration

The surveys for the MTO final evaluation will be administered using computer-assisted
personal interviewing (CAPI) technology. The system that ISR will use is Statistics
Netherland’s Blaise interviewing system. Part of the youth questionnaire will be
programmed to use Audio CAI, where the respondent can choose to either listen to the
question with headphones or read from the screen and enter the answers directly into the
interviewer’s laptop. This will be done to assure confidentiality and encourage accurate
answers to sensitive questions. The Blaise authoring system facilitates the entry of question
text, response categories, and range and logic checks, to ensure data quality.
ISR’s sample management system, SurveyTrak, allows interviewers, questionnaires, and
survey assignments to be managed from one central site, with secure transmission of sample
and completed interviews between the interviewer’s laptop and the central office on a daily
basis. As an added level of security, SurveyTrak removes completed interview data from the
interviewer’s laptop after it has been received and verified in the central office. Interviewers
also have direct e-mail access to supervisors, allowing prompt responses to questions that
arise.

A.4 Efforts to Identify Duplication
The purpose of the surveys for the MTO final impact evaluation is to obtain current
information on the status and well-being of adults and youth in the MTO program
population. Detailed information about key aspects of these respondents' educational
achievement, employment and job skills development, physical and mental health,
delinquency and risky behavior, and neighborhood ties is not available through any other
source.28
28

Some basic information about some of these outcome domains is available from administrative data, as
summarized in the table above, but the administrative records are silent about many key outcomes in each of

32
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Duplication is also being avoided in this study because we will use data collected from
families in the Participant Baseline Survey (and during the 1997 and 2000 canvasses) and
the Interim Survey (conducted in 2002). These data will be combined with the data newly
collected for the final evaluation. For example, there is no need to ask about a variety of
personal characteristics or background factors for known household members, because
these were covered at baseline.
The educational achievement data for this study, to be collected by testing sampled MTO
children who are 10-20 at the time of our field period, represent an important complement to
collection of administrative student-level school records. The advantage of administrative
student-level school records is that they are longitudinal and so enable us to examine how
MTO's impacts on children’s academic achievement has changed since random assignment.
The disadvantage of administrative records is that children in the MTO study are now living
in over 300 school districts across the U.S. These school districts differ widely with respect
to the specific achievement tests that they administer to children for their own pedagogical
purposes (for example, use of the Iowa Test of Basic Skills versus the Stanford 9 versus
some "home grown" state-specific achievement assessment). States also differ with respect
to what grade levels are assessed in any given year. There are ways of standardizing the
results of different types of achievement assessments (for example by converting results to
national percentile ranks or "Z scores") and dealing with missing administrative test data for
some children in some calendar years (if their grades were not tested), but the necessary
adjustments may introduce additional noise to the data in any calculations that pool
information across localities and over time. Administering achievement tests to MTO youth
in the long-term survey sample frame will not provide us with longitudinal information on
achievement, but will have the important benefit to the MTO final evaluation of providing us
with a consistent standardized measure of reading and math achievement across MTO groups
at a particular point in time.

A.5 Involvement of Small Entities
No small businesses or other small entities are involved as respondents in the proposed data
collection effort. Respondents are all members of families participating in the MTO
demonstration.

A.6 Consequences of Less Frequent Data Collection
HUD's original plan for the maintenance and evaluation of the Moving to Opportunity
demonstration program was designed to minimize the frequency of data collection from
participants while at the same time maintaining the longitudinal panel for a ten-year period.
The plan involved significant participant data collection only in the baseline period, at the
these domains. For example student-level school records will include information on grade retention and
perhaps dropout status, but will in most school districts be silent about things like school marks (letter grades),
school engagement, and curriculum choices and academic track placements that are typically not available from
administrative student-level school records. Similarly, administrative UI earnings records provide information
on total quarterly earnings, but do not provide any information about the number of hours worked (a key
measure of labor supply by MTO participants), hourly wages (a key measure of individual worker productivity
and access to good-paying jobs), or fringe benefits such as employer-provided health insurance.

33
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

mid-point of the observation period, and at the end. Sample tracking, primarily with passive
methods would be used to maintain the panel in the intervening years.
This request is for the final data collection, to conduct the final impact evaluation.
There are no plans in place for data collection of a similar scale for the MTO
respondents. We do anticipate small-scale qualitative studies in the future with a subsample of MTO families.

A.7 Special Circumstances
The proposed data collection activities are consistent with the guidelines set forth in 5 CFR
1320.6 (Controlling Paperwork Burden on the Public, General Information Collection
Guidelines). There are no circumstances that require deviation from these guidelines.

A.8 Consultation Outside the Agency
In accordance with the Paperwork Reduction Act of 1995, the Department of Housing and
Urban Development (HUD) published a notice in the Federal Register announcing the
agency's intention to request an OMB review of data collection activities for the MTO Final
Evaluation. The notice was published on April 4 2007 in Volume 72, Number 64, pages
16382-83 and provided a 60-day period for public comments. A copy of this notice appears
in Appendix H.
The MTO final impact evaluation design was developed and is being implemented with the
assistance of National Bureau of Economic Research, the prime contractor. Key members of
the NBER team include Drs. Larry Katz and Ronald Kessler of Harvard University, Dr. Jens
Ludwig of the University of Chicago, Dr. Jeffrey Kling of The Brookings Institution, Dr.
Greg Duncan of Northwestern University, Dr. Lisa Gennetian of The Brookings Institution
and Dr. Lisa Sanbonmatsu of NBER.
HUD staff consults with the NBER team on the design at critical junctures in the study.
The purpose of such consultation is to ensure the technical soundness and usefulness of the
data collection instruments, as well as the accessibility of the data required from the MTO
tracking system for carrying out the evaluation.
We have drawn on the expertise of a team of leading researchers who have agreed to serve as
consultants on the project. In particular the design of health outcomes, including biological
samples that we propose to collect (subject to final HUD approval), has been developed in
close consultation with Thomas McDade and Emma Adam of Northwestern University.
McDade is Associate Professor of Anthropology, Faculty Research Fellow of IPR, and
Associate Director of Northwestern’s Cells to Society Center on Social Disparities and
Health. His research focuses on understanding health and human development in the context
of social and cultural processes, with a particular emphasis on stress, and the integration of
biomarkers into population-based social science research. He is the recipient of NSF’s
Faculty Early Career Development Award and a Presidential Early Career Award for
Scientists and Engineers. Emma Adam is Assistant Professor of Human Development and
Social Policy, Faculty Fellow of IPR, and a leading expert on the study of stress effects on
34
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

human behavior and health through the measurement of the stress hormone cortisol. She has
received prestigious post-doctoral awards from the National Academy of Education / Spencer
Foundation and the William T. Grant Foundation.
The NBER team has also engaged consultants Donald Rock, Senior Associate, Center for
Global Assessment, Educational Testing Service, to advise on the final design of the MTO
achievement assessments and adaptation of the ECLS 5th and 8th grade tests. And, John
Rickford, Professor of Linguistics at Stanford University and a leading expert in
sociolinguistics, specifically the relationship between language and ethnicity, social class and
style, and language variation and change to advise on assessments of language.
HUD and NBER have also formed a Technical Review Panel (TRP) who have commented
significantly on the design of this research: Charles Brown, Professor of Economics,
Program Director of the Institute for Social Research and Co-Investigator of the Health and
Retirement study, University of Michigan; Thomas Cook, Professor of Sociology,
Psychology, Education and Social Policy, Northwestern University; Felton Earls, Professor
of Social Medicine, Harvard Medical School; Ronald Ferguson, Faculty Chair & Director,
the Achievement Gap Initiative, Harvard University; Kathleen Mullan Harris, Gillian T. Cell
Distinguished Professor of Sociology, Director & Principal Investigator, National
Longitudinal Study of Adolescent Health, UNC Chapel Hill; Christopher Jencks, Malcolm
Wiener Professor of Social Policy, Harvard University; Terrie E. Moffitt, Professor of
Psychology, University of Wisconsin and Professor, Centre for Social, Genetic and
Developmental Psychiatry, Institute of Psychiatry, London; Robert Sampson, Henry Ford II
Professor of the Social Sciences, Harvard University; Eldar Shafir, Professor of Psychology
and Public Affairs, Princeton University; and Laurence Steinberg, Distinguished University
Professor and Laura H. Carnell Professor of Psychology at Temple University. These
aforementioned TRP members are further joined by distinguished experts in child and youth
development (such as Jeanne Brooks-Gunn, Virginia and Leonard Marx Professor of Child
Development and Education, Columbia University, and Lawrence Aber, Professor of
Applied Psychology and Public Policy at the Steinhardt School of Education, New York
University) to advise on the youth component of the MTO long-term evaluation. Members
of the TRP will be consulted on an ongoing basis throughout the project, and will be
convened twice during the course of the proposed project.
Significant input was also received from the team of researchers involved with designing and
implementing MTO in the 1990s and undertaking the interim evaluation, including Dr.
Lawrence Orr and Dr. Judith Feins of Abt Associates, Dr. John Goering of the City
University of New York, and Margery Turner and Dr. Susan Popkin of the Urban Institute.
Additional input was provided from the teams of researchers who have completed qualitative
work after the interim evaluation, including Dr. Katherine Edin of Northwestern University,
Dr. Xavier Briggs of MIT, and Dr. Susan Clampet-Lundquist of St. Joseph’s University).
No comments were received as a result of the Federal Register Notice.

A.9 Payments to Respondents
Payments to respondents were authorized by OMB for the MTO canvass in 1997, and
35
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

1999, and for the Interim evaluation in 2002. The incentives were used for household
respondents and contributed to successful canvasses and interviewing efforts. At this time,
HUD is requesting OMB approval of continued use of incentives for MTO
respondents for the final impact evaluation.
The use of incentive payments for the MTO final evaluation can be justified on the same
grounds that were cited when first requesting their use for the MTO canvasses and interim
evaluation:
The MTO panel is small. A total of 4,608 households joined the program and were
randomly assigned to one of the three groups during the course of the demonstration. A total
of 1,676 families in the MTO experimental and Section 8 comparison groups used Section 8
certificates or vouchers issued through the program to move.29 This population size will
permit detection of impacts in the likely size range only if panel attrition is kept very low and
survey response rates are high.
The MTO study period is long. A 10-year study is needed to provide sufficient time to
detect a wide range of program impacts on the education, employment, and social well-being
of the families in the program. It is important for this final impact evaluation to make every
effort to encourage participation in this important final data collection.
The MTO population is responsive to incentive payments. Previous research has shown
that sample members with low incomes and/or low educational attainment have proven
responsive to incentives, as have minority group members. These characteristics are heavily
represented in the MTO panel.30 Experience with MTO canvasses in 1997 and 2000, and the
Interim Evaluation in 2002 bears out the value of the incentive.
Incentive payments can reduce the cost of locating mobile panel members before the
main survey data collection. ISR is planning a focused locating effort leading up to the
survey data collection for the final evaluation. Based upon prior research as well as the
MTO tracking experience, the use of an incentive payment is estimated to significantly
reduce the need for expensive field locating .31 We also plan to offer a small number of
contact persons a ‘finder’s fee’ of $5-$10 for providing the research team with
information that helps us locate respondents who are lost to tracking. ISR has
successfully used finder’s fees to help locate difficult to find and low-income respondents
in studies such as the Panel Study of Income Dynamics. Respondent confidentiality is
maintained at all times – the contact person is simply told that we are trying to locate the
named respondent for “a study being conducted by the University of Michigan” and is not
given the name of the study or the sponsor. If the contact person is able to provide us
29

Families assigned to the third group, the in-place control group, remain in their current public or Section 8
project-based housing.
30
See among the sources documenting this recommendation: Allen P. Duffer et al., "Effects of Incentive
Payments on Response Rates and Field Costs in a Pretest of a National CAPI Survey" (Research Triangle
Institute, May 1994), passim; see also Erica Ryu, Mike P. Couper, and Robert W. Marans, “Survey Incentives:
Cash vs. In-Kind; Face-to-Face vs. Mail; Response Rate vs. NonResponse Error” (International Journal of
Public Opinion Research Vol.18 No.1 July 2005).
31
See Duffer et al., ibid.

36
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

with an updated telephone or address information, or can get the respondent to phone our
800 line, we offer to pay the contact person a small amount of money as a token of
appreciation for taking the time to help us with our research activities.
The final evaluation data collection is the final major step in testing the impacts of
MTO. By late 2008, when the surveys are to be conducted, nine to twelve years will have
elapsed since enrollment, and other positive incentives to cooperate with the data collection
(such as willingness to fulfill the commitment made at enrollment) are likely to be low.
Singer (2002) reports that financial incentives are effective for increasing response rates in
all modes of survey data collection.32 At a 1992 OMB-sponsored symposium on the topic of
incentive payments, “most participants agreed with the general thesis that incentives should
be considered whenever the positive forces to cooperate are low.”33
The final evaluation data collection is extensive. The combination of the household survey
with interviewing and testing youth and children represents a substantial time commitment
for the sample members. It seems necessary to recognize the extent of this data collection
(compared to the brief canvasses) by offering larger incentive amounts. The biomarker data
collection activities will require additional respondent time and commitment to the project.
For all these reasons, HUD is requesting authorization for a coordinated set of incentive
payments for this study:
•

Incentive payments of $50 for the household adults surveyed, who will be asked to
complete a 75-minute interview;

•

Incentive payments of $50 for youth ages 10-20, who will be asked to respond to a
60-minute interview and cooperate with 45 minutes of achievement testing;

•

Additional incentive payments of $25 for adults who are asked to provide blood
spot samples, height, weight and blood pressure measurements; and

•

Increased incentive payments of up to $100 for adults and youth who are selected
for the phase 2 subsample or are difficult to locate. The use of increased
incentives is an important tool in maximizing response rates for long and complex
studies such as MTO.

In addition to these aforementioned incentive payments for data collection, a randomly
selected subset of 1,000 youth respondents ages 13 to 20 will receive payments as part of a
nested decision-making exercise included as part of the long-term evaluation. This decisionmaking exercise will seek to measure time preferences, and will be supported using funding
entirely from private foundations – without any federal support. As HUD and NBER have
discussed with OMB, after completing the survey questionnaire and achievement tests MTO
youth respondents selected for the decision making exercise will be thanked for their
32

See “The Use of Incentives to Reduce Nonresponse in Household Surveys,” in Survey Nonresponse, eds.
Robert M. Groves, Don A. Dillman, John L. Eltinge, and Roderick J. A. Little, (Wiley, 2002) pp. 163-77.
33
See Providing Incentives to Survey Respondents: Final Report (Council of Professional Associations on
Federal Statistics, September 1993), p. 10.

37
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

participation in this important HUD study, and provided with their compensation for
participating. We will then indicate to the respondents that with private foundation support
we are also able to offer them some additional compensation as part of a decision-making
exercise, then reading them the question that asks whether they would prefer that ISR mail
them $20 tomorrow or $25 on their next birthday.
We believe this exercise is important because previous research in behavioral economics
suggests that changes in neighborhood environments that affect mood and stress (based on
findings from the MTO interim evaluation) may affect basic features of decision making
such as the willingness of people to defer gratification, i.e. their rate of “time discounting.”
As a result MTO impacts on time preferences could help explain program impacts on a wide
range of behaviors including decisions to stay in school versus drop out, engage in healthrisk behavior, or participate in criminal activity. As part of our decision making exercise
respondents will be offered a choice between a payment of $20 that would be put in the mail
the day after the long-term interview date, and a payment of $25 put in the mail on the
respondent’s next birthday. Following a brief explanation of the terms of the offer, the
respondent is asked: “Which would you prefer - $20 sent tomorrow, or $25 sent on your next
birthday?” This choice is then realized by remitting the payment for the specified amount on
the chosen date. Variation in time between the survey date and the respondent’s next birthday
will generate variation in the ranges into which the respondent’s choice brackets her time
discount rate.

A.10 Arrangements and Assurances of Confidentiality
Informed Consent and Permission for Youth Data Collection

At the initial intake session for the MTO demonstration program between 1994 and 1998,
applicants heard an explanation of the program and of the research design (including the
random assignment to three groups). Those who then decided to join MTO signed an
Enrollment Agreement acknowledging informed consent and permitting collection of various
data about themselves and their family members. A copy of the MTO Enrollment
Agreement is provided in Appendix I.
For the MTO final impact evaluation, we plan to obtain the permission of the core household
heads for completing the survey, and separately for biomarker data collection that will seek
separate permission for height, weight and blood pressure from collection of f blood spots.
The core household consent will also seek permission for testing and interviewing their
children under the age of 18. For youth ages 18-20 we will seek these youths’ own consent
to collect data at this time. The evaluation contractor will also obtain the assent of those
under 18. Appendix J contains the three consent forms and one assent form for youth 10 to
17 years old. These forms incorporate appropriate language reflecting OMB disclaimers,
NIH Certificate of Confidentiality, and requirements of the University of Michigan’s
Institutional Review Board
Data Confidentiality Protections

The data collected in the surveys for the MTO final evaluation, as well as the educational
38
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

achievement test results and the administrative data from the states, will all be used for
research purposes only (for analysis of the research sample). Mailings to potential
respondents and all in-person introductions will include assurances that participation is
voluntary, that all information will be kept confidential, and that the respondents' answers
will be reported as part of a group only.
In addition, HUD is applying for a National Institute of Mental Health (NIMH)
Confidentiality Certification for the MTO final evaluation. This certification strengthens the
privacy protections otherwise applicable to such research, by virtue of the language in the
Public Health Service Act Section 301(d),34 which says:
The Secretary may authorize persons engaged in biomedical, behavioral, clinical, or
other research...to protect the privacy of individuals who are the subject of such
research by withholding from all persons not connected with the conduct of such
research the names or other identifying characteristics of such individuals. Persons so
authorized to protect the privacy of such individuals may not be compelled in any
Federal, State or local civil, criminal, administrative, legislative, or other proceedings
to identify such individuals.
The certification is being requested for the entire MTO final evaluation. The study’s data
collection plan, this OMB statement, and the proposed survey instruments has been reviewed
and approved by the University of Michigan Institutional Review Board. Pending the receipt
of revised consent forms and clarification of minor questions, copies of the revised consent
forms are presented in Appendix J. The IRB’s approval is required in order to obtain NIMH
certification.

A.11 Sensitive Questions
The household and youth surveys contain several modules with sensitive questions, in the
areas of substance use, mental health, and other risk behaviors. Some of the most sensitive
questions are drawn from the National Co-Morbidity Survey (NCS) of adults and
adolescents. All other questions dealing with these topics also have been drawn from
existing survey instruments, including the MTO Interim Evaluation Survey, the National
Study of Adolescent Health, the Project on Human Development in Chicago
Neighborhoods and the National Longitudinal Study of Youth. These former studies
successfully have collected these data under similar circumstances proposed for the MTO
long-term evaluation. All of these questions have been answered without particularly high
non-response rates in other data collection efforts.
Asking these questions about mental health disorders, victimization and risk behaviors is of
considerable importance to this study given that findings from the MTO interim evaluation
point to effects on these outcomes, and given that each of these outcomes impose very large
costs on the government and society as a whole. The long-term evaluation is an opportunity
to examine whether effects persist—a significant question for policy as social costs from
34

42 U.S. Code Section 241(d).

39
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

criminal activity and mental health disorders, for example, are quite high—and whether
effects on new aspects of behavior emerge. Several of the hypotheses about why MTO’s
effects on youth differ by gender including sexual harassment, adult role-modeling, and
perceptions of discrimination, are another important reason why sensitive questions are part
of the long-term survey design. An extensive literature (summarized in Brock and Durlauf,
1999) posits various theories that neighborhoods may affect social pathologies such as
delinquency, substance use, and early childbearing. For example, as briefly described in
Section A.2 models of “relative deprivation” suggest that well-off neighbors may provoke
resentment among those from poorer backgrounds so that poor youth could be more likely to
develop (or fall into) a deviant sub-culture when living in low-poverty neighborhoods. These
models also suggest youth in the MTO experimental group may also show higher levels of
delinquent behaviors than youth in the control group. Findings from the MTO interim study
indicate that MTO increased criminal activity among adolescent males, for example, and that
these young males continue to have ties to friends from their old neighborhoods. Another
example of how neighborhoods may affect social pathologies comes from collective
socialization models that posit that adults in a neighborhood may influence young people
who are not their children. More affluent adults may act as role models who demonstrate
that success is possible if you work hard and play by the rules; and high-SES adults may act
as “enforcers” who help maintain public order. In this model, as MTO families become more
integrated into their new communities, youth in the MTO treatment groups may have lower
social pathologies than control group members since MTO movers end up in neighborhoods
with a larger proportion of high-SES adults. Some evidence in support of this emerged
during the MTO interim evaluation where female youth showed improvements in mental
health and reductions in risky behaviors.
As stated earlier, we will explain the privacy protections of this study to each respondent and
assure them that their responses will be kept completely confidential and anonymous.
Institutional reviews by the University of Michigan’s, Northwestern University’s and
Harvard Medical School’s Institutional Review Board, and the Certificate of Confidentiality
being sought from NIMH, offer additional protections.
We will explain to respondents that these questions are about just one aspect of their lives
and that their answers will not be treated any differently than other data collected. They will
be treated with the same protections of privacy and confidentiality. In addition, we will offer
special means to make the respondents comfortable with answering these questions. Adult
and youth respondents will be given the chance to enter their answers directly into the
automated CAPI (Computer-Assisted Personal Interviewing) system listening to the
questions on a headset or reading the questions from the screen and entering the answers
directly into the interviewer’s laptop computer.

A.12 Estimate of Annualized Burden Hours and Costs
The data collection for the MTO final evaluation is a one-time effort. The present request
covers only the final data collection effort to be carried out in 2008-2009.
Exhibit 4 shows the actual respondent burden for the MTO population to date. It shows the
time, in hours, initially spent by all applicants who completed the MTO enrollment form and
40
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

the baseline survey. It then shows the actual burden resulting from the two MTO canvasses
conducted to date, in 1997 and 2000, the MTO interim evaluation, and subsequent tracking
of MTO families in 2005 and 2006. The total burden of MTO data collection from
participants to date is 15,194 hours over a period of seven years.

41
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Exhibit 4 Actual Respondent Burden (Through December 31, 2006) Under Prior
OMB Clearances
Form

Enrollment
Form
Participant
Baseline
Survey

Number of
Respondents

Time to
complete
(minutes)

Frequency

Total Burden
(hours)

Eligible MTO
applicants

5,301

5 minutes

1 per
respondent

442 hours

Eligible MTO
applicants

5,301

1 per
respondent

3,534 hours

2,624a

40 minutes
Long form
19 min.;
short form
13 min.

3,808b

13 minutes

Respondent

2000 Canvass

Families randomly
assigned in MTO
through 12/31/96
All families randomly
assigned in MTO

Interim Survey
of Households

Adult head of core
householda

3,526c

65 minutes

1 per
respondent

3,820 hours

Interim Survey
of Youth

Sampled youth ages
12-19 from MTO
core households

2,829c

30 minutes

1 per
respondent

1,415 hours

Interim Survey
of Children

Sampled children
ages 8-11 from MTO
core households

1,780c

15 minutes

1 per
respondent

445 hours

Educational
Achievement
Battery
(WJ-R)

Sampled youth and
children (ages 5-19)
from MTO core
households

2,759 Youth
age 12-19;
1,770 age 811; 735 age 57

45 min. for
youth and
children 811; 30 min.
for children
5-7.

1 per
respondent

3,764 hours

2005 Tracking
Mailing

Adult head of core
household
Adult head of core
households deemed
“hard-to-locate”

1997 Canvass

2006 Tracking
Mailing
TOTAL

1 per
respondent
1 per
respondent

1,304

8 minutes

1 per
respondent

139

8 minutes

1 per
respondent

756 hours
825 hours

174 hours

19 hours
15,194 hours

a

Total sample for the 1997 MTO canvass was 2,883; response rate was 91 percent. A portion of the sample was
administered the long form canvass (at 19 minutes) while the remainder was administered the short form (13 minutes in
length).

b

Total sample for the 2000 MTO canvass was 4,608; response rate was 82.6 percent. The entire sample was administered
the short form of the canvass (13 minutes in length).

c Total sample for the 2001 Interim Evaluation was 4,248 adults (83% response rate); 3,537 for youth (80% response rate);
2,202 for children age 8-11 (81% response rate); 6,683 total sample for WJ-R testing (79% response rate).

Exhibit 5 shows the estimated respondent burden for the data collection associated with the
MTO final evaluation, the data collection for which clearance is being sought in this package.
42
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Exhibit 5 Estimated Future Respondent Burden for the MTO Final Evaluation
Data Collection
Number of
Respondents b

Time to
complete
(minutes)

Frequency

Total
Burden

Form

Respondent

Final
Survey of
Households

Adult respondent a

3,545

75 minutes

1 per
respondent

4,431 hours

Physical
Measurements

Adult respondent

3,545

10 minutes

1 per
respondent

591 hours

Dried Blood
Spots

Adult respondent

3,545

10 minutes

1 per
respondent

591 hours

Final
Survey of
Youth

Sampled youth
ages 10-12 from
MTO core
households

840

30 minutes

1 per
respondent

420 hours

Final
Survey of
Youth

Sampled youth
ages 13-20 from
MTO core
households

4,019

60 minutes

1 per
respondent

4,019 hours

Educational
Achievement
Battery
(ECLS)

Sampled youth
and children
(ages 10-20) from
MTO core
households

4,859

45 minutes

1 per
respondent

3,644 hours

MTO Final
Evaluation
(all)

8,404
respondents total

13,696
hours total

a.

The core household refers to the set of persons expected to move together through the MTO program. This
household’s membership is defined by the applicant for MTO, during the process of completing HUD Form
50058 with the PHA staff. The applicant listed all individuals who planned to move into a new unit with a
Section 8 certificate or voucher, if the family were to be assigned to the MTO experimental or Section 8
comparison group and succeeded in leasing up. An adult respondent will be selected from each core household
using the same criteria that were applied during the interim evaluation. In most, but not all, cases, this is the
same person who completed the Enrollment Agreement and Participant Baseline Survey when applying to join
MTO.
b.

The number of respondents for each form reflect achieving an effective response rate of 88 percent by
interviewing 77 percent of the sample using a two-phase sampling process. For example, we might achieve a
70% response rate during the first phase of interviewing. In the second phase, we would select a random 4-in10 subsample of hard-to-locate cases for more intensive follow-up efforts. If we achieved a 60% response rate
with the hard-to-locate cases, this would yield interviews with 77% of the sample frame (.70 + .30*.40*.60) and
an effective response rate of 88% (.70 + .30*.60). The study’s target response rate is 85 percent, but we will
seek a higher response rate if resources allow.

43
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Total time burden of the adult and youth surveys are derived through a variety of methods.
First, for many of the items we were able to obtain time estimates from the interim survey.
This includes, for example, most of the components of the housing, neighborhood, education
and economic modules. Second, as previously described in Section A.2.4, total time burden
of the new modules such as mental health were estimated through our own careful new
analyses of micro-data from the National Co-Morbidity Survey (NCS). Finally, ISR
conducted a pre-test of the proposed long-term surveys with four low-income, minority
female adults and five low-income minority youth. The results of this pre-test indicated that,
excluding the modules on mental health disorders, victimization and risky behavior, average
time of completion for the adult survey was approximately 65 minutes and average time of
completion for the youth survey was approximately 45 minutes. Given the expected time
required for the mental health sections that were not pre-tested, and that pre-tests generally
require somewhat more time to administer on average than field surveys (because
interviewers are just beginning their familiarity with administering the survey instruments),
the existing survey instruments that submitted in this OMB package should be quite close to
the target average administration time of 75 minutes for MTO adults and 60 minutes for
MTO youth.

A.13 Estimated Recordkeeping and Reporting Cost
Burden on Respondents
The cost to respondents will be the time required to respond to the survey.

A.14 Estimate of Cost to the Federal Government
Exhibit 6 shows the costs to the federal government of past and current data collections for
the Moving to Opportunity demonstration. The first row of the exhibit shows the actual cost
of MTO data collection during the baseline period, when families were joining MTO and
when site agencies were submitting data monthly to HUD’s implementation contractor.
The second and third rows of Exhibit 6 show the actual cost of the MTO canvasses
conducted in 1997 and 2000, which together totaled $1,269,824. For the number of families
in the first canvass sample (only part of the full MTO population, which was not yet
complete at the time), the 1997 canvass cost came to $154 per family. The per family cost in
2000 came to $179. For the MTO interim evaluation, cost to the U.S. Department of
Housing and Urban Development (HUD) was $1,365,526 and cost to other federal agencies
was $2,693,847, for a total of $6,367,246. Ongoing tracking of MTO families through 2005
and 2006 cost $719,238. Estimated cost to HUD of the final evaluation is $1,779,821 and,
approximately $3,669,000 per committed resources from other federal agencies as of July
2007. Pending costs currently being reviewed in proposals to federal entities include one at
the National Institute on Aging of $2,206,000 and one at the Institute for Educational
Sciences of $549,000, for a total additional pending cost of $2,755,294. Private funding for a
request of $1,350,000 for data collection is also pending with The Bill and Melinda Gates
Foundation.

44
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Exhibit 6 Actual and Estimated Costs to the Federal Government
Actual and Estimated Costs to the Federal Government for Data Collection
Cost to the
Federal
Government

Total Cost

Total costs for MTO data collection during program operations
(1994-1999)a

$689,491

$689,491

Total costs for 1997 MTO canvass (including incentive payments)

$444,711

$444,711

Total costs for 2000 MTO canvass (including incentive payments

$825,113

$825,113

Line Item

Costs for Interim Evaluation participant data collection (including
incentive payments)
U.S. Department of Housing and Urban Development

$1,365,526

Other federal agencies (NICHD, NSF, NIMH)

$2,693,847

Subtotal

$4,059,373

$6,367,246

$719,238

$719,238

Costs for the continued participant tracking
Costs for Final Evaluation data collection (including incentive
payments)
U.S. Department of Housing and Urban Development
Other federal agencies (NICHD, NSF, CDC, NIMH)
Subtotal
a

$1,779,821
$3,795,393
$5,575,214

$ 7,326,455

Includes Enrollment Agreements and Participant Baseline Surveys, as well as data collection from site agencies.

The last row of Exhibit 6 shows the estimated costs for the final evaluation data collection
covered in this request for OMB clearance. These estimates were prepared by HUD's current
Contractor, NBER. Costs to be funded by HUD for the evaluation’s survey data collection
(including educational testing) will total $1,779,821. Four other federal agencies, NICHD,
NSF, CDC and NIMH, have agreed to provide $3,669,623 in resources to this data
collection.
Grants from several private foundations for this research bring the total data collection
current levels of funding to $7.326 million.

45
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

A.15 Changes in Burden
This request for clearance does not involve a change in burden due to any program changes
or adjustments. It concerns a new data collection not previously submitted to OMB for
review.

A.16 Plans for Tabulation, Analysis, and Publication
The data collected for the MTO final impact evaluation will be analyzed, tabulated, and
reported to HUD by the evaluation contractor. This section describes the basic analytic
framework for the evaluation.
A.16.1 Impact Estimates: The Basic Model

A central objective of the evaluation is to estimate the impacts of the housing vouchers and
certificates received by the MTO experimental group and the Section 8 comparison group
(the “treatment groups”) on a wide range of outcomes in the domains discussed in the
remainder of this chapter. Random assignment assures that simple comparisons of raw mean
outcomes between each of these groups and the in-place control group will provide unbiased
estimates of these impacts. These differences across groups in average outcomes estimate
the causal effect of eligibility for MTO Experimental or Section 8 subsidies, known as
intention-to-treat (ITT) effects. To improve the precision of the estimates, we will use
regression analysis to control for any chance differences between the treatment and control
groups on a number of characteristics measured at baseline.
We illustrate our analytical approach in a simple regression framework. Let D be an indicator
for use of a voucher to move through the MTO program, or treatment compliance. Let Z be
an indicator for treatment group assignment. Let subsidy use be a function of a set of
observed characteristics from the MTO baseline survey either known prior to randomization
or retrospectively reported as existing prior to randomization (X) as well as other factors ε1,
as in equation (1.1).
1.1

D = Zπ1 + Xβ1 + ε1

The ITT effect is captured by the estimate of the coefficient π2 in a regression of one of the
outcome measures discussed above (Y), which might come from either administrative or
survey data sources, on an indicator for assignment (Z) to a treatment group as in equation
(1.2).
1.2

Y = Zπ2 + Xβ2 + ε2

This ITT parameter is an average of the causal effects for those who do and do not take-up
the treatment. Note that conditioning on baseline characteristics (X) improves the precision
of our estimates (that is, increases statistical power to detect program effects) by accounting
for residual variation in the outcome of interest. These baseline characteristics should have
the same distribution within the treatment and control groups because they are statistically
independent of group assignment. Thus, including them will not change the coefficient π1 or
π2 (unless X happens to differ between groups due to the variability in a small sample).
46
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Because the random assignment probabilities under MTO changed over time for different
cohorts of participating families, our team helped develop sampling weights for the interim
MTO survey that we would also use for the proposed final impact evaluation. Note that our
ITT estimates are not biased by self-selection in MTO treatment take-up within the
Experimental or Section 8 groups, because Controls are compared to all families assigned to
a treatment group – whether or not the latter decide to accept the invitation to participate in
the voucher program. This estimate is known as the “intent to treat” (ITT) estimate, because
it reflects the effect of the treatment on all those to whom it was offered, whether or not they
actually received it.
A.16.2 Impact Estimates: Effects of the Treatment on the Treated

While the ITT estimates produced by the basic model are useful for some purposes, it is also
important to know the effect of the treatment on those who actually availed themselves of the
subsidy, i.e., who leased up and moved. Fortunately, we can derive this estimate of the
impact of the “treatment on the treated” (TOT) directly from the ITT estimates and
knowledge of the proportion of treatment group members who leased up.
In this framework TOT is π2/π1, or ITT divided by the proportion receiving the treatment
(Bloom, 1984; see also Heckman, LaLonde and Smith, 1999). 35The TOT estimate is useful
for gauging the magnitude of the causal effects and assessing the substantive importance of
effects on individuals. Although the estimates from equation (1.3) are average causal effects,
the entire marginal distribution of effects is identified and can be analyzed for continuous
outcomes such as achievement test scores (Imbens and Rubin 1997).
1.3

Y = Dγ5 + Xβ5 + ε5

There are two treatment groups in this application, so we will compute separate estimates for
each (the Experimental and Section 8 groups) by making D and Z matrices of two indicator
variables for these groups. We will stack data for all three MTO groups and present
regression-adjusted ITT and TOT estimates based, respectively, on simple regression
estimates of (1.1)-(1.2), and, as described below, on estimates of (1.4-1.5), using treatment
group assignments as “instrument” or predictor of voucher use through MTO. For an
application of these methods to data from the interim study see Orr et al. (2003), Kling,
Ludwig and Katz (2005) and Kling, Liebman and Katz (2007).
This adjustment provides an unbiased estimate of the impact of the treatment on those who
leased up, under the relatively weak assumption that the treatment had no effect on those who
failed to lease up. It is important to note that this adjustment requires no assumption about
the characteristics of those who leased up and/or those who did not; in particular, the adjusted
estimate will be unbiased even if those who lease up differ markedly from those who do not.

35

When equations (1) and (2) are estimated using ordinary least squares (OLS), this is numerically identical to a
two stage least squares regression of Y on D with Z used as an instrumental variable for D, as in equation (5),
where both the first and second stage equations are estimated as linear probability models with Huber-White
standard errors that are adjusted to account for heteroskedasticity.

47
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

We will produce both “intent-to-treat” and “treatment-on-treated” impact estimates for both
the MTO experimental group and the Section 8 comparison group. Great care must be
exercised in interpreting comparisons of the impacts on the two treatment groups, however,
because the proportion of families who leased up, and therefore the subset of families on
whom the treatment had an effect, differed substantially between the two groups. Thus,
when we compare the intent-to-treat estimates, we might find that the regular Section 8
subsidies had a larger effect on certain outcomes, either because they had a larger effect on
those families who leased up or because a larger proportion of families leased up in the
Section 8 comparison group (or both). And, as noted earlier, in comparing the impact of the
treatment on the treated in the two treatment groups, we must be mindful that these
represent impacts on different subsets of families, corresponding to the different lease-up
rates in the two groups. We might, for example, find that the MTO subsidy had a larger
effect on those who leased up than the regular Section 8 subsidy either because it would
have a larger effect for any subset of families or because the subset of families who leased
up in the MTO experimental group were more susceptible to such effects than those who
leased up in the Section 8 comparison group.
A.16.3 Impacts on Subgroups

Perhaps the main puzzle to arise from the interim MTO evaluation is the gender difference in
program effects among youth. To test these sub-group effects, let W represent a set of
indicator variables for different subgroups (such as gender) of interest. The effects for
different subgroups are estimated by including interaction terms (Z*W) in equation (1.4);
note the main effect of W is subsumed in X. The coefficient α3 represents the difference in
the treatment effect across subgroups (such as between boys and girls).
1.4

Y = Zπ3 + Z*W α3 + Xβ3 + ε3

Of particular importance to housing policymakers may be evidence on which sub-groups of
public housing families are able to benefit most from mobility programs like MTO. These
questions can be examined by re-estimating MTO impacts for other subgroups defined on the
basis of baseline characteristics. It will thus be important to identify in advance those
subgroups that social science and other theories predict might benefit more from the MTO
intervention. Subgroups of particular policy relevance that we aim to examine include
families with relatively younger children in the home at the time of baseline, versus those
with older children; families with chronic physical or mental health conditions at baseline;
families who are chronically unemployed and detached from the labor market at baseline;
and families who are more optimistic and resourceful at baseline.
A.16.4 Variation in Impacts Over Time

We expect that the effects of a change in neighborhood will take some time to materialize,
i.e., that impacts will reflect the cumulative influences of living in a new environment.
Therefore, it will be desirable to analyze the time path of impacts wherever possible. For
some outcomes, this will not be possible because of data collection constraints; we will
simply have point-in-time outcome measures taken at the time of the final evaluation survey.
For these outcomes, equations 1.1 – 1.2 will yield estimates of impact at that point in time.
48
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

For some outcomes, however, we will have continuous histories from the point of random
assignment through the follow-up period.
We plan to exploit the longitudinal information on all MTO participants available from the
administrative records discussed above. We will construct a panel of all post-randomization
periods for MTO participants. Period since random assignment (indexed by t) may be
defined as the calendar quarter for administrative earnings or welfare data, and as the
academic year for student-level school records. The regression model includes a set of
indicators for periods since random assignment (Rit) and indicators for actual calendar period
(Qit) to capture residual variation in outcomes over time. We estimate this model separately
for time periods such as 1-2 periods after random assignment, 3-4 periods after
randomization, etc. The coefficient φ1 is the average difference between the treatment and
control groups in a specified time period.
1.5

Yit = φ0 + Ziφ1 + Xiφ4 + Ritφ5 + Qitφ6 + ψit

The value of this type of longitudinal analysis is highlighted by the interim MTO analysis of
youth arrests from administrative “rap sheets.” Kling, Ludwig and Katz (2005) show that
treatment assignment reduces male arrests for violent crimes during the first few years after
randomization, but this effect dissipates and gives way by 3-4 years after random assignment
to a positive treatment-control difference in property-crime arrests. This finding seems to
rule out stories that focus on the difficulty male youth face dealing with the disruptions of
moving, and might instead suggest a comparative advantage in the competition for anti-social
rewards in these new neighborhoods that takes some time to discover. If this explanation is
correct we would expect to see parallel impacts on educational outcomes, which could give
way again to beneficial MTO effects on delinquency over the long term if accumulated
exposure to “protective factors” in the new neighborhoods makes youth more competitive in
local schools and labor markets.
A.16.5 Adjustments for Varying Random Assignment Ratios

The initial random assignment ratio in all MTO sites was set to yield equal numbers of
leased-up families in the MTO experimental and Section 8 comparison groups, given the best
available estimate of the lease-up rates that could be expected in the two groups (80 percent
in the Section 8 group and 30 percent in the MTO group.)36 Equal numbers of leased-up
families would provide the most statistically efficient (i.e., minimum variance) estimates of
differential impact between the two groups receiving certificates or vouchers.
As the demonstration proceeded, it became clear that the lease-up rates for the MTO
experimental families in several sites were significantly higher than predicted, relative to the
Section 8 lease-up rate. Continuing to assign families at the same random assignment ratio
would have resulted in an unbalanced experimental sample, with substantially more leasedup families in the MTO experimental group than in the Section 8 comparison group. Not
only would this have been statistically inefficient, but it would have exceeded the resources
available to the nonprofit organizations responsible for providing counseling to the MTO
36

The initial ratio was 8 MTO experimental families to 3 Section 8 comparison families to 5 in-place control families.

49
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

experimental families. Therefore, the random assignment ratio was changed to a new ratio
that, on the basis of the experience of the early random assignment cohorts in the site, was
expected to produce equal numbers of leased-up families in the MTO experimental and
Section 8 comparison groups. The random assignment ratio was changed at least once in
every site.
When the ratio of treatment and control families randomly assigned differs among parts of
the sample, a simple comparison of mean outcomes (or, equivalently, a regression of the
form shown in equation 2.1, with a single treatment dummy) may yield biased impact
estimates. This is true because such differences confound assignment to treatment group
with site and time period, so that assignment is no longer random over the entire assigned
sample. In this situation, unbiased impact estimates can still be obtained, however, by
estimating the impact of the program within each “assignment set" (i.e., within each
subsample assigned under the same random assignment ratio) and then computing the impact
on the overall treatment group as the weighted average of the assignment set impacts. Since
the treatment and control groups are well-matched within each assignment set, this yields an
unbiased impact estimate. The impact within assignment sets can be estimated using a
regression model with new variables that interact the treatment status indicator with a set of
dichotomous indicators that represent membership in each random assignment set.
Alternatively, it is possible to weight individual sample members to correct for these
variations in random assignment ratio. This approach is particularly useful for descriptive
analyses where regression analysis is either not appropriate or not convenient. In earlier
work, Abt Associates has developed such individual weights for the overall MTO sample.
Similar weights will be used for the final analysis sample. In addition to adjustments to
accommodate changes in the random assignment ratios, we will create new weights to
adjust for two stage sampling process proposed for the long-term evaluation described in
Section B.2. Whichever approach is taken, regression analysis or weighted descriptive
statistics, care must be exercised in deriving the estimates and, especially, their standard
errors, to ensure that the estimates are unbiased and that appropriate tests of statistical
significance are applied.
A.16.6 Mediating Mechanisms

Another goal is to learn why, as well as how, MTO affects families, and whether any
neighborhood effects on behavior are non-linear with respect to specific neighborhood
attributes. To this end we will employ three other approaches that may help eliminate
candidate mechanisms through which neighborhoods may affect outcomes.
First, we will examine the temporal patterns of MTO impacts on outcomes. By 2008 we will
also have sufficiently long residential histories to explore differences between effects on
long-term outcomes of neighborhood characteristics in 2008 versus the residential duration
weighted averages of these characteristics or other aggregations of cumulative exposure.
Second, we can examine MTO impacts on mediator variables linked to the basic theories of
neighborhood effects. Specifically, we generate ITT estimates using survey measures of
candidate mediating mechanisms as the dependent variables in equation (2) and compare
50
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

patterns of outcomes for the full sample and across sub-groups defined by age and gender
with what are observed for individual outcomes.
Evidence that MTO has an effect on a mediator variable is not conclusive evidence that the
theory associated with that mediator is responsible for the program’s effects on behavior,
since numerous environmental factors are changing at once for families assigned to the MTO
treatment groups. And, many of the candidate mediating variables of interest may in
principle be simultaneously determined with youth outcomes in the schooling, work,
delinquency or health domains. However, lack of an MTO effect on a mediator central to a
given theory can help rule out the importance of that theory. For example, Kling, Ludwig
and Katz (2005) hypothesized that the gender difference in MTO effects on youth arrests
could be due to greater racial discrimination experienced by males than females. Yet data
from the interim MTO youth survey suggests that the Experimental treatment reduces
experiences with racial discrimination for both male and female youth. In this case, analysis
of mediators helped to rule out an otherwise plausible mechanism through which
neighborhoods might affect behavior.
Estimating the role of mediators in this way relies on a series of separate regression equations
to estimate program effects on mediators, rather than on a mediated structural-equations
model. We will do this to capitalize to the extent possible on the experimental nature of the
data. The underlying insight here is that randomization occurred with respect to receipt of the
bundle of program services and not on the basis of mediators. This means that the purity of
the experimental design can be maintained only by treating each outcome and mediator in a
separate set of experimental/control regressions.
Finally, we will use the method developed by Kling, Liebman and Katz (2007) to examine
the effects of specific neighborhood attributes by exploiting variation across MTO sites in the
effects of both the Experimental and Section 8 treatments on neighborhood characteristics.
With this approach a socio-economic measure such as the Census tract poverty rates (P) is
viewed as a summary index for a bundle of neighborhood characteristics that are changed as
a result of MTO. Interactions between treatment group assignments (Z) and site indicators
(S) are used as instrumental variables to isolate the experimentally-induced variation in P
across sites and groups, as in equation (2.7), where the main site effects are subsumed in X.
To explore non-linear effects they also use these interactions to instrument for higher-order
terms in P.
1.6

P = Z*S μ6 + Xβ6 + ε6

1.7

Y = Pλ7 + Xβ7 + ε7

Kling, Liebman and Katz test for nonlinear effects of neighborhood socioeconomic
composition (poverty rates as well as fraction college graduates, median income and share of
households headed by a female) in the interim evaluation and find little evidence of
nonlinearities. Using models with two endogenous variables, compliance and poverty rate,
and ten excluded site-by-treatment instruments, they do find evidence that ITT effects appear
to be driven by the poverty rates of neighborhoods rather than simply by use of a voucher at
all. Ludwig and Kling (2007) extend this approach to also disentangle the effects of
51
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

neighborhood class and race composition, as well as neighborhood crime rates, each of
which may have conceptually distinct effects on youth behavior. In preliminary analyses
with the interim MTO data we find that there is enough independent variation in each across
the ten separate MTO treatment/site “experiments” to generate meaningful estimates of
multiple measures – although there are obviously limits on statistical power for the number
of mechanisms that can be distinguished with these instruments. We can also use candidate
mediating measures in the long-term evaluation as second-stage dependent variables and
compare these results to those for youth outcomes to learn more about neighborhood
processes. For example, Ludwig and Kling (2007) find in the interim data that tract poverty
rates are more strongly associated than is tract racial composition with mediators implicated
by leading theories such as community disorder, collective efficacy and policing quality. Yet
tract racial composition is predictive of MTO youth arrests while poverty is not. These
findings do not seem supportive of many of the leading theories about neighborhood effects
on crime.
The strength of MTO is the demonstration’s randomized experimental design, and so our
proposed scope of work prioritizes the analysis of long-term data on MTO participants that
exploits random assignment.
A.16.7 Threats to Internal Validity

The greatest threat to the internal validity of this study is the potential bias from attrition
caused by failure to locate or complete interviews with some respondents. SRC’s efforts to
maximize the survey response rate will include interviewing a sub-sample of hard-to-find
cases, which has been quite successful in previous research.
We will also explore several sensitivity checks to explore potential bias from missing data.
Sample attrition that is systematically related to our outcomes of interest (Y) would
presumably also be related to the distribution of baseline characteristics (X), and so bias from
sample attrition would be reflected by sensitivity of our estimates to conditioning on baseline
characteristics. We can examine this quite easily by comparing baseline characteristics of
respondents to non-respondents, overall and by the random assignment research group status.
We will also pursue more complicated analyses. For example, we can examine the
sensitivity of our results to worst-case bounds, which enable us to bracket the true effects of
MTO without imposing any assumptions about the unobserved outcomes of MTO
participants for whom outcome measures are not available.37 A final approach to addressing
the problem of missing data would be to impute values using the data that are available on
respondents from other data sources, such as the interim MTO surveys or administrative data
sources on related outcome domains.

37

To take a simple example, suppose that data on welfare receipt are available for every adult in the Control
group but are missing for 5% of adults in the Experimental group. A no-assumptions upper bound for any
effects of MTO Experimental assignment to reduce welfare receipt rates comes from imputing outcomes of
non-receipt of welfare to missing Experimental adults; logically, any beneficial effects of MTO on welfare
receipt cannot be any larger than those obtained under this assignment of missing outcomes. See Manski (1989,
1990, 1995), Ludwig et al. (2005).

52
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

A.16.8 Multiple Testing and the “False Positive” Problem

One strength of the long-term MTO study proposed here is the collection of survey and
administrative data on a wide range of outcomes and mediating factors. However, the
availability of so many measures also presents a challenge. In particular, with so many
outcomes, the probability of rejecting a true null hypothesis for at least one outcome is much
greater than the significance level used for each test. Therefore, it is important to use
methods that address this possibility of false positives.
We plan to use the techniques discussed in detail by Kling and Liebman (2004) to conduct
hypothesis tests in three ways. First, following standard practice we will consider the
statistical significance of individual treatment-effect estimates in isolation, or “percomparison significance.” Second, we will make summary statements about the effects of
MTO on a given family of outcomes, such as the entire domain of mental health, by
constructing summary measures equal to the sum of standardized treatment effects within the
family of outcomes. Our calculation of statistical significance for summary measures will use
a series of seemingly unrelated regressions that accounts for the covariance across estimates
within the set of mental health outcomes. Third, we will discuss the statistical significance of
the entire family of related hypotheses in each of our five outcome domains (“familywise
significance”) compared to the null hypothesis of no effect on any of the outcomes in the
domain.
A.16.9 Analytic Techniques, Tabulations, and Reporting

The experimental design of MTO allows for use of fairly straightforward analytic techniques.
The difference in mean outcomes between the in-place control group and either the MTO
experimental group or the Section 8 comparison group provides an unbiased estimate of the
impact of the treatment. To improve the precision of the estimates, OLS regression will be
used to control for chance differences between groups in characteristics that affect the
outcomes. For dichotomous outcomes, logistic regression will be employed.
The analytic results will be presented in tables in the same way that findings were presented
for the MTO interim evaluation (see Orr et al., 2003) that show the control mean, the means
for the MTO experimental and Section 8 comparison groups, the (regression-adjusted)
differences in means and their statistical significance, and the MTO and Section 8 impacts as
percentages of the control mean. The outcomes to be analyzed were discussed in detail in
Section A.2. Exhibit 7 shows one way of displaying these results.

53
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Exhibit 7 Sample
Table Shell for
Presentation of
Impact Estimates

Control Mean

Experimental vs.
Control
ITT

Section 8 vs. Control
ITT

TOT

TOT

Outcome measure

The final report of the final evaluation, to be submitted to HUD in October 2009, will present
a comprehensive analysis of all the data collected over the course of the evaluation. A draft
outline of the report is shown in Exhibit 8. The report will include an Executive Summary
suitable for dissemination to policy makers and the general public, as well as a more detailed
explication of the results in the text and a series of appendices containing documentation of
estimation methods and statistical results, data sources, and additional descriptive
information. The text of the report will be written in language accessible to the layman.
A.16.10 Time Schedule for Analysis and Reporting

Collection of survey data from MTO participants is expected to begin in April 2008 and be
completed by April 2009. The analysis of these data will be carried out between May 2009
and September 2009. A final report is due to HUD at the end of October 2009.

A.17 Expiration Date Display Exemption
All data collection instruments created for the MTO final impact evaluation will display
prominently the expiration date for OMB approval.

A.18 Exceptions to Certification
This submission describing data collection requests no exceptions to the Certificate for
Paperwork Reduction Act (5 CFR 1320.9).
Exhibit 8 Draft Outline: Final Report
Executive Summary
Chapter 1 - The Final Evaluation

The Moving to Opportunity Demonstration
Previous Studies of Mobility Programs and the Effects of Neighborhood
Previous Analyses of the MTO Demonstration
Research Questions
Overview of This Report
54
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Chapter 2 - Geographic Mobility in the MTO Final Evaluation Sample

Hypotheses about mobility in MTO
Mobility data sources and measures
Baseline conditions and initial lease-ups
Sample mobility in the follow-up period
Geographic mobility impacts
Interpretation of results
Chapter 3 – Impacts on Housing, Neighborhoods, and Safety
Hypotheses about housing, neighborhood, and safety
Data sources and measures
Baseline housing and neighborhood status of MTO participants and control
group context
Impacts on housing, neighborhoods, and safety
Interpretation of results
Chapter 4- Impacts on Adult and Youth health
Hypotheses about adult and child mental and physical health
Data sources and measures
Context and baseline status of the sample
Mediators for health impacts in MTO
Impacts on adult and youth mental and physical health
Interpretation of results
Chapter 5- Impacts on delinquency, criminal and risky behavior
Hypotheses about adult and youth involvement with delinquent / criminal or
risky behavior
Data sources and measures
Context and baseline status of the sample
Effects on mediators for delinquent / criminal or risky behavior in MTO
Impacts on delinquent / criminal or risky behavior
Interpretation of results
Chapter 6 – Impacts on adult and youth education
Hypotheses about adult and youth education in MTO
Data sources and measures
Baseline education experiences and control group context
Impacts on hypothesized mediators of educational impacts
Impacts on hypothesized outcomes
Interpretation of results
55
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification

Chapter 7 – Impacts on adult and youth employment and earnings
Hypotheses about adult and youth employment and earnings
Data sources and measures
Context and baseline employment status of the sample
Impacts on hypothesized mediators
Impacts on employment and earnings of adults
Impacts on employment and earnings of youth
Interpretation of results
Chapter 8 – Impacts on income and receipt of public assistance
Hypotheses about income and public assistance
Data sources and measures
Context and baseline employment status of sample
Impacts on hypothesized mediators
Impacts on income and public assistance
Chapter 9 – Summary and Implications of Estimated Impacts of MTO
Summary of Impact estimates
Assessing the Impact estimates
Implications of the Final Evaluation Results for Public Policy
Appendix A - Estimation Methods and Derivation of Outcome Measures
Appendix B - Data Sources and Data Collection Methods
Appendix C - Descriptive Tables
Appendix D - Detailed Estimation Results

56
National Bureau of Economic Research
University of Michigan – ISR

PART A: Justification