MTOFinalEval_Part B

Part B: Collection of Information Employing
Statistical Methods
B.1 Respondent Universe and Sampling Methods
Universe of Core Households and Final Evaluation Sample

The purpose of the MTO final impact evaluation is to assess the effect of the experimental treatment
on the lives of participants on average between 10 and 12 years after the time of random assignment.
The final evaluation will include the full program population (4,604 families).
Universe of Core Household Members

Within the 4,604 core households that are the sample for the final evaluation, there are 17,000
members ranging in age (measured at the end of calendar year 2007) from 10 to over 100. The core
household membership is fixed by definition, since core household members are those who lived
with the MTO applicant during the process of Section 8 eligibility determination prior to MTO
random assignment. Core members are those members of the family who were identified as
planning to move together if awarded a voucher or certificate through the MTO program. Thus,
core household membership is exogenous to the experiment (that is, not affected by which of the
three MTO mobility groups to which an applicant was randomly assigned), while current household
composition is not (since MTO assignments could for example affect the propensity of MTO
applicants to be married or live with other relatives).
Through passive tracking, the 1997 and 2000 MTO canvasses, and the 2002 Interim survey, data are
available on the whereabouts of nearly all core household members. Sampling of youth for the final
evaluation will be carried out regardless of where they are located (including those who are no
longer living with the original MTO adult applicant), and arrangements will be made to test and
interview youth wherever they are currently living.
Sampling of Youth

Among core household members, 6,647 persons will be between the ages of 10 and 20 on December
31, 2007. These are the members eligible to be selected for the youth survey sample frame. Exhibit 9
shows their distribution by age group and by number in core family.
The sampling plan calls for random selection of up to three children ages 10 to 20 per core
household. Over 75 percent of the 4,604 core households in the final evaluation still have children in
this age range. (The remainder had children under 18 when they joined MTO but now have only core
children who will be over the age of 20 at the end of 2007). The distribution of numbers per
household (shown in the lower panel of Exhibit 9) implies that, for the 70 percent of households in
the MTO study who have from one to three children in the relevant age range, there need be no
selection of children – we will seek to interview all of the children ages 10-20 in the core household.
The random sampling of youth will only be relevant for the remaining 257 households that contain
more than three youth ages 10-20. Because the children in these 257 households are of different
56
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

ages, we will not know the precise age composition of the overall child/youth sample frame for the
MTO long-term evaluation until it is actually drawn (that is, until we have randomly selected 3
children from households with more than 3 children in the relevant age range). In cases where our
youth survey sample includes two or more children per MTO household, we will adjust the standard
errors in our analysis to account for the fact that observations drawn from the same home will not be
statistically independent.
Exhibit 9 Child and Youth Population for MTO Final Evaluation
Number

Percent

Ages 10-13

1219

18.34%

Ages 14-18

3923

59.02%

Ages 19-20

1505

22.64%

TOTAL

6647

100.00%

Age Group on 12/31/2007

Number of Children or Youth in Core Household
None

1079

23.44%

One

1552

33.71%

Two

1160

25.20%

Three

556

12.08%

Four or more

257

5.58%

B.2 Information Collection Procedures
Sample Design

All households enrolled in the MTO demonstration will be eligible for inclusion in the study. If
sufficient additional funding is secured 1 , we will include each adult and up to three children between
the ages of 10 and 20 (without further stratification on age) from among the core MTO households.
In this way, analyses can be conducted combining children of different ages without having to weight
the data specifically for this step.
We plan to employ a two-stage stratified sampling design for MTO. For the first stage of the
sample, MTO households will be stratified into four groups, based on location and cost factors.
Working with SRC’s Statistical Design Group, we will identify optimal sampling rates for each
group (with slightly lower sampling rates in the more expensive strata, allowing us to maximize the
effective sample size for a given data-collection budget). The Stage 1 sample will be released for
interviewing in April 2008. Our response rate goal for the first stage of interviewing is 70%.

1

See pending Federal costs described in Section A.14.

57
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

When the Stage 1 response rate goal has been achieved, we will review and stratify the remaining
outstanding cases, taking into account cost, survey errors, and propensity models created during
Stage 1 interviewing. Optimal sub-sampling rates will be selected (again in consultation with SRC’s
Statistical Design Group) and approximately 40% of the remaining sample will be randomly
selected and released for Stage 2 interviewing. To maximize the success of Stage 2 interviewing,
we plan to increase our respondent interview incentives from $50 to $100, and use our most
experienced and successful MTO interviewers to obtain our response rate goal of 60% for this
second stage of interviewing. Note that because the 40% of cases selected in Stage 2 are randomly
selected, their survey responses provide a representative picture of what is happening among all of
the cases that were not interviewed in Stage 1. Put differently, random sampling of cases in Stage 2
lets us “weight up” their responses to project the outcomes of the entire set of cases not interviewed
in Stage 1.
The final sampling weight will be the product of the Stage 1 and Stage 2 weights.
Estimation Procedures

As described in Section A.16 above, the data to be collected for the final evaluation will be used to
estimate impacts of the demonstration on a wide range of outcomes. For each outcome, impacts will
be estimated by regressing the outcome on a set of baseline characteristics and a dummy variable that
indicates whether the sample member belongs to the treatment or control group. Impacts will be
estimated separately for the MTO experimental group and the Section 8 comparison group, relative to
the in-place control group. With random assignment, ordinary least squares will produce unbiased
estimates.
Degree of Accuracy Required

It is important to consider the precision with which we will be able to measure impacts on the
proposed outcomes and mediators, given the sample sizes available, since imprecise estimates
reduce our ability to draw meaningful inferences about the ways in which MTO impacts the lives of
participating families. The best way to assess the precision of the estimates that can be derived from
an experimental design is to examine the minimum detectable effects (MDEs) obtainable under
that design (Orr, 1999). The minimum detectable effect is the smallest true program impact that has
a good chance of being identified with data from a given sample. The MDE is defined as 2.8 times
the estimated standard error (see Orr, 1999), and represents the smallest true impact that has an 80
percent chance of being statistically significant, using a two-tailed hypothesis test at the 10 percent
level. The smaller the MDE, the more precise the estimate. We focus on calculating MDEs for the
intent to treat (ITT) estimate, that is, the regression-adjusted difference in mean outcomes for
respondents assigned to the MTO experimental group versus the control group. We focus on ITT
impacts because they are simpler and require fewer statistical assumptions than the TOT impact
estimates described earlier in our OMB submission.
Exhibits 10 and 11 show MDEs as a percent of the control mean for selected outcomes, for adults and
youth, respectively. 2 Separate MDEs are shown for the ITT effects of assignment to the MTO
2

The MDEs shown here do not take into account two offsetting influences on the precision of the estimates. First, to the
extent that covariates in the impact regression explain some of the variation of the dependent variable, the precision of the

58
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

experimental group and to the Section 8 comparison group. The MDEs shown in the exhibit are
calculated using data from a variety of sources, including from the MTO interim evaluation (see Orr
et al., 2003) or for new outcomes that were not included on the interim study, from nationally
representative data sources.
As the exhibits demonstrate, for the adult outcomes we can be reasonably confident of detecting
impacts that range from 1.8 to 24.0 percent of the control mean, with slightly better precision (smaller
MDEs) for the MTO experimental group than for the Section 8 comparison group. If the true impacts
from the MTO demonstration are similar in size to those found by Rosenbaum (1992) for the
Gautreaux mobility program, then we would have adequate statistical power (precision) to detect
these MTO impacts. For the most part the MDEs that we calculate for youth outcomes suggest at
least as much precision with the youth analysis as for the adult analysis, consistent with the idea that
our MTO youth survey sample frame for the long-term evaluation will be even larger than our adult
sample. Previous waves of the MTO evaluation have found program impacts on a variety of youth
outcomes that are as large or sometimes larger than the MDEs that we calculate in Exhibit 11 for the
long-term MTO evaluation (see Katz, Kling and Liebman, 2001, Ludwig, Duncan and Hirschfield,
2001, Ludwig, Ladd and Duncan, 2001, Orr et al., 2003, Kling, Ludwig and Katz, 2005, and Kling,
Liebman and Katz, 2007).

estimates will be somewhat better than shown here. Second, because the random assignment ratio varied across sites and
over time, to obtain unbiased impact estimates the sample observations must be weighted to reflect their different
assignment probabilities; this will reduce the precision of the estimates. We do not expect either of these factors to change
the MDEs by more than 5-10 percent, and for a typical dependent variable, we expect them to be roughly offsetting.

59
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Exhibit 10 – MDEs on Selected Adult Outcomes (ITT Estimates)
Mean 3

Outcome

MDE as % of mean
MTO Experimental

Section 8

Housing
Receiving any housing assistance

.71

7.5%

8.3%

Rate Housing as Excellent or Good

.52

11.8%

12.9%

Satisfied or Very Satisfied with Housing

.48

12.9%

14.1%

Problem With Vermin

.53

11.6%

12.7%

Problem With Heating/Plumbing

.39

14.9%

16.3%

Problem With Peeling Paint/Plaster
Any Household Member Crime Victim (Last 6
Months)
Economic

.49

12.5%

13.6%

.21

22.6%

24.0%

Currently employed

.52

11.2%

12.3%

Receiving TANF

.29

18.5%

20.5%

Currently employed & Not Receiving TANF

.45

10.4%

10.9%

Annual earnings

$8899

$1216

$1341

Household Income

$15536

$1522

$1696

General Health (very good/good vs. fair/poor)

.67

8.3%

9.2%

High Blood Pressure

.30

18.7%

20.4%

Currently Obese

.47

12.8%

14.6%

Total Cholesterol

204

2.4%

2.6%

C-Reactive Protein

5.7

16.4%

17.8%

Glycosylated Hemoglobin

5.6

1.8%

2.0%

Body Mass Index

29.3

2.8%

3.1%

Psychological Distress (K6) Index

.33

12.7%

13.6%

5

.24

19.9%

21.7%

.39

14.2%

15.4%

Physical Health

Biomarker

4

Mental Health
Any DSM-IV Disorder- 12-Month
Any DSM-IV Disorder- Lifetime

3

MDEs are presented as a percent of the control mean for Interim items and as a percent of the source mean for nonInterim items (the four biomarker and two DSM-IV disorder measures).
4
MDEs derived from data collected in the 2003-04 edition of the National Health and Nutrition Examination Survey.
5
MDEs derived on DSM-IV diagnoses derived from data collected as part of the National Co-morbidity Survey-Revised.
The disorders included in the calculation for adults included major depressive disorder, bipolar disorder, post-traumatic
stress disorder, generalized anxiety disorder, intermittent explosive disorder, and panic disorder.

60
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Exhibit 11 – MDEs on Selected Youth (Age 12-19) Outcomes (ITT Estimates)
Mean 6

Outcome

MDE as % of mean
MTO Experimental

Section 8

Achievement and School Progress
ECLS 5th Grade Reading Scores

137

1.9%

2.1%

ECLS 5 Grade Math Scores

111

2.2%

2.4%

Educationally On Track

.74

6.8%

7.5%

Behavior Problems Index

.26

13.3%

14.2%

Delinquency Index

.09

21.9%

24.6%

.71

8.5%

9.4%

Psychological Distress (K6) Index

.26

14.2%

15.1%

7

.54

10.1%

11.0%

.59

9.2%

10.0%

.45

12.1%

13.2%

.35

41.7%

46.5%

.45

12.2%

13.2%

th

Social Behavior

Physical Health
General Health (very good/good vs. fair/poor)
Mental Health
Any DSM-IV Disorder- 12-Month
Any DSM-IV Disorder- Lifetime
Mediators
Parents Rarely or Never Help with Homework 8
Comfortable Talking to 3+ Adults

9
7

Friends Think Grades Very Important

It may be possible to obtain even more precise estimates of impacts on these outcomes by focusing
the analysis in some cases on MTO sub-groups for whom the outcome in question is most relevant –
for example, males are responsible for committing the large majority of all criminal offenses in the
U.S. 10 Another way to define sub-groups for this type of analysis is to use baseline data to identify
those most at risk of such behaviors or conditions. To the extent that appropriate baseline data are
available, we will do so in the analysis.

6

MDEs are presented as a percent of the control mean for Interim items and as a percent of the source mean for nonInterim items (the ECLS test score, DSM-IV disorder, parent help with homework, number of adults to whom respondent
can speak comfortably, and importance of grades to friends measures).
7
DSM-IV diagnosis data come from the National Comorbidity Survey-Revised. The disorders included in the calculation
for adults included the six disorders used in the adult calculation as well as attention deficit disorder, oppositional defiant
disorder, and conduct disorder.
8
Data come from the National Educational Longitudinal Survey (NELS88).
9
This measure was based on Interim data for youth aged 15-20 (not 12-19 as with the other Interim measures).
10
The MDEs in Exhibit 10 do not take account of the inclusion of baseline variables as covariates in the impact
regression. This effect will vary across outcomes, but will generally be relatively small. For outcomes that are persistent
over time, however (e.g., chronic medical conditions), inclusion of the baseline value can substantially improve the
precision of the impact estimate.

61
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

In summary, then, it appears that our target MTO sample frames of around 4,600 MTO female adults
and 6,600 youth ages 10-20 should provide adequate precision to detect plausible MTO program
impacts.
Procedures with Special Populations

The MTO population contains a considerable number of respondents whose first language is Spanish.
As in all previous MTO data collections, HUD’s Contractor will translate each of the adult survey
instruments and modules into Spanish, for administration in the language most comfortable for the
respondent. All preliminary contacting materials and consent forms will also be made available in
Spanish.
For the educational testing of youth and children, however, HUD's Contractor will only administer
the ECLS tests in English. We have been advised that the test is not well translated, with respondents
likely to get confused and provide unreliable data. In addition, prior data collection and focus group
experience indicates that the children in MTO families are proficient in English even when the
parents are not. The results of the analyses of these test data will be interpreted as the long-term
effects of the MTO intervention on the ability of participating youth to read and conduct math
exercises in English.

B.3 Methods to Maximize Response Rates
As in the interim study, we will employ two-phase sampling to obtain responses from a representative
sub-sample of hard-to-locate respondents (Groves 2004). The target response rate for all elements of
the MTO final evaluation data collection is 85 – 90 percent. 11 Such a high response rate for the longrun MTO survey is feasible given the ongoing efforts by HUD to track MTO families and our success
in obtaining a response rate of this level for the interim MTO evaluation in 2002. Due to the tracking
and locating efforts HUD has sponsored since the demonstration began, there has been very little
attrition and the location of the sample is known to a very high degree.
Several methods will be used to maximize the response rates for the MTO final evaluation data
collection. These methods involve preliminary tracking and locating, incentive payments, and sample
control during the data collection period.
Preliminary Tracking and Locating

In order to maximize response to the MTO final evaluation, HUD’s Contractor has planned a focused
locating effort before the main data collection begins. The purpose of this locating effort is to obtain
updated address, phone, and secondary contact information for all MTO heads of household, as was
done in advance of each MTO canvass. ISR will also attempt to collect the same information for
sampled youth (ages 10-20) no longer living with the core household head.

11

We plan to employ two-phase sampling, with a goal of a 70% Response Rate during the initial phase and a goal of a
60% Response Rate for a 4-in-10 subsample of hard-to-locate respondents not interviewed during the first phase. If the
phase one response rate was 70% (e.g., 3149/4498), and a 4-in-10 subsample of hard-to-locate cases were worked in
phase two with a response rate of 60% (324/540), then the effective response rate would be .80+.20*.50=.88, or 88%.

62
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

The locating effort will be carried out largely by mail, with some reminder phone calls and some inperson visits to obtain updated address and telephone number information for selected respondents.
The initial mailing will include a letter explaining the study, a toll-free number to call if the
respondent has questions, and a post card to be returned with address or telephone number updates.
If the letter is returned as undeliverable, ISR will send the case to our tracking team who will use a
combination of active and passive tracking methods to obtain an updated address or telephone
number for the selected respondent.
The introductory letter will be sent to all household heads. Approximately 2-3 weeks after the initial
mailing (if it is not returned as undeliverable), the assigned interviewer will contact the respondent to
schedule an interview.
Use of Incentive Payments

When respondents are being contacted to schedule appointments, the interviewers will offer incentive
payments as discussed fully in Section A.9 above. The substantial amounts being offered for
cooperation with the surveys and educational testing should help gain cooperation from most of the
sample members.
Sample Control During the Data Collection Period

During the data collection period, non-response levels and response bias will be minimized in the
following ways:
¾ The Contractor will use trained field interviewers and testers who are skilled at maintaining
rapport with respondents, so that the number of break-offs and the incidence of item nonresponse will be kept low.
¾ The data collection procedures will limit intrusiveness by conducting the adult and youth
interviews and testing simultaneously (to reduce the amount of time in the home) where
feasible.
¾ Respondents will have a choice of time for the data collection.
¾ If the respondent prefers to have the interview conducted at a location other than his/her
home, the interviewer will arrange for a location that is convenient for the respondent, and
provides privacy (e.g., a conference room at a local library) so the respondent’s answers are
not overheard by others.
¾ Additional tracking and locating steps will be taken, as needed, when sample members are not
found at the phone numbers or addresses previously collected.
¾ Finally, the Contractor’s survey director and field supervisors will manage the sample to
ensure that the target response rates are achieved (or approached) evenly for all three
randomly assigned groups in each MTO site and for movers and non-movers alike.
By these methods, the Contractor anticipates being able to achieve an 85 to 90 percent effective
response rate for all data collection components associated with the final evaluation.
63
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

B.4 Test of Procedures
Prior to submission of the OMB package, HUD's data collection contractor, NBER and ISR
conducted pretests with a convenience sample of 4 low-income, minority female adults, and 5 lowincome, minority youth respondents. The pre-test was of a paper and pencil version of the survey
excluding the modules on mental health disorders drawn from the National Co-Morbidity survey.
The purpose of this very small sample pre-test was to allow experienced ISR interviewers to become
acquainted with the survey instrument and offer preliminary and early guidance on revisions. The
pre-test also provided us with rough benchmark time estimates for particular modules (described in
Section A.12).
Because of the complexity of the survey and proposed data collection at the time of the survey,
NBER and ISR will also conduct post-OMB clearance pre-testing of the survey instruments and
assessments covered by this clearance request. After clearance by OMB for our proposed survey
work, two formal pretests will be conducted before our final MTO evaluation fieldwork commences.
The first pretest is scheduled for October 2007, with a convenience sample of low-income volunteer
respondents in the Detroit, Michigan area (proximate to ISR’s offices in Ann Arbor), the second pretest is scheduled for January 2008 with a small set of MTO respondents. The pretest interviewers
will keep close track of elapsed time and will take detailed notes on places in the instruments where
skip patterns needed revision or where respondents did not understand the question wording.
As a result of the pretest, we anticipate minor adjustments will be necessary. Because the majority of
the items in the instruments are taken from the MTO Interim evaluation and other well-tested
instruments, we expect the changes to include the following:
¾ Omitting lower-priority questions, to conform with goals for survey length.
¾ Simplify the language in instruments, letters, or consent forms, to be sure that all respondents
will understand what is being asked.
¾ Clarify skip patterns, or modify question order to make the questions flow more smoothly and
logically for the respondents.

B.5 Individuals Consulted on Statistical Aspects of the Design
The individuals shown in Exhibit 12 assisted HUD in the statistical design of the final evaluation.
Exhibit 12 Individuals Consulted on the Study Design
Name

Telephone Number

Role in Study

Dr. Greg Duncan

847-467-1503

NBER team, Northwestern University

Dr. Lisa Gennetian

914-834-2200

NBER team, The Brookings Institution

Dr. Lawrence Katz

617-588-0304

NBER team, Harvard University

Dr. Ronald Kessler

617-432-3587

NBER team, Harvard University

Dr. Jeffrey Kling

202-797-6000

NBER team, The Brookings Institution

64
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

NBER team, University of Chicago
Dr. Jens Ludwig

773-702-3242

Dr. Larry Orr

301-913-0520

Abt Associates, Principal Investigator of the
MTO Interim Evaluation

In addition to the individual listed above, HUD and NBER work closely with many of the world’s
leading survey statisticians at the University of Michigan Survey Research Center. SRC’s Statistical
Design Group (SDG) has been led by Steven Heeringa who has more than 27 years of sample design
experience and has been responsible for the design, development and implementation of probability
samples for major SRC projects. Heeringa has served as a sample design consultant to a number of
government and non-government organizations including the National Heart Lung and Blood
Institute (NHLBI), the Agency for Health Care and Prevention Research (AHCPR), and the General
Motors Corporation. HUD and NBER also receive consultation from James Wagner, Senior Project
Manager, has over 10 years of experience in managing projects and staff in the field of survey
research and works closely with the MTO project team on sampling, estimation of weights and
variance, development of budget models, and forecasting of costs.
Inquiries regarding the statistical aspects of the study's planned analysis should be directed to:
Dr. Jens Ludwig
University of Chicago
969 East 60th Street
Chicago, Illinois 60637
Phone: (773) 702-3242
Fax: (773) 702-0874
E-mail: [email protected]

65
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

References
Achenbach, T. M., & Edelbrock, C. S. (1981). “Behavioral problems and competencies reported by
parents of normal and disturbed children aged four through sixteen.” Monographs of the Society
for Research in Child Development, 46 (1, Serial No. 188).
American Psychiatric Association (1994) Diagnostic and Statistical Manual of Mental Disorders,
(DSM-IV), Fourth Edition. Washington, DC: American Psychiatric Association.
Bader, Michael (2007) “Report on the Analysis of Systematic Social Observations from Block and
Street Level Data”. Population Studies Center, Institute for Social Research, University of
Michigan.
Becker, Gary S. and Kevin M. Murphy (2000) Social Economics: Market Behavior in a Social
Environment. Cambridge, MA: Belknap Press.
Bloom, Howard (1984) “Accounting for No-shows in Experimental Evaluation Designs.” Evaluation
Review. 8 (April): 225-46.
Brock, William A. and Steven N. Durlauf. (2001). “Interactions-Based Models,” in James Heckman
and Edward Leamer, eds, Handbook of Econometrics Volume 5, Amsterdam: North Holland,
forthcoming.
Browning, Christopher R. and Kathleen A. Cagney (2003) “Moving Beyond Poverty: Neighborhood
Structure, Social Processes, and Health.” Journal of Health and Social Behavior. 44(4): 552571.
Carneiro, Pedro and James J. Heckman (2003) “Human Capital Policy.” In Inequality in America:
What Role for Human Capital Policies? Edited by James J. Heckman and Alan B. Krueger.
Cambridge, MA: MIT Press. pp. 77-239.
Chrousos, G. P., & Gold, P. W. (1992). The concepts of stress and stress system disorders. Journal of
the American Medical Association, 267, 1244-1252.
Clampet-Lundquist, S., Edin, K., Kling, J., and Duncan, G. (2006) “Moving At-Risk Teenagers Out
of High-Risk Neighborhoods: Why Girls Fare Better Than Boys.” Princeton, NJ: Princeton
University Industrial Relations Working Paper # 509.
Coleman, James S. (1988) “Social Capital in the Creation of Human Capital.” American Journal of
Sociology. 94:S95-S120.
Coller, Maribeth; Harrison, Glenn W., and Rutström, E. Elisabet, “Are Discount Rates Constant?
Reconciling Theory and Observation,” Working Paper 3-31, Department of Economics, College
of Business Administration, University of Central Florida, 2003.

66
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Cook, Philip J. and Kristen Goss (1996) “A selective review of the social-contagion literature.”
Working Paper, Sanford Institute of Public Policy, Duke University.
Crane, Jonathan (1991) “The Epidemic Theory of Ghettos and Neighborhood Effects on Dropping
Out and Teenage Childbearing.” American Journal of Sociology. 96(5): 1226-1259.
Cukor-Avila, Patricia and Guy Bailey (2001) “The effects of the race of the interviewer on
sociolinguistic fieldwork.” Journal of Sociolinguistics. 5(2): 254-270.
Demyttenaere, K., Bruffaerts, R., Posada-Villa, J., Gasquet, I., Kovess, V., Lepine, J-P., Angermeyer,
M.C., Bernert, S., de Girolamo, G., Morosini, P., Polidori, G., Kikkawa, T., Kawakami, N., Ono,
Y., Takeshima, T., Uda, H., Karam, E.G., Fayyad, J.A., Karam, A.N., Mneimneh, Z.N., MedinaMora, M.E., Borges, G., Lara, C., de Graaf, R., Ormel, J., Gureje, O., Shen, Y., Huang, Y.,
Zhang, M., Alonso, J., Haro, J.M., Vilagut, G., Bromet, E.J., Gluzman, S., Webb, C., Kessler,
R.C., Merikangas, K.R., Anthony, J.C., Von Korff, M.R., Wang, P.S., Brugha, T.S., AguilarGaxiola, S., Lee, S., Heeringa, S., Pennell, B.E., Zaslavsky, A.M., Chatterji, S., Ustun, T.B.
(2004) “Prevalence, Severity, and Unmet Need for Treatment of Mental Disorders in the World
Health Organization World Mental Health Surveys.” Journal of the American Medical
Association. 291: 2581-2590.
Devine, Deborah J., Robert W. Gray, Lester Rubin, and Lydia B. Taghavi (2003) Housing Choice
Voucher Location Patterns: Implications for Participant and Neighborhood Welfare. Washington,
D.C.: U.S. Department of Housing and Urban Development.
Duffer, Allen P. et al. (1994). ‘ "Effects of Incentive Payments on Response Rates and Field Costs in
a Pretest of a National CAPI Survey" Chapel Hill, NC: Research Triangle Institute.
Edelman, D., Olsen, M.K., Dudley, T.K., Harris, A.C. & Oddone, E.Z. (2004). “Utilitity of
hemoglobin A1c in predicting diabetes risk.” Journal of General Internal Medicine 19: 1175-80.
Eckel, Catherine C., Johnson, Cathleen, and Monmarquette, Claude. "Saving Decisions of the
Working Poor: Short and Long-Term Horizons," in J. Carpenter, G. W. Harrison, and J. A.
List, Field Experiments in Economics. Greenwich and London: JAI Press, 2005.
Ellen, Ingrid Gould and Margery Austin Turner (2003) “Do Neighborhoods Matter and Why?” In
Choosing a Better Life? Evaluating the Moving to Opportunity Social Experiment. Edited by John
Goering and Judith D. Feins. Washington, DC: Urban Institute Press. pp. 313-338.
Fischer, Mary J. and Douglas S. Massey (2004) “The Ecology of Racial Discrimination.” City and
Community. 3(3): 221-241.
General Accounting Office (2001) Federal Housing Assistance Programs: Costs and Housing
Characteristics. Washington, D.C. Report GAO-01-901R.
Glaeser, Edward L. and José A. Scheinkman (1999) “Measuring Social Interactions.” Unpublished
paper, Harvard University, August.
67
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Goodyer, I. M., Herbert, J., & Tamplin, A. (2003). Psychoendocrine antecedents of persistent firstepisode major depression in adolescents: a community-based longitudinal enquiry. Psychological
Medicine, 33, 601-610.
Groves, Robert M. (2004) Survey Errors and Survey Costs. Hoboken, NJ: Wiley-Interscience.
Groves, Robert M. and Steven .
Harrison, Glenn W.; Lau, Morten Igel, and Williams, Melonie B., “Estimating Individual Discount
Rates for Denmark: A Field Experiment,” American Economic Review, 92(5), December 2002,
1606-1617
Heckman, James J., Robert LaLonde, and Jeffrey Smith (1999) “The Economics and Econometrics of
Active Labor Market Programs.” In Handbook of Labor Economics, vol. 3A. Edited by O.
Ashenfeleter and D. Card. Amsterdam: North Holland. pp.1865-2097.
Holt, C. and S. Laury. 2002 “Risk Aversion and Incentive Effects.” American Economic Review,
92(5): 1644-1655
Imbens, Guido W. and Donald B. Rubin (1997) “Estimating Outcome Distributions for Compliers in
Instrumental Variables Models.” Review of Economic Studies. 64(221): 555-575.
Jargowsky, Paul. “Stunning Progress, Hidden Problems: The Dramatic Decline of Concentrated
Poverty in the 1990s.” Washington, DC: Brookings Institution Press, May 2003.
Jencks, Christopher, and Susan E. Mayer (1990) "The Social Consequences of Growing Up in a Poor
Neighborhood." In Inner-City Poverty in the United States. Edited by L.E. Lynn, Jr. and M.G.H.
McGeary. Washington, DC: National Academy Press. pp. 111-186.
Jones, LeAlan, and Loyd Newman (1998) Our America: Life and Death on the South Side of
Chicago. New York: Washington Square Press.
Kain, John F. (1968) “Housing Segregation, Negro Employment, and Metropolitan Decentralization.”
Quarterly Journal of Economics. 82(2): 175-197.
Katz, Lawrence F., Jeffrey R. Kling, and Jeffrey B. Liebman (2001) “Moving To Opportunity In
Boston: Early Results of a Randomized Mobility Experiment.” Quarterly Journal of Economics.
116: 607-654.
Kawachi, Ichiro and Lisa Berkman (2001) “Social ties and mental health.” Journal of Urban Health
78, 458-467.
Kessler RC, Chiu WT, Demler O, Walters EE (2005) “Prevalence, Severity and Comorbidity of 12month DSM-IV Disorders in the National Comorbidity Survey Replication.” Archives of
General Psychiatry. 62(6): 586-696.

68
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Kessler, R. C., Ustun, T. B. (2004). “The World Mental Health (WMH) Survey Initiative Version of
the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI).”
International Journal of Methods in Psychiatric Research 13, 93-121.
Kessler, R. C., Merikangas, K. R. (2004). “The National Comorbidity Survey Replication (NCS-R).”
International Journal of Methods in Psychiatric Research 13, 60-68.
Khadduri, Jill, Mark Shroder, and Barry Steffen (1998) “Welfare Reform and HUD- Assisted
Housing: Measuring the Extent of Needs and Opportunities.” Paper presented at June 26
Conference on Managing Affordable Housing Under Welfare Reform: Reconciling Competing
Demands. Washington, D.C.
Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz (2007) “Experimental Analysis of
Neighborhood Effects.” Econometrica. 75(1): 83-119.
Kling, Jeffrey R., Jeffrey B. Liebman, and Lawrence F. Katz (2001) “Bullets Don’t Got No Name:
Consequences of Fear in the Ghetto.” Discovering Successful Pathways in Children's
Development: Mixed Methods in the Study of Childhood and Family Life. Edited by Thomas S.
Weisner. Chicago: The University of Chicago Press.
Kling, Jeffrey R., Jens Ludwig, and Lawrence F. Katz (2005) “Neighborhood Effects on Crime for
Female and Male Youth: Evidence from a Randomized Housing Voucher Experiment.” Quarterly
Journal of Economics. 120(1):87-130.
Kling, Jeffrey R. and Jeffrey B. Liebman (2004) “Experimental Analysis of Neighborhood Effects on
Youth.” Princeton, NJ: Princeton University Industrial Relations Working Paper # 483.
Knudsen, Eric I., James J. Heckman, Judy Cameron and Jack P. Shonkoff (2006) “Building
America’s future workforce: Economic, neurobiological and behavioral perspectives on
investment in human skill development.” Proceedings of the National Academy of Sciences (in
press).
Kotlowitz, Alex (1992) There are no children here. New York: Anchor Books.
Labov, William. 2001. Principles of linguistic change: Social factors. Oxford and NY: Blackwell.
Labov, William, Paul Cohen, Clarence Robbins and John Lewis. 1968. A Study of the NonStandard English of Negro and Puerto Rican Speakers in New York City. Cooperative Research
Report 3288, vols I and II. Philadelphia: US Regional Survey (Linguistics Laboratory,
University of Pennsylvania).
Labov, William. 1972. Sociolinguistic Patterns. Philadelphia: University of Pennsylvania Press.
Labov, William. 1966. The Social Stratification of English in New York City. Washington, DC:
Center for Applied Linguistics.

69
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Leventhal, Tama, and Jeanne Brooks-Gunn (2000) “The Neighborhoods They Live In: The Effects of
Neighborhood Residence on Child and Adolescent Outcomes.” Psychological Bulletin.
126(2):309-337.
Lippi-Green, R. (1997). English with an Accent: Language, Ideology and Discrimination in the
United States. New York: Routledge.
Loewenstein, George and Ted O’Donoghue (2004) “Animal Spirits: Affective and Deliberative
Processes in Human Behavior.” Working Paper, Department of Social and Decision Sciences,
Carnegie Mellon University.
Ludwig, Jens, Greg J. Duncan, and Paul Hirschfield (2001) “Urban Poverty and Juvenile Crime:
Evidence from a Randomized Housing-mobility Experiment.” Quarterly Journal of Economics.
116(2): 655-680.
Ludwig, Jens, Greg J. Duncan and Joshua C. Pinkston (2005) “Housing Mobility Programs and
Economic Self-Sufficiency: Evidence from a Randomized Experiment.” Journal of Public
Economics. 89(1): 131-156.
Ludwig, Jens, Helen Ladd, and Greg J. Duncan (2001) “The Effects of Urban Poverty on Educational
Outcomes: Evidence from a Randomized Experiment.” In Urban Poverty and Educational
Outcomes. Brookings-Wharton Papers on Urban Affairs. Edited by William Gale and Janet
Rothenberg Pack. Washington, DC: Brookings Institution Press. pp.147-201.
Ludwig, Jens and Jeffrey R. Kling (2007) “Is Crime Contagious?” Journal of Law and Economics.
Forthcoming.
Luttmer, Erzo F.P. (2005) “Neighbors as Negatives: Relative Earnings and Well-Being.” Quarterly
Journal of Economics. 120(3): 963-1002.
Manski, Charles F. (1989) “Anatomy of the Selection Problem.” Journal of Human Resources.
24(3):343-360.
Manski, Charles F. (1990) “Nonparametric Bounds on Treatment Effects.” American Economic
Review Papers and Proceedings. 80(2):319-323.
Manski, Charles F. (1995) Identification Problems in the Social Sciences. Cambridge: Harvard
University Press.
Manski, Charles F. (2000) “Economic Analysis of Social Interactions.” Journal of Economic
Perspectives. XIV:115-136.
Massey, Douglas S. and Garvey Lundy (2001) “Use of Black English and Racial Discrimination in
Housing Markets: New Methods and Findings.” Urban Affairs Review. 36(4): 452-469.

70
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Moffitt, Robert A. (2001) “Policy Interventions, Low-level Equilibria, and Social Interactions.” In
Social Dynamics. Edited by Steven Durlauf and Peyton Young. Cambridge, MA: MIT Press.
Murnane, R., Levy, F. and Willett, J. (1995) “The Growing Importance of Cognitive Skills in Wage
Determination.” Review of Economics and Statistics. 44(3): 451-66.
Newman, Sandra J. and Ann B. Shnare (1997) "‘... and a Suitable Living Environment’: the Failure
of Housing Programs to Deliver on Neighborhood Quality.” Housing Policy Debate. 8(4): 70341.
Olsen, Edgar O. (2000) “The cost-effectiveness of alternative methods of delivering housing
subsidies.” Department of Economics, University of Virginia, Charlottesville, VA.
Olsen, Edgar O. (2003) “Housing Programs for Low-Income Households.” In Means-Tested Transfer
Programs in the United States. Edited by Robert A. Moffitt. Chicago: University of Chicago
Press. pp. 365-442.
Orr, Larry, Judith D. Feins, Robin Jacob, Erik Beecroft, Lisa Sanbonmatsu, Lawrence F. Katz,
Jeffrey B. Liebman, and Jeffrey R. Kling (2003) Moving to Opportunity Interim Impacts
Evaluation. Washington, DC: U.S. Department of Housing and Urban Development, Office of
Policy Development and Research.
Orr, Larry (1999) Social Experiments. New York: Russell Sage.
Pollack, Judy, Michelle Najarian, Donald Rock, Sally Atkins-Burnett, Elvira Germino Hausken
(2005). Early Childhood Longitudinal Study, Kindergarten Class 1998-99 (ECLS-K):
Psychometric Report for Fifth Grade. Washington, DC: National Center for Education Statistics,
U.S. Department of Education.
Providing Incentives to Survey Respondents: Final Report (Council of Professional Associations on
Federal Statistics, September 1993), p. 10.
Quigley, John M. (2000) “A Decent Home: Housing Policy in Perspective.” In Brookings-Wharton
Papers on Urban Affairs. Edited by William G. Gale and Janet Rothenberg Pack. Washington,
DC: Brookings Institution Press. pp. 53-100.
Ridker, P.M., Buring, J.E., Cook, N.R., Rifai, N. (2003). C-reactive protein, the metabolic syndrome,
and risk of incident cardiovascular events. Circulation 107: 391-7.
Rivkin, Steven G., Eric A. Hanushek and John F. Kain (2005) “Teachers, Schools, and Academic
Achievement.” Econometrica. 73(2): 417-458.
Rosenbaum, James E. (1992). “Black Pioneers: Do Their Moves to the Suburbs Increase Economic
Opportunity for the Mothers and Children?” Housing Policy Debate. 2:4, 1179-1213.

71
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Rosmond, R. (2005). “Role of stress in the pathogenesis of the metabolic syndrome.”
Psychoneuroendocrinology, 30, 1-10.
Rubinowitz, Leonard S. and James E. Rosenbaum (2001) Crossing the Class and Color Lines: From
Public Housing to White Suburbia. Chicago: University of Chicago Press.
Ryu, Erica, Mike P. Couper, and Robert W. Marans (2005). “Survey Incentives: Cash vs. In-Kind;
Face-to-Face vs. Mail; Response Rate vs. NonResponse Error” International Journal of Public
Opinion Research Vol.18 No.1 (July)
Sampson, Robert (2003) “The Neighborhood Context of Well-Being.” Perspectives in Biology and
Medicine. 46(3): S53-64.
Sampson, Robert J., Jeffrey D. Morenoff, and Thomas Gannon-Rowley (2002) “Assessing
`Neighborhood Effects’: Social Processes and New Directions in Research.” Annual Review of
Sociology. 28: 443-478.
Sampson, R. J., S. Raudenbush, and F. Earls (1997) “Neighborhoods and Violent Crime: A Multilevel Study of Collective Efficacy.” Science. 277 (August 15):918-924.
Sanbonmatsu, Lisa, Jeffrey R. Kling, Greg J. Duncan and Jeanne Brooks-Gunn (2006)
“Neighborhoods and Academic Achievement: Results from the Moving to Opportunity
Experiment.” Journal of Human Resources. 41(4): 649–691.
Sherman, Lawrence (2003) “Fair and Effective Policing.” Crime: Public Policies for Crime Contro.
Edited by James Q. Wilson and Joan Petersilla. Oakland, CA: ICS Press. pp.383-412.
Shonkoff, Jack P. and Deborah A. Phillips (2000) From Neurons to Neighborhoods: The Science of
Early Childhood Development. Washington, DC: National Academy Press.
Shroder, Mark, and Arthur Reiger (2000) “Vouchers Versus Production Revisited.” Journal of
Housing Research. 11(1): 91-107.
Silverman, Irwin W. (2003) “Gender Differences in Delay of Gratification: A Meta-Analysis,” Sex
Roles. 49: 451-463.
Singer, E. “The Use of Incentives to Reduce Nonresponse in Household Surveys.” In: Groves RM,
Dillman DA, Eltinge JL, Little RJ., editors. Survey Nonresponse. New York: Wiley; 2002. pp.
163–78.
“The Use of Incentives to Reduce Nonresponse in Household Surveys,” in Survey Nonresponse, eds.
Robert M. Groves, Don A. Dillman, John L. Eltinge, and Roderick J. A. Little, (Wiley, 2002) pp.
163-77.
Wilson, William Julius (1987) The Truly Disadvantaged: The Inner City, The Underclass, and Public
Policy. Chicago: University of Chicago Press.
72
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods

Wolfram, Walt. 1969. A Sociolinguistic Description of Detroit Negro Speech. Arlington,
VA: Center for Applied Linguistics.

73
National Bureau of Economic Research
University of Michigan – ISR

PART B: Collection of Information Employing
Statistical Methods