Research Design, Data Collection And Analysis Plan

CHOICE NEIGHBORHOODS DEMONSTRATION
STUDIES – TASK ORDER #1

RESEARCH DESIGN, DATA COLLECTION
AND ANALYSIS PLAN
MARCH 2, 2012

Submitted to:
U.S. Department of Housing and Urban Development
451 Seventh Street, SW
Washington, DC 20401

Contract No. C-CHI-01127/CHI-T0001; GS23F-8198H
UI No. 08322-021-00

2100 M Street, NW ● Washington, DC 20037

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Table of Contents
Part 1: Introduction and Overview ............................................................................................................... 3
1.1

Purposes, Framework, Structure, and Data Sources .................................................................... 3

1.2

Schedule ........................................................................................................................................ 4

Part 2: Research Design ................................................................................................................................ 6
2.1

Implementation ............................................................................................................................ 6

2.1.1

Goals, Objectives, and Planned Activities of the Intervention ............................................. 6

2.1.2

Implementation .................................................................................................................... 9

2.1.3

Leverage and Catalytic Investment ..................................................................................... 11

2.1.4

Random Assignment Opportunities .................................................................................... 13

2.2

Residents of the Assisted Project and the Choice Neighborhood .............................................. 13

2.2.1

Resident Conditions to be Monitored................................................................................. 15

2.2.2

Objectives and Research Questions.................................................................................... 15

2.3

Housing ....................................................................................................................................... 20

2.3.1

Housing Condition and Site Design ..................................................................................... 21

2.3.2

Housing Quality of Assisted Development.......................................................................... 22

2.3.3

Housing Quality in the Neighborhood ................................................................................ 25

2.3.4

Housing Finance for Redevelopment .................................................................................. 25

2.3.5

Housing Management ......................................................................................................... 26

2.4

Neighborhoods............................................................................................................................ 27

2.4.1

Neighborhood Conditions to be Monitored ....................................................................... 27

2.4.2

Objectives and Research Questions.................................................................................... 28

2.4.3

Data Sources ....................................................................................................................... 31

2.4.4

Analytic Approach and Data Sources by Research Topic .................................................... 31

2.5

Analysis Plan and Reporting ........................................................................................................ 37

2.5.1

Transformation Plans: Characteristics and Critique ........................................................... 38

2.5.2

Identifying Opportunities for Embedded Experiments ....................................................... 40

2.5.3

Documenting Baseline Conditions ...................................................................................... 41

1

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2.5.4

Documenting Near-Term Changes for Residents, the Project, and the Neighborhood ..... 42

2.5.5

Progress in Implementing the Transformation Plan ........................................................... 43

2.5.6

Reports ................................................................................................................................ 44

Part 3: Data Collection Plan ........................................................................................................................ 47
3.1

Document Review ....................................................................................................................... 48

3.1.1
3.2

Description .......................................................................................................................... 48

Baseline Surveys of Residents of the Assisted Project and the Neighborhood .......................... 49

3.2.1

Description .......................................................................................................................... 49

3.2.2

Data Collection Instruments & Questions .......................................................................... 50

3.2.3

Sampling and Fielding the Surveys...................................................................................... 52

3.2.4

Resident Tracking ................................................................................................................ 54

3.3

Key Informant Interviews............................................................................................................ 55

3.3.1

Description .......................................................................................................................... 55

3.3.2

Topics .................................................................................................................................. 56

3.4

Observations of Key Community Meetings ................................................................................ 58

3.4.1

Description .......................................................................................................................... 58

3.4.2

Rationale ............................................................................................................................. 58

3.4.3

Data Collection Instruments & Questions .......................................................................... 58

3.5

Focus Groups............................................................................................................................... 60

3.5.1

Description .......................................................................................................................... 60

3.5.2

Data Collection Instruments & Questions .......................................................................... 60

3.6

Secondary Quantitative Data ...................................................................................................... 61

3.6.1
3.7

Description .......................................................................................................................... 61

Block Front and Pedestrian Routes Surveys ............................................................................... 66

3.7.1

Description .......................................................................................................................... 66

3.7.2

Data Collection Instruments & Questions .......................................................................... 67

References .................................................................................................................................................. 71
Appendices A-F appear in a separate document.

2

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Part 1: Introduction and Overview
1.1 Purposes, Framework, Structure, and Data Sources
This document is the Research Design and Data Collection and Analysis Plan (RD/DCAP) proposed by the
Urban Institute (UI) team for completing BPA Task Order No. C-CHI-00127/CHI-T0001, comprising
baseline data collection and process evaluation for the first Choice Neighborhoods Initiative
implementation sites. As specified in HUD’s Statement of Work for the study:
“The objective of this Task Order is to document baseline conditions in the neighborhoods receiving
implementation grants under the FY 2010 Choice Neighborhoods Demonstration, describe the
revitalization efforts in those neighborhoods, and document the early outcomes of these efforts.”

The purpose of the RD/DCAP is to provide a detailed plan for the research, data collection, and analysis
that will guide the research team’s decision-making over the next two and one-half years of the study.
We use a combined RD/DCAP rather than separate documents because the Task Order focuses so
centrally on background data collection.
Our research sites include three of the five FY2010-11 implementation grantees: Chicago (Grove Parc
Plaza in the Woodlawn neighborhood), New Orleans (Iberville/Tremé), and Boston (Woodledge/Morrant
Bay in the Quincy Corridor). The budget for this Task Order does not include sufficient resources for all
five implementation sites. San Francisco (Alice Griffith in the Eastern Bayview neighborhood) was
excluded because of the inclusion of that project within HOPE SF, whose progress is already being
documented by a partially overlapping evaluation sponsored by the Enterprise Foundation. Seattle
(Yesler Terrace) was excluded because its original residents have already been relocated and because its
implementation grant funds only the first phase of its long-term vision.
To address HUD’s research objectives for this task order, we will employ a multi-method data collection
strategy aimed at providing us with all the information necessary to answer HUD’s research questions.
Our process begins with HUD’s research questions. We have reframed many of these research questions
3

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

as described at length in the Research Design to account for the complexity of the implementation, the
logic of data collection, and the richer understanding of the implementation projects that has emerged
since the Task Order’s research questions were written.
Each of our reframed questions requires multiple methods of data collection, but many of these
methods contribute to more than one research question. The development of the data collection
instruments (presented in the Appendices to this RD/DCAP) reflects their service to multiple research
questions. Analysis of the data collected using these instruments, likewise, will reflect their contribution
to answering multiple research questions. Finally, the data analyzed from multiple data collection
methods will be synthesized in baseline and final reports to HUD. The reports will refer back to our
reframed research questions, organized to correspond to the sections laid out in this Research Design
(implementation, housing, residents, and neighborhood).
The main sections of this RD/DCAP follow this reclassification. The RD/DCAP is complex because some
data collection methods contribute not only to more than one research question in the same major
group but often to questions in more than one group. (For example, the resident survey will contribute
to baseline data collection on housing, support systems, and opportunity and livability.) To account for
this, each section in Part 2 of the RD/DCAP describes the data that will be gathered from each data
collection method used to answer those questions. A more comprehensive description of each method
appears in Part 3 of the RD/DCAP. The data collection instruments themselves are available in
Appendices A-E.
We will collect primary data in surveys, interviews, focus groups, and site observations; we will also
gather and analyze primary data collected by others and review and incorporate findings of secondary
studies.

1.2 Schedule
This project began on September 28, 2011 and is scheduled to be completed three years later on
September 27, 2014. The detailed schedule appears in the project’s Management and Work Plan (Urban
Institute, 2011). Major milestones are as follows:
03/02/2012 – complete final Research Design, Data Collection and Analysis Plan and OMB
Package.
03/23/2012 – complete first site visit
09/28/2012 – complete draft of Baseline Report
11/09/2012 – complete final Baseline Report (based on document review and first site visit)
11/21/2012 – estimated start of baseline residents survey (assuming OMB approval by
8/31/2012)
02/15/2013 – complete residents survey data collection
03/15/2013 – complete second site visit

4

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

05/31/2013 – complete follow-up data collection instruments for site visit (e.g., focus group
guides, interview guides) if necessary
02/21/2014 – complete third site visit and all primary and secondary data collection
03/28/2014 – briefing to HUD on analysis and early results
05/09/2014 – complete first draft of Final Report
06/20/2014 – complete second draft of Final Report
08/01/2014 – complete final version of Final Report
09/26/2014 – complete post-report briefings and closeout activities

5

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Part 2: Research Design
Part 2 describes our approach to the research on this first Choice Neighborhoods Demonstration Studies
Task Order. In it, we regroup the Research Questions from the Request for Quotation into four key
areas: the implementation of the Initiative; the residents in both the focal development and in the
Choice neighborhood; the focal housing developments; and the Choice neighborhoods, linking each
domain to the research questions and the data collection methods we will use to answer them. Part 3
will describe the data collection methods in further detail.

2.1 Implementation
The research design for Implementation questions reorganizes the eight questions in the Task Order into
a series of different questions. We have further regrouped the research questions into four main sets of
questions: Goals and activities, implementation, leverage and catalytic impact, and opportunities for
random assignment. As redevelopment projects sometimes take unexpected directions, which also
influence what is possible to research within those projects, these questions represent desired
directions for the qualitative research, data permitting. The overarching goals of the implementation
section are to establish a baseline for future observers and to answer some of the most critical policyrelevant questions represented in the early stages of the Initiative.

2.1.1 Goals, Objectives, and Planned Activities of the Intervention
Task Order research questions: What are the goals of each grantee’s program? What particular
problems do they intend to remedy? What types of activities are funded?
The first set of research questions capture the grantees’ goals, the problems they intend to remedy,
their concrete objectives, and the activities included in their applications as mechanisms to reach those

6

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

objectives. In all, the answers to these questions will allow us to present a clear and detailed description
of the current design of a treatment that is meant to have the impact on people, housing, and
neighborhoods. Put more simply, the question asks: What activities do you intend to carry out, and
why? The answer to this question may change over time for a number of reasons. Some activities may
become unfeasible (for example, because of insufficient funds). Some activities may lose appeal because
they no longer seem likely to achieve the stated goals. In some cases, goals may shift, rendering some
activities irrelevant. Documenting the grantees’ logic at the point of departure, therefore, is important,
but it is not definitive. On the contrary: changes to activities are to be expected as resources, problems,
and goals shift, and our research design (with repeated site visits and ongoing engagement with the site
via an on-site employee) reflects that expectation.
The Choice Neighborhoods Initiative (CNI) is both a place-based and a person-based approach, and its
interpretation and goals vary from one site to another. We account for this diversity first by the
formulation of subject-specific research teams whose charges are not only to document conditions at
baseline but also to understand the interventions that are proposed. The initial review of grantees’
planning documents will be undertaken in a systematic way both to catalog conditions and to identify
the principal investments made. The issue of level of resources is important both for Choice and for
other place-based initiatives, where it has been observed that a certain threshold or saturation of
investment may be necessary to achieve population-level spillover effects (Accordino, Galster and Tatian
2005). This cataloging will be exploited for the development of our illustration of the “theory of change”
underlying each grantee’s program, as described in the Analysis section (2.5) below.
Different stakeholders both within and outside the Initiative will differ in how they frame goals and
objectives. They may also differ in their assessment of feasibility of each activity and the likelihood of its
having the desired outcome. These differences will not necessarily undermine the program; rather,
these frames represent orientations toward community work which may need to be negotiated over the
course of program implementation (Schon and Rein 1995). One frame that may be particularly
important is the extent to which local goals relate to the revitalization of the site in question, and how
much they extend to broader, neighborhood-level strategies; these may inform subsequent research
questions on coordination, policy reform, and leverage. Another is how varied stakeholders see the goal
of creating a mixed-income neighborhood; while a powerful component of Choice’s overall objectives,
perceptions about this goal can vary widely, be understood in different ways, and mean different things
in practice (Joseph and Chaskin 2010, Joseph, Chaskin and Webber 2007). We will attempt to develop
faithful theories of change for each site based on what we learn about goals, objectives, activities, and
anticipated outcomes. In some sites, there may be competing narratives, which we will not attempt to
reconcile but will simply discuss. The discussion of theories of change will appear in the Baseline report.
Because of Choice’s flexible design, the rationale and justifications for the local intervention will differ in
varied settings. An advantage to the three-site study is its ability to compare how the CNI
Implementation plan addresses different neighborhood problems according to local challenges, and as
they may relate to specific opportunities. Understanding variation in context-specific problems may help

7

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

provide additional evidence about the Initiative’s potential application to different settings. Our
understanding of the way in which setting matters will be reflected in a cross-site section of the Final
Report.
Much of the data related to goals may be collected through document review of planning materials and
other public information. However, in-depth interviews with key stakeholders may reveal additional
detail not captured in these plans. Interviews may also capture how different sets of stakeholders
(including residents, private developers, mayoral and housing administrations) view goals as they relate
to Choice’s neighborhood, housing, and people components. These questions may also be addressed
through focus groups.
Available documents: How are the grantee’s objectives for the Choice Neighborhood
transformation made clear in the application and other internal and public documents? How
widely distributed and easy to access are these documents? How much of the applicant’s idea of
the “treatment” is captured within the application versus contained in supporting documents?1
Plans and programs: In each of the sites, according to grantee plans and informants, what are
specific program goals and programmatic features related to people, neighborhoods, and
housing? Topics to be explored more fully here include, for example:
o What are the goals, objectives, and programmatic features for the housing
redevelopment? What are the grantee’s goals for homeownership and more broadly for
tenure in the new community?
o What are the goals, objectives, and programmatic features for the residents of the focal
development and people elsewhere in the neighborhood? How do sites see tenant rates
of return, and the goal of influencing resident economic trajectories? How do they view
the need for physical integration of residents within the development and within the
neighborhood?
o What are the goals, objectives, and programmatic features for the neighborhood as a
whole? What are the grantee’s community-building goals? What kind of community or
communities does the grantee envision? What are the presumed links between the
bricks and mortar investments in the neighborhood and changes in community, safety,
and other social elements?
o How do these three sets of goals, objectives, and implementation support one another
or conflict with one another? Where are the most resources proposed to be committed,
and what justifies these different emphases?
Influences and context: How did the grantees arrive at their transformation plan and its
implementation programs, and how is it distinctive? Some key topics here may include:
1

Throughout this document, we refer to the grantee’s application and their transformation plan interchangeably.
In the baseline and final reports, we will distinguish the materials in the application from supporting materials and
plans. The relevant binding document for each site, furthermore, is the grant agreement, which we will also review
as we develop the baseline report assuming its availability to us.

8

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

o

How do the local grantee’s experiences with other affordable housing, community
development, and neighborhood transformation processes (both locally and federally
generated) influence their strategies for redevelopment in the Choice Neighborhood?
o In what ways do the goals held by local actors for the Initiative resemble or depart from
their goals for the city’s other low-income neighborhoods?
o How did public participation and resident involvement influence the plan and programs?
Framing: How do various stakeholders differ in their perspectives on the intervention?
Important topics here may include:
o How are goals of the Initiative broadly framed by different actors within the
neighborhood? What are the justifications for the intervention, and how do these vary
among CN sites?
o How do different actors envision Choice’s overarching vision of creating sustainable,
mixed-income communities?
o In each of the sites, do different stakeholders frame these broad goals differently?
Where are balances struck between social and market goals, where is there the greatest
amount of consensus, and where is the greatest disagreement?
o Are resources adequate to achieve these goals, in the views of different stakeholders?

2.1.2 Implementation
Task order research questions: How effectively are the various activities coordinated? What policy
reforms and innovations accompany the intervention? What is the impact of the flexible design of the
Choice Neighborhoods program’s performance? How does the grantee utilize partnerships to improve
program performance?
Our approach to evaluate the early years of implementation will answer all these Task Order questions,
but we embed our answers in a set of other questions about progress in implementation. These topics
roughly correspond to a sequence or chronology related to early implementation – 1) planning for
Choice and how it influenced early implementation, 2) early decisions required to move Choice toward
its broad objectives; 3) programmatic coordination within Choice and accompanying systems reform.
A grounding implementation question is how have Choice planning processes affected early
implementation? Illustrative sub-questions in this topic, addressed through document analysis and key
informant interviews, include:
Which important interagency relationships were established in planning that have advanced or
slowed early implementation efforts? Which relationships were not established that are now
important to build as preconditions to implementation?
How did the engagement of residents in planning (or the lack thereof) affect their current level
of engagement? How has resident engagement in planning affected progress in
implementation?

9

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

A second research question addresses Choice’s broadest objectives, and early implementation steps
taken to achieve them: how do implementers move toward Choice’s overarching goal of establishing
sustainable, mixed-income neighborhoods of opportunity? Illustrative operational questions that
pertain to this research topic, which we will answer through document analysis and key informant
interviews, may include:
What are the grantee’s previous experiences with building mixed-income communities,
including, for example, under HOPE VI? Do other partners on the CNI team have experience
building such communities?
What are different actors’ overarching visions for a sustainable, mixed-income neighborhood?
What logical sequence of decisions is necessary for satisfactory progress in the redevelopment?
What implementation activities were already underway at the local site when HUD announced
the award? Which local partners were most engaged, and what was their role?
What other implementation activities took place up to early 2014?
A third research question on implementation aligns with two Task Order questions: How effectively are
the various activities coordinated, and how does the grantee utilize partnerships to improve program
performance? This line of inquiry will examine how these partnerships overcome any limitations among
core implementing actors, the challenges of coordination among many partners, and how they are
overcome. Another area of interest related to coordination and partnerships relates to how local
coordination adds value to the program as a whole, distinguishing co-located from coordinated
activities. Here, illustrative questions may include:
What capacities do implementing actors have to manage or carry out major components of the
plan? To what extent do actors (private developers, housing authorities) need to take on new
roles?
How, if at all, do partnerships complement existing capacities, especially in the areas of
education and workforce development?
What are formal mechanisms and governance structures for promoting these partnerships, and
how do they function?
What are challenges of coordination among multiple partners and agencies, and how do
grantees manage these tensions? How do they overcome regular challenges of engagement,
communication, and consensus building within the Choice context?
How are actors able to coordinate either individual programs or coordinate within larger
projects to maximize advantages to participants? Where are shortfalls in this coordination?
What, if any, are the critical missing partners whose absence or even opposition threatens the
success of the grantee’s initiative? Were efforts made unsuccessfully to engage these partners
in the past? Are efforts currently underway to bring these critical missing partners into
collaboration?

10

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

A final research question reflects another Task Order question: What policy reforms and innovations
accompany the intervention? Finally, we will explore two types of policy reforms and systems changes
that may be associated with Choice: the identification of plans, policies, regulations, and procedures
that need changing as a precondition to implementation of the grantee’s initiative, and how well
different types of systems (such as education and housing) are integrated and coordinated as a result of
the CNI intervention. Since many policy reforms and systems changes will occur at the same time as the
Choice intervention, we will necessarily be limited in the scope of this analysis, but we will profile a small
number of key changes that appear to be very closely connected with each grantee’s CNI work.
Illustrative questions include:
What reforms or system changes were necessary to allow implementation to proceed? If any
reforms or system changes were necessary but did not occur, why not?
To what extent did the CNI intervention foster attempts in the first two years of implementation
to integrate and coordinate different systems, such as housing and education? To what degree
did such attempts succeed?

2.1.3 Leverage and Catalytic Investment
Task Order research questions: To what extent does the Choice Neighborhoods investment leverage
other funding? To what extent does the Choice Neighborhood investment catalyze private investment in
unassisted units?
“Leverage” is an important component of Choice and other place-based initiatives (Mooney 2010).
However, leverage is often difficult to track, as various definitions have been applied to the measure.
Some define leverage simply as additional investments associated with a particular project, even those
that would occur in the absence of that project. Others define leverage more strictly, as investment that
is induced by another. That definition would be met, for example, if a foundation committed resources
in year 2 of the implementation to help create a mixed-income neighborhood. By an even stricter
definition, leverage involves funding that is not simply redirected or reallocated from other public or
private sources but rather added to the sum of investments made for the transformation. This might
apply to a private developer who would not otherwise have considered making an investment in the
neighborhood (or elsewhere) were it not for the presence of Choice.
This Task Order tracks only the first two and one-half years of implementation, reducing our ability to
observe much in the way of leverage or catalytic impacts, which will presumably ensue over the course
of many years. We propose taking on the leverage questions in two distinct ways, in two distinct
deliverables as described in the Analysis section (2.5).
First, we will address the leverage questions in part as we answer questions about the activities planned
by each grantee for its Choice Neighborhoods Initiative (Section 2.1.2 above). In addressing that section,
we will ask the key informants whether and when particular planned investments in both housing and

11

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

neighborhoods more generally would have occurred in the absence of the Choice Neighborhoods
Initiative. An additional question for the Initiative is the way that different stakeholders perceive this
potential influx of new housing investments. Unlike redevelopment of the project itself, where one–forone replacement and the right of return may provide assurances to those who are concerned about
potential displacement, Choice grantees have presumably less control about the potential implications
for existing residents of additional, private investment that may occur in the Choice Neighborhood.
Perspectives of some stakeholders about the impact of the investment may anticipate the potential for
displacement of established tenants. They may also fear that relocation of residents of the focal
development to other parts of the neighborhood (which may occur in all three of the sites under study
here) will impede new investments elsewhere because they believe these tenants may increase levels of
disorder on their new blocks. We will develop models of the grantees’ and other stakeholders’ theories
of change illustrating understandings of connections among the planned activities, emergent synergies
among activities (i.e., the way in which the whole is greater than the sum of the parts), and potential
impacts on neighborhood conditions that might moderate the possibility of leverage. The “theory of
change” discussion will appear in the baseline report.
Second, we will address the leverage questions by observing what actually occurs over the course of the
Task Order. Here, we will concentrate more explicitly on whether (a) planned activities within the
Initiative not funded by HUD have advanced as fast as or faster than expected thanks to HUD funding,
and (b) public and private investments not described in the grantees’ applications have materialized that
can be logically attributed in some way to the grantee’s initiative. Our analysis of leverage as “progress
on the ground” will be integrated as appropriate into the sections of the Final Report on the
Neighborhood and Assisted Housing.
Illustrative sub-questions to be answered through stakeholder interviews and document review may
include, for example:
Where, over the first two and one-half years of implementation, are resources actually spent?
How do they line up to initial plans as captured in the applications? If the spending does not
match initial plans, then why not?
Overall, what are the various funding streams associated with the Choice investment? To what
extent are there in-kind or shifted public resources, and how much are private investments?
What has been the sequence of investment decisions, especially but not only those in housing
(i.e., do Choice investments precede others)? How much does the Choice investment induce
these other investments? Can this inducement be attributed to the grantee’s programs and
investments?

12

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2.1.4 Random Assignment Opportunities
Task order questions: N/A.
An array of potentially innovative “people”-based strategies is planned as part of local CNI
programming. As part of its initial scan of developments and funded activities, the research team will
examine whether any of these are suitable for random assignment or quasi-experimental evaluation.
The scan will be particularly attentive to programs that may test an innovative new model for service
delivery, or programs that test more established models but within a housing-assisted, relocation
context. Researchers will focus on programs that are likely to administer a stable or consistent
treatment over a relatively long period of time (several years) to allow exploration of impacts that may
take time to unfold, where there is likely to be a treatment differential or difference between the
services provided and those accessible to control group members, and where there is sufficient internal
tracking to ensure program participation data may be collected and that participants in a control
condition do not inadvertently receive program services.
Sub-question:
What are potentially innovative programs that are likely to provide high-quality, long-term, and
data-driven services to residents?

2.2 Residents of the Assisted Project and the Choice Neighborhood
This section describes our research design related to collecting baseline data on the residents of the
focal development targeted for redevelopment and of residents of the surrounding community.
The Choice Neighborhoods Initiative seeks to build on the lessons of the HOPE VI program, expanding
focus beyond improving the focal development and following the principles of responsible relocation
and providing adequate support for residents.
There is a considerable body of research on outcomes for HOPE VI relocatees. Research on the first
phase of the program (1992-1998) documented that HOPE VI sites often lost track of original residents
and did not fully implement community supportive services (Popkin, et al. 2004, Buron, et al. 2002).
After 1998, HUD put in place stronger monitoring and oversight of relocation and resident services. The
most comprehensive research on resident outcomes from the second phase comes from the five-site
HOPE VI Panel Study (Popkin, Levy and Buron 2009).2 Overall, relatively few original residents returned
to live in the new, mixed-income communities; most either relocated with vouchers or moved into
other, traditional public housing developments. Still, most are faring better—most are living in higherquality housing in dramatically safer neighborhoods and many report improved mental health. However,
while the neighborhoods they have moved to are not as distressed as their original developments, they
2

A number of single-site evaluations have found similar results. See (Popkin, Levy and Buron 2009) for a review.

13

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

are still high poverty and racially segregated. Further, many former residents are experiencing material
hardship, particularly in being able to afford utility payments. Finally, there is no evidence that HOPE VI
initiatives improved residents’ employment or overall physical health.
The Choice Neighborhoods Initiative seeks to improve on this record, requiring grantees to:
Implement a “mobility” approach to relocation to assist residents with finding housing in areas
with low concentrations of poverty and minority populations.
Put greater emphasis on identifying special needs of public and assisted housing residents and
on providing services and supports to increase families’ well-being, including access to quality
educational institutions from pre-school though secondary school and access to quality health
services.
Provide supportive services to increase employment.
Guarantee right of return for lease compliant residents.
Track relocatees to ensure maintained communication and support.
Choice also seeks to go beyond HOPE VI and improve conditions in the neighborhood around the focal
development. That means that this research needs to go beyond the HOPE VI approach, and explore
conditions for residents of the broader community, both to obtain information on baseline conditions
and resident characteristics to compare to those of the residents of the focal development, and to track
how successful the Choice intervention is in improving conditions over time. There has been less
research that has looked at both revitalization of an assisted development and the broader community;
one early study of public housing revitalization in Chicago compared residents of the focal development
and other neighborhood residents’ perceptions of neighborhood conditions and safety, finding that
even though the other residents were also largely poor, minority households, they were less distressed
on average than the families in the focal development and also reported far lower rates of problems
with social and physical disorder in their neighborhood. Further, residents in the neighborhood reported
much higher rates of collective efficacy (Popkin, Buron, et al. 2000). The Making Connections Cross-site
Survey tracked residents from low-income communities targeted for comprehensive community
initiatives; a key finding from that study was high rates of residential mobility among the lowest-income
residents, raising questions about how much they will benefit from a community-wide intervention
strategy (Coulton, Theodos and Turner 2009).
The work under this task order involves gathering baseline data on residents. The main sources of data
will include the grantees’ documents and administrative records, key informant interviews, our own
baseline survey of tenants, de-identified administrative data with location information collected from
state and local agencies when available, and focus groups among tenants. We provide further detail
about all these data collection methods in the appropriate Data Collection sections in Part 3 of this
document.

14

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

A key challenge for this research effort is that relocation is already underway on the Chicago site and will
have progressed considerably by the time the baseline resident survey is implemented. In order to
gather information from a sufficiently large sample of original residents, we recommend a sampling
strategy that includes both residents still on-site as well as those relocated since January 2012. Although
we used retrospective data collection strategies in the HOPE VI Resident Tracking Study (Buron, et al.
2002), they are not ideal since the passage of time affects accuracy and recall. Instead, we will assess the
housing and neighborhood conditions and well-being of residents from the target development,
whether they are on-site in their original unit, a temporary unit, or their permanent replacement unit; or
in a temporary or permanent unit outside of the focal development.

2.2.1 Resident Conditions to be Monitored
The work under this task order related to residents has three major purposes. First, we will gather
baseline information about the characteristics of the residents of the focal development and
neighborhood residents across a range of domains (e.g. basic demographics, social support,
employment, education, child well-being, health). The goal of collecting this information is so that later
research will be able to assess the impact of the Choice intervention on residents’ social and economic
well-being. Second, we will collect data from both residents of the focal development and neighborhood
residents about housing quality and baseline neighborhood conditions, including perceptions of safety,
collective efficacy, and services and amenities. Finally, because Choice involves relocation and
redevelopment, we will gather baseline information from the residents of the focal development about
interest in returning to the site and satisfaction with relocation services. We will use this information in
our final report to paint a comprehensive picture of the Choice communities at baseline, as well as to lay
the groundwork for future comparisons.

2.2.2 Objectives and Research Questions
Our research questions about residents fall into three broad categories: housing and neighborhood,
relocation and support systems, and opportunity and livability. We have reframed the questions from
the RFQ for the task order to reflect the fact that we are collecting baseline information on the tenants
and some of the key questions of interest about the rate of return and the impact of Choice on
residents’ housing and neighborhood circumstances will need to be addressed under future task orders.
Instead, our design will lay the groundwork for longitudinal follow up and analysis. Because we are
seeking to gather a range of types of information, we expect to use several of the core data collection
strategies, including: the baseline surveys of residents of the focal development and neighborhood
residents; interviews with key informants from the Choice agencies collected during the team’s site
visits; focus groups with residents of the focal development, neighborhood residents, and service
providers; grantees’ documents and administrative records; de-identified administrative data with
location information collected from state and local agencies when available; housing agency data; and

15

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

other secondary data. We discuss our research questions regarding residents below, key challenges, and
our strategies for addressing each.
2.2.2.1 Housing and Neighborhood

As discussed above, the Choice Neighborhoods Initiative seeks to learn from and improve on the HOPE
VI experience. Many of the questions in this section overlap with questions in other domains—housing
conditions (section 2.3) and neighborhoods (section 2.4). The research questions discussed here are
those that reflect issues related to the potential effects of the Choice intervention on residents’ overall
well-being, i.e. whether Choice succeeds in improving housing and neighborhood conditions for the
residents of the focal development and the broader neighborhood.
The first group of research questions has to do with residents’ perceptions of and satisfaction with their
current housing conditions.
What are residents’ perceptions of their housing conditions?
Do they report problems with their housing?
How satisfied are residents of the focal development with their units? With the development?
The baseline research will also gather information about residents’ views about the neighborhood as a
place to live and raise their children. These questions are important not only for assessing how Choice
ultimately affects well-being, but also to help inform our understanding of why residents of the focal
development might choose to return to the redeveloped site. Research on HOPE VI relocatees found
that many who chose not to return were those who had the most negative perceptions of their
neighborhood at baseline. In particular, relocatees’ views about crime and safety were closely related to
their relocation choices (Popkin, Levy and Buron 2009, Comey 2007).
How do residents view their neighborhood as a place to live and raise children?
What are their perceptions of crime and disorder in the neighborhood?
Have they or others in their household been victims of crime in the neighborhood?
What are their perceptions of social cohesion and trust (collective efficacy)?
What are their perceptions about local schools? About other services and amenities (parks,
transportation, stores, etc.)?
What are focal development residents’ perceptions of crime and disorder in the focal
development? Have they or others in their household been victims of crime in the focal
development?
Information on residents’ views about their housing, neighborhood, and the redevelopment effort will
come from several data sources: the baseline survey, which will include items on support networks and
on views about the redevelopment effort; focus groups, which present an opportunity to probe views
about the redevelopment effort in greater depth; and interviews with key informants during site visits.

16

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

We provide greater detail about our plans for gathering information about housing conditions in Section
2.3 below; information about our plans for gathering comprehensive information from both residents of
the focal development and neighborhood residents about neighborhood conditions is described in
Section 2.4.
2.2.2.2 Relocation and Support Systems

Choice grantees are charged with undertaking responsible relocation initiatives that encourage original
residents who are lease-compliant to return to the redeveloped site and that provide tenants with
relocation services that emphasize mobility and opportunity. We will not be able to determine how
successful these efforts are within the timeframe of this task order. The goal of this research is to gather
information on tenants’ characteristics, experiences with the relocation services, social networks, and
attitudes that might affect their choices about relocation and mobility over time. In addition, we seek to
understand how residents—both residents of the focal development and neighborhood residents—
perceive the redevelopment effort and its potential impact on their lives.
The first set of questions in this domain has to do with how both residents of the focal development and
the surrounding neighborhood perceive the Choice Neighborhoods Initiative.
What are resident perceptions of the planned redevelopment effort? Do they view the redevelopment
effort as something intended to benefit them?
How long have they lived in the development? In the surrounding neighborhood?
What are their perceptions of the key actors involved in implementing the Choice intervention?
The second set of questions has to do with relocation strategies and services for the residents of the
focal development. As noted above, relocation will already be underway in Boston, Chicago, and New
Orleans by the time the baseline resident survey is fielded. Assuming we use the proposed strategy of
interviewing those who have relocated within the twelve months prior to the fielding of the baseline, we
will also be able to gather information about actual relocation experiences.
What are residents’ experiences with either temporary or permanent relocation?
What services/supports did the housing agency provide during and after relocation?
What were residents’ choices for temporary or permanent relocation?
How satisfied were residents with their options and the services provided?
For residents who have already relocated, what proportion returns to redeveloped properties?
Where are residents who do not return to the redeveloped property living?
What is their housing assistance status (voucher, public housing/PB S8/unassisted)?

17

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

What are the neighborhood characteristics of the communities where former residents are
living (race, poverty, amenities)
2.2.2.3 Institutional Supports and Services

A third set of questions have to do with the adequacy and use of institutional supports—schools,
supportive services, child care, and health care services—in the community. A key difference between
Choice and HOPE VI is the goal of partnering with and strengthening community institutions beyond the
focal housing development. For the purposes of this baseline research, it is important to understand
what was available in the community at the outset of the Choice Neighborhoods Initiative.
What institutional supports are available to residents of the focal development at the focal housing
development and in the Choice Neighborhood?
What supports are available to community residents?
Do residents of the focal development and neighborhood residents use the same services
(schools, health clinics, child care, etc.?)
What are residents of the focal development and neighborhood residents’ perceptions of the
services? Are they satisfied with the level and types of services provided?
Are there types of services or supports that are missing (i.e., child care, grocery stores, health
services)?
The final group of questions in this domain has to do with residents of the focal development and
neighborhood residents’ informal social networks and support systems.
Do residents of the focal development and other neighborhood residents have informal networks and
support systems (non-institutional, non-governmental)?
To what extent do residents rely on informal support systems within the immediate
neighborhood at baseline?
Do they have connections outside the community?
Are these support networks inclusive of a variety of incomes and racial/ethnic/social groups?
Information on residents of the focal development’ views about institutional supports and social
networks will come from several data sources: the baseline resident survey, which will include items on
support networks and on views about the redevelopment effort; focus groups with residents of the focal
development and neighborhood residents, which present an opportunity to probe views about the
redevelopment effort and the community in greater depth; and interviews with key informants during
site visits.

18

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
2.2.2.4 Opportunity and Livability

Like HOPE VI, the Choice Neighborhoods Initiative seeks to improve the life chances for residents of the
focal development and help them move toward self-sufficiency. At baseline, our design calls for
gathering information about residents’ overall well-being—physical and mental health, as well as about
their educational and economic status. One of the key findings from HOPE VI research was the level of
severe, chronic health problems among assisted tenants, a situation that did not improve over time
(Popkin, Levy and Buron 2009). However, a demonstration program that provided intensive, wrap
around services to vulnerable assisted residents stabilized participants’ health (Popkin, Theodos and
Getsinger, et al. 2010). And the findings from the final evaluation of MTO found that participants who
moved to lower-poverty neighborhoods experienced lower rates of obesity and diabetes than those
who remained in high-poverty areas (Sanbonmatsu, et al. 2011). If the Choice grantees build on the
lessons of HOPE VI and MTO and provide strong, comprehensive supportive services along with the
improvements in the built environment, there is the potential for improvements in residents’ health and
well-being.
At baseline, we will address the following questions:
What is the mental and physical health status of residents of the focal development and Choice
Neighborhood at baseline?
What types of health conditions (asthma, diabetes, obesity) are most prevalent?
What is residents’ access to health care in the community?
What is residents’ level of physical activity? What are the barriers to greater activity (safety, lack
of facilities, quality of parks, etc.)?
Do residents have access to affordable, high-quality food, particularly fruits and vegetables?
What are focal development and neighborhood residents’ attitudes toward their own and their children’s
opportunities at baseline?
If the Choice intervention is successful, it should increase opportunities for local residents—both
residents of the target development and other neighborhood residents. At baseline, we will collect
information from residents about their perceptions of their own and their children’s opportunities in
order to be able to track how these views might change over time.
What is employment status, educational status, and income of residents of the focal development and
Choice Neighborhood at baseline?
We will also collect information from residents about their economic status in order to be able to track
changes over time. For example:
What types of jobs do residents hold?
Do they work in the community or elsewhere?

19

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

What do residents report are the barriers to employment (lack of jobs in the community, lack of
education or skills, lack of child care, poor health, etc.)?
Do children attend school in the community?
What is the level of material hardship experienced by residents of the focal development and
neighborhood residents in the Choice Neighborhood at baseline?
Finally, a key finding from research on HOPE VI relocatees and from MTO is that many former public
housing residents experience substantial economic hardship when they move into the private market
(Popkin, Levy and Buron 2009, Levy 2010, Sanbonmatsu, et al. 2011). In particular, former residents
struggle with paying utility bills and rent payments. To track any impact on hardship, we will gather
information from residents of the focal development about their experience of material hardship at
baseline. Questions include:
What types of hardship do residents report (utility, food, etc.)?
What is the housing cost burden for residents of the focal development? For residents in the
surrounding community?
As described in our data collection plan, information on health and economic status of residents of the
focal development will come from several data sources: the baseline resident survey; focus groups with
residents of the focal development and neighborhood residents, and interviews with key informants
during site visits.
We will also use information from grantees’ documents and administrative records and de-identified
administrative data with location information collected from state and local agencies when available to
track employment and income.

2.3 Housing
This section presents our baseline research design related to the housing units targeted for
redevelopment (including substantial rehabilitation, demolition, or new construction) in conjunction
with local CNI efforts. The housing component of the research primarily focuses on public and assisted
housing units that are considered central to the respective Choice Implementation grants – housing we
refer to as the “focal” development. Where feasible and within scope and budget, however, it takes into
account other public, assisted or unassisted housing units that also are included in local Choice
applications and plans (as discussed in section 2.4, below).
Research questions and data collection pertaining to the housing component fall roughly into three
categories, including:
1. Existing housing conditions and site design, and prospective changes to them as a result of the
respective Choice interventions;

20

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2. The feasibility of the housing redevelopment financing packages that were developed in
conjunction with the Choice grants; and
3. The prospective housing management plans that are anticipated for the redeveloped properties.
The third category was added to the topic areas provided by HUD because of the importance of housing
management planning (and, ultimately, implementation) to the long-term success of Choice
interventions.
Several of the specific housing questions originally stipulated by HUD involve a before-after comparison.
Because most of the housing redevelopment will not have been completed prior to the end of the initial
study period, however, we will not be in a position to observe actual changes brought about through
Choice. Consequently, our research design and data-collection instruments take this into account in
their focus on the provision of primarily descriptive information that can serve as a baseline for
subsequent before-after comparisons.
We have also revised certain research questions to reflect our focus on projected or prospective
changes to housing – including our assessment of the viability of focal redevelopment pro formas and
housing finance projections, of potential energy savings to be realized in the redeveloped focal
properties, and of the prospective housing management plans for the redeveloped focal properties. A
review of projected or prospective financial, energy and management plans allows for (a) baseline
assessment against appropriate benchmarks, (b) tracking of actual efforts (and any deviations) during
implementation, and (c) subsequent empirical comparison of the extent to which they were
implemented to achieve eventual outcomes. Note, however, that it is not useful in all cases to document
current conditions, such as current financial arrangements of the existing focal developments, inasmuch
as they will be substantially overhauled in the process of redevelopment.

2.3.1 Site Design
What is the projected change in the site plan?
Site plans for proposed redevelopments, as incorporated in architectural plans, landscape designs and
engineering drawings, do not necessarily convey the nature and extent of change from the current
configuration. For baseline purposes, therefore, it is important to understand what changes are being
proposed from the current situation and the grantees’ rationale for proposing such changes.
To describe the projected changes in site plans, we intend to answer the following sub-questions:
What is the current layout of the assisted housing site to be redeveloped?
What are the proposed changes to the site plan?
What rationale underlies the changes in the site plan?
What has been the involvement of original residents in planning for redevelopment, and what
effect has resident input had on actual plans?

21

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Determining the current layout of the assisted development will involve reviewing Round 1 and Final
Applications for each site. Data to be collected may include (a) square footage; (b) building types (high
rise, barracks-style, etc.), (c) sidewalks and public spaces, (d) walkways and parking; (e) drainage
facilities, sanitary sewer lines, water lines; (f) lighting, landscaping and garden elements; and (g)
dis/connection with surrounding street patterns and housing. Data on the proposed changes to the site
plan will be drawn from a review of the sites’ final applications, collecting information on these same
data elements.
Where a review of sites’ application documents does not provide information on the rationale for
changing the site plans or where key information is missing, interviews (or focus groups) with those
involved in preparing the applications will be used to fill in the gaps.
We will begin by reviewing the respective sites’ Round 1 and final applications to extract information on
the goals associated with the proposed site plans and other relevant information regarding the rationale
for changing or reconfiguring the sites. Following this review, we will conduct interviews or focus groups
(as appropriate) with one or more of the following: key architectural personnel; public/assisted housing
managers; design professionals, etc. Through both the document review and interviews/focus groups,
we will explore the motivations and goals that guided site planning, such as whether design focused on
physical integration with the surrounding area, social integration among residents of the redeveloped
site, creation of safe spaces, etc.
Data to evaluate the extent of resident involvement and input into the site plans will be drawn from
minutes of planning meetings (where available); interviews or focus groups with lead planners; and
interviews or focus groups with a small number of residents who were involved in planning meetings or
discussions—identified by reputation. These discussions will focus on: (a) the opportunities that were
available for resident involvement in site, housing or management planning; (b) levels of resident
engagement during the planning stages; and (c) the perceived effects of resident input with respect to
site and housing plans (including management plans).
Additional research into resident engagement is discussed previously in section 2.1.2 on
Implementation.

2.3.2 Housing Quality of the Focal Development
What is the projected change in the quality of public and assisted housing?
For baseline assessment purposes, it is important to document the physical quality (condition) of all
properties that are undergoing redevelopment under Choice. Of interest are the conditions of:
residential units; common areas; electrical, plumbing and heating and air-conditioning systems; roofs,
windows and walls; and mechanical systems such as elevators. Physical condition indicators include,
among others, whether units and structures: meet current code requirements; are of a suitable size and

22

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

configuration for the intended occupants; contain mold or rodent infestations; are compatible with
resident safety and security needs; and provide energy efficiently.
To describe projected changes in the physical quality of public and assisted housing developments, we
intend to answer the following sub-questions:
What are the physical conditions of the units and the structure(s) in which they are located?
What are the proposed changes to the units and the structures?
Does the unit meet the bedroom needs of the occupants?
What are the estimated savings in energy costs expected from housing rehabilitation or
redevelopment?
Data will be drawn from a number of sources to collect information on pre-redevelopment conditions.
From the Round 1 and final applications, we will collect the following information if it is available in
application documents: date of construction of the existing structure; the estimated cost of repairing the
existing units; housing quality conditions related to window panes and screens, walls, paint, water
damage, electrical and plumbing systems; and building quality conditions related to the same items.
We will attempt to obtain scores from HUD’s Real Estate Assessment Center (REAC) and score details,
where available. To the extent that Housing Choice Vouchers have been used in the assisted housing
developments, we will attempt to obtain results of recent Housing Quality Standards (HQS) inspections
by the appropriate housing agencies.
From the resident survey we will collect information on the presence of: mold or mildew in respondents’
units or in common areas; rats, mice or cockroaches in respondents’ units or common areas; plumbing
problems or water leaks in respondents’ units or in common areas; damaged windows or doors in
respondents’ units or in common areas; insufficient lighting in common areas’ entryways, stairwells and
hallways, if applicable; and poorly or non-functioning elevators in common areas, if applicable.
Our original task order approach suggested that we would include inspections of housing units in
conjunction with the resident survey in which staff of the survey firm visiting residents would inspect the
units while an interview was taking place. Further investigation has led us to reject this approach,
however. It was unclear whether such staff would have direct access to resident units during the survey
(or, for example, would be asked to wait in the hallway) or, if they did enter the unit, whether they
would be able to inspect spaces other than, perhaps, a living room. The previous experience of our
survey subcontractor in doing similar types of surveys suggested that survey staff cannot always
consistently observe and rate the physical conditions of units. At this point, therefore, we plan to rely on
the applications, REAC data, HQS inspections, and the resident survey to assess housing quality at
baseline.

23

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Additionally, data will be drawn from observations of the building structure made by research staff
during initial site visits. These observations will focus on: (a) lack of building accessibility as evidenced by
broken or missing handrails leading to entryways, if applicable, and ramps or other means of accessing
building entry; (b) evidence of roof damage and damaged or missing gutters and downspouts; (c)
exterior lighting around the building; and (d) weeds and trash on the property.
Data on the adequacy of the existing focal units to meet the bedroom needs of occupants (at baseline)
will be drawn from the resident survey, which will include the following information on preredevelopment conditions: the number of people living in respondents’ apartments; and the number of
rooms in respondents’ apartments.
After discussing options for estimating energy-cost savings with our subcontractor, CNT Energy, it was
decided that establishing a baseline of current energy use across sites will not be of value for future
research. This is because focal developments will change to such an extent as to make pre- and postredevelopment comparisons of little use and because the considerable variation in housing types and
plans across sites would make it difficult to develop a consistent approach to energy analysis. CNT
Energy has recommended an approach that establishes a baseline benchmark of expected energy use.
The benchmark will be based on utility cost projections to be developed based on each site’s focal
development plans for construction and renovation. The cost projections will be compared to actual
energy costs of similar, existing developments. This benchmark can then be used in a future evaluation
as the point of comparison for actual energy costs in the new focal development.
Apart from projections, CNT Energy also will examine actual post-redevelopment energy performance in
the Grove Parc site, in which a number of buildings are already occupied. CNT Energy has data-sharing
agreements currently in place with the appropriate local utility companies, which allow them to request
authorization for utility information related to this development. The assessment of energy
performance CNT Energy will produce for this site can serve as a template for future assessments.
Data to produce the baseline benchmark for anticipated savings generated from reduced energy costs
after redevelopment will be drawn from a number of (a) primary documents, including the Round I and
final applications, building specs, and engineering and energy models, and (b) secondary sources,
including local utility companies. Data will be used to understand and estimate: the energy efficiency
plans for the redeveloped housing; the anticipated post-redevelopment energy cost based on a
consideration of the energy efficiency plan; estimated occupancy; local utility rates and structures, and
data from other similar structures, to be identified once on-site research has begun.

24

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2.3.3 Housing Quality in the Surrounding Neighborhood
What is the projected change in the quality of other (assisted or unassisted) housing units in the target
neighborhood that are part of the Choice applications and plans?
CNI plans may include housing investments beyond the particular focal development, including projector tenant-based assisted and unassisted properties. We also expect housing conditions in the areas
surrounding the focal developments to improve as the Initiative’s housing and commercial investments
proceed. In order to capture these direct and indirect influences in the future, we will collect baseline
data on current housing conditions in the neighborhood to support the future study of change. We will
also project housing quality changes for the properties beyond the focal developments that are included
as part of the Choice Neighborhood plans.
Our findings will be incorporated into the discussion of neighborhood conditions in the baseline report,
so the presentation of the specific sources to support this work is found is in section 2.4.4.3.

2.3.4 Housing Finance for Redevelopment
Is the rehabilitation or redevelopment plan financially feasible?
This question has been revised from the one listed in the RFQ to focus on the financial feasibility of the
redevelopment plans rather than a comparison of redevelopment costs to other programs. The analysis
will include, to the extent possible, the financial feasibility of all housing units included in
redevelopment plans. In order to describe the financial feasibility of the redevelopment plan, we intend
to answer the following sub-questions:
Is the financial plan sound?
How do hard and soft cost projections of the planned housing rehabilitation and redevelopment
compare to other similar projects in the same metropolitan area or region?
Data on the soundness of financial plans will be drawn from Round 1 and final applications as well as
from other available financial documents. Intended as an independent, third party assessment of the
financial feasibility of the planned focal redevelopments, the review will be conducted by our
subcontractor, Strategic Economics, and cover both near-term financial viability and longer-term
financial sustainability. Taking into account local market contexts, Strategic Economics will:
Critically review grantees’ assumptions with respect to projected operating revenues and costs;
Evaluate grantees’ ability to leverage funds from multiple sources; and
Conduct sensitivity analyses of grantees’ financial models, given varying market assumptions.
Data for cost comparison analyses will come from the same financial documents used to address the
financial soundness of the plans and from actual cost data from other developments in areas that are

25

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

similar to the focal developments with respect to scale, income mix, and/or financing structures. The
comparison properties may be wholly unsubsidized or tax-credit developments. Comparisons will be
made on the basis of cost per square foot.

2.3.5 Housing Management
What are the prospective plans with respect to management structures, policies and practices in the
redeveloped properties, and how do these differ from current structures, policies and practices?
We added this question because of the undeniable importance to the success of a housing development
of a range of property management issues (Kingsley, et al. 2004, Abravanel, Levy and McFarland 2009).
The type of management structures, policies and practices that will be put into place will not be
observable until after the baseline effort has been completed. Yet it is important to be able to
document, as part of the baseline study, what thought has been given and what plans have been made
with respect to property management. These will be assessed using professional property management
best practices as benchmarks—to the extent they are relevant to the redevelopment objectives at each
site. The Round 1 and final application documents are generally silent on these issues, although it is
conceivable other documents lay out the plans of subsequent owners, managers, and partners with
respect to how the redeveloped properties will be managed and how this differs from current practice.
To describe current and prospective plans to the management structures, policies and practices, we
intend to answer the following sub-questions:
What are the current management structures, policies and practices in place in focal
developments?
What are the proposed changes to the management structure and policies?
What are the goals, benchmarks and performance measures that grant recipients and
redevelopment owners/managers have explicitly established, otherwise set, or have in mind for
assessing property management outcomes.
To the extent available, information to answer these questions will be gleaned from the Round 1 and
final applications. More importantly, however, information will be collected through interviews with
current (pre-development) and post-development property ownership and management personnel.
Emphasis will be placed on learning about: the current and prospective management models; partner
relationships related to management; and practices related to occupancy (tenant selection), tenant
satisfaction, tenant interactions, turnover, routine maintenance, safety, etc.
We will probe for the logic models that grantees and owners/managers have developed (or have in
mind) for the purpose of being able to measure property management performance. The outcomes of
interest are primarily intermediate, including such indicators as vacancy rates, turnover times, tenant
satisfaction, resident organization, extent of late payments, and other key aspects of management. To

26

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

the extent available, information will be gleaned from the Round 1 and final applications but, more
importantly, it will be collected through interviews with post-development property ownership and
management personnel, including key partners where appropriate.

2.4 Neighborhoods
This section presents our design for research on the neighborhoods that the Choice process is intended
to impact and ameliorate. Sections 2.2 and 2.3 provide in-depth discussions of our research the focal
development and on residents (mainly those in the focal development, but also including selected
aspects of the research on neighborhood residents). The research described in this section is often not
as deep but it will cover a much broader range of topics.

2.4.1 Neighborhood Conditions to be Monitored
This component of the work will document and interpret a host of neighborhood conditions from the
baseline period (2011/2012) until the end of the data collection period of this project (early 2014). The
characteristics selected for monitoring are listed below (6 main topics, 15 subtopics). All of the
neighborhood level topics identified in the research questions and outcomes of interest provided by
HUD in the RFQ are incorporated in this list, but they have been rearranged to relate more directly to
proposed structures for data collection and analysis. Three topics have been added: (1) resident
mobility; (2) changing business activity (commercial services); and (3) more explicit and complete
description of social/human development services.
Residents
Basic demographic conditions (e.g., racial/ethnic diversity, age, household types)
Economic circumstances (e.g., income, employment, education)
Physical and mental health, child wellbeing, and attitudes toward the future
Resident mobility
Housing
Housing stock (quantity, tenure, unit size)
Housing market activity (e.g., vacancies, mortgage market activity, new investment)
Housing quality (facilities and abandoned housing)
Housing hardship (affordability and overcrowding)
Education and Social/Human Development Services
Educational opportunities for youth
Other social/human development services

27

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Social Conditions and Public Safety
Social cohesion and interaction - includes informal networks and support systems (noninstitutional, non-governmental)
Public Safety - Crime, victimization, signs of disorder (graffiti, noise, etc.)
Economy and Commercial Services
Employment opportunities for adults
Commercial and professional services
Built Environment
Neighborhood physical conditions ( road conditions, vacant lots, parks, open space,
playgrounds, urban agriculture, community gardens, sidewalks)
The quality of the pedestrian environment, including actual and perceived pedestrian safety
We recognize that these topics need to be examined together because they interact in the process of
changing outcomes. For example, new residents with much higher incomes moving into the
neighborhood might spur new investment in housing and business establishments, but cause
displacement of existing residents. Alternatively, deterioration of existing housing conditions or
increasing crime rates (brought about by foreclosures and/or other forces) could spur selective
outmigration that would change the composition of the resident base and reduce investment flows into
the neighborhood.

2.4.2 Objectives and Research Questions
The central objective pertaining to neighborhoods in HUD’s evaluation of its Choice Neighborhoods
Initiative is to determine how the interventions that are implemented affect conditions (particularly
measures of well-being) in the surrounding neighborhoods. Because it will not monitor conditions for a
long enough time period and will not incorporate randomized trials, the work called for in this Task
Order cannot address that objective definitively. However, it can provide considerable information that
will be needed for longer term research and that will in itself support reasonable, if rough, assessments
of effects in the interim.
Two research questions will guide this work:
What are the measurable and perceived neighborhood’s conditions and change trajectories
from baseline through early 2014?
What are the mechanisms and forces driving change in the neighborhood through early-2014,
and how are those dynamics likely to impact Choice implementation?

28

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
2.4.2.1 Neighborhood Conditions and Change Trajectories

To address a question like the first one listed above, researchers increasingly recognize that an adequate
“baseline” analysis for a neighborhood must go beyond a presentation of data on conditions at a single
point in time. Rather it needs to cover:
A multi-topic description for the neighborhood of conditions (see list above) and trends in
conditions over a recent period of time, and
A similar description of conditions and change trajectories for: (1) the surrounding city and
region; and (2) several other neighborhoods, selected because of their similarity to the Choice
Neighborhood.
As will be discussed more below, this analysis will be based predominantly on data from secondary
sources (federal, local, and proprietary) as supplemented by information from document reviews, the
resident survey, the block front survey, the pedestrian routes survey, key informant interviews, and
focus groups. As such, it will be able to touch on most, although not all, of the subtopics in the list of
relevant conditions presented above.
In this work, we expect to characterize pre-baseline neighborhood change over the period roughly from
2000 through as close to 2012 as possible. In that overall period, we will attempt where possible to
differentiate conditions and trends before and after the onset of the national Great Recession and
housing market collapse. For New Orleans, we will of course differentiate trends pre- and post-Katrina
to the extent possible.
Our first results will be presented in the Baseline Report, due in September 2012. This will rely mostly on
neighborhood level data from the Decennial Census, American Community Survey (ACS) and other
national sources. For most of the national secondary sources, the most recent data available when this
report is being prepared will be for times prior to 2012. For example, the most recent data available
from HMDA (indicators of mortgage market activity) will be for calendar year 2010. The data from the
decennial census (resident and housing characteristics) will be as of April 2010 and the most recent data
from the ACS will present average conditions for the 2006-2010 period.
During this period, we will be negotiating access to de-identified local administrative data with location
information including property sales, crime, and vital statistics. The dates and breadth of data will vary
from site to site. Given the timing, the first report will rely primarily on national data, but we will include
analysis of any local data obtained in time. We will also include any qualitative information or insights
gained in the first site visit in Spring 2012 and in the document reviews.
Our second round of analysis regarding neighborhood conditions and change will be presented in the
Final Report (first draft due May 2014). It will update of the information provided in the baseline report
(for the Choice Neighborhood, other neighborhoods in the CNI city and the city/region), but will also

29

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

include additional data sources obtained after the baseline report, particularly in regards to local
administrative data.
The primary data collected over the project period will offer another set of information. The
Neighborhood Residents Survey (discussed in Section 2.2) will entail interviews with a random sample of
residents in the Choice Neighborhoods not living in the development slated for transformation. This
survey is being funded by the Annie E. Casey Foundation. It will be conducted at the same time as the
survey of residents of the assisted housing projects, from November 2012 through February 2013. The
content of this survey will be much the same as the survey of residents in the focal development except
for cases where differences are obviously warranted (e.g., questions on relocation).
In addition, two other data collection tools focused on obtaining data on the Built Environment will be
administered during the study period. The block front survey using Google Street View will be conducted
in winter 2012/2013, and the pedestrian routes survey will be conducted during the second site visit.
Like the Neighborhood Residents Survey, these sources will not be available for the other
neighborhoods in the CNI cities or larger geographies. Given the timing of these and the Neighborhood
Residents Survey, results reported by May 2014 will still be more in the nature of an amplification of
baseline conditions rather than results influenced by the intervention.
Finally, the key informant interviews and focus groups will provide more nuanced information on the
perception of neighborhood conditions and how they have been changing.
2.4.2.2 Mechanisms Driving Neighborhood Change and Impact on CNI Implementation

Once the information from the various sources has been synthesized, the research team will address its
second research question and will meet to identify hypotheses (theories of change) that could explain
the differences in trajectories that have been identified. Based on the local implementation schedule
and data sources, we will identify appropriate time periods for characterizing the trends before and
after the initiation of Choice Neighborhoods Initiative. Our analysis will include the identification of
mechanisms and forces that could drive the changes and may affect the implementation of Choice.
Some of these are likely to be outside factors, e.g., conditions in metropolitan housing and labor
markets. Others may be motivated by internal actions, including Choice Neighborhoods interventions
and important actions by others in the past few years (e.g., the closing of a school, new private
investments in retail establishments or housing).
This initial analysis will be presented to select local individuals knowledgeable about the Choice
Neighborhoods Initiative in each of the three cities and the forces that affect them. They will be asked
for: (1) their explanations of the key changes that have been identified (hypotheses concerning key
causal forces); and (2) their hypotheses concerning possible impacts of these forces and trends on
Choice implementation and outcomes. This interaction will be supplemented by responses from the key

30

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

informant interviews and focus groups regarding perceptions of change in the area. This analysis and the
reactions of the grantees will be presented in the final report (first draft due May 2014).

2.4.3 Data Sources
This research will take advantage of 8 data sources, described in detail in Section 3 of this document.
1.
2.
3.
4.
5.
6.
7.
8.

Neighborhood Residents Survey
Key Informant Interviews
Document Review
Observations at Community Meetings
Focus Groups
Secondary Quantitative Data
Block Front Survey
Pedestrian Routes Survey

2.4.4 Analytic Approach and Data Sources by Research Topic
(Research Questions 1 and 2)
In the paragraphs below, we explain our approach to analysis and the data sources we expect to use for
each of the research topics in the (1) baseline and (2) final reports.
Maps for the locally defined boundary of the Choice Neighborhoods are included in Appendix F. We will
also consult with the local grantees if there are locally meaningful subareas within the selected
neighborhoods. The smallest geographic level of the secondary quantitative data will vary by source.
When possible, we will request de-identified administrative data at the address level to be able to
report on the “official” Choice Neighborhood. For the Decennial Census in 2000 and 2010, block-level
data will similarly be used to report information for the Initiative’s area whenever possible. For any
secondary source data provided at the census tract level, we will use tract boundaries as defined for the
2010 Census. Where tract boundaries have changed, data for prior years will be adjusted to conform to
2010 boundaries using methods employed in constructing the Urban Institute’s Neighborhood Change
Database. For tracts that overlap boundaries of the selected neighborhoods, estimates will be made for
the portions of the tract inside the official neighborhood boundaries for variables that are key to the
analysis. Tracts in the three neighborhoods are as follows:
Boston: 5 tracts total (4 overlap neighborhood boundaries)
Chicago: 12 tracts total (none overlap neighborhood boundaries)
New Orleans: 14 tracts total (5 overlap neighborhood boundaries)

31

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
2.4.4.1 Neighborhood Residents – Baseline Report

Basic demographic conditions
Decennial census SF1 (2000 and 2010): race/ethnicity, age, household types
Economic circumstances
Decennial census SF3 (2000) and ACS (2006/10): income, employment, education
Local Employment Dynamics (LED) (2002-2010): employed residents by wage level, age,
economic sector, distance travel to work.
Public Assistance Records (dates will vary by city): rates of TANF and SNAP receipt from deidentified state and local administrative data
Physical and mental health
Vital Statistics (dates will vary by city): rates of low-birth weight births, infant mortality, teen
pregnancy, inadequate prenatal care, death (by age and cause) from de-identified state and
local administrative data
Residential Mobility
Decennial census SF3 (2000) and ACS (2006/10): Number of residents moved over past year
No data on child wellbeing or attitudes toward the future in the baseline report
2.4.4.2 Neighborhood Residents – Final Report

Basic demographic conditions
Neighborhood Residents Survey (November 2012-March 2013): household composition,
race/ethnicity, age
ACS (2007/11 and 2008/12): race/ethnicity, age, household types
Economic circumstances
Neighborhood Residents Survey (November 2012-March 2013): education, employment,
income, financial management, hardship, public assistance
ACS (2007/11 and 2008/12): income, employment, education
LED (2011): employed residents by wage level, age, economic sector, distance travel to work
Public Assistance Records (dates will vary by city): rates of TANF and SNAP receipt from deidentified state and local administrative data

32

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Physical and mental health, child wellbeing, and attitudes toward the future
Neighborhood Residents Survey (November 2012-March 2013): physical and mental health,
child well-being, resident attitudes toward the future
Vital Statistics (dates will vary by city): rates of low-birth weight births, infant mortality, teen
pregnancy, inadequate prenatal care, death (by age and cause) from de-identified state and
local administrative data
Residential Mobility
Neighborhood Residents Survey (November 2012-March 2013): residential mobility
ACS (2007/11 and 2008/12): Number of residents who moved over past year
2.4.4.3 Neighborhood Housing – Baseline Report

Housing stock
Decennial Census (2000 and 2010): number of units by tenure
Decennial census SF3 (2000) and ACS (2006/10): distributions of housing types, sizes, year built
Assessors’ file (dates will vary by city): number of parcels, tenure, structure type, year built
Code enforcement data (dates vary by city): violations and fines as recorded by the city
Building permits (dates vary by city): new construction, rehab, demolition, units and dollars from
de-identified address-based local administrative data
Housing market characteristics and activity
Decennial Census (2000 and 2010): housing tenure
Decennial census SF3 (2000) and ACS (2006/10): home values and rents, vacancy
Home sales data (dates vary by city): number of sales and prices from de-identified addressbased local administrative or proprietary sources
HMDA (2000-2010): mortgage applications and originations (by race, gender, income), loan
amounts
U.S. Postal Service (Q3 2010): housing vacancies
HUD or local housing authorities: presence and mix of assisted housing (public housing, projectbased assisted housing, and housing choice vouchers)
Foreclosures (dates vary by city): notices and sales from de-identified address-based local
administrative or proprietary sources
Housing quality
Decennial census SF3 (2000) and ACS (2006/10): facilities

33

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Housing hardship
Decennial census SF3 (2000) and ACS (2006/10): overcrowding and cost burden
2.4.4.4 Neighborhood Housing – Final Report

Housing stock
Neighborhood Residents Survey (November 2012-March 2013): Data on unit characteristics
Block front survey (November 2012-March 2013): dilapidated housing
ACS (2007/11 and 2008/12): number of units, distributions of housing types, sizes
Assessors’ file (dates will vary by city): number of parcels, tenure, structure type, year built
Housing market characteristics and activity
Neighborhood Residents Survey (November 2012-March 2013): Data on tenure, housing
assistance status, utilities and costs, perception of vacancy and abandonment
ACS (2007/11 and 2008/12): home values, rents, vacancy
Home sales data (dates vary by city): number of sales and prices from de-identified addressbased local administrative or proprietary sources
HMDA (2011-2012): mortgage applications and originations (by race, gender, income), loan
amounts
HUD or local housing authorities (dates vary by city): presence and mix of assisted housing
(public housing, project-based assisted housing, and housing choice vouchers)
U.S. Postal Service (TBD): housing vacancies
Building permits (dates vary by city): new construction, rehab, demolition, units and dollars from
de-identified address-based local administrative data
Any relevant data or analysis from the Strategic Economics neighborhood housing market
analysis
Housing quality
Neighborhood Residents Survey (November 2012-March 2013): Data on unit and building
conditions, tenant satisfaction with management/services
ACS (2007/11 and 2008/12): facilities
Real Estate Assessment Center (REAC) (dates vary by city): Scores and score details for public
and privately-owned project-based housing
Housing Quality Standards (HQS-dates vary by city): inspections for housing units where
voucher-holders live
Block Front Survey (November 2012-March 2013): dilapidated housing

34

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Housing hardship
Neighborhood Residents Survey, (November 2012-March 2013):Data on cost burden and
eviction; Overcrowding (calculated from roster and number of rooms)
ACS (2007/11 and 2008/12): overcrowding and cost burden
2.4.4.5 Education and Other Social/Human Development Services – Baseline Report

Educational opportunities for youth
National Center for Educational Statistics (NCES) (2005/06-2010/11): location of schools in the
area, school-level enrollment by race and free/reduced price lunch status.
School Performance (dates vary by city): school-level math and English proficiency scores,
absences, high school graduation from local administrative data
Child care facilities (dates vary by city): location of facilities, number of total and subsidized slots
from local administrative data
Key informant interviews, focus groups, and document reviews (2012 – early 2014):
assessments of quality of schools; identification of key events and forces influencing change.
Other social/human development services
Key informant interviews, focus groups, and document reviews (2012 – early 2014):
identification of other social and human development services offered in the neighborhood
(scope and dollars); assessment of quality, accessibility; identification of key events and forces
influencing change
2.4.4.6 Education and Other Social/Human Development Services – Final Report

Educational opportunities for youth
Neighborhood Residents Survey (November 2012-March 2013): experience with schools of own
children, perception of availability of quality schools
National Center for Educational Statistics (NCES) (2011/12 and 2012/13): location of schools in
the area, school-level enrollment by race and free/reduced price lunch status.
Student Performance (dates vary by city): school-level math and English proficiency scores,
absences, high school graduation, capital investment from state and local administrative data
Child care facilities (dates vary by city): location of facilities, number of total and subsidized slots
from local administrative data
Key informant interviews focus groups, and document reviews (2012 – early 2014): assessments
of quality of schools; identification of key events and forces influencing change.

35

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Other social/human development services
Neighborhood Residents Survey (November 2012-March 2013): receipt of services
Key informant interviews, focus groups, and document reviews (2012 – early 2014):
identification of other social and human development services offered in the neighborhood
(scope and dollars); assessment of quality, accessibility; identification of key events and forces
influencing change
2.4.4.7 Social Conditions and Public Safety – Baseline Report

Social cohesion and interaction
Document review
Public safety and disorder
Crime incident data (dates vary by city): Part I and II crimes by type from de-identified addressbased local administrative sources
2.4.4.8 Social Conditions and Public Safety – Final Report

Social cohesion and interaction
Neighborhood Residents Survey (November 2012-March 2013): neighborhood cohesion and
interaction (including collective efficacy)
Public safety and disorder
Neighborhood Residents Survey (November 2012-March 2013): neighborhood crime, disorder,
victimization, and perception of safety
Crime incident data (dates vary by city): Part I and II crimes by type from de-identified addressbased local administrative sources
Block front survey (November 2012-March 2013): graffiti, trash, loitering
2.4.4.9 Economy and Commercial Services – Baseline Report and Final Report

In this category, data sources and research approaches are the same for the baseline and final reports.
Employment opportunities for adults
LED (2011): persons working in neighborhood by wage level, age, economic sector, distance
travel to work.

36

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Key informant interviews focus groups, and document reviews (2012 – early 2014): assessments
employment outlook in neighborhood, assessment of factors driving change.
Commercial and professional services
Business surveys (dates vary by city): number of establishments and employment by type ($
sales and investment) from address-based local administrative or proprietary sources
Key informant interviews focus groups, and document reviews (2012 – early 2014): assessments
of trends in the availability and quality of commercial and professional services in the
neighborhood; key events and factors shifting the trends
Neighborhood Residents Survey (November 2012-March 2013): presence of services
2.4.4.10 Built Environment – Final Report

No information will be provided pertaining to the built environment in the baseline report. The topics
will be covered in some depth, however, in the final report.
Neighborhood Residents Survey (November 2012-March 2013): infrastructure and community
amenities
Block Front Survey (November 2012-March 2013): All items
Pedestrian Routes Survey (January 2013-March 2013): perceptions of public safety, conditions
contributing to traffic safety and livability

2.5 Analysis Plan and Reporting
Our analysis plan describes the ways in which we will use the data collected by our team to document,
understand, and evaluate:
the characteristics of the transformation plan itself and associated anticipated investments and
procedural innovations;
the process of change for residents and the neighborhood in the early years of the
transformation process;
the baseline conditions of assisted residents, the assisted housing under redevelopment, the
surrounding Choice Neighborhood, and its city and regional context; and
the progress, obstacles, innovations, and lessons learned from the early years of the
transformation process.
Given the small number of implementation sites, the research team can focus considerable attention on
each, create a tailored approach to unique situations, and dig deep so that we tell each site’s particular
story. In all cases, we will build our evaluation with sensitivity to the “but for” question: that is, but for
the intervention, how might conditions have changed for the residents, the focal developments, the
neighborhoods, the grantees, and the other stakeholders in the process? In the absence of random
assignment, it is impossible to tell precisely, but we will be constantly alert to the possibility that

37

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

random-assignment experiments and quasi-experimental research opportunities may present
themselves.

2.5.1 Transformation Plans: Characteristics and Critique
Evaluating a comprehensive community initiative like the Choice Neighborhoods Initiative is particularly
complex because each site begins at a different point and each addresses multiple, complex issues
across the three broad areas of interest (people, housing, and neighborhood). Although the overall goal
is for the Initiative to improve conditions in the community and to create a neighborhood of choice and
opportunity, each site will face a different set of challenges, set different goals, and develop different
approaches for achieving those goals.
The first step in our evaluation will be to understand each site’s conceptual model or “theory of change”
about how the sites’ CNI plan will lead to community transformation. As Figure 1 shows, the end result
of this process will be a logic model that shows the hypothesized links between the sites’ planned
activities and their desired end outcomes (Smith, et al. 2010). This logic model will specify site goals,
planned activities (e.g. demolition, relocation, community improvements, relocation and mobility
services, etc.), outputs (e.g. number of planned units, number of residents relocated, etc.), and shortand long-term outcomes.
To develop the individual logic models for each Choice Neighborhoods site, we will draw on information
gathered from our document review and interviews conducted during our first visit to the site. As
described above, during this visit, staff will conduct detailed administrative interviews with key
informants who played the lead roles in developing the application and plan and representatives from
participating agencies. We will use the information from the document review and first site visit to
develop a set of linked hypotheses and develop a full logic model. We will ask key informants from the
site to review and comment on the logic model; this will provide an opportunity for us to work with
grantees to develop a mutual understanding of their and our hypotheses about the impacts of the
intervention.
Each site’s theory of change will guide and structure the design of key informant interviews after the
first site visit. In identifying activities, outputs and outcomes, the logic model helps target appropriate
respondents and topics of conversation. The change theory also fleshes out hypotheses and research
questions for subsequent analysis by highlighting key elements and actors in local plans.

38

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Figure 1: Stylized Choice Neighborhoods Logic Model (example)

OUTPUTS

END OUTCOMES

Demolition,
construction,
rehabilitation,
preservation.
Community
building.

Number of units
at diff. rents:
demolished,
rehabilitated,
preserved.
Mix of units.

High quality, energyefficient, affordable
housing units in mixedincome developments.

Transform
neighborhoods.

Community
improvements
(e.g., transit,
banks, schools,
services, parks);
public safety.

Number of
community
improvements;
less crime.

Safe, healthy, mixedincome neighborhoods
with high-performing
schools and services.

Support
positive
outcomes
for
residents.

Resident services
(e.g., relocation,
mobility, job
training, work incentives, educational supports).

GOALS

Revitalize
distressed
properties.

Operate
high quality
transformation
process.

ACTIVITIES

Resident
engagement,
sustainability,
community
building, recruiting
and supporting a
mix of incomes,
property
management.

Number of
resident
services and
participants.

Number of
activities and
residents
involved,
sustainability
measures,
inclusive
resident
associations.

39

Self-sufficient,
healthy residents
with access to
opportunities.

Residents are engaged in
the process, relocation is
effective and equitable
with original residents
allowed to return, and the
property is financially,
operationally, and socially
sustainable over the longterm.

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2.5.2 Identifying Opportunities for Embedded Experiments
As part of the Choice Neighborhoods Initiative, a number of innovative—but unproven—new programs
may be launched to improve the lives of local residents, both those living in focal developments as well
as those living in the surrounding community. These programs may target various issues related to
family self-sufficiency faced by all community residents, such as employment and human capital
development, employment advancement, asset-building, and financial counseling. They may also be
focused on issues that are particularly important to improving outcomes for residents of the focal
development (for example, relocation counseling). The programmatic strategies that evolve to address
these types of issues may offer opportunities to use individual or group-level random assignment, or an
appropriate quasi-experimental design to determine whether, in fact, investments in these programs
actually produce their intended benefits and are worthy of sustaining, expanding, or even exporting to
other similar neighborhoods.
As part of the first site visit and document analysis, MDRC will conduct a “program scan” to identify
potentially innovative programs that might be suitable candidates for such an embedded study. Followup interviews by MDRC with administrators and staff will help develop a deeper understanding of the
program goals and objectives, services and implementation strategies, staff opinions, funding sources,
time line, and the general receptivity to random assignment-based evaluation. The feasibility
assessment will also examine whether selected interventions are ready for a more rigorous assessment.
For example, some of the factors to consider include the program’s stability and ability to reach the
target population, the strength of the intervention, and the availability of alternative services in the
community. It is possible that through such an assessment, researchers may conclude that (a) we have
been able to identify one or more programs sufficiently strong and distinctive to merit a rigorous
evaluation using random assignment or a strong quasi-experimental design; (b) programs with some
promise exist, but they need to mature to be ready for a rigorous evaluation; or (c) a program suited for
rigorous evaluation cannot be mounted within the available options, or does not exist.
For example, programs may focus on improving the employment and earnings of families living within
the redeveloped properties. Non-experimental evidence from years of HOPE VI research appear to
suggest that despite the improvements in neighborhood quality, many HOPE VI residents remained
economically at risk, and many of those who moved to traditional public housing experienced no gains
at all (Popkin, Levy and Buron 2009). This might suggest the need for more employment-focused
support services, to help families progress towards economic self-sufficiency. The experiments over the
last twenty years offer some evidence about strategies that might be adapted within the context of the
Choice Neighborhoods Initiative. Similarly, grantees may want to test strategies that help families save,
build assets, and improve their financial well-being. Large numbers of low-income families, including
many who are likely to be touched by the Choice Neighborhoods Initiative, have low or negative net
worth, putting them at risk of greater economic peril in the event of job loss, illness, or other family
hardship. Because building assets is vital to economic security, and is a central feature of HUD’s Family
Self-Sufficiency program, the team could explore the feasibility of embedding a test of an economic

40

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

security and asset-building strategy for households in Choice communities. The specific aspects of such a
test could be worked out during a design phase, but such an embedded study could be evaluated
through a random assignment test using a broad range of data on assets and financial behaviors – for
example, savings, debt, banking behaviors, unfavorable banking events, credit scores, payday loans,
check-cashing outlets, and housing and material hardships. Finally, the Choice Neighborhoods Initiative’s
focus on education might also offer potential opportunities for a randomized test.

2.5.3 Documenting Baseline Conditions
Residents of the Focal Housing Development
To develop an integrated understanding of the conditions and characteristics of residents, we will rely
on grantee documents, stakeholder interviews, the focal development residents’ survey, and focus
groups. The first two of these sources will be reviewed during the first year of the evaluation to
contribute to the baseline report. The resident survey and focus groups will be completed, coded, and
analyzed after the baseline report is complete; analyses of these data sources will be integrated with
findings from the baseline report in the final report. Together, the data will allow us to describe of a full
range of topics: demographic, economic, health, education, transportation and travel, recreation, social
networks, food security, resident satisfaction, and so on. Much of this portion of the analysis is
descriptive and aims to produce a rich, multi-dimensional portrait of the local intervention. Wherever
available, the portrait will compare the residents to other public and assisted housing residents in the
jurisdiction and its region.
The Focal Housing Development
Key benchmarks will be established about the focal development to allow long-term evaluation of the
changes in housing quality. Our integrated understanding of the development will be derived from REAC
reports about housing conditions, grantee documents, stakeholder interviews, the resident survey, and
focus groups. Data on the civic realm will be collected by Urban Institute staff inside and around the
project using the pedestrian routes survey (Appendix E), which is based on protocols based on the Irvine
Minnesota Inventory (Day, et al. 2005). CNT Energy has advised UI not to collect data from local utilities
to document baseline energy use, due to the differences in regional regulatory and cost structures, lack
of standardization, and privacy and legal hurdles. CNT Energy will compile and analyze grantee energy
use projections from building specifications for planned new construction and major renovation
projects, comparing plans against appropriate LEED standards, Enterprise Green Communities checklists,
and supporting documents, such as energy cost budgets and energy modeling outputs. CNT Energy also
will benchmark energy use and costs for focal housing compared to similar housing in the area, drawing
on public and private national and regional data sources including but not limited to the Energy
Information Agency’s Residential Energy Consumption Survey, EPA’s Portfolio Manager, and local data
as available. The projection and the comparison analyses will provide benchmarks against which future
analysis of actual energy use can be compared.

41

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

The Choice Neighborhood
Baseline conditions in the Choice Neighborhood environment will be documented and evaluated using a
wide array of data from grantee planning documents and other documents, archival records, secondary
quantitative and image data, the resident survey, the block front survey, the pedestrian routes survey,
stakeholder interviews, and focus groups. The evaluation team will develop full, independent, and
consistent long-term portraits of neighborhoods and their surroundings focusing on environment, social
conditions, residents’ economic and demographic characteristics and self-assessed health status,
businesses, infrastructure, building stock, parks and recreational facilities, crime, schools and education,
community facilities, and social networks. The portrait will situate the neighborhood within the context
of other neighborhoods in the jurisdiction and its region.

2.5.4 Documenting Near-Term Changes for Residents, the Project, and the
Neighborhood
Residents of the Focal Housing Development
Using the tracking data described above, we will replicate our methods from recent analysis of
movements of MTO participants to identify different sub-populations within which we can identify a
shared set of mobility experiences from the start of the transformation process. Since we do not
anticipate deployment of a second survey of the residents of the focal development or of the
neighborhood before the end of this task order, we will rely on additional rounds of interviews and
focus groups to gather information about how residents fare in early months of the transformation and
in the context of other changes that may affect their neighborhoods. Where comparison data are
available for other public and assisted housing tenants, we will compare changes in the tenants in
transformation developments to those in other housing developments during the early years of the
transformation. In sum, we will (a) develop mapping and metrics of householders’ mobility and changing
neighborhood condition and (b) develop from multiple sources a qualitative portrait of the magnitude
and distribution of challenges and progress for residents during implementation.
The Focal Housing Development
Conditions of the housing development itself may also change during the first years of implementation.
We will report on the ways in which conditions are maintained or deteriorated by referring to
stakeholder interviews, focus groups, and the research teams’ observations of the conditions on the
grounds and in the corridors of the development. Where comparison data are available for other
housing developments, we will compare changes in the transformation targets to those in other projects
during the early years of the transformation.
Two additional analysis tasks will yield estimates of post-redevelopment performance of the focal
housing development. First, subcontractor Strategic Economics will assess the near-term, and estimate
the longer-term, financial stability/sustainability of the redeveloped housing projects within the local
42

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

market context. Strategic Economics will assess grantees’ assumptions with respect to operating
revenues and costs, based on independent research on metropolitan and local market conditions,
especially with respect to market rate units and commercial space included in the project. They will
evaluate the grantees’ ability to leverage funds from multiple sources, including but not limited to local
government, philanthropic sources, and private capital. They will test the sensitivity of the grantees’
financial models to a range of market assumptions including abrupt downturns or shocks, long-term
strains or other factors that could impact operating revenues or costs. Finally, they will report on the
assessment of assumptions and sensitivity tests. Second, CNT Energy will use a series of assumptions to
project post-redevelopment energy demand. Their analysis will help owners/developers understand
what energy efficiency they can achieve and to set realistic targets for building energy consumption. For
one focal development, Grove Parc in Chicago, CNT Energy will use a statistically significant sample of
owner and tenant energy use data from utilities to analyze and compare actual to predicted energy use
for new buildings and rehabilitations completed by September 2011, and that are at full or near full
tenant lease-up for at least one year. This should include approximately 100 units in four buildings. CNT
Energy will use post-occupancy energy use data from the period of August 2012 through August 2013.
This analysis, to be included in the final report, will serve as an Energy Snapshot template for future
post-occupancy evaluation in other developments. CNT Energy will collect all necessary data for this
analysis.
The Neighborhood during Transformation
Finally, conditions in the broader neighborhood and its metropolitan context are likely to change over
the first two and one-half years of the transformation process. Using the wide array of data we have
already discussed, we will discuss neighborhood change in the Final Report for each Choice
Neighborhood. Where citywide and regional data are available, we will also analyze the position of the
Choice Neighborhood with respect to other neighborhoods in the same city and the same metropolitan
area. A series of broad and shallow indicators (e.g., crime, home sales, city investment) will be used for
comparison to all city neighborhoods.

2.5.5 Progress in Implementing the Transformation Plan
Our analysis of site implementation progress will draw on information from the key informant
interviews gathered during site visits, our monitoring calls, notes from local community meetings, the
two rounds of focus groups, and our analysis of secondary data. Each site will have a lead analyst at
either UI or MDRC who will be responsible for writing two interim memos on the implementation
process at approximately six- to eight-month intervals after delivery of the final Baseline Report. As
described above, we will keep extensive notes from our interviews and the focus groups; for our
analysis, we will review the notes and summarize key themes and trends that emerge. In addition to this
qualitative data, we will draw on secondary sources including progress reports and site performance
measures, REAC scores, and any available indicator data. We will use all of these data to assess how well

43

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

the sites have succeeded in meeting the initial goals specified in their logic models during the first years
of implementation, successful strategies, and where they have encountered challenges and barriers.
We will also use interviews and planning documents to provide context for the Choice Neighborhoods
Initiative within the jurisdiction’s overall community development activities. This contextualization will
allow us to highlight ways in which the Choice Neighborhoods Initiative affects neighborhood
redevelopment processes.

2.5.6 Reports
The Task Order specifies that two main reports will be produced. The Baseline Report is expected in
draft form in approximately week 52 (September 28, 2012) of the evaluation. The Final Report is
expected in draft form in week 136 (May 9, 2014), a little more than two and a half years after the
announcement of the Choice implementation grants for FY2010-11.
2.5.6.1 Baseline Report

We do not expect OMB clearance to be received for the resident survey until just before the due date
for the Baseline Report. As a consequence, the baseline report will discuss only those aspects of the
transformation plan and the conditions of residents, housing, and neighborhoods that can be obtained
through secondary data collection, document review, our planned initial site visits in March 2012, and
subsequent interviews by our site-based employees.
The baseline report will include separate chapters detailing conditions in each of the three sites. It will
also include a generalized overview that compares the three sites and identifies common and unique
features of each transformation plan. This overview will not constitute a fully realized “cross-site”
report.
Implementation
The baseline report will answer in preliminary form our first research question, on the grantee’s plans
and programs at baseline. Based on the answers to these questions, the baseline report will provide our
initial description and critique of each grantee’s explicit or implicit theory of change. The baseline report
will also include a discussion of the planning process that resulted in the development of the grantee’s
program as well as other important federal, state, or local planning and community organizing processes
that paralleled or preceded the Choice Neighborhoods planning process. This review will identify
opportunities for synergy among planning activities; it will also provide background for later evaluation
of how the planning phase can establish (or fail to establish) partnerships that help make progress in the
implementation phase.

44

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Residents, Housing, and Neighborhoods
The baseline report will also answer in preliminary form a series of research questions about residents,
housing, and neighborhood. Referring to the grantee’s documents and other archival materials, initial
interviews, and secondary data collection, we will describe the baseline conditions for many of our
questions as well as expectations of the key actors on how these baseline conditions will change,
because of the combined impact of the intervention and other factors affecting the neighborhood.
(These expectations are part of the theory of change described above.) To the extent possible, the
baseline report will also relate the Choice Neighborhood to other neighborhoods in the same city and
conditions in the city and metropolitan area, as discussed above in section 2.4.
Opportunities for Quasi-Experimental and Random Assignment Research
Relying on program documents and initial interviews with the grantees, the baseline report will also
include a section on whether and how the intervention as planned presents opportunities for research
using quasi-experimental and random assignment methods.
2.5.6.2 Final Report

The final report, like the baseline, will include separate chapters detailing conditions on each of the
three sites. It will also include a cross-site chapter comparing the sites, identifying common and unique
experiences among the sites, and reflecting on the contributions of the Choice Neighborhoods Initiative
to neighborhood transformation across the sites. In this sense we intend to make this chapter a crosssite analysis of progress toward implementation.
Implementation
The final report will revisit the implementation questions answered by the baseline report about goals,
programs, and activities; by the second year of implementation, the answers to these questions may
have changed. In addition, the final report will answer—as well as possible, after two years of
progress—the other three sets of research questions on implementation. We will discuss the
characteristics of the sites, as well as the features of the Choice Neighborhoods initiative itself, that yield
more and less positive results for all these questions.
Residents, Housing, and Neighborhoods
The final report will include answers to the remaining research questions, again as well as possible given
two years of progress. We will revise and supplement our baseline conditions report using new
observations from the resident survey, block front survey, pedestrian routes survey, focus groups, and
additional interviews. In addition, we will report on how the neighborhood has changed since the start
of implementation, relying on two years of local data, news reports, and observations by our site-based
employees. Finally, for those sites with substantial relocation before the end of 2013 and with reliable

45

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

resident tracking data, we will identify characteristics (of residents, housing and neighborhood)
associated with moving vs. remaining in place.
Opportunities for Quasi-Experimental and Random Assignment Research
The final report will revisit the initial report on opportunities for research using quasi-experimental and
random assignment methods, relying on new data collected from tenants and in focus groups as well as
progress in implementation.

46

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Part 3: Data Collection Plan
The following seven sections provide a description of and approach for each element in our data
collection plan for the Choice Neighborhoods Implementation Evaluation and baseline data collection.
Data collection (Task 5 for this Task Order) is expected to begin on February 24, 2012 and be completed
by February 21, 2014. Primary and secondary data collection activities will be undertaken by research
teams at UI and MDRC supported by part-time site-based employees and by three subcontractors: DIR
(baseline survey and block-front survey), Strategic Economics (document review), and CNT Energy
(document review and collection of energy data).
Table 1 below shows the estimated schedule for the data collection activities, several of which occur
during site visits and others which are ongoing throughout the data collection period. It also lists
participants in each data collection effort.
Several data-collection activities will be ongoing throughout the course of the evaluation
(beginning immediately and ending February 21, 2014). Activities based at UI’s and MDRC’s
offices include document compilation and review, secondary quantitative data collection, and
periodic telephone interviews with site-based actors. Ongoing activities based in the field and
undertaken by our on-site part-time employees (one per site) include observation of key
community meetings, additional key informant interviews, and periodic visits to observe the
neighborhood.
Key informant interviews, focus groups, the pedestrian conditions survey, and general
observation of the focal development and the neighborhood will occur during the three planned
site visits at each of the three sites in this study. Each site visit will be conducted by a
combination of UI and MDRC research staff; UI will lead the visits in Boston and Chicago and
MDRC will lead the visits in New Orleans. It is expected that two staff members of the lead
organization for the site and one staff member from the other organization will make the site
visits. The UI site-lead for each site will be one of the staff members attending the visit. Before
each site visit, background briefing materials drawn from various primary and secondary sources

47

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

will be prepared by the organization leading the site visit for all the researchers going to the site.
The part-time site-based employees will provide ground support in advance of the site visits.
The surveys of residents and block-front conditions will be conducted by subcontractor DIR,
currently scheduled to begin in November but with potential to begin earlier depending on OMB
clearance.
Table 1: Data Collection Activities and Schedule for the Choice Neighborhoods Implementation Evaluation and
Baseline Data Collection

Data Element and Site Visits
Data Collection Begins
Document Review
Secondary Quantitative Data
Observations of Key Community Meetings
Site Visit #1
Key Informant Interviews
Residents Survey
Block Front Survey
Site Visit #2
Key Informant Interviews
Focus Groups
Pedestrian Conditions Survey
Site Visit #3
Key Informant Interviews
Focus Groups
Data Collection Ends

Primary Responsibility for
Data Collection
UI/MDRC/Strategic
Economics/CNT
UI
Site-Based Employees
UI/MDRC/Site-Based
Employees
DIR
DIR
UI/MDRC/Site-Based
Employees

UI/MDRC/Site-Based
Employees

Start Date
3/5/2012
3/5/2012

End Date
2/21/2014

3/5/2012
4/1/2012
3/12/2012

2/21/2014
2/21/2014
3/23/2012

11/21/2012
11/21/2012
2/25/2013

2/15/2013
2/15/2013
3/15/2013

1/27/2014

2/21/2014

2/21/2014

3.1 Document Review
3.1.1 Description
Document review plays an important role in our data collection efforts for this Task Order. We will
amass document libraries to assemble a detailed portrait of each implementation site’s background and
the transformation plan and extract information in a standardized template that categorizes information
by each research question regarding implementation, housing, people, and neighborhoods.
The first key document is the transformation plan itself as defined in the Round 1 and Round 2 Choice
Neighborhood Initiative Implementation grant application materials. These documents contain a host of
information, particularly relevant to the implementation and housing research teams, on the needs of
the focal development, the residents and the neighborhood; the goals, activities, and vision of the

48

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

grantees; the housing investments and site plans; and the capacity of the grantees and their partners to
implement CNI.
In addition to the Implementation grant applications, we will assemble and review other key reference
materials, drawing from a variety of sources that may include:
Local planning documents (e.g., comprehensive plans, consolidated plans, PHA plans, zoning
ordinance and maps, housing code, building code);
Histories of the neighborhood (stand-alone studies and accounts within other histories of the
city and metropolitan area);
Books and refereed articles about the neighborhood, the city, and the local history of public
housing and community development; and
E-newsletters of the city and neighborhood organizations and key agencies implicated in or
affected by the transformation plan.
In addition, we will establish a series of Google Alerts to supplement the historic record and to monitor
unfolding events. Through this we will identify mentions in media and blogs of the neighborhood and
the focal development. Key items from the search will be summarized and archived monthly for access
by the full team.
In addition to providing insights for the two formal reports, these materials will be reviewed and
incorporated as appropriate into the memos written in preparation for the three site visits and for
tailoring the interview guides discussed in section 3.3.

3.2 Baseline Surveys of Residents of the Focal Developments and the
Neighborhoods
3.2.1 Description
We designed and will implement a baseline survey of residents of the focal developments, creating the
basis for a long-term panel study that documents how the Choice Neighborhoods Initiative affects
residents across a range of domains. With additional funding provided by the Annie E. Casey foundation,
we also will survey the residents of the Choice Neighborhoods outside of the focal development. The
neighborhood resident survey will allow us to compare the residents of the focal development to other
neighborhood residents, and track the impact of the Choice Neighborhoods Intervention on
neighborhood residents.
The UI-MDRC team has designed the survey, drawing on surveys we have designed for other studies,
including the HOPE VI Panel Study, Making Connections Demonstration, Chicago Family Case
Management Demonstration, and the Camden Community Survey. Claudia Coulton and Mark Joseph

49

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

from Case Western Reserve University (CWRU) and other consultants on our research team have
reviewed the survey content.

3.2.2 Data Collection Instruments & Questions
The Choice Neighborhoods Initiative’s complexity requires that the survey cover a broad range of
domains, including those listed and described below. The content of the surveys for residents of the
focal development and for the neighborhood residents will overlap substantially, but there will be some
differences. In particular, neighborhood residents will not be asked about CNI-funded services and
relocation experiences.
Housing Mobility
When the survey is fielded (currently expected to be late Fall 2012), the process of relocating residents
from the target developments will have already begun to some extent in Boston, Chicago, and New
Orleans. In order to ascertain the length of time that the respondent has been in their current unit, we
will ask a short series of housing mobility questions that were developed for the HOPE VI Panel Study.
These questions will provide important information about how long the recent relocatees have been
away from the original development.
Housing Conditions and Utility Costs
The survey includes questions on interior housing conditions that were adapted from the American
Housing Survey (AHS) and used in the Moving To Opportunity (MTO) Final Evaluation. We drew survey
questions from other studies, including questions from the HOPE VI Panel Study, and the Chicago Family
Case Management Demonstration related to housing assistance status and rent/utility hardship.
Neighborhood Conditions, Safety and Social Cohesion
With the assistance of CWRU, we included questions that measure the level of neighborhood cohesion
at baseline, residents’ formal and informal social networks, as well as their level of involvement in
community activities. We drew on our previous research for items on resident perceptions of and
experiences with crime, victimization, social and physical disorder, and collective efficacy (Sampson,
Raudenbush and Earls 1997). To understand residents’ social networks beyond and within the
neighborhood, we incorporated survey questions from the MTO Final Evaluation and the Camden
Community Survey. We will also ask residents a series of question to understand their involvement in
community and CNI related activities.
Neighborhood Services and Amenities
The survey includes questions that will allow the UI-MDRC team to understand focal development
residents’ housing relocation experiences during and after relocation from the focal development during
the demolition and construction period; specifically, the survey asks what services were available and

50

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

how residents would rate the quality of those services. We include items from the Chicago Family Case
Management Demonstration survey that gather information about institutional supports provided to
the residents of the focal development, such as case management services, classes, and job training.
All survey respondents, from the focal developments and the Choice Neighborhoods, are asked about
their perceptions of and access to neighborhood services, including recreation centers, parks, libraries,
and grocery stores. As described in Section 3.6, these survey items will supplement commercial data on
residential services and amenities (e.g., grocery stores, restaurants) and GIS layers (e.g., schools, parks,
transportation).
Household Composition
The survey respondents will be asked to supply basic information about all members of the household,
including names, ages, employment status (if appropriate), and educational attainment.
Physical and Mental Health
As in the HOPE VI Panel Study, we use items adapted from the National Health Interview Survey (NHIS)
to document residents’ overall health and specific health conditions, access to medical and dental care,
and ability to carry out activities of daily living. We collect information from respondents about their
height and weight in order to calculate their Body Mass Index. We also included measure on average
sleep to assess sleep adequacy, and a mental health screener (the Mental Health Inventory, MHI-5) to
detect depression.
Employment, Education, and Public Assistance
We drew on employment items from the HOPE VI Panel Study surveys to gather information on
educational attainment, current employment status, earnings, and barriers to work (e.g., child care,
health, transportation, literacy). We ask about receipt of benefits, such as TANF, SSI, Social Security
Disability, and Food Stamps. We also included questions that will capture whether focal development
residents experience economic hardship, such as trouble paying bills, getting medical care, or
experiencing food insecurity.
Child Well-Being
In this study, two focal children will be randomly selected from each household, one younger (age 5 or
under) child and one older child (age 6 to 14). The interview respondent (the head of household) will be
asked to answer a short series of items about the childcare arrangement or grade level for the younger
focal child. The series of questions for the older focal child will cover grade level, behavior,
extracurricular activities, school engagement, school performance, and health. We will also gather the
name of the school each child attends.

51

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Consent for Administrative Data Collection
To the extent feasible, we will request respondents’ consent to gain access to administrative data
records that can be linked to their survey responses. We are still exploring the costs and benefits of
requesting consent and, in the absence of more complete information, cannot commit to requesting
consent in all cases. Key concerns include the following:
State and local agencies usually have specific language that they require to request consent
from individuals to access their records. This language differs among states/jurisdictions and
among their respective agencies. Generalized language developed thus far in our draft survey
could prove insufficient for data requests.
A record of verbal consent is likely not to be sufficient to satisfy agency protocols, but adding a
role to the survey recruiters—explaining the process and getting a signature—will add cost and
time to the administration of the survey.
Requests for consent to access administrative records will displace some questions from the
survey, given the need to keep the average time to 45 minutes.
If these concerns are alleviated, then we will request respondents’ consent to gather administrative
records from the housing authority and state employment and welfare agencies and request parental
consent to collect school records. We anticipate that blanket language about a variety of sources will not
give adequate control to the respondent nor will it satisfy some agencies. We will rely on a sample
consent form provided by NORC with checkboxes to give the respondent control over data sources. Any
questions about consent will be asked at the end of the survey to ensure that the request does not
affect the respondent’s willingness to participate in the survey at all. Finally, we will not collect survey
respondents’ social security numbers. The survey and OMB package reflect this approach, but the
inclusion of consent questions in the survey as administered in the field must still be finalized pending
our full investigation of agencies’ protocols for records release.
Note on Administrative Data Collection on Survey Respondents
We will not request individuals’ identified administrative records from public agencies. We believe these
requests are better done under separate task orders with specific research questions and after enough
time to expect some effects from CNI to emerge. State administrative agencies generally maintain
historic records so future analysis will be able to include information from the baseline period. We will,
however, request the address or other geographic information available for administrative records in
order to aggregate data to the Choice Neighborhood or subareas within the Choice Neighborhood.

3.2.3 Sampling and Fielding the Surveys
Among the five FY 2010-11 finalists for Choice Neighborhoods implementation grants, Seattle, Chicago,
and New Orleans have large focal developments, with 556, 325, and 440 units occupied as of February

52

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

2012, respectively. We aim to collect a random sample of 200 completed surveys in Chicago and 225 in
New Orleans. The Boston and San Francisco sites have 127 and 229 occupied units in their focal
developments as of February 2012, respectively. We will obtain 80 completed surveys in Boston. We will
obtain data from the CNI grantee’s partners that includeresidents’ names and addresses to identify
sampling frames for residents of the focal development. For these households, the head of household
will have already been defined by HUD.
The neighborhood resident survey will be implemented in the three sites with funding from the Annie E.
Casey Foundation. Because we do not know neighborhood residents’ names and addresses, our survey
firm will randomly select addresses at each site from address-based sampling frames. Our survey firm
(DIR) will contact selected households by phone or in person and recruit an adult member of the
household who is responsible for financial matters and knowledgeable about the household in general
(i.e., head of household). The Annie E. Casey funding is adequate to obtain samples for each site of 225
neighborhood residents. In the event that further funding is obtained, we intend to increase the sample
sizes for these three neighborhoods to 300 in Boston and 400 in both New Orleans and Chicago. In the
event that further funding is secured, the larger sample sizes in New Orleans and Chicago will allow us to
randomly select households for 300 completed surveys and to oversample households near the focal
development for an additional 100 completed interviews. Our Boston approach does not require
stratification because its neighborhood is less populous and has a smaller land area than the other two
sites.
Both the survey of residents of the focal development and that for residents of the broader Choice
Neighborhood will be fielded simultaneously in each site. Our schedule for the surveys calls for a 12week data collection period, beginning in all three sites in week 60 (to allow time for the OMB clearance
process). DIR will have responsibility for all field data collection and, to ensure consistent and highquality data, will collect as much of the data as possible through telephone interviewers working from
their centralized, monitored telephone interviewing facility. The interview approach is a call-out/call-in
survey method whereby staff from DIR’s Computer Assisted Telephone Interviewing (CATI) center call
residents to complete a CATI-based interview conducted by DIR staff or, if needed, on-site recruiters
(called “field locators”) make direct contact with residents to facilitate their participation in a CATIbased interview. This method allows DIR to hire field staff who are most comfortable and effective
working in low-income and disadvantaged neighborhoods and convincing potential respondents to
participate in surveys. These are not necessarily the same individuals who are most effective in
conducting complex structured interviews. Therefore, DIR finds the approach of using those staff to
facilitate respondents contact with DIR’s CATI center to be most effective. DIR has used this approach
successfully for surveys in housing developments in Los Angeles, Dayton, Baltimore, and other cities, and
is currently using it in New York City.
Landlines are the first choice for conducting all field-initiated interviews that will be completed by
telephone. All calls will be made into the toll-free phone number in DIR’s CATI center. Field locators are
trained to first determine whether the potential respondent has an available landline before attempting

53

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

to initiate a call from the cell phone. If no landline is available, then the field locator is prepared with a
DIR-provided cell phone for the respondent to use to call into the CATI center.

3.2.4 Resident Tracking
A major goal of the research is to track where residents of the focal development move over the course
of each grantee’s Choice implementation process, i.e., how many return to the revitalized site, how
many move to other opportunity neighborhoods, and how many leave housing assistance. Choice
grantees are required to track housing outcomes and addresses for all original residents starting from
the first-round application; our team will put in place a system for obtaining updates on this information
from grantees at regular intervals. In addition, to ensure high and comparable response rates for the
panel of residents over time, tracking and locating will be a critical component of the research effort.
DIR will use a variety of creative and tailored tracking methods throughout the study period to update
potential respondents’ contact information to achieve the response-rate goals in subsequent surveys.
DIR will create a database with contact information from the baseline survey and keep these records as
updated as possible with information obtained from the Choice Neighborhoods grantee, supplemented
by contact attempts directly with the respondents so that they remain connected to the study and early
evidence of incorrect or outdated addresses can be uncovered. Specifically, the following tracking
methods will be used: 1) batch National Change of Address (NCOA) searches; 2) birthday cards (or some
other greeting) sent periodically to respondents with request for forwarding addresses; 3) study
reminder postcards sent at 6-month intervals; and 4) advance letters sent with request for forwarding
address at 3 months prior to any follow-up survey conducted as part of future research. If no updated
contact information can be obtained through the alternate contacts, trackers will use online nationwide
searching databases such as online directory assistance (DA), 555-1212.com, Lexis-Nexis, and Intellius as
well as searching Internet sites such as MySpace and Facebook.
Finally, DIR will use field-based locating; locators will use original contact information or any updated
information for the sample members and the alternates to conduct in-person home visits and will use
standard field methods, such as communicating with gatekeepers and neighbors, postal delivery
persons, and other residents in the area to help identify local gathering places where potential
respondents may be found.
It is anticipated that funding from the Annie E. Casey Foundation in the second year of this project will
provide resources for tracking the respondents from the neighborhood resident survey.

54

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

3.3 Key Informant Interviews
3.3.1 Description
We will design and conduct key informant interviews to monitor the progress of the Choice
Neighborhoods Initiative in each site, including the program, the residents, and the surrounding
neighborhoods. In addition to this primary purpose, the interviews also will serve as primary and
supplemental data collection related to baseline and projected changes in housing and neighborhood
conditions to cover gaps from document reviews, surveys, and focus groups. The UI-MDRC team will
develop topic lists and discussion guides focused on local conditions, plans, and information from
respondents. Mark Joseph, one of our subcontractors from Case Western Reserve University, will also
review and provide input on the guides.
We plan to conduct the interviews in three main ways—formal field visits by a central research team,
occasional telephone interviews of CNI stakeholders by staff from the UI-MDRC team in Washington and
New York, and ongoing updates from site-based UI staff. Key informant interviews will take place during
all three site visits: the initial visit in March 2012; a visit after the baseline resident survey (early 2013);
and a final visit in early 2014. In addition, the part-time site-based employee will conduct progress
interviews with key participants in the transformation process throughout the study. Through this
approach, we combine the benefits derived from a focused site visit and key-informant strategy with a
more constant ethnographic presence in which we monitor processes over time.
Interviews will be conducted in the offices of participants or at some more remote location such as a
coffee shop, according to the preference of the interviewee, and will last approximately 1.5 hours.
Interviewers will take comprehensive notes on each interview and, with permission of the respondents,
selected interviews may be recorded for reference purposes but no transcription of the recordings will
occur. Participants will be given the option to stop recording or to not answer any sections of the
protocol that they do not wish to answer. Interviewers will then summarize the content of the interview
for review by the relevant team members. Although the data collected during these interviews will be
used to answer the research questions described in Part 2 and to develop publicly available case studies,
researchers will not share notes with actors outside the research team and will make efforts to ensure
that specific quotes cannot be attributed to individual actors. It will also be important to assure
participants that raw data will not be available to HUD or to other actors.
Research staff will use semi-structured protocols to guide the interview. These protocols will be tailored
to the site and to individual actors and ask questions about site-specific programs and strategies. Given
the variable nature of the instruments and multiple types of respondents, we do not expect a
substantially similar instrument will be used with 10 or more individuals and therefore, do not believe
the guides require OMB clearance. Accompanying each protocol will be a write-up and analysis guide,
whereby interviewers will summarize the content of the interview as related to the topics discussed.

55

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Data collection will involve a range of local actors, so as to capture a variety of potential responses to
CNI and interpretations of local events. Potential informants include representatives from all agencies
involved in planning and implementing the redevelopment, e.g., the housing authority (HA), the Mayor’s
Office, schools, police, local transportation agency, public health agency, and other city agencies. Other
key informants could include housing development owners, property managers, developers, architects
and designers, community leaders, resident leaders, representatives from social service agencies or
mobility programs, and business owners. These first interviews will be identified through document
review and through preliminary calls with site leads; subsequent interviews will be identified through
snow-balling at the end of interviews (i.e., asking whether there are important people with whom we
should speak about Choice) and through careful attention to implementation progress to observe any
new, relevant actors that emerge as Choice stakeholders.

3.3.2 Topics
The semi-structured topic guides for these interviews will be based on a generic template for each
respondent category that is then tailored to specific sites. The precise content of the interviews will
vary, depending on each key informant’s role and degree of involvement in the Choice Neighborhoods
Initiative. Given limited resources and time during interviews, the overall strategy will be to probe on
topics most important for that actor to answer. Customization of protocols will follow document review
so that specific aspects of the local plan and context can be incorporated into the interviews.
Potential topics include (but are not limited to): details about plans and activities to address major
redevelopment goals (e.g., demolition, construction, ensuring energy efficiency); relocation and mobility
programs for assisted residents; resident turnover; new businesses or other development in the
surrounding neighborhood; coordination among partner agencies; redevelopment financing; financial
forecasts for the revitalized development; and opportunities for quasi-experimental or random
assignment research modules. See table 2 below for a matrix of possible interview candidates and the
topics we would cover in those interviews.
The first round of interviews, to be conducted as part of the initial site visit, focus on identifying the
major players in the Choice Neighborhoods Initiative, understanding the goals of the intervention, local
data capability and plans, the planned intervention strategies and the anticipated outcomes for assisted
residents, neighborhood residents, housing, and neighborhood conditions. Key informants for this round
of interviews will be members of the grantee team and, where possible, a resident representative from
the focal development. These interviews will place emphasis on obtaining information necessary for the
baseline report (draft due September 2012). The second site visit will focus on changes to the original
plan, challenges encountered, and successes so far; it will also be devoted to conducting the first round
of focus groups discussed in section 3.5 below. The final site visit and key informant interviews focus on
further progress in implementing the Choice Neighborhoods Initiative. The issues addressed in this last
round of interviews will depend on how much progress each site has made in implementation.

56

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Service partners

Business/key private
sector stakeholders

x

x

x

x

x

x

x

x

Planning process and early implementation

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

x

Goals and framing for CNI

x

Types of activities

x

Leverage of CNI on other investments

x

Coordination of activities and partnerships
Flexible design of CNI and HUD's role in CNI
Opportunities for random assignment or quasiexperimental evaluations

x

x

x

x

x
x

x
x

x

Assisted residents' health and economic status

x
x

x
x

x

x

Residents
Residents' views on housing, neighborhood and
redevelopment.
Assisted residents' views on relocation,
institutional supports and social networks

stakeholders

Grantee / People
team staff

x

Development
partner
representatives
Resident

City officials

Implementation
Choice's programmatic emphases and
innovations

Research Teams and Topics

Grantee /
Neighborhood team
staff
Grantee / Housing
team staff

Local HUD officials

Table 2: Key Interview Topics by Research Team and Interview Participants

x

x

x

x

x

x

x

x

x

x

x

x

x

Housing
x

x

x

x

x

x

x

x

x

x

Projected change in site plan
Current physical condition of assisted units and
units in the neighborhood
Proposed changes to units outside of assisted
development
Management structure, policies and plans

x

Neighborhoods
Perception of neighborhood conditions and
change
Mechanisms driving neighborhood change and
impact on CNI
Educational opportunities for youth
Social/human development services
Employment opportunities for adults
Commercial and professional services in the
neighborhood

x
x

x
x

x

x
x
x
x

57

x

x

x

x
x
x
x
x

x
x
x
x

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

3.4 Observations of Key Community Meetings
3.4.1 Description
Our site-based UI employee at each site will attend and take notes on key neighborhood meetings and
other gatherings of community stakeholders important to the Choice Neighborhoods Initiative.
Relationships will be established with the lead agency head and other critical agency leads during the
initial site visit and will be mined in order to find about out about and gain admittance to relevant
community meetings. The research team will ask to be placed on any email lists related to Choice
Neighborhoods and to receive publications (newsletters, etc.) intended for original residents of the focal
development. The site-based UI employee will send field notes from key meetings to designated
members of the UI-MDRC team as they are produced and debrief with the central team members at UIMDRC on a monthly basis where ongoing activities, partnerships and developments germane to the local
Choice Neighborhood effort will be discussed.

3.4.2 Rationale
Observation of key community meetings will provide an important complement to other forms of data
collection. A primary objective is to collect direct observation data about the local design and
implementation process to complement the information that will be gathered through document
review, interviews, and focus groups. A limitation of interviews and focus group data is that it relies on
individual perceptions and recall. Direct observations provide an additional, more “objective” and
systematic perspective on the local implementation process. Furthermore, direct observation allows for
collection of data about meetings and events that is guided by a specific set of priority topics for
documentation, including information about the content, participants, interactions, and other
observable dynamics of meetings.
The meeting observation methodology is intended to:
capture data that will be used in analysis and reporting on evaluation research questions;
capture information that can help the research team stay up-to-date on local project progress
and any key changes or events that may have implications for the research study; and
promote stakeholder identification as well as relationship and network-building and positioning
for the research team that will facilitate further collection of various forms of relevant data.

3.4.3 Data Collection Instruments & Questions
3.4.1.1 Sample

We anticipate a growing and dynamic list of possible meetings for research staff to observe. This list will
likely include:

58

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Internal business meetings
o Choice planning task force/workgroup meetings
o development company meetings
o management team (development, prop. mgr, service provider) meetings
o housing authority meetings
Development resident meetings
o renters meetings
o local advisory council meetings
o homeowners/condominium association meetings
Public meetings
o development open houses and marketing meetings
o informational meetings/public hearings
o design charrettes
o housing authority board meetings
o political meetings (local ward, council meetings)
Neighborhood meetings
o neighborhood association meetings
o community policing meetings
o local school council, parent/teacher association meetings
o local park council meetings
Community activities
o community building activities and events: celebrations, picnics, gatherings
o adult and youth programming
3.4.1.2 Implementation

The site-based UI employee will generate a universe of potential meetings and activities and will contact
appropriate individuals to seek access. Given the anticipated range and variety of local meetings and
activities, and limited evaluation team capacity for observation, it will be important to have a systematic
way of determining which meetings to prioritize for attendance and direct observation. Meetings will be
divided into high-priority meetings which should be attended as regularly as possible and lower-priority
meetings and activities which should be attended periodically. There may also be some activities
designated for one-time observation, just to get a sense of the meeting/event content and participation.
Where meetings cannot be attended by the site-based employee, there may be a value in a follow-up
phone conversation with a key stakeholder who was in attendance, to document that individual’s
perspective on the content and relevance of the meeting.
The evaluation team will consider whether to have the site-based UI employee recruit additional lowcost capacity for meeting observation in the form of local university students or volunteer interns who
can be trained in meeting observation.
The site-based employees will receive some training on the observation protocol including a review and
discussion of protocol and review and discussion of examples of field notes. Mark Joseph will review the
first few sets of field notes with the site-based employees to improve the process. As the project
proceeds, there will be periodic review and discussion of field notes as needed.

59

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

3.5 Focus Groups
3.5.1 Description
To complement the data collected through other methods, especially through key informant interviews,
we plan to conduct focus groups at each site with residents of the focal development, other
neighborhood residents, and service providers from the community surrounding the focal development.
We plan to conduct two rounds of focus groups, at approximately one and two years into the study
during the second and third site visits. We will conduct one focus group with each category of
respondent at each wave, for a total of three groups per site per year. We used this methodology for
our process study of the Madden/Wells HOPE VI redevelopment in Chicago (Levy and Gallagher 2006),
and found it an excellent and relatively low-cost way to gather information on community perceptions
of problems and of the changes in the community over time. The focus groups will cover a range of
topics, including: perceptions of neighborhood conditions; challenges, adequacy of services and
amenities; perceptions of the focal development; resident turnover; new residents moving into the
community; views about the Choice Neighborhoods redevelopment; and opportunities and value of
resident participation in redevelopment planning meetings.

3.5.2 Data Collection Instruments & Questions
3.5.1.1 Recruitment and sample

Focus groups will occur during the second and third site visits by the UI-MDRC research teams. Our sitebased Urban Institute employee will recruit focus group participants for three focus groups at each site:
residents of the focal development, other neighborhood residents, and service providers.
To recruit participants for the resident groups, we will distribute letters and post fliers in the target
development and other public places in the neighborhood. Interested individuals will be asked to
contact UI and participate in a screening to confirm their eligibility. Criteria for eligibility are based on
address at the time of screening and availability on the date of the focus group. Participants in the focal
development residents’ group will include individuals who lived in the development for at least one year
at the time of screening. Participants in the neighborhood residents’ group will be recruited from within
the Choice Neighborhood boundary. UI staff will send a confirmation letter to the first nine assisted
residents and neighborhood residents that agree to participate. Three additional residents will be placed
on a waiting list in the event of a cancelation.
To recruit service providers, we will develop a list of service providers from within the Choice
Neighborhood boundary. UI staff will contact the organizations on the list to invite them to participate in
the focus group and screen them for eligibility and availability on the date of the focus group. UI staff
will send a confirmation letter to the first nine service providers that agree to participate. Three
additional service providers will be placed on a waiting list in the event of a cancelation.

60

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
3.5.1.2 Implementation

Staff from the Urban Institute and MDRC will design and facilitate the focus groups. Focus group
participants will be encouraged to speak openly about the issues which are raised during the focus
group and will be asked not to share any information discussed with others. Participants will be given
$40 for their participation and drinks will be provided to those participating in the focus group.
3.5.1.3 Key Questions

Resident groups will be asked about the following topics:
What makes a good neighborhood
People and institutions in the neighborhood
Changes in the neighborhood
There will be slightly different modules for residents of the focal development and
neighborhood residents on the following topics:
o Plans to stay in the neighborhood
o Changes that they would like to see
o Whether they have already observed changes
o Whether they think they will benefit from changes
Service providers will be asked about the following topics:
Role in the Choice Neighborhoods Initiative
Caseloads, referrals, partner agencies
How the Choice Neighborhoods Initiative has affected clients and organization
How the relocation process is going
How they keep in touch with clients who have moved
Challenges in engaging clients
Suggestions for improving the process

3.6 Secondary Quantitative Data
3.6.1 Description
As noted in the Part 2 on the research design, a number of the research questions can be answered with
data from secondary sources. We will develop secondary quantitative data and potentially images of the
Choice Neighborhoods from several main known sources.

61

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
3.6.1.1 Grantee and HUD databases

Grantees are expected to develop their own databases and metrics of a series of indicators, according to
the NOFA. We are building an excellent working partnership with all grantees to ensure a free flow of
data between our team and the main actors in the implementation process. We are in the midst of
establishing relationships with grantees to assist in their identification of key data. When possible, we
will rely on the grantees’ own performance measurement activities to obtain and update some baseline
data on neighborhoods and residents. We will also use our interactions to document the extent of
grantees’ independence from external assistance in data development and performance measurement.
We will collect relevant property management data and reports from the PHAs, assisted multifamily
housing owners, or HUD. This information will include the public housing and multifamily focal
developments’ scores and sub-score details from the Real Estate Assessment Center (REAC). We can
obtain the overall scores and centroid of the developments through the publicly-available data on the
HUDUser website, but we also need the non-published sub-scores to assess the particular elements
related to housing conditions. These performance assessment scores are based on property inspections,
analysis of financial and management reports, and resident satisfaction surveys. We will also attempt to
obtain Housing Quality Standards (HQS) inspections results for units whose tenants receive Housing
Choice Vouchers (HCVs) from the local housing authorities—including HCV units within HUD-assisted
properties.
We will also request access to the individual-level or census tract summaries of the 50058 data on the
number and characteristics of voucher households in the metropolitan area in order to learn how
important the Choice Neighborhood is in the city’s and region’s stock of HUD-assisted housing over
time.
3.6.1.2 Publicly Available Federal Government Sources

All these data will be collected for the cities and metropolitan areas surrounding the site to allow
tracking of comparison neighborhoods.
Decennial Census: Every ten years, the Census Bureau conducts a national household survey. The
federal government uses Decennial Census data for apportioning congressional seats, for identifying
distressed areas, and for many other activities. Short form information is collected on every person and
includes basic characteristics, such as age, sex, and race (data for 2010 are currently being released).
Through 2000, the long form was sent to one out of every six households and collected more detailed
information, such as income, housing characteristics, and employment. The American Community
Survey has since replaced the Long Form data – see below.
Dates for analysis: 2000, 2010
American Community Survey (ACS): The ACS is an ongoing statistical survey run by the U.S. Census
Bureau, replacing the long form in the decennial census. The ACS has approximately 250,000
respondents monthly, totaling three million per year. Because of the smaller sample size compared to

62

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

the decennial census, we need to pay much more attention to standard errors and confidence intervals
with the ACS. ACS publishes one-year estimates for geographies with more than 65,000 people, threeyear average estimates for geographies with more than 20,000 people, and five-year average files for all
geographies down to block groups.
Dates: 2006/10, 2007/11, 2008/12 (five-year estimates)
A Picture of Subsidized Housing (APSH): APSH describes the households living in HUD-subsidized
housing in the United States by census tract. The data provides characteristics of HUD-subsidized
housing units and residents. We will use APSH If more complete or more recent data is not available
from the local housing authority or from HUD.
Date: 2008 (update schedule unknown)
Low Income Housing Tax Credit (LIHTC) database: LIHTC is the most important resource for creating
affordable housing in the United States today. The publicly available LIHTC database contains
information on developments and housing units placed in service.
Date: 1987-2009
Multifamily Assistance and Section 8 Contracts Database: This database has address-level information
on privately-owned publicly-assisted properties, including the geocoded location and the number of
units.
Date: 2011 (update schedule unknown)
Home Mortgage Disclosure Act (HMDA): HMDA requires most lending institutions to report mortgage
loan applications, including the outcome of the application, information about the loan and applicant,
and location of the property. In 2004, the Federal Financial Institutions Examination Council (FFIEC)
expanded the data to include structure type, lien status, and if the loan had high interest rates. FFIEC
collects the data in order to determine whether financial institutions are meeting a community’s
housing credit needs; to target community development funds to attract private investment; and to
identify possible discriminatory lending patterns. The reporting requirements are based on the level of
institutional assets and the number of loans originated in metro areas. The loan-level data are
summarized for various geographic levels into indicators on the racial and income distribution of
borrowers, denial rates by race and income, and loans from subprime lenders by race.
Dates for analysis: 1997-2012
Local Employment and Household Dynamics: This time series data created under the federal-state Local
Employment Dynamics (LED) Partnership provide details about America's jobs, workers, and local
economies and communities. LED creatively integrates existing data from state-supplied administrative
records on workers and employers with existing censuses, surveys, and other administrative records.

63

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

State-of-the-art methods to protect the confidentiality of the original respondents allow LED to release
valid data for local or regional areas beyond traditional boundaries for public use on the Internet. There
are three general sets of information released annually in March: number of workers by income, age,
and industry based on where they live; number of workers by income, age, and industry based on where
they work; and a file detailing the origin and destination of workers. Only data based on residence is
available for Massachusetts. LED data is available at the block level, allowing for aggregation up to the
Choice Neighborhood.
Dates for analysis: 2002-2011
National Center for Education Statistics (NCES): The Department of Education conducts an annual
survey of state education agencies to obtain data for every public elementary and secondary school in
the United States and its territories, which it then compiles and publishes as the Common Core of Data
Public Elementary/Secondary School Universe Survey (CCD). The CCD has two main purposes: 1) to
provide an official listing of public elementary and secondary schools and school districts in the nation as
a basis for samples for other NCES surveys; and 2) to provide basic descriptive statistics on public
elementary and secondary schools. Mostly derived from administrative records, data cover school
characteristics such as the school level, grades taught, student-teacher ratio, and federal Title I funding
eligibility, and also provide information on enrolled student characteristics, including race/ethnicity,
free/reduced price lunch eligibility, migrant status, and gender. The CCD will be used as a data source on
the schools physically located in the Choice Neighborhood (where latitude-longitude coding in the CCD
is accurate) and to compare these schools to others in the jurisdiction.
Dates for analysis: 2005/06-2011/12
United States Postal Service (USPS) Vacancy data: The USPS tracks data for every address on whether
mail is received there or whether the unit is vacant. In the past, HUD has made this data available at the
census tract level quarterly on its web site. We have this data from the third quarter 2010 that will
provide a baseline picture of vacancies in the neighborhood. HUD is currently negotiating with USPS
about access and dissemination of future time periods, so availability of future data will depend on the
conditions of that agreement.
Dates for analysis: Third quarter 2010
American Housing Survey (AHS): The AHS is conducted by the Bureau of the Census for the Department
of Housing and Urban Development (HUD). It collects data on the Nation's housing, including
apartments, single-family homes, mobile homes, vacant housing units, household characteristics,
income, housing and neighborhood quality, housing costs, equipment and fuels, size of housing unit,
and recent movers. National data are collected in odd numbered years, and the sample covers an
average 55,000 housing units. The primary use for the AHS in this analysis is to have a national
comparison for selected housing quality indicators from the baseline survey of residents.

64

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Dates for analysis: 2007 Boston Metropolitan Area; 2009 (National, New Orleans & Chicago
Metropolitan Areas); 2011 (National – with metropolitan estimates available for 30 areas).
3.6.1.3 State and Local Government Sources

Our priority will be to negotiate access to state or local administrative data for the entire city or county.
If the source is regional or statewide, we will also request data for the remainder of the metropolitan
area. Dates will vary by site, but we will request access to up to three years’ historic data as well as
ongoing access to semi-annual or annual releases through 2013. Whenever possible we will obtain the
local data set at the address-level or aggregated to the smallest available geographic level where
addresses are not available. We will not request names, social security numbers, or other personal
identifiers. In our experience, successful local data requests take an average of three to six months to
fulfill.
We have already scanned online sources for publicly available data. Most of the data for the three sites
are not in the current holdings of our NNIP partners, but we are seeking their guidance to learn of any
experience in requesting the data or if they have personal contacts in the local agencies. We have had a
productive phone call with the Department of Neighborhood Development in Boston, verifying that
many of the housing and crime files are available either at record level or for the neighborhood as a
whole. We have a call scheduled for early March with the New Orleans grantee, and in Chicago, we will
work with Chapin Hall as the primary data collector to coordinate overlapping data requests. We will use
our standard protocols for gaining access to up to three years’ historic data as well as ongoing access to
reports during the evaluation period. The list of relevant local data sources are below (see section 2.4.4
for specific indicators sought). We will prioritize the first-tier data sources (home sales, crime, GIS
shapefile coverages, and school data). These are ones that we think will have the greatest payoff for
tracking the Choice Neighborhoods Initiative, and generally are obtainable. The second-tier sources also
offer important information. We will investigate and request these data sources, but will conduct
intensive follow-up and analysis as time allows.
First-tier data sources
Property-sales data
Reported Crime
Assessors’ parcel data
School-level performance and enrollment data (most likely through public online sources)
GIS coverages, such as land use, parcel boundaries, building footprints, streets, sidewalks, parks, and
school catchment boundaries (most likely through public online sources)
Second-tier data sources
Foreclosure notice and/or sale data
Building permits

65

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Code enforcement
Student-level performance and enrollment data
Vital Statistics
Public Assistance records
3.6.1.4 Google StreetView Images

We decided that data collection for the block front survey with Street View is preferable to the on-site
coding and video collection by DIR described in the original proposal. We have confirmation from HUD
that use of Google Street View for this research is allowable, but we will not be able to archive the
images due to Google license restrictions. Its relatively lower cost will allow us to collect data on the all
of the neighborhood blocks with Street View coverage. The data for the Boston and New Orleans
neighborhood are fully covered and from 2009 and 2007/2008 respectively. Only half of the Chicago
neighborhood is covered (including major streets and throughways), and these images are dated
partially from 2007 and partially from 2009. We estimate that there are 300, 500, and 1200 observable
block fronts in the Choice Neighborhoods of Boston, Chicago, and New Orleans, respectively.
3.6.1.5 Private, For-sale Data

Given the limited resources for data purchase, we do not plan to rely heavily on commercial data.
However, we do plan to purchase business data to inform our understanding of the retail and
professional services in the neighborhood. We have experience with InfoUSA, a commercial database
marketing firm that maintains a proprietary database of over 14 million U.S. businesses organized by
industrial classifications. The company’s researchers compile information from telephone and business
directories and state and local government records, verifying records by telephone and updating records
with U.S. Postal Service information files. Before purchasing the data from InfoUSA, we will consult HUD
staff and NNIP colleagues for advice about possible alternative companies.

3.7 Block Front and Pedestrian Routes Surveys
3.7.1 Description
To the extent feasible, the data collected from secondary sources, such as parcel maps, and primary
sources such as the baseline survey and focus groups, and key informant interviews will be used to
characterize the neighborhood conditions of the Choice Neighborhoods as described in previous
sections. In the resident survey, we ask about residents’ perceptions of neighborhood conditions. We
will supplement these results with primary data collection of neighborhood conditions through the block
front and pedestrian routes surveys.
In both the block front and pedestrian routes surveys, we will be recording notes about the conditions of
block fronts in the Choice Neighborhoods. In both, we will create training protocols to ensure
consistency in ratings among the observers. The block front survey captures the physical appearance of
the block and the presence of key safety, disorder, and other neighborhood characteristics. The

66

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

pedestrian survey focuses on neighborhood conditions as they relate to a resident’s common walking
route, such as the way to a transit hubs or a grocery store.

3.7.2 Data Collection Instruments & Questions
3.7.1.1 Methodology

Block Front Survey
The block front survey will be conducted from November 2012 to March 2013, and uses methods
informed by a number of different studies including Robert Sampson’s research in Chicago (Earls, et al.
1995) and the Irvine Minnesota Inventory (Day, et al. 2005). Using a Google StreetView application
developed by Professor Andrew Rundle and his team at Columbia University, trained observers will
complete a short standard data collection form for each side of the street, resulting in two block front
observations for each pass down a street. The coding data are archived and downloadable for analysis.
Pedestrian Routes Survey
Drawing on the protocols from the Irvine Minnesota Inventory, at least one member of the grantee
team will accompany the UI-MDRC research team on a key pedestrian route within the Choice
Neighborhood. The survey will be conducted during the second site visit (scheduled for February/March
2013). The protocol allows observers with modest levels of training to document a variety of features of
the built environment that affect walkability. The walking tours will include one route from within the
focal development to a key destination (e.g., schools, commercial areas, or transit hubs). For
developments with several buildings, this will include blocks as pedestrians exit the development area.
We will consult with grantees during the first site visit to learn the strategic importance of different
routes in the overall Choice Neighborhoods Initiative, and then will select the route based on that
information. If resources allow and major changes have occurred, we will consider having the site-based
employee do a follow-up survey for the same route at a later time or for additional routes.
Before the tour, we will prepare block maps with locations and landmarks that we are able to identify
from secondary data sources. Each tour will be conducted once during daylight hours to record physical
conditions. If the chosen route is to a commercial area or transit hub that is also accessed at night, we
would also conduct the survey once after dark on a Friday or Saturday night to observe social activities,
lighting, and other elements that are expected to differ substantially after dark. The research team will
complete the protocol for each block segment along the route and take field notes and confirm
neighborhood features, including barriers to walkability like highways, on maps of the route. The
research team will also measure the width of the sidewalk on each segment. The results of this protocol
will be the basis of the analysis for the final report. We also plan to record the daytime pedestrian
conditions tour on video for future research.

67

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE
3.7.1.2 Concepts and measures

To operationalize the research questions discussed in section 2.4, we have identified the specific aspects
of the built environment and street conditions to be measured. Certain questions may be removed or
replaced if the information can be gathered through secondary data sources. In addition, we recognize
that the time of year, the day of the week, and the time of the day can influence the results. For
example, it is less likely teens will be loitering in winter in Boston and Chicago. Once the final schedule
for the second site visit has been determined, we will review both proposed protocols to see if the
questions are appropriate for the month when the survey will be conducted. In addition, we will work
with DIR staff to determine the time of day and day of the week of the block front survey, and evaluate
questions in light of the determined day and time. See Appendices D and E for the detailed protocols.
Physical Conditions
Questions on the physical condition of blocks and buildings in the neighborhood will capture the quality
and state of the built environment in the Choice Neighborhood. In particular, we should note the main
uses of the neighborhood space.
Block front survey:
How is the land used on this block?
Do any buildings have major cracks or missing bricks, siding, or other outside wall material?
Are there abandoned buildings or lots on this block?
What is the condition of the road?
Is any significant barrier to walking present?
Pedestrian routes survey:
How is the land used on this block?
Public Safety
A sense of physical safety is an important quality of a neighborhood designed for walking and livability.
Concern for one’s own safety erodes trust in the neighborhood, limits the ability to travel in and out of
the neighborhood freely, and reduces access to key neighborhood amenities and services.
Block front survey:
How many buildings have windows with bars?
Are there window bars / gratings on residential doors or windows?
Are pull-down metal security blinds or iron gates on the fronts of commercial industrial
properties?
Do any buildings have broken or boarded up windows?

68

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

What kind of street lighting is there?
Is there visible electric wiring overhead?
Pedestrian routes survey:
How safe do you feel walking on this segment?
Are there other pedestrians present?
How many buildings have windows with bars?
What kind of street lighting is there?
Are there liquor stores, bars, or adult entertainment shops on the segment?
Are there homeless or begging people on the segment?
Traffic Safety and Pedestrian Conditions
Having a safe, easy walking routes enable pedestrians to access services, schools, and employment
centers around the city. We will be looking for barriers along common pedestrian routes, the existence
and condition of sidewalks, and safety at intersections.
Block front survey:
What kind of traffic signal is provided?
Are places that are intended for pedestrians to cross the street marked for pedestrian crossing?
Is there a speed bump or something else that is intended to slow down traffic?
Is there a bus stop or cable car stop available in the block front? If so, is there a shelter or bench
at the stop?
How far is the sidewalk or path from the curb?
What kind of on-street parking is there?
What type of sidewalk or path is there?
In what condition is the sidewalk or pedestrian path?
Pedestrian routes survey:
Are there clearly marked places for pedestrians to cross the street?
If there a permanent buffer (e.g. landscaped buffer strip, street trees, etc.) between sidewalk
and the street?
Are there parked cars on the street?
Are there barriers to walking present on the segment?
What is the general condition of the sidewalks?

69

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Neighborhood Livability
Visible evidence of disorder could signal more serious problems in the neighborhood. Do neighborhood
residents care about or have pride in where they live? Similarly, vacant lots or abandoned homes, cars,
or commercial establishments are signs of an unstable neighborhood and a weak local economy.
Block front survey:
How much litter is apparent on this segment?
Is there graffiti on buildings, signs, or walls?
Pedestrian routes survey:
Are there public spaces on the segment?
How much litter is apparent on this segment?
How much graffiti is apparent on this segment?

70

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

References
Abravanel, Martin D., Diane K. Levy, and Margaret McFarland. The Uncharted, Uncertain Future Of HOPE
VI Redevelopments: The Case for Assessing Project Sustainability. Washington, D.C.: The Urban
Institute, 2009.
Accordino, John, George Galster, and Peter Tatian. The Impacts of Targeted Public and Nonprofit
Investment on Neighborhood Development: Research based on Richmond, Virginia’s
Neighborhoods in Bloom Program, Richmond, VA. Richmond: Community Affairs Office of the
Federal Reserve Bank of Richmond, 2005.
Buron, Larry, Susan Popkin, Diane Levy, Laura Harris, and Jill Khadduri. The HOPE VI Resident Tracking
Study: A Snapshot of the Current Living Situation of Original Residents from Eight Sites. A Report
prepared by Abt Associates, Inc. and the Urban Institute for the U.S. Department of Housing and
Urban Development., Washington, D.C.: The Urban Institute, 2002.
Comey, Jennifer. HOPE VI'd and On the Move. Washington, D.C.: The Urban Institute, 2007.
Coulton, Claudia J., Brett Theodos, and Margery Austin Turner. Family Mobility and Neighborhood
Change: New Evidence and Implications for Community Initiatives. Washington, D.C.: The Urban
Institute, 2009.
Day, K., M. Boarnet, M. Alfonzo, and A. Forsyth. Irvine Minnesota Inventory (paper version). Accessed at:
, 2005.
Earls, Felton J., Stephen W. Raudenbush, Albert J. Reiss Jr., and Robert J. Sampson. "Project on Human
Development in Chicago Neighborhoods (PHDCN): Systematic Social Observation. Instrument for
ICPSR 13578." 1995.
Greenberg, David, Nandita Verma, and David C. Seith. The Double Bind of Redevelopment: Camden
During Receivership. Working Paper, New York: MDRC, 2009.
Greenberg, David, Nandita Verma, Keri-Nicole Dillman, and Robert Chaskin. Creating a Platform for
Sustained Neighborhood Improvement: Interim Findings from Chicago’s New Communities
Program. New York: MDRC, 2010.
Healey, Patsy. Collaborative Planning. London: Macmillan, 1997.
Joseph, Mark L, Robert J Chaskin, and Henry S Webber. "The Theoretical Basis for Addressing Poverty
Through Mixed-Income Development." Urban Affairs Review 42, no. 3 (2007): 369-409.

71

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Joseph, Mark, and Robert Chaskin. "Living in a Mixed-Income Development: Resident Perceptions of the
Benefits and Disadvantages of Two Developments in Chicago." Urban Studies 47, no. 11 (2010):
2347-236.
Kato, Linda Y., James A. Riccio, and Jennifer Dodge. Building New Partnerships for Employment:
Collaboration Among Agencies and Public Housing Residents in the Jobs-Plus Demonstration.
New York: MDRC, 2001.
Kingsley, G. Thomas, Martin D. Abravanel, Mary Cunningham, Jeremy Gustafson, Arthur J. Naparstek,
and Margery Austin Turner. Lessons from HOPE VI for the Future of Public Housing. Washington,
D.C.: The Urban Institute, 2004.
Kubisch, Anne C, P. Auspos, P. Brown, and T. Dewar. Voices from the Field III: Lessons and Challenges
from Two Decades of Community Change Efforts. Washington, D.C.: The Aspen Institute, 2010.
Levy, Diane K. The Limits of Relocation: Employment and Well-being Among Former Madded/Wells
Residents. CHA Families and the Plan for Transformation Brief 6, Washington, D.C.: The Urban
Institute, 2010.
Levy, Diane K., and Megan Gallagher. HOPE VI and Neighborhood Revitalization. Washington, D.C.: The
Urban Institute, 2006.
Mooney, Andrew J. "Building the Platform for Community Development." The Journal of the Institute for
Comprehensive Community Development, December 2010: 15-17.
Popkin, Susan J, Brett Theodos, Caterina Roman, and Elizabeth Guernsey. The Chicago Family Case
Management Demonstration: Developing a New Model for Serving the “Hard to House” Public
Housing Families. Washington, D.C.: The Urban Institute, 2008.
Popkin, Susan J, Brett Theodos, Liza Getsinger, and Joe Parilla. An Overview of The Chicago Family Case
Management Demonstration. Supporting Vulnerable Public Housing Families Brief No. 1.,
Washington, D.C.: The Urban Institute, 2010.
Popkin, Susan J., Bruce Katz, Mary K. Cunningham, Karen D. Brown, Jeremy Gustafson, and Turner A.
Margery. A decade of HOPE VI: Research findings and policy challenges. Washington, D.C.: The
Urban Institute and the Brookings Institution, 2004.
Popkin, Susan J., Diane K. Levy, and Larry Buron. "Has HOPE VI Transformed Residents’ Lives? New
Evidence From the HOPE VI Panel Study." Housing Studies 24, no. 4 (2009): 477-502.
Popkin, Susan J., Larry Buron, Diane K. Levy, and Mary K. Cunningham. "The Gautreaux Legacy: What
Might Mixed-Income Dispersal Strategies Mean for the Poorest Public Housing Tenants?"
Housing Policy Debate 11, no. 4 (2000): 911-942.

72

FINAL RESEARCH DESIGN, DATA COLLECTION AND ANALYSIS PLAN
CHOICE NEIGHBORHOODS DEMONSTRATION STUDIES TASK ORDER #1
URBAN INSTITUTE

Sampson, Robert J., Stephen W. Raudenbush, and Felton Earls. "Neighborhoods and violent crime: A
multilevel study of collective efficacy." Science 277 (1997): 918-924.
Sanbonmatsu, Lisa, et al. Moving to Opportunity for Fair Housing Demonstration Program - Final Impacts
Evaluation. Washington, D.C.: U.S. Department of Housing and Urban Development, 2011.
Schon, Donald A, and Martin Rein. Frame Reflection: Toward The Resolution Of Intractable Policy
Controversies. New York: Basic Books, 1995.
Smith, Robin E, et al. Monitoring Success in Choice Neighborhoods: A Proposed Approach to Performance
Measurement. Washington, D.C.: The Urban Institute, 2010.
Susskind, Lawrence E., Sarah McKearnen, and Jennifer Thomas-Lamar. The Consensus Building
Handbook: A Comprehensive Guide to Reaching Agreement. Thousand Oaks, CA: SAGE
Publications, Inc., 1999.

73