B_Research objectives and questions by data source

ATTACHMENT B

RESEARCH QUESTIONS, DATA SOURCES, AND KEY OUTCOMES

This page has been left blank for double-sided copying.

Attachment B. Overview of evaluation approach

Study component	Data sources	Key outcomes
Objective 1. 1. Document how each of the participating sites operates the pilot by describing project design, operations, and outcomes; by describing the exact nature of each treatment delivered; and by identifying specific lessons to improve policy and practice and promote replication of successful treatments in other locations.
Implementation	Site visits (1) planning and early implementation, 2) operations, and 3) full-implementation and close-out In-depth interviews with grantee/state-level staff, SNAP eligibility staff, and E&T provider staff Focus groups of participants Observations of activities Document review	Program features, processes, structure, and context Implementation activities Staff experiences and challenges Program measurement and performance
Objective 2. 2. Document short- and long-term causal impacts on participants’ activities, outputs, and outcomes, overall and as they vary with participants’ characteristics and program contexts
Impact	State Administrative records: Earnings, UI, SSI, SNAP, TANF, WIC, etc. Registration Document Follow-up surveys at 12 months and 36 months	Primary outcomes: Employment (job search behavior, quarterly employment status, job duration, number of jobs, occupation/industry, type of employment, advancement, etc.) Earnings (wages, hours, fringe benefits, earnings from off-the-books jobs, and transportation and child care costs) Receipt of public assistance (SNAP, TANF, WIC, SSI, UI, Medicaid, EITC, etc.) Secondary outcomes: Household food security, physical and mental health, self-esteem and self-efficacy, and housing status
Objective 3. 3. Examine the participation patterns—in offered employment and training services and in SNAP—in each pilot treatment and determine if participation varied within and across the pilots.
Participation	Administrative data on SNAP participation Registration Document Follow-up surveys at 12 months and 36 months	SNAP participation rate Number of SNAP applications Support services Eligibility for and referral to activities Start and end dates of service participation and completion status Receipt of support services, certifications, and post-employment services Perceptions of SNAP E&T
Objective 4. Provide a full social accounting from the perspectives of governments, other program partners (such as educational institutions and employers), and participants of the consequences of undertaking the policy alternatives implicit in each pilot project, both needed inputs and their costs and resulting outputs and benefits.
Cost - Benefit	Cost data Time use data Site visits MIS data Administrative data Baseline, 12 months, and 36 months survey	Costs: Total costs Per-component and pre-participations costs Benefits: Increased earnings and tax payments Reduced public assistance receipt Value of output produced during program Unmeasured benefits Cost-benefit summary: Net benefits Cost-benefit ratio