Instrument 6: Analysis plan for impact evaluations

THE PAPERWORK REDUCTION ACT OF 1995

This collection of information is voluntary and will be used to provide the Administration for Children and Families with information to help refine and guide SRAE program development. Public reporting burden for the collection of information is estimated to average 480 minutes per response, including the time for reviewing instructions, gathering and maintaining the data needed, and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The OMB number and expiration date for this collection are OMB #: XXXX-XXXX, Exp: XX/XX/20XX. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Susan Zief [email protected].

Instrument 6: SRAE Impact Evaluation Analysis Plan Template

Impact Evaluation Analysis Plan for [Grantee Name] – [DATE]

The Evaluation of [Intervention Name] in [Geographic Area]

1. Impact Study Research Questions

This section presents the primary and secondary research questions that will be assessed in the impact evaluation of [Intervention Name].

a. Primary research questions

[Copy and paste text/Start writing here.]

b. Secondary research questions

[Copy and paste text/Start writing here.]

2. Impact Study Design

This section provides a brief description of the study design and the process for creating intervention and comparison groups.

[Copy and paste text/Start writing here.]

3. Impact Analysis

This section lays out plans for constructing outcomes, defining the analytic sample, assessing baseline equivalence, cleaning data and handling missing data, addressing potential crossover and contamination, and finally the analytic models for estimating program impacts and planned sensitivity analyses.

a. Data cleaning

[Copy and paste text/Start writing here.]

b. Outcome measures

[Copy and paste text/Start writing here.]

Table 2. Outcomes used for primary research questions

Outcome name	Source item(s)	Constructed measure	Timing of measure
[Your outcomes here]			[Your outcomes here]

Table 3. Outcomes used for secondary research questions

Outcome name	Source item(s)	Constructed measure	Timing of measure
[Your outcomes here]			[Your outcomes here]

c. Analytic sample(s)

[Copy and paste text/Start writing here.]

d. Assessment of baseline equivalence

[Copy and paste text/Start writing here.]

e. Benchmark analytic approach for primary research questions

[Copy and paste text/Start writing here.]

i. Estimation approach

[Copy and paste text/Start writing here.]

ii. Adjustment for baseline differences

[Copy and paste text/Start writing here.]

iii. Additional covariates

[Copy and paste text/Start writing here.]

Table 4. Covariates included in impact analyses

Covariate	Description of the covariate

f. Analytic approach for secondary research questions

[Copy and paste text/Start writing here.]

g. Sensitivity analyses

[Copy and paste text/Start writing here.]

h. Bayesian interpretation

[Copy and paste text/Start writing here.]

4. Program Implementation Analysis

This section lays out plans for analyzing implementation data for understanding and documenting program implementation.

[Copy and paste text/Start writing here.]

Table 1. Planned implementation analysis

Implementation element	Research question	Measure	Data source(s)
[Your data here]

5. Additional planned analyses

[Copy and paste text/Start writing here.]