Z_MOU template

ATTACHMENT Z

SNAP E&T PILOTS MEMORANDUM OF UNDERSTANDING

This page has been left blank for double-sided copying.

EVALUATION OF SNAP EMPLOYMENT AND TRAINING PILOTS

MEMORANDUM OF UNDERSTANDING

between

MATHEMATICA POLICY RESEARCH, INC.

and

[FULL GRANTEE NAME]

Memorandum of Understanding No. [NUMBER]

1. PURPOSE OF AGREEMENT

The purpose of this Memorandum of Understanding (the “Agreement”) is to define the terms under which [FULL GRANTEE NAME] (herein referred to as “[SHORT GRANTEE NAME]”) with assistance from [LIST ALL PARTNERS] will work with Mathematica Policy Research, Inc. (Mathematica) with assistance from subcontractors MDRC, Insight Policy Research, Koné Consulting, and Decision Information Resources (hereinafter referred to as the “Mathematica Team”) to strengthen the pilot project design, conduct random assignment, recruit participants for the pilot project, identify site support and technical assistance (TA) needs, and provide requisite data to answer research questions. A detailed statement of work is provided in Exhibit A. This MOU also includes a Data Use Agreement in Exhibit B to provide additional information about the permitted uses, disclosures, and protections of data provided by [SHORT GRANTEE NAME] to the Mathematica Team.

2. BACKGROUND AND LEGAL AUTHORITY

The Agriculture Act of 2014 (P.L. 113-79, Section 4022) (the “Act”) authorizes the U.S. Department of Agriculture (USDA) to enter into cooperative agreements with state agencies to carry out pilot projects designed to raise employment, increase earnings, and reduce reliance on public assistance, including the benefits provided by the Supplemental Nutrition Assistance Program (SNAP). Pursuant to the authorizing statute, the USDA awarded grants to pilot projects in California, Delaware, Georgia, Illinois, Kansas, Kentucky, Mississippi, Virginia, Vermont, and Washington State.

The Act also directs the USDA to undertake an independent longitudinal evaluation of each pilot project using statistical methods that can determine differences in employment, earnings, and public assistance expenditures between those who receive the employment and training programs and services offered under the pilots and a control group that does not receive such services. The USDA, Food and Nutrition Services (FNS), has contracted with Mathematica (under Contract No. AG-3198-B-15-0002/Task Order No. AG-3198-K-15-0003) to conduct the Evaluation of the SNAP Employment and Training (E&T) Pilots. This congressionally-mandated evaluation will rigorously test innovative strategies for increasing employment and earnings among SNAP participants and reducing their dependence on SNAP and other public assistance programs. The evaluation includes four components: 1) an implementation analysis, 2) a random-assignment impact evaluation, 3) a participation analysis, and 4) a cost-benefit analysis. Data sources will include unemployment insurance wage records, SNAP caseload files and other administrative data, service receipt data, baseline and follow-up household surveys, interviews from site visits and focus groups, and pilot cost data and frontline staff time-use surveys.

[SHORT GRANTEE NAME] will complete the specified work under this Agreement as part of the terms and conditions of its contract with FNS, under USDA FNS Pilot Projects to Reduce Dependency and Increase Work Requirements and Work Effort under the Supplemental Nutrition Assistance Program, Grant CFDA 10.596. Under separate subcontract agreements with Mathematica, the Mathematica Team members - MDRC, Insight Policy Research, Koné Consulting, and Decision Information Resources - will assist Mathematica in the execution of this Agreement by providing guidance and direction to [SHORT GRANTEE NAME] to facilitate completion of the statement of work.

This Agreement contains several integral components which are outlined below to provide an overview of the structure and topics covered, attached hereto, and incorporated into this Agreement by reference:

Exhibit A: Statement of Work. Exhibit A provides provide background on the evaluation’s research objectives and components of the study design. Additionally, it outlines the responsibilities of [SHORT GRANTEE NAME] and the Mathematica Team, presented separately for each activity related to the evaluation. Activities related to the evaluation include managing participating providers and partners, planning and overseeing implementation of the study, obtaining consent from eligible participants and collecting information through a registration document, conducting random assignment of participants and maintaining the integrity of random assignment, and collecting administrative and other data on pilot participants. Finally, the schedule of study activities provides detail on the responsibilities of [SHORT GRANTEE NAME] and the Mathematica Team throughout the phases of the evaluation.

Exhibit B: Data Use Agreement (DUA). Exhibit B details the purpose and variables needed from the four data sources required for the evaluation, including (1) SNAP administrative data files, (2) pilot service receipt data files, (3) pilot cost data files, and (4) unemployment insurance data (through separate agreement with Mathematica’s subcontractor MDRC). The DUA provides information on secure transmission of these data and the protections used for maintaining data security and confidentiality, including the Mathematica Team’s use of these data.

Attachments: The attachments to this Agreement include: (1) a pilot project flow chart illustrating [SHORT GRANTEE NAME] plans for pilot services and point of random assignment, (2) a summary of target pilot sample enrollment goals by month for the duration of pilot services, (3) a list of performance targets for the duration of pilot services, (4) the draft registration form required to enroll each eligible pilot participant in the evaluation, and (5) the draft consent form for pilot participants.

3. TERM OF AGREEMENT

This Agreement is effective upon the date of last signature by authorized representatives of each of Mathematica and [SHORT GRANTEE NAME]. It will terminate on March 31, 2021 unless otherwise extended by the mutual written agreement of the parties.

4. POINTS OF CONTACT

The following individuals will serve as the points of contact responsible for managing the performance of each party’s necessary functions and responsibilities under this Agreement:

MATHEMATICA POLICY RESEARCH, INC.

For administrative/contractual matters:

William (Bill) Farino, CPCM

Sr. Contracts Administrator

Mathematica Policy Research, Inc.

955 Massachusetts Avenue, Suite 801

Cambridge, MA 02139

617-715-6940 (P)

[email protected]

For technical matters:

James Mabli

Associate Director of Human Services Research

Mathematica Policy Research, Inc.

955 Massachusetts Avenue, Suite 801
Cambridge, MA 02139
617-491-7900 (P)

[email protected]

[FULL GRANTEE NAME]

NAME

TITLE

[FULL GRANTEE NAME]

ADDRESS

PHONE

[email protected]

5. MODIFICATION

Any changes, amendments, or modifications to this Agreement must be made and agreed to by the parties in writing. Notwithstanding the foregoing, the parties may mutually agree to modify the data tables set forth in Exhibit B without modifying this Agreement.

6. INDEMNIFICATION AND HOLD HARMLESS CLAUSES

To the extent permitted by applicable law, each party agrees to indemnify, defend, and hold harmless the other party, its officers, agents, and employees from and against any and all claims, actions, losses, damages, injuries, liabilities, suits, and costs and expenses, including reasonable attorneys' fees and expenses, arising out of or related to the performance of this Agreement by the indemnifying party, its officers, agents, or employees.

7. TERMINATION PROCESS

Should Mathematica's contract with FNS be terminated, Mathematica shall provide written notice to [SHORT GRANTEE NAME]. This termination will not affect the [SHORT GRANTEE NAME] contract with FNS and [SHORT GRANTEE NAME] is still responsible to continue with work specified under USDA FNS Grant CFDA 10.596.

8. ENTIRE AGREEMENT

This Agreement, including the exhibits incorporated herein by reference, constitutes the parties’ entire agreement with respect to the subject matter hereof and supersedes any and all prior statements or agreements, both written and oral.

9. COUNTERPARTS

This Agreement may be signed in one or more counterparts, each of which will be deemed an original, but all of which will be deemed one instrument.

IN WITNESS WHEREOF, the parties hereto have caused this Memorandum of Understanding to be executed by their duly authorized representatives as of the date of the last signing party below.

MATHEMATICA POLICY RESEARCH, INC.

______________________________

Signature and Date

______________________________

Name

______________________________

Title

[FULL GRANTEE NAME]

______________________________

Signature and Date

______________________________

Name

______________________________

Title

EXHIBIT A: STATEMENT OF WORK

This Exhibit A describes the scope of work to be conducted under the Memorandum of Understanding (the “Agreement”) to which it is attached by [FULL GRANTEE NAME], herein referred to as “[SHORT GRANTEE NAME]”, for the Evaluation of SNAP Employment and Training Pilots project led by Mathematica Policy Research, Inc. (Mathematica) with assistance from subcontractors MDRC, Insight Policy Research, Koné Consulting, and Decision Information Resources (hereinafter referred to as the “Mathematica Team”), for the U.S. Department of Agriculture, Food and Nutrition Service (Task Order AG-3198-K-15-0003). As part of this project, [SHORT GRANTEE NAME] will work with the Mathematica Team to strengthen the pilot project design, conduct random assignment, recruit participants for the pilot project, identify site support and technical assistance needs, and acquire requisite data to answer research questions.

To support evaluation activities, the Mathematica Team will provide evaluation technical assistance on the pilot and evaluation designs and guidance on providing necessary data for the evaluation. [SHORT GRANTEE NAME] will work with FNS and the Mathematica Team to finalize the pilot design, and will provide the data in accordance with the timeline specified in this Exhibit, in addition to other activities considered integral to the evaluation.

1. BACKGROUND AND RESEARCH OBJECTIVES

1.1 Background

The Supplemental Nutrition Assistance Program (SNAP) is a critical work support for low-income people and families. SNAP benefits help eligible low-income families put food on the table in times of need. It also supports critical and needed skills and job training so that recipients can obtain good jobs that lead to self-sufficiency. To help unemployed and underemployed recipients and their families achieve self-sufficiency, strategies are needed to impart the skills employers want, and to help address other barriers to employment. Some participants need assistance developing a resume and accessing job leads, others need education and training, and still others need help overcoming barriers that prevent them from working steadily. The SNAP Employment and Training (E&T) program provides assistance to unemployed and underemployed participants in the form of job search, job skills training, education (basic, post-secondary, vocational), work experience or training, and workfare, but limited information exists on what is most effective in connecting these participants to gainful employment.

The Agriculture Act of 2014 (PL 113-79, Section 4022), otherwise known as the 2014 Farm Bill, authorized grants for up to 10 pilot sites to develop innovative SNAP E&T strategies for engaging more SNAP work registrants in unsubsidized employment, increasing participants’ earnings, and reducing reliance on public assistance. The pilots’ significant funding can expand the reach of employment and training services and enable States to experiment with promising strategies to increase engagement and promote employment. The USDA awarded grants to pilot projects in California, Delaware, Georgia, Illinois, Kansas, Kentucky, Mississippi, Virginia, Vermont, and Washington State.

1.2 Research Objectives

The Farm Bill also provides funding for rigorous evaluation of the 10 pilots. For each pilot, the study’s research questions are:

• How do pilot project services affect SNAP recipients’ employment rates, earnings, and receipt of public assistance? How do they affect other outcomes such food security, well-being, health and housing status? What is the effect of these services on subgroups of recipients defined by participant and program characteristics?

• What are the characteristics and service paths of pilot project participants? Do the presence of the pilots and their offer of services or requirements to participate affect whether people apply to SNAP?

• How are these services implemented, and how do differences in implementation affect participants’ employment, earnings, and related outcomes?

• Are the benefits of these services measured in dollars greater than their costs?

1.3 Study Design

An experimental research design will rigorously test pilot project strategies for increasing employment and earnings among SNAP participants and reducing their dependence on SNAP and other public assistance programs. The design includes four primary research components: (1) an implementation analysis, (2) a random-assignment impact evaluation, (3) a participation analysis, and (4) a cost-benefit analysis.

(1) An implementation analysis will document the context and operations of each pilot as well as help to interpret and understand impacts within and across pilots. The Mathematica Team will conduct three rounds of process study data collection, one visit per year with a specific focus. Visits will last three to four days and will include interviews with staff at all levels (grantees/State-level staff, SNAP eligibility staff, E&T provider staff, and DOL staff), structured observations of SNAP offices and E&T service provider operations, in-depth interviews with participants, and focus groups with E&T participants and employers. The first round of site visits will collect data on planning and early implementation and is scheduled after the beginning of random assignment. The second round of site visits will collect data on operations and is scheduled for the mid-point of operations. The third will collect data on each pilot’s full implementation and their pilot close-out processes near the end of operations. These data will be supplemented with data from participant case files, document reviews, and program administrative data. The three rounds of site visit data also will be supplemented with data collected through the TA/monitoring site visits and multiple grantee calls.

(2) A random-assignment impact evaluation will identify what strategies work and for whom, through estimating program impacts on primary outcomes of employment, earnings, and public assistance receipt; and secondary outcomes of food security, health, and housing status. In each pilot, SNAP participants eligible for the SNAP E&T pilot project services will be randomly assigned to at least two study groups: one or more control groups that receives no enhanced pilot intervention services, and one or more treatment groups that receive variations of enhanced pilot intervention services. Random assignment will typically be conducted at the individual level. Data sources used to estimate program impacts will include UI wage records, SNAP administrative data, service receipt data, and baseline and follow-up household surveys.

(3) A participation analysis will examine the characteristics and service paths of pilot participants to determine if participation varied within and across the pilots. Where feasible, it also will assess whether the presence of the pilots and their offer of services or requirements to participate affect whether people apply to SNAP (entry effects). This analysis will use management information system (MIS) data on service receipt, information from SNAP administrative records, and baseline and follow up survey data.

(4) A cost-benefit analysis will estimate the return to each dollar invested. All participating pilots will provide cost data at the completion of the planning period, and then quarterly thereafter and participate in an annual frontline staff time-use survey. The Mathematica Team will analyze these data together with outcome and service receipt data to determine the overall cost-benefit to the individual and to society. Data will be collected and analyzed for pilot treatment services, and for services provided to the control group under the existing SNAP E&T program to the extent possible.

2. UNDERSTANDING OF THE RESPONSIBILITIES OF [SHORT GRANTEE NAME] AND THE MATHEMATICA TEAM

2.1 Participating providers and partners

a. [SHORT GRANTEE NAME]

[SHORT GRANTEE NAME] will name a primary liaison to work with the evaluation team to support and ensure efficient program implementation. The primary liaison will participate in program planning and implementation, supervise all [SHORT GRANTEE NAME] research activities, and be responsible for all program site operations. The primary liaison should be authorized to speak for the [SHORT GRANTEE NAME] to the Mathematica Team, the [SHORT GRANTEE NAME] management group, and [SHORT GRANTEE NAME] research and site staff.

[SHORT GRANTEE NAME] also agrees to designate a project‐wide Research Liaison who will coordinate all [SHORT GRANTEE NAME] research activities. The Liaison will ensure that all site staff implement SNAP E & T research procedures as described in this statement of work. The duties of the Research Liaison include but are not limited to:

1. Supervise collection and storage of electronic data and paper documents containing data collected from study participants;

2. Support the Program Manager in serving as a liaison with the Mathematica Team to manage the secure transmission of paper and electronic data to appropriate evaluation partners;

3. Answer questions from staff about research procedures;

4. Produce aggregate reports separately for pilot activities; transmit electronic files in a secure manner, according to evaluation partners’ specifications;

5. Document participant’s consent to participate in the research study according to procedures developed by the Mathematica Team;

6. Conduct random assignment using an online assignment tool and database maintained by the Mathematica Team and ensure individuals are clearly identified by and treated programmatically according to the research group they have been assigned to;

7. Ensure that all records (such as study informed consent forms, baseline data, and contact information) completed by study participants are handled and submitted to the Mathematica Team according to the guidelines provided;

8. Participate in required check‐in calls with the Mathematica Team;

9. Ensure that all staff receive required pilot and research training, general and specific to service operations;

10. Ensure that staff market each of the programs in a consistent and accurate way;

11. Ensure that pilot staff implement the program services as described in [SHORT GRANTEE NAME] implementation plan, monitor program performance, and take into account feedback and technical assistance provided by the Mathematica team.

12. Supervise all data management, collection and reporting activities and ensure that all staff adheres to required security protocols, including the proper transmission and storage of both data and documents.

[SHORT GRANTEE NAME] will engage with a number of partners, service providers, and other organizations connected to the pilot in the implementation of the pilot project and the study, in accordance with the proposal under USDA FNS Grant CFDA 10.596. [SHORT GRANTEE NAME] will ensure that all service providers and other partners under the pilot project (defined below) adhere to the responsibilities outlined in this agreement for the full term of the agreement. Participating entities include:

[NAMES/ADDRESS OF ALL PROVIDERS (BULLET LIST)]

2. Planning and implementation of the study

a. Phases of the study

There are three phases to the study, all involving close collaboration between the [SHORT GRANTEE NAME], the Mathematica Team, and FNS to determine when each phase will transition to the next:

• Phase I is the planning phase during which the program activities and flow are defined and the evaluation design is determined. This phase includes training of [SHORT GRANTEE NAME] staff on random assignment and research procedures, as well as other aspects of the study.

• Phase II is an initial implementation phase of random assignment and pilot activities, during which the [SHORT GRANTEE NAME] and the Mathematica Team, in close collaboration with FNS, will have the opportunity to assess pilot and research activities, discuss and agree upon any needed changes to either the research or program flows and activities, and assess whether these activities are sufficiently strong that this Phase II sample will be included in the full evaluation study. Important performance goals that will be assessed include meeting sample build-up targets for Phase II, creating a sufficiently large service contrast between the research groups, and reaching adequate pilot participation rates according to agreed upon performance goals (see Attachment 3).

• Phase III is the full evaluation period, including continuing random assignment and sample build-up, collecting data for monitoring of program performance, and collecting data for the impact, implementation, participation, and cost studies.

b. Planning and Implementation Responsibilities of [SHORT GRANTEE NAME]

[SHORT GRANTEE NAME] agrees to work with the Mathematica Team to implement the study in accordance with the procedures laid out in this Agreement. [SHORT GRANTEE NAME] will inform the Mathematica Team about the pilot project design and specific intervention services. [SHORT GRANTEE NAME] will work with the Mathematica Team to determine any changes needed to local computer systems to accommodate the study, including the web-based information systems detailed below in section 2.4. [SHORT GRANTEE NAME] agrees to demonstrate flexibility in working with the Mathematica Team regarding the design and implementation of the pilot project, including the addition or enhancement of pilot project activities or services which will serve to strengthen the pilot and/or national evaluation design. [SHORT GRANTEE NAME] will collaborate with the Mathematica Team to determine the optimal point in the program to randomly assign pilot project participants to the treatment group(s) or the control group. [SHORT GRANTEE NAME] will implement procedures to ensure legally effective and prospectively obtained informed consent from pilot participants that is valid for up to six years after the participants enter the pilot. [SHORT GRANTEE NAME] will provide and support the interpretation of data on pilot participants (described in Exhibit B) for up to six years after the participants enter the pilot. Detail on the pilot project flow specific to [SHORT GRANTEE NAME] is provided in Attachment 1.

[SHORT GRANTEE NAME] will support the Mathematica Team in the completion of a state-specific or provider specific institutional review board (IRB) research applications as needed by facilitating communications with relevant entities and working to expedite the review process where possible.

[SHORT GRANTEE NAME] will allow time for all staff who will be involved in the study to be trained by the Mathematica Team, including time for new staff trainings and refresher trainings as needed. [SHORT GRANTEE NAME] will train staff, partners, and service providers on the pilot project and evaluation procedures.

[SHORT GRANTEE NAME] will collaborate with the Mathematica Team to develop strategies for effective recruitment of the target population(s) to reach agreed upon research sample targets. (Attachment 2 provides additional detail on sample size targets). During Phase II, [SHORT GRANTEE NAME] will provide to the Mathematica Team bi-weekly data on specific performance goals agreed upon by [SHORT GRANTEE NAME] and the Mathematica Team. These data will be used by both parties to determine if random assignment and the program flow are being executed as anticipated. Data include sample build-up and pilot service receipt. During Phase III, these data will be provided on a monthly basis.

[SHORT GRANTEE NAME] and their partners will participate as needed in on-site visits conducted by the Mathematica Team. [SHORT GRANTEE NAME] will facilitate access to interviews with grantee staff, agency staff, pilot partner and provider staff, and evaluation sample participants. These visits will include conversations regarding the planning and design of pilot project services and the evaluation, as well as monitoring of random assignment throughout the period of random assignment. [SHORT GRANTEE NAME] will host the Mathematica Team site liaisons and help arrange for visits during the period of this agreement. During or in preparation for these visits, [SHORT GRANTEE NAME] will furnish information requested by the study team.

[SHORT GRANTEE NAME] agrees to maintain frequent contact with FNS and the Mathematica Team, and to inform FNS and the Mathematica Team within one week of identifying a challenge which may impact the program or evaluation design or implementation. Regular standing phone calls between [SHORT GRANTEE NAME], the Mathematica Team and FNS will be set up at a mutually agreeable time.

c. Planning and Implementation Responsibilities of the Mathematica Team

The Mathematica Team will support [SHORT GRANTEE NAME] in developing a strong pilot project and evaluation design, through (1) identification of additions or enhancements to pilot project activities or services and; (2) technical support for developing efficient recruitment strategies, including how to monitor recruitment, minimize drop off, improve locating the target population(s), and design strong recruitment messaging. The Mathematica Team will discuss with [SHORT GRANTEE NAME] the optimal point in the program to randomly assign participants to the treatment group(s) or the control group to maximize what can be learned from the evaluation.

The Mathematica Team, in collaboration [SHORT GRANTEE NAME], will lead the completion of state or provider-specific IRB applications.

The Mathematica Team will offer training and training materials to all staff who will be involved in the study, including how to administer the consent form and registration document, how to enter data into the random assignment system (discussed further in sections 2.3 – 2.4 below), and the importance of implementing random assignment correctly. This may include customized procedure manuals where needed and in-person and web-based trainings on study procedures.

The Mathematica Team will provide real-time technical assistance and formative feedback as necessary to [SHORT GRANTEE NAME] through (1) on-going monitoring of reports and information gathering visits; (2) targeted technical assistance visits providing feedback on program operations and research-related practices, as needed; and (3) regular teleconferences with [SHORT GRANTEE NAME].

The Mathematica Team will conduct a number of visits to [SHORT GRANTEE NAME] during the evaluation. During the period of preparation for the study (Phase I), the Mathematica Team will schedule and conduct approximately four visits, and during the period of random assignment (including the Phase II pilot testing of procedures and Phase III full implementation) will schedule and conduct approximately nine visits. In addition, the Mathematica Team will conduct three rounds of process study data collection, one round per year with a specific focus. Visits will typically last three to four days and will include interviews with staff at all levels (grantees/State-level staff, SNAP eligibility staff, E&T provider staff, and DOL staff), structured observations of SNAP offices and E&T service provider operations, and focus groups with E&T participants and employers. The first round of site visits will collect data on planning and early implementation and is scheduled after the beginning of random assignment. The second round of site visits will collect data on operations and is scheduled for the mid-point of operations. The third will collect data on each pilot’s full implementation and their pilot close-out processes near the end of operations. These data will be supplemented with data from participant case files, document reviews, and program administrative data. Site liaisons will work with [SHORT GRANTEE NAME] to schedule visits in order to minimize burden on the site staff. Activities may include (1) interviews with administrators, staff, partner staff, and participants; (2) observations of program activities; (3) a review of a sample of individual pilot-participant case files; and (4) a review of program documents. The site liaisons will provide ongoing feedback to [SHORT GRANTEE NAME] based on visits, data, and other communications.

The Mathematica Team will maintain frequent contact with [SHORT GRANTEE NAME] and identify solutions to challenges presented. Regular standing phone calls between [SHORT GRANTEE NAME], the Mathematica Team and FNS will be set up at a mutually agreeable time.

2.3 Completion of consent, the registration document

a. [SHORT GRANTEE NAME]

[SHORT GRANTEE NAME] staff, partners, service providers, and other relevant organizations will use the Mathematica Team’s web-based system to (1) document an individual’s consent to participate in the study and (2) complete the registration document, for all those consenting to participate in the pilot. Once a potential participant is deemed eligible for pilot services, [SHORT GRANTEE NAME] staff, partners, service providers, and/or other relevant organizations will provide potential participants’ with the evaluation consent form (Attachment 5), describe the evaluation, and obtain the participant’s consent. Program staff will record consent in the web-based registration document (see Attachment 4). Participants who do not consent will be excluded from the study and will not be eligible to receive any pilot-funded treatment services.

Once a participant has consented to participate in the study, the staff member will walk the participant through a short registration document, which the staff person will complete via the Mathematica Team’s web-based information system (see Attachment 4). The document will collect basic information on all participants’ demographic and household characteristics; baseline measures of outcomes such as earnings, employment status, and public assistance program participation; staff predictions on likely services received by the participant if assigned to the treatment group(s), and how to locate the participant for follow-up data collection. These activities will occur prior to random assignment of the participant to the treatment group(s) or control group.

[SHORT GRANTEE NAME] will work with the Mathematica Team to set up a centralized process to optimize efficiency, accuracy, and timely completion of these activities. Detail on this process specific to [SHORT GRANTEE NAME] is provided in Attachment 1.

b. The Mathematica Team

The Mathematica Team will provide access to a web-based information system which [SHORT GRANTEE NAME] staff, partners, service providers, and other relevant organizations will use to (1) document an individual’s consent to participate in the study and (2) complete the registration document, for all those consenting to participate in the pilot. See Attachments 4 and 5 for drafts of related documents. The Mathematica Team will obtain and maintain approval of the above forms from the Office of Management and Budget (OMB) and relevant institutional review boards. The Mathematica Team will provide technical support to [SHORT GRANTEE NAME] to train users on how to introduce and complete the documents with study participants, including providing scripts in easy-to-understand language. The Mathematica Team will provide timely technical support to [SHORT GRANTEE NAME] as needed in the event of problems involving the web-based system.

2.4 Conducting random assignment and maintaining the integrity of random assignment

a. [SHORT GRANTEE NAME]

Once the registration document is complete, the staff member will conduct random assignment through the Mathematica Team’s web-based system. The random assignment system will validate information as it is entered and indicate any incomplete or duplicate data entries. The completion of data entry into the random assignment system will occur only after completion of the consent and registration document.

Once data are entered into the random assignment system, the system will randomly assign the participant to a research group and assign a unique identifier that will be used to track the participant throughout the study. The system will indicate the results of random assignment immediately, and the staff member will inform the participant of the result. The system will produce a letter that can be given to the participant that notifies them of their random assignment result.

On a case-by-case basis, [SHORT GRANTEE NAME] can request from the Mathematica Team an exemption from random assignment. This request will be made only in extraordinary circumstances and must be made by [SHORT GRANTEE NAME] and approved by the Mathematica Team prior to random assignment.

To ensure a strong random assignment evaluation design, [SHORT GRANTEE NAME] will recruit a targeted number of eligible pilot participants so that sample sizes in each research group will be large enough to produce precise impact estimates. The total target sample sizes for [SHORT GRANTEE NAME] are [##] for the treatment group that receives services [XX], [##] for the treatment group that receives [XX], and [##] for the control group. The anticipated start date for sample enrollment is [Month DD, YYYY] and the anticipated end date is [Month DD, YYYY] or until the Phase II and Phase III target samples for each research group are achieved. Attachment 2 provides additional detail on target sample sizes during Phases II and III.

[SHORT GRANTEE NAME] will collaborate with the Mathematica Team to implement and maintain a clear contrast between the services received by each treatment and control group. In addition, [SHORT GRANTEE NAME] will ensure that staff, partners, service providers, and other relevant organizations only provide pilot-funded services to study participants for which their assigned research group is eligible. The service embargo period is the time during which participants must stay in the research group to which they were randomly assigned. Each sample member’s service embargo period is six years starting from the point he or she was randomly assigned. During this period, sample members already assigned to a research group may not receive services that are outside of those permitted for their research group. If program staff are unsure of an individual’s research status, they can look it up using the Mathematica Team’s random assignment system. In addition, if an individual re-applies for services and is re-sent for random assignment, the random assignment system will identify this individual and return the original random assignment result.

b. The Mathematica Team

The Mathematica Team will provide technical support to [SHORT GRANTEE NAME] to train users on how to inform participants of their assignment status and explain the types of services they are entitled to receive given their group assignment, including a script in easy-to-understand language. The Mathematica Team will provide technical support to [SHORT GRANTEE NAME] on the implementation and maintenance of a strong random assignment evaluation design, including maintaining a clear contrast between the research treatment group(s) and control group services and developing adequate sample sizes and/or accelerated sample build-up. The Mathematica Team will review data from the web-based information system on random assignment and on service receipt (as detailed below in section 2.5 and in Exhibit B) to validate the accurate execution of the random assignment evaluation design. The Mathematica Team will similarly be conducting case file reviews as part of in-person follow-up visits during pilot testing and full implementation phases to ensure pilot services and evaluation procedures are implemented correctly.

2.5 Collection of administrative data and other types of data

a. [SHORT GRANTEE NAME]

[SHORT GRANTEE NAME] agrees to provide the Mathematica Team with data on pilot activities and participants in both the treatment and control groups, including (1) SNAP electronic administrative data files, (2) electronic pilot service receipt data files, and (3) pilot cost data files. See Exhibit B, Data Use Agreement for further detail on the data to be provided to the Mathematica Team.

[SHORT GRANTEE NAME] will describe existing or new SNAP, State agency, or other provider data systems to the Mathematica Team and will connect staff data users, programmers, and experts to the Mathematica Team to answer questions and to develop a plan for providing data to the Mathematica Team. [SHORT GRANTEE NAME] will discuss the optimal way to capture necessary data either in the MIS with additional data items or through a supplemental data tracking system. [SHORT GRANTEE NAME] or other relevant state agency will enter into separate agreement with Mathematica’s subcontractor MDRC to provide unemployment insurance data. [SHORT GRANTEE NAME] will transmit these data to the Mathematica team for all pilot participants in the treatment and control groups until the date specified in the data use agreement, Exhibit B. [SHORT GRANTEE NAME] will provide assistance to the Mathematica Team on data use and systems for up to six years, the term of this agreement.

b. The Mathematica Team

The Mathematica Team will provide guidance to [SHORT GRANTEE NAME] on data collection needs in each of the four types of data: (1) SNAP administrative data, (2) service receipt data for treatment and control groups, (3) pilot cost data for treatment and control services and (4) unemployment insurance data (as mentioned, through separate agreement with MDRC). The Mathematica Team will provide a secure web-based system for the transfer of all data and technical support to [SHORT GRANTEE NAME] and other relevant partners. To minimize burden on [SHORT GRANTEE NAME], the Mathematica Team will coordinate the timing of data requests to coincide as much as possible with routine data-related activities, such as data submissions to the state.

3. SCHEDULE

The schedule for the activities specified in this agreement is outlined below. These dates are subject to change and the Mathematica Team and [SHORT GRANTEE NAME] will come to agreement on any revisions to the dates below.

TABLE 3.1. TIMELINE SNAP E&T PILOTS PROGRAM AND EVALUATION ACTIVITIES [CUSTOMIZE TABLE]

	Activity	Expected Completion Date
PHASE I	Collaborate with FNS and the Mathematica Team to develop program and evaluation design	April 2015 – [Month, YYYY] target: October 2015
PHASE II	Start random assignment of eligible participants to “pretest” service delivery and evaluation processes	[Month, YYYY] target: November 2015, 2 – 4 months pretest
PHASE III	Continue random assignment, service delivery, and evaluation processes for Grantees	[Month, YYYY] - [Month, YYYY] target: December 2015 12 – 18 months, 24 months maximum
PHASE IV	Mathematica team releases study findings	Interim 2019 Final 2021

⁺ Dates and time commitment estimates are subject to change.

EXHIBIT B: DATA USE AGREEMENT

THIS DATA USE AGREEMENT (“DUA”) is entered into by and between the [FULL GRANTEE NAME] (herein referred to as “[SHORT GRANTEE NAME]”) who, with assistance from [LIST ALL PARTNERS], will work with Mathematica Policy Research, Inc. (Mathematica) with assistance from subcontractors MDRC, Insight Policy Research, Koné Consulting, and Decision Information Resources (hereinafter referred to as the “Mathematica Team”) to perform the obligations set forth in the Memorandum of Understanding (the “Agreement”) to which this Exhibit B is attached. Hereinafter, [SHORT GRANTEE NAME] and the Mathematica Team may be referred to singularly as a “Party” and collectively as the “Parties.” This DUA stipulates the conditions upon which [SHORT GRANTEE NAME] will provide confidential information to the Mathematica Team for use in performance of the Evaluation of SNAP Employment and Training Pilots.

INFORMATION TO BE PROVIDED. [SHORT GRANTEE NAME] agrees to provide Mathematica Team with (1) SNAP administrative data files, (2) pilot service receipt data files, (3) pilot cost data files, and (4) unemployment insurance data (through separate agreement with Mathematica’s subcontractor MDRC). The requirements of data files 1-3 are outlined in Tables 1 - 3 below. If a specific data requirement does not exist in the [SHORT GRANTEE NAME] data system, [SHORT GRANTEE NAME] and Mathematica will discuss solutions and alternatives.

(1) SNAP administrative data will be used to define primary outcomes and subgroups for analysis. The data request is for unit-level data, not individual-level data, and includes only evaluation sample members (all pilot project participants, each member of treatment and control), not the full caseload. For each evaluation sample member, the SNAP administrative data request includes the 12 months before random assignment, the month of random assignment, and each month after random assignment. [SHORT GRANTEE NAME] will provide monthly data files [monthly, quarterly] [additional schedule detail as needed]. The data will be provided to Mathematica’s subcontractor, Insight Policy Research for cleaning and preparation of the analysis file. [SHORT GRANTEE NAME] will transmit data to Insight Policy Research through a secure file transfer protocol as agreed upon with Mathematica. Separate requests for aggregate count data on SNAP applicants or participants by geographic areas such as counties may also be made. Table 1 provides further detail on the timeline and variables requested.

TABLE 1. SNAP, TANF, AND MEDICAID ADMINISTRATIVE DATA VARIABLES [CUSTOMIZE TABLE]

Timeline
Monthly data to be collected every three months starting in first month of random assignment, with the first extract containing the past 12 months of data, then three months of data each following quarter.
Variables
Basic unit information	Unit Benefit amounts for each program for each month
Case ID – unique identifier of unit’s case	SNAP indicator, month X (whether unit currently has active case)
Date of SNAP certification	SNAP monthly benefit amount, month X
Date of most recent SNAP recertification	Medicaid indicator, month X
Length of certification period	Medicaid claims, month X
County where application was processed (FIPS code)	TANF indicator, month X (presence of TANF income in month X)
	TANF monthly benefit amount, month X
Unit income (as of most recent certification)
Unit’s gross countable monthly income	Contact information
Unit’s net countable income	First and last name of unit head
Presence of earned income	Address (residential and mailing)
Presence of unearned income	Telephone numbers (include all available numbers on the file)
Presence of GA income	Email
Presence of SSI income
Presence of Social Security income	For cases that are closed
	Reason for case closure
	Date of case closure
	For cases that are open, an indicator for whether the case is currently under sanction
Notes
Information from the TANF and Medicaid systems, including data on each SNAP participant, will be obtained for a single outcome measure from integrated systems only

(2) Pilot service receipt data from [SHORT GRANTEE NAME] will be used to assess pilot performance and describe participation and services received by evaluation sample members. During Phase II, [SHORT GRANTEE NAME] will provide bi-weekly data on specific performance measures and targets agreed upon by [SHORT GRANTEE NAME] and the Mathematica Team. These data will be used by both parties to determine if random assignment and the program flow are being executed as anticipated. Data include sample build-up and pilot service receipt. During Phase III, these data will be provided on a monthly basis. Performance data will be aggregate reports based on the MIS data collected by [SHORT GRANTEE NAME]. In addition to the aggregate reports, [SHORT GRANTEE NAME] will provide individual-level data on service receipt [monthly, quarterly] [additional schedule detail as needed]. These data will be transmitted to Mathematica through a secure file transfer protocol as agreed upon with Mathematica. Table 2 provides further detail on the universe, purpose, timeline, and variables requested.

TABLE 2. PILOT SERVICE RECEIPT DATA VARIABLES [CUSTOMIZE TABLE]

Universe
Each person randomly assigned to a treatment or control group as part of the pilot project
Purpose
These data will be used for 1) assessing data fidelity for the evaluation, 2) the participation analysis, and 3) the cost/benefit analysis.
Timeline
We will work closely with each site to develop a feasible plan for collecting data on service receipt. The data for monitoring will be needed periodically after the start of the pilot, but data for the participation and cost/benefit analysis will be required less frequently.
Variables
Demographic information	Service receipt information
Unique identifier to link to SNAP case	Service/activity start date (overall and by activity)
Name	Service/activity end date (overall and by activity)
Gender	Date completed/withdrew by activity
Age	Reason for withdrawal by activity
Homeless	Cost of training/service
Veteran	Amount of subsidy paid for employment
Offender	Amount and frequency of support services received
Low-income
Limited English language proficiency	Providers
	Provider code—to identify the provider by service
Education	Location where each service was provided—zip code
Highest school grade completed at entry/exit
School status at entry	Credentials and degree/certifications
	Type of credentials/degrees/certifications earned
Type of service received	Date earned
(Varies by site—should include discrete training and	Status at program exit (if not complete)
services that best describe what happens at each site)
Self-initiated services	Employment
Assessment (document results)	Employment status at entry into program
Case management received/most recent date of service or frequency	Occupation/industry codes of most recent employment
Type of training service	Employment status at exit from program
Industry for which receiving training	Occupational and industry codes for employment at exit/follow-up
Type of education program in which enrolled	Entered training-related employment
Type of support service received	Employed by training employer
Type of follow-up services received (over what period
of time)
Notes
For sites that capture the required data within their MIS, or can add needed data elements to their systems, we will arrange for regular extracts from the existing system. In other sites, where existing systems do not or cannot capture all of the needed data on service receipt, we will work with the site to develop a supplemental tracking system that will allow providers to track and submit data consistently and systematically.

(3) Pilot cost data will be used to describe the overall cost of operating the pilot and its components and services. Additionally, the pilot cost data will be used to conduct a cost benefit analysis of the pilot project. [SHORT GRANTEE NAME] will provide cost data from [SHORT GRANTEE NAME], partners, and providers through Mathematica’s customized Excel workbooks immediately following the planning period and quarterly thereafter for the duration of the pilot. [SHORT GRANTEE NAME] will transmit data to Mathematica through a secure file transfer protocol as agreed upon with Mathematica. Data will be collected and analyzed for pilot treatment services, and for services provided to the control group under the existing SNAP E&T program to the extent possible. Table 3 provides further detail on the universe, timeline, and variables requested.

TABLE 3. COST DATA VARIABLES [CUSTOMIZE TABLE]

Universe
Collect information on costs from grantee, partner, and provider organizations
Timeline
Administer cost data collection workbooks to collect costs (for treatment and control services) incurred during start-up/planning phase and each quarter throughout pilot implementation and operations.
Variables
Staff (all those involved in pilot)	General supplies & equipment
Job title and pilot-related job responsibilities	Cost of general supplies and equipment used during the reporting period
Salary and fringe benefits	Percentages of cost paid for by grant and other sources
Hours worked
Percent time worked on pilot during reporting period
Percentages of salary paid for by grant and other sources
Percentage of time spent on evaluation-related activities
Service provider contracts	Indirect and other costs
Amount paid to service providers during the reporting period	Indirect costs incurred during reporting period
	Any travel expenses incurred
Facilities
Cost of facilities (including donated facilities) used for pilot during the reporting period
Percentages paid for by grant and other sources
Service costs
Payments to vendors for services provided to pilot participant (for example, tuition, child care)
Payments to participants during the reporting period (for example, child care, transportation)
Purchase prices of other service-specific supplies (for example, assessment test booklets and licensing fees)
Percentages paid for by grant and other sources
Notes
Workbooks will be tailored to collect appropriate costs for each grantee and, after first data collection, many workbook fields will be prepopulated to minimize burden.
Three annual time use surveys of grantee and partner frontline staff (first one administered spring 2016) will estimate staff costs for pilot services during pilot operations.

USE OF THE DATA PROVIDED. As set forth in the Agreement, Mathematica will use the data provided by [SHORT GRANTEE NAME] solely for the purpose of monitoring and analysis activities under the SNAP Employment and Training Pilots Evaluation and as otherwise required under Mathematica’s contract with FNS. Mathematica may publish the data in public use files, provided that the data are aggregated in a manner that does not reveal the identity of any individual or employer. Mathematica will also provide restricted use files to FNS.

SPECIFIC USERs. Mathematica specifies the following users of this information, who are authorized to receive and work with the data received from [SHORT GRANTEE NAME]:

Mathematica

• Michael Ponza, Senior Fellow and Associate Director

• James Mabli, Associate Director

• Gretchen Rowe, Senior Researcher

• Peter Schochet, Senior Fellow

• Nick Beyler, Statistician

• Linda Rosenberg, Senior Researcher

• Nora Paxton, Senior Manager

• Jessica Ziegler, Researcher

• Lisa Dragoset, Senior Researcher

• Quinn Moore, Senior Researcher

• Dallas Dotter, Researcher

• Julia Lyskawa, Research Analyst

• Brittany English, Research Analyst

• Mary Anne Anderson, Research Analyst

• Kristen Joyce, Research Analyst

• Julie Worthington, Research Analyst

• John Tyler, Program Analyst

MDRC

• Barbara Goldman, Vice President

• Gayle Hamilton, Senior Fellow

• Richard Hendra, Senior Associate

• Frieda Molina, Senior Operations Associate & Deputy Director

• Donna Wharton-Fields, Senior Operations Associate

• Jonathan Bigelow, Research Analyst

• Keith Olejniczak, Research Assistant

• Alexandra Pennington, Research Associate

• David Navarro, Research Associate

• Kelsey Schaberg, Research Analyst

• Betsy Tessler, Research Associate

• Stephen Freedman, Senior Associate

Insight Policy Research

• Claire Wilson, Executive Director of Research

• Carole Trippe, Associate Director

• Brian Estes, Senior Researcher

• Brittany McGill, Senior Researcher

• Anne Peterson, Founder and Principal

Koné Consulting

• Alicia Koné, President

• Alicia Huguelet, Senior Consultant

• Sue McGinn, Consultant

TRANSMISSION OF INFORMATION. [SHORT GRANTEE NAME] agrees to provide the Mathematica Team with the information specified in this Agreement though a secure online file transfer protocol (FTP) site or a File Exchange (FX) site, encrypted in transit and at rest, and maintained by Mathematica.

CONFIDENTIALITY ASSURANCE. The Mathematica Team shall maintain the data received hereunder in strict confidence and shall only use, access or disclose the data for purposes of the evaluation. The Mathematica Team shall adhere to all applicable state and federal statutes, rules, and regulations governing the use and disclosure of information received hereunder. Furthermore, Mathematica hereby attests that it has internal security measures sufficient to protect the confidentiality of this information and to restrict access to the information to only those who have a valid need to access the information.

PROPRIETORSHIP OF INFORMATION. The data or information provided by this Agreement remains the property of [SHORT GRANTEE NAME], even if Mathematica has paid for its production or any related production. [SHORT GRANTEE NAME] retains the right to request that Mathematica return to [SHORT GRANTEE NAME] the data or information, in the same form or manner as provided to Mathematica. [SHORT GRANTEE NAME] will provide such notice in writing and will afford Mathematica at least thirty (30) days after receipt of notice to return the data or information. [SHORT GRANTEE NAME] retains the right to request any copies or other representations of the information provided under this Agreement made by Mathematica, its employees or agents, in addition to the original data

DATA DISPOSITION. Upon termination or expiration of the Agreement, regardless of the reason for such action, the Mathematica Team shall destroy all data received from [SHORT GRANTEE NAME] hereunder, certifying in writing to such destruction.

ATTACHMENTS

ATTACHMENT 1. Pilot project flow

ATTACHMENT 2. Pilot sample build-up goals for Phase II and III by month

ATTACHMENT 3. Pilot performance measures and targets

ATTACHMENT 4. Registration document

ATTACHMENT 5. Consent form

ATTACHMENT 1

PILOT PROJECT FLOW

PILOT PROJECT FLOW

[ADD PILOT-SPECIFIC DIAGRAM – NO TEMPLATE]

ATTACHMENT 2

PILOT SAMPLE BUILD-UP GOALS FOR PHASE II AND III BY MONTH

PILOT SAMPLE BUILD-UP GOALS FOR PHASE II AND III BY MONTH

[SHORT GRANTEE NAME] agrees to recruit a sufficient number of individuals for the study as indicated in Table A below.

[FILL IN] TABLE A: MINIMUM RESEARCH SAMPLE REQUIREMENTS

MONTH	Phase II: Treatment Group	Phase II: Control Group	Phase III: Treatment Group	Phase III: Control Group
	#	#	#	#

ATTACHMENT 3

PILOT PERFORMANCE MEASURES AND REPORTING

PILOT PERFORMANCE MEASURES AND REPORTING

[FORTHCOMING; FURTHER DEVELOPMENT WITH PILOT SITES NEEDED]

ATTACHMENT 4

REGISTRATION DOCUMENT

[ATTACH BY PDF]

ATTACHMENT 5

CONSENT FORM

[ATTACH BY PDF]