Evaluation Plans

OMB Control No. 0551-New

Evaluation Plan

According to draft Local and Regional Procurement (LRP) regulations (§ 1590.13), all Program Participants must develop an evaluation plan for the project. Applicants must also submit baseline data information for performance monitoring indicators and an evaluation baseline report. This evaluation plan will detail the evaluation purpose and scope, key evaluation questions, evaluation methodology, time frame, evaluation management, and cost as described below.

The recipient must ensure that the evaluation plan:

1. Is designed using the most rigorous methodology that is appropriate and feasible, taking into account available resources, strategy, current knowledge and evaluation practices in the sector, and the implementing environment;

2. Is designed to inform management, activity implementation, and strategic decision-making;

3. Utilizes appropriate analytical approaches and methods, such as mixed methods, panel surveys, baseline data collection, and methodologies that seek to establish an appropriate comparison or control group, including, but not limited to, randomized field experiments and quasi-experimental designs;

4. Adheres to generally accepted evaluation standards and principles;

5. Uses participatory approaches that seek to include the perspectives of diverse parties and all relevant stakeholders; and

6. Where possible, utilizes local consultants and seeks to build local capacity in evaluation.

Further, unless otherwise provided in the agreement, the evaluations must be conducted by an independent, third party that:

1. Is financially and legally separate from the recipient's organization; and

2. Has staff with demonstrated methodological, cultural and language competencies, and specialized experience in conducting evaluations of international development programs involving agriculture, trade, education, and nutrition.

Applicants must submit a draft evaluation plan as an attachment. Each evaluation plan should include a comprehensive approach to evaluating the project’s performance and impact. The evaluation plan, must address all of the project-level strategic objectives. Depending on the project’s results framework and stated objectives and activities this may include any or all of the following USDA LRP Strategic Objectives:

STRATEGIC OBJECTIVE 1: Increase the capacity of local and regional food market systems in support of school feeding programs and responses to food crises.

STRATEGIC OBJECTIVE 2: Improved nutritional status of children and families.

STRATEGIC OBJECTIVE 3: Reduced short-term food insecurity in food crises and disasters.

The evaluation plan should be developed as a stand-alone document that can be shared with key project partners, stakeholders and the public. USDA expects the evaluation plan submitted as part of the application process to be no more than 10 pages (excluding annexes). The applicant must include an evaluation plan that includes, at a minimum, sections A-H as specified in more detail below:

1. Introduction

Provide a brief description of the purpose of the evaluation plan and how it will be used by the project and its partners.

2. Project Overview

Provide a summary description of the project strategy including the project strategic objectives and expected results. The Project-Level Results Framework should be included here. Provide a brief description of the project activities and corresponding targeted project beneficiaries. The project overview will provide important context to the evaluation plan and methodology proposed.

3. Baseline Study

Baseline data will be collected for two purposes: (1) to measure progress on performance indicators and (2) to assess project outcomes and impacts using evaluation methods. The evaluation plan should provide a description of the organization’s plan to establish performance indicator baseline information and targets for which the project will regularly measure performance every six months of the fiscal year in required progress reports. The baseline information for performance indicators and evaluation assessments must be measured and established prior to the start of project activities. Please also describe data collection techniques and methodologies proposed for establishing baseline information for evaluation activities.

The evaluation plan should describe the quantitative and qualitative evaluation methods that will be used throughout the length of the project (i.e. an evaluation design may include a randomized control trial (RCT), propensity score matching (PSM) design, repeat cross-sectional designs, or panel studies and also may include direct observations, key informant interviews, and secondary data analysis). These methodologies should be described in detail including sample design, expected sample sizes, and key informants. The methodological description of the baseline should also be linked to the midterm and final evaluations. For example, the evaluation plan should describe in detail if the project plans to use a PSM design with data collected at the baseline, midterm and final stages.

4. Midterm Evaluation

Provide a description of the project’s midterm evaluation strategy and activities. The evaluation plan should identify the purpose and scope of the evaluation, preliminary key evaluation questions, methodology, selection of the evaluation team, and key audience for the evaluation. These questions should be organized according to standard evaluation criteria of relevance, effectiveness, efficiency, sustainability, and impact. The evaluation plan should include a timeline for the conduct of key evaluation activities and a description of how the project plans to utilize the evaluation findings and recommendations.

5. Final Evaluation

Provide a description of the project’s final evaluation strategy and activities. The evaluation plan should identify the purpose and scope of the evaluation, preliminary key evaluation questions, methodology, and the key audience for the evaluation. The evaluation questions should be organized according to standard evaluation criteria of relevance, effectiveness, efficiency, sustainability, and impact. The evaluation plan should describe the methodology selected and the strengths and weaknesses in the proposed methodology for measuring impact and assessing attribution. The evaluation plan should include a timeline for the conduct of key evaluation activities and address issues of independence, coordination and the use of participatory methods. The evaluation plan must include a description of the expected qualifications of the evaluation team and provide a list of key stakeholders.

6. Special Studies (where applicable)

Proposals may include plans to conduct special studies focused on a particular intervention, sector or thematic area that may aid in identifying project effectiveness, impact, or lessons learned complementary to the required midterm and final evaluations. Proposals may also conduct qualitative or anthropologic studies that help to triangulate evaluation information, provide context to evaluation findings, or offer a better understanding of evaluation findings.

7. Evaluation Management

Briefly describe an evaluation management structure that reflects standards and principles of evaluation independence and credibility. If the organization maintains an evaluation unit, USDA requires that the evaluation is managed by the organization’s evaluation unit. If the organization does not have a dedicated evaluation unit the review should be managed by a project staff person or organizational staff person with significant knowledge and expertise concerning evaluation. Ideally, the organization would maintain an evaluation unit that was separated from the staff or line management function of the project being evaluated. Such a structure helps to ensure the independence and impartiality of the evaluation process and report of findings, conclusions and recommendations.

The evaluation plan should also address the roles and responsibilities of the project partners and key stakeholders throughout the evaluation process. Additionally, the evaluation plan should address the regular review and updating of the evaluation plan throughout the life of the agreement, and should describe the project’s dissemination strategy for improving the knowledge base and sharing evaluation findings and lessons learned

More detailed descriptions of the roles and responsibilities of the M&E staff should be included in the staffing plan and organizational chart as described in the Organizational Capacity and Staffing section above

8. Evaluation Budget

An evaluation budget is directly related to the purpose, scope, timeline, and approach of the evaluation (includes all baseline, midterm and final evaluations). It is dependent on the required skills and expertise, specified deliverables, and any provisions provided by the evaluation commissioners. Costs vary considerably across evaluations--typical budget line items for evaluation include costs for employee salaries, expenses, and per diem; travel (international and in-country); costs for third-party evaluation contractors; costs associated with the development of a beneficiary monitoring system or data collection equipment and tools; and other costs for translators, data processors, meeting space, and support staff. The USDA Food Assistance M&E policy states that Applicants should include monitoring and evaluation key personnel in labor costs. Applicants should consider allocating, at a minimum, three percent (3%) of the project budget toward monitoring and evaluation. The minimum three percent is exclusive of the Applicants’ M&E employee staff costs. For evaluation plans which include the conduct of impact evaluations, USDA expects the M&E costs to range between five to ten percent (5-10%) of the project budget. Evaluation budget information should be included as described in the project Budget Narrative.

Post-Award

As noted in the USDA Food Assistance Monitoring and Evaluation Policy, the implementer is responsible for submitting an updated evaluation plan within 3 months of agreement signature for USDA review and approval.