Fns 885 Snap-ed Toolkit Intervention Scoring Tool

Print

Instructions

UNITED STATES DEPARTMENT OF AGRICULTURE

Food and Nutrition Service

SNAP-ED TOOLKIT INTERVENTION SCORING TOOL

OMB APPROVED NO. 0584-0639

Expiration Date: XX-XX-XXXX

• Please enter your score next to each question in the "Score" column.

• The Intervention Submission Tool Questions or Materials to Review are suggestions, please feel free to use all information provided about the intervention to determine a score for each question.

• Please feel free to make comments for each question, these will only be shared with other reviewers if scores need to be aligned.

• If an intervention is not chosen to be included in the SNAP-Ed Intervention Toolkit, intervention developers will be provided the reasons it was not included, and the additional information or actions to be taken for inclusion. This feedback will be de-identified.

• Mandatory questions on the Intervention Submission Tool are indicated with an asterisk (*).

• For more information about the RE-AIM Framework, please visit https://snapedtoolkit.org/training/online-training/

Review Question	Score	Intervention Submission Tool Questions or Materials to Review	Maximum Possible Points	Factors for High Score	Reviewer Comments
Reach How many people are exposed or served and are they representative?			12
Did the intervention reach the intended target audience?		Questions 21*, 25*, & supporting documents	5	Total persons/institutions reached High proportion of eligible persons/institutions reached Persons/institutions reached are representative of target audience
Is the intervention appropriate for the audience for which it was intended?		Questions 21*, 22*, 23*, supporting documents & intervention materials	7	Cultural needs and preferences were thoughtfully considered and integrated Language level and availability (e.g. translation, format) Resource commitment (time, space, capital, human resources)
Reach Total:

Effectiveness What is the impact of your intervention on the intended outcomes?			35
Was the target audience (or community partner) involved in the development of the intervention?		Questions 22*, 23*, 23a, & supporting documents	3	Depth and quality of involvement (true partnership versus consultation) Demonstrated integration of target audience or community partner feedback
If applicable, does participant or partner feedback indicate acceptability of the intervention?		Questions 23*, 23a, & supporting documents	1	- Reports from participants, stakeholders, and partners indicating acceptability
Do the intended outcomes indicate that objectives were appropriately addressed?		Questions 25*, 26, & supporting documents	10	Number of outcomes achieved Extent of achievement (proportion of participants/ organizations reporting positive results, statistical significance, difference from baseline) Who conducted the evaluation (external versus internal evaluators)

Form Approved OMB No. 0584-NEW ^{| According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information} unless it displays a valid OMB control number. The valid OMB control number for this information collection is [0584-xxxx]. The time required to complete this information collection is estimated to average 6 hours per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.

Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 1320 Braddock Place Alexandria, VA 22314, ATTN: PRA (0584-xxxx*). Do not return the completed form to this address. Expiration date: XX/XX/XXXX

FORM FNS-885 Version 2 (04-21) Previous Editions Obsolete

^SBU Electronic Form Version Designed in Adobe 11 Version

Review Question	Score	Intervention Submission Tool Questions or Materials to Review	Maximum Possible Points	Factors for High Score	Reviewer Comments
Does the intervention address multiple levels of the SNAP-Ed Evaluation Framework (or multiple levels of the Socio-Ecological Model)?		Questions 27*,30*, 31*, supporting documents & intervention materials	5	Type of intervention Intervention setting(s) Outcomes and the extent to which they occurred
Does the supporting documentation indicate that the intervention is evidence-based at a level that is appropriate for the intervention's stage of development (Research-tested, Practice-tested, Emerging)?		Questions 15*, 16*, 27*, supporting documents, & intervention materials	8	Evaluation methods used Evaluation type for lifespan of intervention Quality of the supporting materials and conclusions as appropriate for a low-income audience
Does the evidence provided support that the intervention would be effective if adopted by other SNAP-Ed agencies? Is it reasonable to expect that this intervention will be effective in the field?		Questions 15*, 16*, 27*, supporting documents, intervention materials, & possible outside research by reviewer	6	Theory of behavior change Extent to which behavior change theory is addressed through intervention methods Extent to which intervention has been implemented and evaluated by other SNAP-Ed agencies -
Are process evaluation materials provided?		Questions 28*, 28a, 28b, & intervention materials	2	- Yes/No
Effectiveness Total:

Adoption How many settings/sectors are involved and are they representative?			14
Has the intervention previously been been used with a low-income audience?implemented with people eligible for SNAP-Ed (on limited incomes or earning low wages)?		Question 332*	1	- Yes/No

Review Question	Score		Intervention Submission Tool Questions or Materials to Review	Maximum Possible Points	Factors for High Score	Reviewer Comments
How appropriate is the intervention for the setting for which it was intended?			Questions 29*, 30*, supporting documents, & intervention materials	5	Resources needed for adoption (materials, staff, time, space) Ability of setting to reach SNAP-Ed target audience Availability of setting in communities of need
Did most of the sites/settings/ partners engaged complete the intervention?			Questions 30*, 31*, & supporting documents	2	Number of sites/settings/partners approached that completed all components of the intervention Sites/settings/partners who expressed desire to continue but were unable to complete due to reasons beyond the scope of the intervention (such as closure of business)
Does the intervention collaboratively engage partners who can affect change in multiple levels of the SNAP-Ed Evaluation Framework (or multiple levels of the Socio-Ecological Model)?			Questions 30*, 31*, 32*, supporting documents, implementation materials, & possible outside research by reviewer	6	Mix of partners across the sectors of influence Ability of setting to reach SNAP-Ed target audience Availability of setting in communities of need
Adoption Total:

Implementation Were the required activities of your intervention successfully implemented?			20
Are training materials available for staff, partners, and/or volunteers?		Questions 36*, 36a, & intervention materials	2	- Yes/No
Are implementation directions and materials clear and easy to follow?		Intervention materials	5	Language level Logical flow of implementation steps Materials are provided to support successful implementation with fidelity Materials are appropriate for knowledge and experience level of intended user (for example, materials for lay persons avoid technical jargon)
Are the intervention's main components (critical features) reasonably feasible to replicate with fidelity?		Questions 35*,36*, 36a, 37*, & intervention materials	8	Resources needed for implementation (including cost) Availability and feasibility of methods for ensuring intervention fidelity Ability of organizations with limited resources to implement the intervention
Are the methods described to ensure program fidelity appropriate for the interventions?		Questions 35*, 36*, 36a, 37* & intervention materials	5	Data collection methods Resources needed to ensure intervention is completed with fidelity (such as staff time for observations or physical materials)
Implementation Total:

Review Question	Score	Intervention Submission Tool Questions or Materials to Review	Maximum Possible Points	Factors for High Score	Reviewer Comments
Maintenance What are the long-term effects of your intervention? Is the intervention sustainable?			18
Any evidence of maintenance of outcomes? (After 6 months for individuals)		Questions 25*, 26 & supporting documents	2	Feasibility of maintaining outcomes Are maintenance outcomes expected at this point in the lifespan of the intervention (esp. if emerging)
Are resources or materials reusable or available to participants/partners at no/low cost on an ongoing basis to facilitate outcome maintenance?		Questions 10*, 11, 25*, 40*, 43 & intervention materials	3	Feasibility of maintaining outcomes Comparison of outcome maintenance to similar interventions Are maintenance outcomes expected at this point in the lifespan of the intervention (esp. if emerging)
Are the core components of the intervention clearly described and realistic for the audience and setting for which it is intended? Can components of the intervention be adapted to be used in settings or communities other than those explicitly described in the submission criteria?		Questions 35*, 40*, 41, 42, supporting documents & intervention materials	5	Resources needed for implementation (including cost) Appropriateness of the intervention for multiple audiences/settings
Has the intervention been adopted by partners/in settings not directly supported by SNAP-Ed?		Question 39*	1	- Yes/No
Are sustainability concerns reasonable and able to be addressed through routine operation, including expressed or expected partnerships or diversified funding mechanisms? Consider both expressed and intuited concerns.		Questions 40*, 43 supporting documents & intervention materials	7	Number and extent of sustainability concerns Total resources needed for intervention adoption, implementation, and maintenance Diversity of potential partners or funding streams Number of potential partners or funding streams
Maintenance Total:

BONUS: Does the intervention reach an underrepresented audience in the toolkit?

Questions 19*, 20* supporting documents & intervention materials

5

Less than 30% of interventions currently represented in the Toolkit address this population: Middle School High School Pregnant/Breastfeeding Women Homeless/Food Pantry Clients African Americans Asian Americans/Pacific Islanders Native Americans/Alaskan Natives

- Language other than English or Spanish

Review Question	Score			Intervention Submission Tool Questions or Materials to Review	Maximum Possible Points	Factors for High Score	Reviewer Comments
BONUS: Does the intervention reach an underrepresented setting in the toolkit?				Questions 29* & 32*	5	Less than 30% of interventions currently represented in the Toolkit address this setting: Community Gardens Farmers Markets School Gardens Faith-based community Food pantries Health Care Indian Tribal Organizations Food Retail USDA Program Sites Worksites
BONUS: Does the intervention use an approach/strategy that is currently underutilized in the toolkit?				Question 14*	5	Less than 30% of interventions currently represented in the Toolkit address this implementation strategy: - Social marketing **Consider if this intervention is appropriate for the target audience and settings, would reach the SNAP- Ed target audience, and if it is feasible for organizations to adopt and implement
BONUS: How would you rate the overall quality of the intervention responses and materials?					5	Responses that are comprehensive and use specific, concise language Materials that provide relevant supporting information and are clearly referenced
Bonus Total:
Total Score (No Bonus):

If no to above, please describe your reasoning for exclusion from the Toolkit:

If no to above, please describe what additional information or actions would be needed to recommend this intervention for inclusion in the Toolkit:

If yes to above, please describe your reasoning for inclusion in the Toolkit: