Process Evaluation

Eval of SNAP Nutrition Edu Practices Study - Wave II

APPENDIX B. Data Collection Instruments for Process Evalua SLO 3.1.11

Process Evaluation

OMB: 0584-0554

Document [docx]

Download: docx | pdf

Appendix B.

D ata Collection Instruments
for Process Evaluation

PRIMARY DATA COLLECTION INSTRUMENTS

All Sites

SNAP-Ed Wave II: Nutrition Education Observation Form
SNAP-Ed Wave II: Discussion Guide for Demonstration Project Evaluation Manager [Pre-implementation]
SNAP-Ed Wave II: Discussion Guide for Demonstration Project Evaluation Manager [Post-implementation]
SNAP-Ed Wave II: Discussion Guide for Demonstration Project Program Administrator [Pre-implementation]
SNAP-Ed Wave II: Discussion Guide for Demonstration Project Program Administrator [Post-implementation]
SNAP-Ed Wave II: Discussion Guide for Onsite Nutrition Educators [Pre-implementation]
SNAP-Ed Wave II: Discussion Guide for Onsite Nutrition Educators [Post-implementation]
SNAP-Ed Wave II: Summary Form for Demonstration Project-Led Evaluation [Post-implementation]
SNAP-Ed Wave II: Discussion Guide for School Principals and Senior Center Program Managers [Post-implementation]

INN

SNAP-Ed Wave II: Discussion Guide for Classroom Teachers [Pre-implementation]
SNAP-Ed Wave II: Discussion Guide for Classroom Teachers [Post -implementation]

SNAP-Ed Wave II: Questionnaire for Lead Teachers in Classrooms Receiving the Iowa Nutrition Network Building and Strengthening Iowa Community Support Program
SNAP-Ed Wave II: Discussion Guide for Retail Stores: Produce Managers [Post-implementation]
SNAP-Ed Wave II: Group Discussion Guide for Parents/Caregivers (INN) [Post-implementation ONLY]

UKCES

SNAP-Ed Wave II: Discussion Guide for UKCES FCS Agents [Pre-implementation]
SNAP-Ed Wave II: Discussion Guide for UKCES FCS Agents [Post-implementation]
SNAP-Ed Wave II: Questionnaire for Lead Teachers in Classrooms Receiving the LEAP2 Program
SNAP-Ed Wave II: Group Discussion Guide for Parents/Caregivers (UKCES) [Post-implementation ONLY]
SNAP-Ed Wave II: Discussion Guide for School Classroom Teachers [Post-implementation]

MSUE

SNAP-Ed WAVE II: Discussion Guide for Demonstration Project: MSUE Regional/Area-Level Extension Staff [Pre-implementation]
SNAP-Ed WAVE II: Discussion Guide for Demonstration Project: MSUE Regional/Area-Level Extension Staff [Post -implementation]
SNAP-Ed Wave II: GROUP Discussion Guide for Eat Smart, Live Strong Participants [Post-implementation ONLY]

SNAP-Ed Wave II: Nutrition Education Observation Form

The purpose of this observation tool is to describe the intervention as it is being implemented and inform the process evaluation of this project. This observation is not intended to evaluate the teaching abilities of the instructor.

Name of observer: Date of class observed:

Name of intervention:

Name of instructor:

Name and type of site:

PART A: BACKGROUND INFORMATION ABOUT THE NUTRITION INTERVENTION (to be filled out prior to class)

Name of lesson to be taught:

Lesson topic(s):

Intended lesson objective(s):

Target audience(s):

Children Yes No Grade/age range of children in class:

Parents/guardians Yes No

Older adults Yes No

PART B: CLASS OBSERVATION

Length of Class

Class start time:

Class end time:

Reach

Number of participants:

How many of the participants were exposed to the complete class:

Description of the Setting

Physical location

In a traditional classroom

Indoors, in a general purpose room in the building (describe briefly)

Indoors, in an informal area of the building not structured for group classes (describe briefly; e.g., in the hallway, in the front waiting area)

In an outdoor area

Adequacy of space

Space is very ample for the number of participants and activities planned

Space is sufficient, but somewhat limited for the number of participants and activities planned

Space is insufficient for the number of participants and activities planned

Any other facilitators or barriers related to classroom setting:

Facilitators to teaching the lesson, carrying out planned activities, and engaging participants:

Barriers to teaching the lesson, carrying out planned activities, and engaging participants:

Other observations about adequacy of space or class environment/setting:

Teaching Methods

Teaching techniques used: Check the teaching techniques used in teaching the lesson.

Lecture/verbal presentation

Educator engages the children in discussions

Story reading

Food preparation demonstration

Food tasting

Movement activity

Student performance (e.g., dance)

Small group discussions or activities (likely relevant only with large classes of parents)

Other

Types of teaching aids used: Check the types of teaching aids used in the lesson.

Food models

Storybooks

Posters

Music

DVDs or videos

Handouts

Foods for demonstration purposes and tasting

Other

Materials distributed: Check the materials that were distributed during the lesson.

Recipes

Nutrition education newsletters

Handouts:

Weekly logs

Other:

Participant Engagement in the Lesson

Describe the level of engagement of participants in the lesson as presented. For example, did it appear that the participants were engaged in the lesson? Was the lesson age appropriate? Was the literacy level appropriate? Was it culturally appropriate? Did it appear that this was new information for the participants?

PART C. LESSON TAUGHT AS PLANNED IN THE PROJECT

Overall, did the instructor follow the curriculum for this lesson as developed? If not, how was it different and what are the apparent reasons for this deviation?

Observer comments/notes:

PART D. ENVIRONMENTAL REINFORCEMENTS/INFLUENCES

Classroom Teacher Involvement [for UKCES and INN only]

What role(s) did the school/child care teacher(s) play during the intervention class?

N/A—absent from the classroom during the lesson

Silent observer who did not participate or support the educator during the lesson

Assistant to the nutrition educator in handing out materials

Assistant to the nutrition educator in activities beyond handing out materials

Other roles, if any, that the teacher played in supporting the intervention messages:

Senior Center Director Involvement [for MSUE only]

What role (s) did the senior center director play during the intervention class?

N/A—absent from the room during the lesson

Silent observer who did not participate or support the educator during the lesson

Assistant to the nutrition educator in handing out materials

Assistant to the nutrition educator in activities beyond handing out materials

Other roles, if any, that the director played in supporting the intervention messages:

Availability of Fruits and Vegetables at the Intervention Site

Request and review the current weekly or cycle menu to see the extent and variation in fruits and vegetables offered at the school/senior center for meals and snacks. Below, provide a general description of the number of the fruits and vegetables on menu each day and the variety of fruits and vegetables offered on menu. Attach a copy of the menu.

Supportive or Conflicting Indirect Nutrition Messages Visible at the Intervention Site

Note any posters, displays, bulletin boards at the intervention site that relate to nutrition and physical activity.

Description of nutrition messaging at intervention site:

PART E. LESSONS LEARNED FOR IMPROVEMENT AND REPLICABILITY

These are four questions for observers to ask educator after the lesson:

Did you deviate from the written lesson plan for today? Yes No

[IF YES]

What did you do differently?
Why did you decide to make this change (or changes) today?

What do you think works best today about this lesson and why?

What if anything made it challenging to teach the lesson as you had planned today?

What recommendations would you have for improving this lesson if you or others are teaching it another time?

Additional observer comments/notes:

SNAP-Ed Wave II: Discussion Guide for Demonstration Project Evaluation Manager

[Pre-implementation]

State:

Respondent/Title/Organization:

Address:

Phone:

Fax:

E-mail:

Interviewer:

Date of Interview:

Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of [NAME OF PROGRAM] that is offering information to older adults/children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education models around the country and to provide recommendations for how these interventions could be improved to better serve the older adults/children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Although there are only a select number of programs participating in this evaluation, we will do our best to aggregate data wherever possible in order to avoid information being tied back to a particular respondent. Nothing that is said today will be attached to you, and nothing that you say will affect your job or be shared with your employers. I expect that our discussion today will take 30 minutes. Before I begin, do you have any questions?

Evaluation-Planning Phase

I would like to ask you briefly about your experiences in the design and planning phase for this evaluation.

What challenges, if any, have you faced during the design and planning phases of this evaluation?
What factors do you feel have contributed to a successful design and planning phase?
What lessons have you learned during this key phase of the evaluation design?

What would you do differently?
What would you do the same?

How will data be documented and entered from the various evaluation instruments? Please describe forms and software.

Anticipated Challenges for Implementation and Quality Control Efforts

What challenges do you anticipate for this evaluation as you now approach your initial evaluation data collection phase?
Please describe any quality control or monitoring that will take place during data collection?

Who will conduct these?
With what frequency?
What methods will be used?

Evaluation of the Social Marketing Campaign [INN ONLY]

I would like to ask you briefly about your experiences in the design and planning phase of the evaluation of the social marketing campaign.

What challenges, if any, have you faced during the design and planning phases of the social marketing campaign evaluation?
What factors do you feel have contributed to a successful design and planning phase of this evaluation?
What factors do you feel have been barriers to a successful design and planning of the evaluation of the social marketing campaign?
If this social marketing campaign is part of a larger Iowa Department of Public Health social marketing campaign, has this collaboration helped or hindered your work on this project?
What lessons have you learned during this key phase of the evaluation design?

What would you do differently?
What would you do the same?

How will data be documented and entered from the various media outlets/retail stores [include the demonstrations in the stores]?
Please describe forms/software/other types of tracking methods.

Anticipated Challenges for Implementation and Quality Control Efforts

What challenges do you anticipate for this evaluation as you now approach your initial evaluation data collection phase?
Please describe any quality control or monitoring that will take place during data collection?

Who will conduct these?
With what frequency?
What methods will be used?

Dissemination of Evaluation Results

When do you expect to complete data collection?
When do you anticipate that you will complete data analysis?
Who will conduct the data analysis?
How do you intend to use and/or disseminate your evaluation results?
Do you have an updated evaluation plan to share with us? If not, please send any changes to the evaluation plan, no matter how minor, to my attention.
Is there anything else you would like to share about your evaluation plans, methodologies, or staffing?

That ends my formal interview questions. Do you have any information about your evaluation plans, comments, or recommendations that you would like to add?

Thank you very much for your time and input on this very important project.

SNAP-Ed Wave II: Discussion Guide for Demonstration Project Evaluation Manager

[Post-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. As you know, the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the [NAME OF INTERVENTION] that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping to improve the health and nutrition status of children, families, and adults.

This study will include not only outcome evaluation information but also process information on how it is being implemented and how you are evaluating the intervention. All of this will be useful to both FNS and to other Supplemental Nutrition Assistance Program-Education (SNAP-Ed)-implementing agencies that are planning to evaluate their own SNAP-Ed interventions.

As I mentioned during our last meeting, we will be using first names only today. Everything you say will be kept private except as otherwise required by law. After we conduct several of these interviews, we will write a report for the FNS. Your name will not appear anywhere in the report. Nothing that is said today will be attached to your name at any point. Nothing that you say will affect your job or be shared with your employers.

Today we will specifically discuss how the implementation of the program differed from your expectations. We also will discuss lessons learned and your feedback on how the program might be improved. I expect that this discussion will take about 40 minutes. I appreciate you taking the time to speak with me today. Before I begin, do you have any questions?

Specific Changes From Planned to Actual Evaluation

We would like to know about the specific aspects of your evaluation that might have changed along the way. We want to be able to describe any deviations from the evaluation plan you described to us during our first meeting, and also know why you had to make any specific changes from your plans.

Let’s start with the evaluation design. What changes, if any, occurred from your planned evaluation design? What caused these changes?
What changes, if any, occurred in your process measures, outcome measures, your data collection tools, and/or your planned data collection techniques? What caused these changes?
What changes, if any, did you make in the methods for protecting participant privacy? What caused these changes?
What changes, if any, did you make [or are you planning to make] in your data analysis plan? What caused these changes?
What changes, if any, did you make in the staffing for your data collection or staffing for your data analysis?
Did you need more or less time than budgeted for staff to spend on the data collection? On the data analysis? Why do you think you needed more/less time than budgeted for these evaluation tasks?
Did you have or are you anticipating any increased nonpersonnel costs or resources required for the evaluation? If so, what additional costs or resources have been or will be needed compared to what you planned for?

Social Marketing Campaign

What changes, if any, occurred from your planned evaluation design for the social marketing campaign? What caused these changes?
What changes, if any, occurred in your process measures, outcome measures, data collection tools, and/or planned data collection techniques for the social marketing campaign? What caused these changes?
What changes, if any, did you make [or are you planning to make] in your data analysis plan for the social marketing campaign? What caused these changes?
What changes, if any, did you make in the staffing for your data collection or for your data analysis for the social marketing campaign?
Did you need more or less time than budgeted for staff to spend on the data collection? On the data analysis? Why do you think you needed more/less time than budgeted for these evaluation tasks?
If this campaign was part of a larger effort within the Iowa Department of Public Health, can you describe the shared staffing for data collection and analysis?
Did you have or are you anticipating any increased nonpersonnel costs or resources required for the evaluation? If so, what additional costs or resources have been or will be needed compared to what you planned for?

Questions Related to Analysis

With many programs, there are alternative explanations of program outcomes that need to be ruled out due to plausible threats to validity. If you saw changes in the program outcomes, what other factors could explain the changes you see? [Probe as needed on validity threats such as competing programs, concurrent media campaigns, and the effects of maturation among evaluation participants.]

Lessons Learned

Next let’s talk about your overall experience in carrying out this evaluation and what you see as lessons learned and recommendations for the future.

Other than those that we discussed above, what challenges, if any, have you faced during the implementation of this evaluation? [Refer back to the anticipated challenges cited by the interviewee prior to beginning the demonstration project led evaluation.]
What do you think worked very well in the implementation of this evaluation? What factors contributed to what worked well?
What do you think did not work well, and what factors contributed to this?
What lessons have you learned from this evaluation design?
1. What would you do differently?
2. What would you be sure to do the same?
Are you planning a future evaluation of your program?
Whether or not you are planning a future evaluation, what would you do differently?

Dissemination Plans

How do you plan to use and/or disseminate your evaluation results?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this important project.

SNAP-Ed Wave II: Discussion Guide for Demonstration Project Program Administrator

[Pre-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of [NAME OF PROGRAM] that is offering information to older adults/children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education (SNAP-Ed) models around the country and to provide recommendations for how these interventions could be improved to better serve the older adults/children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Today we will specifically be discussing the planning process and your expectations for the intervention. Once it has been implemented, we will follow up with you to find out whether the intervention met your expectations and how it might be improved. I expect that this interview will take about 45 minutes. Thank you for taking the time to speak with me.

Before I begin, do you have any questions?

Can you please describe your role as program administrator?
Do you also play a role in the budget management for the project? If not, who is responsible for the project budget?
Can you please describe your role in the program design/evaluation?
What challenges, if any, have you faced during the design and planning phases of this nutrition education program?
What factors do you feel have contributed most to a successful design and planning phase (e.g., using education materials that were already developed, good communication between contributors, knowledgeable staff, establishment of strong partnerships)?
What lessons have you learned during this key phase of program development?
1. What would you do differently? Why?
2. What would you do the same? Why?

Now I would like to shift our focus to the upcoming implementation of your SNAP-Ed project.

Now that you are ready to transition from the planning and design phase of your project to the implementation phase, what challenges, if any, are you anticipating? Why? How do you think you will address these challenges?
Do you feel that the environment in which the intervention will take place will be able to support the intended change in behavior, knowledge, and/or attitudes? [For INN and UKCES] For example, do you have any sense of the teachers and schools buy-in and/or enthusiasm about the intervention and what impact this might have on the children?
Does the school/senior center offer the children/older adults healthy food options, or are healthy foods otherwise available?
What, if any, other nutrition education messages are the older adults/children in the intervention sites being exposed to (that you are aware of)? Did the program have any difficulty recruiting adequate staff for the nutrition education delivery? If so, what were the recruitment challenges/problems?
Please describe the training the nutrition educators have received or will receive (e.g., frequency and duration of training, training agenda and objectives).
1. Who will do the direct training?
2. When will these trainings be provided?
3. What topics will be covered in the training
4. What is the training outline/agenda?
5. What format will the training be conducted
6. Qualifications of trainer(s):
Do the educators have flexibility in how they deliver the program, or are they directed to follow the curriculum strictly as written? How will that be assessed?
Please describe any quality control and monitoring efforts that will take place during implementation (e.g., of nutrition education delivery, of nutrition education data collection).
What specific guidance and materials are planned to be provided to direct educators to work with the sites to recruit the adult participants for the intervention?

[QUESTIONS 15 AND 16 FOR MSUE ONLY]

What specific guidance and materials are planned to be provided to direct educators to work with the sites to help retail older adult participants in the intervention and attend all four Eat Smart, Live Strong classes?
How will direct educators be asked to document/track the enrollment and attendance of individual seniors to assist participants in accessing all four intervention classes? What forms will be used? How will data be analyzed? (Obtain copies of enrollment and attendance forms to be used.)
How will the demonstration project be tracking the number of children/adults enrolled in each class at each intervention site?
Will the demonstration project be tracking dosage at the individual level (i.e., which lessons participants take part in)? How will this be tracked?

Social Marketing [INN ONLY]

Can you please describe your role in the program and evaluation design of the social marketing campaign?
Could you describe the steps you have taken to design the social marketing campaign?

PROBE: With whom did you work to design the campaign? What media are used in the social marketing campaign, and how are they used? How are the media used (e.g., what specific messages are used with which media)?

Describe the process of selecting the retail grocery stores for the social marketing campaign.
Describe the social marketing activities that you have planned for the grocery stores.

PROBE: How often will these activities take place? Who will implement these activities? How will these staff interact with the grocery stores? What role will the produce manager have in the social marketing activities?

What challenges, if any, have you faced during the design and planning phases of the social marketing campaign?
What factors do you feel have contributed most to a successful design and planning phase (e.g., building on the previous Pick a better snack social marketing plans/designs, good communication between contributors, knowledgeable staff, establishment of strong partnerships)?
What lessons have you learned during this key phase of program development?
1. What would you do differently? Why?
2. What would you do the same? Why?
Now that you are ready to transition from the planning and design phase of the social marketing campaign to the implementation phase, what challenges, if any, are you anticipating? Why?

PROBES: Identify the challenges. How do you think you will address these challenges?

Now I’d like to focus on partnerships you have developed to assist with the implementation of your project.

I brought the Key Program Staff and Partnering Agencies form you completed for the April kickoff meeting in Alexandria and wanted to check for any updates to this form. If there are any, ask them to revise form.
How do these partnerships enhance your intervention?
Have there been any challenges in developing these partnerships?
Would you recommend these partners to other States who might replicate your project?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. As I mentioned, we will follow up and talk with you after the intervention and evaluation period are over.

SNAP-Ed Wave II: Discussion Guide for Demonstration Project Program Administrator

[Post-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. As I told you during our last meeting, the U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the [NAME OF INTERVENTION] that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults.

As mentioned during our last meeting, nothing that is said today will be attached to you, and nothing that you say will affect your job or be shared with your employers.

Before I begin, do you have any questions?

Formative Research and Program Design

I’d like to briefly discuss how, if at all, the implementation of your nutrition education intervention differed from what was originally planned. There are several aspects of implementation that I would like to cover.

Were the nutrition education messages for the intervention modified at any point during implementation? If so, how and why were they modified?
Did the target audience differ from what was originally planned? If so, how and why did they differ?
Were the methods of delivery (i.e., direct education, indirect education) modified during implementation for any reason? If so, how and why were they changed?
Did the dose of nutrition education vary from what was originally planned (e.g., the number of lessons, the length of each lesson)? If so, how and why did this vary from what was planned?
Were you able to implement the intervention at the originally proposed number of sites and do you feel that you reached the intended number of participants? Were there any factors that affected your ability to achieve the full, intended reach?
Were the nutrition education materials modified at any point during implementation? If so, how were the materials modified and why?
To what extent were the original implementation timelines met? What are the reasons for and implications of any departures from the original timelines?

Operational Steps Involved in Program Implementation

Did you find the level of staff, in terms of both qualifications and the total number of staff (and types of staff), adequate for optimally delivering your nutrition education intervention?
What changes, if any, were made to planned key staff involvement and what were the reasons for any such changes?
Were any quality control and monitoring processes employed to maximize the fidelity/quality of the intervention delivery?
How effective were staff in delivering the intended nutrition education messages?
1. Why do you think these staff were effective/ineffective?
2. What could they have done differently to improve their effectiveness?
Please describe the nutrition education training provided for the implementation of this intervention and how it was different from what you had planned.
Do you think the nutrition educator training was sufficient?
1. What worked well?
2. What could have been improved?
Were planned recruitment (of older adult participants/parents) efforts modified during implementation? If so, how were recruitment efforts modified and for what reasons?
What recruitment methods did you find to be most effective/least effective?
In your opinion, how well was the direct program able to track participation in the direct education?
Did previously identified partners remain engaged throughout the intervention?
Were these partnerships successful?

[IF YES]

How were they successful?
What would you say contributed to their success?

[IF NO]

Why not?

Social Marketing Campaign [INN ONLY]

Was the social marketing campaign modified at any point during the intervention?

PROBE: Modification could include methods used to implement the social marketing campaign, nutrition messages used in the campaign, retail stores identified for the implementation of the campaign, types and frequency of food demonstrations conducted in the retail stores, etc.

PROBE: If the campaign was modified from the original plan, why was the modification necessary? [Describe any barriers to implementation of the original plan.]

Resources Devoted to Intervention

What were the actual time commitments for key staff (full-time employees) if different than planned? Why did they differ?
How closely did the actual program cost components reflect the budgeted costs? If there was a difference between budgeted and actual, what factors might have contributed to this?
Were the necessary type and quantity of materials, technology, etc. available to carry out the implementation as planned? If not, what else was needed?

Lessons Learned for Improvement and Replicability

Next I’d like to talk about lessons learned during implementation of the study.

Overall, what factors were key to the success of this nutrition education program?
What factors hindered or limited the success of this nutrition education program?
Looking back over the past [NUMBER OF MONTHS] months, what lessons have you learned? What would be most valuable for another State or implementing agency to know if they were considering using this model?
In your opinion, are there any aspects of this Supplemental Nutrition Assistance Program-Education program that would make it difficult to implement on a larger scale?
How did the FNS requirements for this demonstration project influence the design of your intervention project in ways that you had not anticipated when you applied to become a demonstration project?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project.

SNAP-Ed Wave II: Discussion Guide for Onsite Nutrition Educators

[Pre-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of [PROGRAM NAME] that is offering information to older adults/children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping to improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education models around the country and to provide recommendations for how these interventions could be improved to better serve the children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

We will be using first names only today. Everything you say will be kept private except as otherwise required by law. After we conduct several of these interviews, we will write a report for the FNS. Your name will not appear anywhere in the report. Nothing that is said today will be attached to your name at any point. Nothing that you say will affect your job or be shared with your employers.

Today we will specifically be discussing your background and other qualifications as an educator for this education program, the planning process that has already begun with the intervention sites, and your expectations for the reach and design of the program. Once you have completed teaching one complete session of [PROGRAM NAME], we will follow up with you for one more interview to find out how things may have changed from what you planned to do and to obtain your experiences and views on what worked well or not and why, and what you might change to improve the program.

I expect that our discussion today will take about 30 minutes. Before I begin, do you have any questions?

Educator’s Job Title, Qualifications, and Capabilities

First I would like to ask you a few questions about your position and your background for this type of work.

What is your job title in this role as educator for [PROGRAM NAME]?
Do you also provide nutrition education or community education for any other programs?

[IF YES]

Please tell me a little bit about your other related work.
How long have you been a nutrition educator?

What percent time are you working as an educator for this intervention? (are you full-time or part-time)
Prior to this role as an educator for [PROGRAM NAME], have you had any other job or volunteer experience in nutrition or health education for older adults/children and families?

[IF YES]

Please describe these job or volunteer experiences?
How many total years of experience in nutrition or health education for older adults/children and families did you have before you came to be an educator in [PROGRAM NAME]?

What is the highest level of education that you have completed to date? If you have a college or graduate school degree, what subject was your major or degree in?
Outside of any formal education, have you had any specialized training or certification either in nutrition education or health education? If so, please describe this training for me.
What else from your life experience do you think makes you a good educator for [PROGRAM NAME]?
What are some of the challenges that you or others like you might face in being a good educator for [PROGRAM NAME]?

Training Provided by the Demonstration Project

Did the demonstration project provide training for you to implement this curriculum? If so, please describe the training you received (who provided, number of hours, where the training was held, what materials were used).
Do you think that the training provided you with the skills and materials to effectively implement the curriculum? Please describe why you think this.
What recommendations, if any, do you have for how the training could be improved?

Recruitment and Implementation Plans

Next I would like to discuss what is being planned to recruit sites and participants for the intervention and how many sites, classes and students you plan to be working with.

Do you know yet at which sites you will be teaching the [PROGRAM NAME] classes?

[IF NO, SKIP TO QUESTION 19]

[IF YES]

Please name centers/schools.
When do you plan to start the intervention at the senior centers/schools you will work with?

Do you know yet how these sites were recruited?

Who did the site recruiting, and how did they reach out to enroll the sites?
Do you think this was an effective way to select the sites? Why or why not?

Have you visited or otherwise been in contact with the site(s) yet to talk about your plans for the intervention?

15. How will you be recruiting older adults/teachers at these sites to participate in the intervention?

16. Aside from yourself as the nutrition educator, will there be anyone else involved in recruiting older adults/teachers to participate in the [PROGRAM NAME] program at these sites?

[IF YES]

What are their roles?
Do you have any sense of their buy-in and/or enthusiasm about the intervention and what impact this might have on participation?

17. Aside from yourself as the nutrition educator, will there be anyone else involved in teaching the [PROGRAM NAME] curriculum at these sites?

[IF YES]

What are their roles?
Do you have any sense of their buy-in and/or enthusiasm about the intervention and what impact this might have on participation?

18. What physical resources will you need at the sites to implement the intervention (e.g., space, audiovisual equipment, computers)?

Scheduling

In order to plan our site visits we need to know specific information about the scheduling of your classes.

19. How many classrooms or groups of children/ older adults will you be teaching at each of these sites?

(a) [FOR INN AND UKCES ONLY] Will you have any joint classes combining classrooms or teach each classroom of children separately? How often (days per week/month) will you be going out to the sites teach these groups? How long will each class or activity be? What time of day will you be providing the education? Is that a good time for the target population?

(b) [FOR INN AND UKCES ONLY]Will you have one joint class or a separate class for each classroom?

How many children/adults do you expect will be involved in each class?
Do you have a written schedule yet of the dates and times for all the classes? If so, could you provide a copy of this schedule to us?
How can we best stay in touch with you to firm up your schedule for teaching at your sites (e.g., phone, e-mail)?

Are you planning on doing any direct training of the teachers or other staff at the schools/centers?
Is there anything unique about the sites where you will be teaching [NAME OF PROGRAM] or the population of older adults/children at these sites that you think will require you to tailor the program to better meet the needs of the older adults/children and/or their parents at this center? If so, how are you planning to tailor the program to address these needs?

Perceived Facilitators and Challenges to Intervention Success

Based on what you know about the curriculum, materials, and other aspects of [PROGRAM NAME], what components of this curriculum do you think will be most effective with the target audiences you are trying to reach?
Before we close, I would like to ask you whether you foresee any challenges in implementing the intervention as designed or planned. If so, what are those potential challenges and how might they be overcome?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this important project. My colleagues and I at Altarum will get back in touch with you to schedule a follow-up interview after you finish teaching [PROGRAM NAME]. I am looking forward to talking with you then.

SNAP-Ed Wave II: Discussion Guide for On-Site Nutrition Educators

[Post-Implementation]

State:

Respondent/Title/Organization:

Address:

Phone:

Fax:

Email:

Interviewer:

Date of Interview:

Time of Interview:

Thank you for taking the time for this interview. As I told you during our first meeting, the U.S. Department of Agriculture’s Food and Nutrition Service has contracted with our Altarum Institute to conduct a study of the [NAME OF INTERVENTION] that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping to improve the health and nutrition status of children, families, and adults.

The purpose of the study is to describe how several SNAP-Education program models are being carried out across the country and evaluate their impact on nutrition behaviors. The study will also highlight recommendations for how to replicate and improve these SNAP-Education models –based on what we observe and learn from the program planners, from the people who are implementing these interventions—like yourselves—and from the intervention participants.

We will be using first names only today. Everything you say is private except as otherwise required by law.. After we conduct several of these interviews, we will write a report for the U.S. Department of Agriculture’s Food and Nutrition Service. Your name will not appear anywhere in the report. Nothing said today will be attached to your name at any point. Nothing that you say will affect your job or be shared with your employers.

Today we will talk first about the training and assistance you were provided, then about differences between what your planned implementation versus what actually happened. After we cover that information, I want to spend most of our today hearing what you think worked well and your suggestions for any revisions or improvements to [NAME OF INTERVENTION].

I expect that our discussion will take about 30 minutes today. Before I begin, do you have any questions?

Experience and Satisfaction with Training

Let’s start with your views on the he training you received from (NAME OF ADMINISTERING AGENCY) before you began implementing the [NAME OF INTERVENTION].

Please describe the format of initial training you received from [NAME OF INTERVENTION] staff to teach this curriculum. (e.g., was it in a large or small group or one-on-one, was observation of implementation involved, etc.)
Who provided this training for you?

How much training did you initially receive (number and length of sessions)?
What aspects of the training did you find most useful?
What additional information, tools, skills, or other training do you think should be provided in the initial training to help you or other educators like yourself be more effective in delivering [NAME OF INTERVENTION]?
Are there any other changes you would suggest to improve the content or format or other aspects of the educator trainings for [NAME OF INTERVENTION]?

Did you receive any ongoing training or assistance—in a structured or unstructured format— after your initial training for [NAME OF INTERVENTION]?
[IF YES]

What was the format?

What was the content?
How much of this assistance did you receive?
What was helpful about this follow-up training or assistance?
What other follow-up training or assistance could have helped your or other educators like you teach the nutrition education curriculum to this target population more effectively?

[IF NO]

What kind of follow-up training or technical assistance do you think could have helped you more effectively teach the [curriculum]?

Reach, Dosage and Intensity of Intervention- Actual Compared to Planned

How did the number of sites, groups, classes and activities you implemented with the children/seniors differ, if at all, from what you had planned?
How did the amount of time you spent in direct education with the children/seniors differ, if at all, from what you had planned?

Differences between Actual and Planned Implementation

In addition to any changes in the number, size and length of your educational activities you mentioned earlier, were there other differences in how you implemented the [NAME OF INTERVENTION] compared to what you and the program planners had intended?

[IF YES]

a) In what ways was it implemented differently from what was planned?

Why did these changes from the original plan occur?
In what ways were the changes positive?
In what ways were the changes negative?

Lessons Learned for Improvement and Replicability

What do you think worked well about the direct education/in-classroom education format of [NAME OF INTERVENTION] for children/seniors and why do you think it worked well?
What could be improved about the direct education/in-classroom education format of [NAME OF INTERVENTION] for children/seniors and why would you suggest this change?
What do you think worked well about the nutrition education materials and lesson activities designed for the children/seniors and why do you think it worked well?
What could be improved about the nutrition education materials and lesson activities designed for the children/seniors and why would you suggest this change?
Do you think that the nutrition educational materials and lessons and other aspects of the [NAME OF INTERVENTION] are tailored to be culturally-appropriate to the racial and ethnic groups that are in the target audience?

[IF YES]

a) What features of the materials and lessons make them culturally-appropriate?

[IF NO]

b) What do you think specifically could be changed or tailored in the materials and/or lessons or class form to make them more culturally appropriate for the racial and ethnic groups that are in the target audience for [NAME OF INTERVENTION]?

In addition to what we have already talked about already, are there any other specific aspects of the [NAME OF INTERVENTION] that you think worked well?
Are there other particular aspects of the program do you think did not work well?
Do you have any other suggestions for ways that [NAME OF INTERVENTION] could be improved to be more effective in improving the nutrition behaviors of its target audiences?

Scheduling

In order to plan our site visits we need to know specific information about the scheduling of your classes.

19. How many classrooms or groups of children/seniors did you teach at each site?

(a) [FOR INN AND UKCES ONLY] Did you have any joint classes combining classrooms or teach each classroom of children separately? How often (days per week/month) did you be go out to the sites to teach these groups? How long did each class or activity take? What time of day did you provide the education? Was that a good time for the target population?

b) [FOR INN AND UKCES ONLY] Did you have one joint class or a separate class for each classroom?

c) How many children/seniors were involved in each class?

Did you have a written schedule of the dates and times for all the classes? If so, could you provide a copy of the final schedule?

Did you do any direct training of the teachers or other staff at the schools/centers?
Is there anything unique about the sites where you taught [NAME OF PROGRAM] or the population of seniors/children at these sites that you think required you to tailor the program to better meet the needs of the seniors/children?

Perceived Facilitators and Challenges to Intervention Success

Based on your experience with the curriculum, materials, and other aspects of [PROGRAM NAME], what components of this curriculum do you think were the most effective with the target audiences you taught?
Before we close, I would like to ask you whether you found any challenges in implementing the intervention as designed or planned. If so, what are those potential challenges and how might they be overcome?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this important project.

SNAP-Ed Wave II: Summary Form for Demonstration Project-Led Evaluation

[Post-implementation]

(This form will be tailored for each demonstration project and responses first completed by the project’s evaluation manager. A follow-up telephone interview will be conducted with the project evaluation manager to clarify written responses and fill in information gaps. During this interview, the post-implementation Discussion Guide for Evaluation Managers will also be implemented.)

A. Research Objectives and Hypothesis

1. Provide hypotheses (research questions) addressed by the evaluation.

2. Specify each impact (outcome variable) assessed by the evaluation.

B. Sample Size/Sampling Strategy

1. Provide the number of units (schools or senior sites) and individuals (children or older adults) in the intervention and control groups at the start of the intervention.

2. Provide the sample size and describe the method used to select sample participants from the population.

3. Provide any additional information on the power analysis that was conducted other than that previously provided in your application.

4. Describe steps taken to increase the likelihood that members of the target population approached by the program would participate (i.e., recruitment strategies used to increase the program response rate).

C. Outcome Measures

1. For each impact (outcome variable) being assessed by the evaluation (including intermediate factors in the behavior change process, if appropriate):

(a) Describe key measures or indicators used to assess the intervention’s impact (outcome variable).

(b) State whether the measures were scales or single-item measures.

(c) Provide information on the reliability (internal consistency [alpha], test-retest reliability, and/or reliability across raters) and construct validity of each measure.

D. Data Collection

1. Describe the data collection methods and timing of pre- and post-intervention data collection.

2. Note and describe any differences in data collection for the intervention and control group participants.

3. Describe procedures used to track participants longitudinally.

4. Describe the training provided to data collectors.

5. Provide information on survey response rates pre- and post-intervention.

E. Data Analysis

1. Provide a table showing demographic information for all participants and the number of participants in the intervention and control group. Describe tests of statistical significance to assess baseline comparability across treatment and control groups. If your evaluation did not include a control group, then provide information for the intervention group only. Table 1 provides a suggested format for providing this information.

2. For each outcome measure, compare intervention and control groups pre- and post-intervention, the number of participants measured at each period, and the program impact (i.e., the difference in the change for the intervention and control groups). Describe tests of statistical significance and their results. If your evaluation did not include a control group, then provide information for the intervention group only. Table 2 provides a suggested format for providing this information for means, and Table 3 provides a suggested format for providing this information for percentages.

3. Describe the modeling approach (model specification) used, including variables in the model, the software package used, and estimation procedures.

F. Attrition

1. Describe analyses and methods used to handle attrition bias, if any.

2. If attrition analyses were conducted, provide the results. For example, indicate whether any characteristics distinguished between participants lost to attrition and those who completed the post-intervention data collection.

G. Missing Data (Item Nonresponse)

1. Describe procedures used to account for missing data, if any.

2. Provide the amount of missing data on an item-by-item basis for the demographic and outcome variables included in the model (the number of cases and the percentage missing).

Table 1. Suggested Format for Providing Information on the Demographic Characteristics of the Full Sample and Comparisons Between Intervention and Control Groups at Baseline

Characteristic	Full Sample (n = 484)	Intervention (n = 246)	Control (n = 238)	2	p
Age in years M (SD)	48.29 (14.08)^a	48.34 (13.74)^a	48.30 (14.50)^a	0.07^b	0.981
Gender %				3.97	0.052
Female	77.69	81.30	73.73
Male	22.31	18.70	26.27
Etc.

^aMean (standard deviation)

^b t-values from Student’s t-test

Table 2. Suggested Format for Providing Information on Outcome Measures (Means)

	Intervention				Control				Estimated Impact (95% CI)^a	Wald Chi-Square p-Value
	Pre	Post	t	_p	Pre	Post	t	_p	Estimated Impact (95% CI)^a	Wald Chi-Square p-Value
Outcome
Variable 1
Sample size	246	175			238	169
Mean (SE)	1.42 (0.14)	1.69 (0.15)	1.92	0.057	1.68 (0.21)	1.71 (0.22)	0.17	0.861	0.23 (0.22, 0.24)	0.355
Etc.

^a Program impact (with 95% confidence limits) estimated via difference-in-difference models comparing change across time in the intervention versus control groups. Ratios of impact estimates of 1.00 indicate no interaction between time and program group (i.e., no program impact).

Table 3. Suggested Format for Providing Information on Outcome Measures (Percentages)

	Intervention				Control				Estimated Impact (95% CI)^a	Wald Chi-Square p-Value
	Pre	Post	2	_p	Pre	Post	2	_p	Estimated Impact (95% CI)^a	Wald Chi-Square p-Value
Outcome
Variable 2
Sample size	246	174			238	168
Percent (SE)	53.91 (4.41)	67.92 (4.13)	7.45	0.059	59.0 (6.33)	62.3 (6.23)	1.50	0.683	10.8 (9.8, 11.8)	0.090
Etc.

SNAP-Ed Wave II: Discussion Guide for School Principals and Senior Center Program Managers

(with tailored questions noted for each demonstration project)

[Post-implementation]

State:
Respondent /Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time to participate in this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of [NAME OF PROGRAM] that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping to improve the health and nutrition status of children, families, and adults.

This study will provide information on how the [NAME OF PROGRAM] works from the perspective of the people who planned the program, the program teachers, you and your staff and some of the parents whose children participated. We also will use what you tell us today to provide recommendations for how [NAME OF PROGRAM] can be improved to better work with organizations like yours and the children and families you serve.

Any answers you provide for this study will be kept private except as otherwise required by law and your name will not be identified with any answers you provide. The estimated amount of time required to complete this interview is 30 minutes. I want to thank you for taking the time today to speak with me.

Before I begin, do you have any questions?

Tell me about your involvement in overseeing the implementation of [NAME OF PROGRAM].

REQUIRED PROBES:

Have you observed any of the classes for the children/older adults?
Have you been able to read any of the [NAME OF PROGRAM] materials that were sent home with children to their parents/the participants?

Now that the intervention is over, tell me your views about the educator who led the classes?
What would you say are the most useful aspects of the [NAME OF PROGRAM] program overall for the age groups it is targeting?
How did you promote the program and recruit teachers/adults to participate in the [NAME OF PROGRAM] at your school/center?

What worked well? Why?
What could be changed or improved to promote interest and participation in the program?

Were other teachers in the school/adults who come to the center interested in participating in [NAME OF PROGRAM] once they saw the program in action?

[QUESTION 6 FOR (INN) BUILDING AND STRENGTHENING IOWA COMMUNITY SUPPORT FOR NUTRITION AND PHYSICAL ACTIVITY (BASICS) AND UKCES’S LEAP2 ONLY]

How effective do you think the various strategies that were used by the [NAME OF PROGRAM] program to encourage parent involvement (e.g., take-home materials, activities targeted to parents and caregivers)? If you are not familiar with the strategies used, please feel free to skip this question.

What worked well? Why?
What could be changed or improved to increase parent or other caregiver engagement in the program’s nutrition education components?

[QUESTION 7 FOR MSUE EAT SMART, LIVE STRONG ONLY]

How did you work with the educator to help retain adults in the four-week program once they enrolled?

(a) What worked well? Why?

(b) What could be changed or improved to increase participation in the program?

What challenges or issues did you face in implementing this program at your school/site? How did you address these? Did you need to communicate with the [NAME OF PROGRAM] program staff to address any of these issues? If so what did you need to communicate to them about and how were those issues addressed?
What could be done to make the [NAME OF PROGRAM] program more appealing to schools/senior centers like yours?
Do you have any other suggestions for ways that this educational program could be improved?

[QUESTIONS 11 AND 12 FOR INN BASICS AND UKCES LEAP2 ONLY]

[NAME OF PROGRAM] aside, do you have any suggestions for other ways that schools like yours can encourage children to eat more fruits and vegetables at home and encourage their parents to serve more fruits and vegetables?
Do you think the classroom teachers involved with [INSERT NAME OF PROGRAM] would:

Be interested and able to continue some of the lessons and activities with the students in the classroom?
Need assistance?
Need outside resources?

[QUESTION 13 FOR INN BUILDING ANDS STRENGTHENING IOWA COMMUNITY SUPPORT ONLY]

From a program administrator perspective- how did it work for the classroom teachers to lead four lessons of the BASICS programs themselves? Did the teachers have to be “covered” for other duties they are responsible in for the school to be able to be trained and teach the BASICS lessons?

[QUESTION 14 FOR MSUE EAT SMART, LIVE STRONG ONLY]

[NAME OF PROGRAM] aside, do you have any suggestions for other ways that centers like yours can encourage older adults to eat more fruits and vegetables?
Are you interested in incorporating the concepts and or lessons of [NAME OF PROGRAM] into your school /senior center without the presence of [NAME OF PROGRAM]?

[IF YES]

How might you do this?
How feasible would it be to incorporate the concepts into your school?
What kind of help might you need from [NAME OF PROGRAM] if it were available?

My final and very straightforward question for you today is, would you want the [NAME OF PROGRAM] to come to your school next year? Why or why not?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. We have a gift card to thank you for your time

SNAP-Ed Wave II: Discussion Guide for Classroom Teachers

(Building and Strengthening Iowa Community Support for Nutrition and Physical Activity)

[Pre-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the Building and Strengthening Iowa Community Support for Nutrition and Physical Activity (BASICS) program that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping to improve the health and nutrition status of children, families, and adults. This study will include not only outcome evaluation information but also process information on how it is being implemented and how you are evaluating the intervention. All of this will be useful to both FNS and to other Supplemental Nutrition Assistance Program-Education (SNAP-Ed)-implementing agencies that are planning to evaluate their own SNAP-Ed interventions.

The purpose of my interview today is primarily to ask you about your experiences with perceptions of BASICS at your school. I will use what you tell us today to provide recommendations for how the BASICS program could be improved to better serve the children and families in your community and those in other communities like yours.

I expect that our discussion today will take about 30 minutes. Before I begin, do you have any questions?

Background Information

First, I would like to confirm that you are a teacher who works in a class room at [NAME OF SCHOOL].
What is your current job title at the school?
How long have you worked in this position at this school?
On a scale of 0–5, with 0 being unimportant and 5 being extremely important, how important would you say eating more fruits and vegetables is for school age children and their families? Why do you think this?
On a scale of 0–5, with 0 being unimportant and 5 being extremely important, how important would you say choosing 1 percent or non-fat dairy products is for school age children and their families? Why do you think this?

Exposure and Satisfaction with Intervention Classes Targeted to the Children

Now I would like to ask you about any experience you might have had in the past with BASICS and experience you may have had in the past with the nutrition educator who will be teaching the class

Has the BASICS curriculum been taught in your classroom in the past? If so, what has your experience been with the BASICS curriculum?
Do you know/have you observed the nutrition educator who will be teaching the class the eight BASICS lessons in your classroom in the past? If so, what has your experience been?
Do you anticipate staying for the BASICS lessons taught by the nutritionist? [FOR RESPONDENTS WHO ANSWER FEWER THAN THE MAXIMUM NUMBER OF CLASSES OFFERED] What would make it easier for you to come to these classes (e.g., scheduling issues, length of class, language barriers)?

Feedback on Teacher Training and Teacher Use of Take-Home Materials (e.g., parent pages/informational materials and recipes)

9. Have you received training on the four BASICS lessons that you will be teaching? If so, could you describe the training (e.g., who provided the training, where it was provided, how many hours it was offered, how many hours you participated)?

Considering your available time, how much have you been able to look over and read BASICS (particularly for the four lessons that you will be teaching):

Teacher training materials?

□ Not looked over or read at all

□ Glanced at materials

□ Browsed through most materials

□ Read thoroughly

Take-home materials designed for parents or guardians of the children in your classroom?

□ Not looked over or read at all

□ Glanced at materials

□ Browsed through most materials

□ Read thoroughly

11. What do you think were the most helpful aspects of these teacher training materials?

12. What changes or improvements, if any, would you suggest to the teacher training materials?

13. What do you think were the most helpful aspects of these take-home materials?

14. What changes or improvements, if any, would you suggest to the take-home materials?

15. Do you feel prepared to teach the four lessons of the BASICS program?

16. What changes or improvements, if any, would you suggest to the four lessons which will teach in the classroom?

17. Would you need training from the INN again next year in order to teach the BASICS lessons next year at your school?

Exposure and Feedback on Parent/Family Classes

Now, I would like to hear your feedback on the classes that were provided for the parents and guardians of the children in your classroom by the BASICS and any other recommendations you would suggest for reaching parents with the messages of this program.

18. How many of the BASICS parent/family activities do you plan to attend? [PLEASE PROVIDE A CONCRETE NUMBER.]

19. Do you think these parent events will be well attended? Why or why not?

How well attended are other parent activities at the school?
What changes or improvements would you suggest to better reach the parents with the messages of the BASICS?

Feedback on the Program Overall and Other Ways to Meet Program’s Nutrition Objectives

20. At this point in the planning process do you have any recommendations or suggestions for ways that the BASICS could be improved?

21. [ADDITIONAL QUESTION NOT ON MAIL TEACHER SURVEY] Do you have any other suggestions for how schools like yours can encourage school age children to eat more fruits and vegetables at home and encourage their parents to serve more fruits and vegetables? If so, what are they?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. We have a gift card to thank you for your time.

SNAP-Ed Wave II: Discussion Guide for Classroom Teachers

(Building and Strengthening Iowa Community Support for Nutrition and Physical Activity)

[Post-implementation]

State:
Interviewer:
Respondent:
Date of Interview:
Title:
Center Name:
Address:
Phone:
Fax:
E-mail:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the INN (INN) Building and Strengthening Iowa Community Support for Nutrition and Physical Activity (BASICS) program that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute, and our work focuses on helping to improve the health and nutrition status of children, families, and adults. This study will include not only outcome evaluation information but also process information on how it is being implemented and how you are evaluating the intervention. All of this will be useful to both FNS and to other Supplemental Nutrition Assistance Program-Education (SNAP-Ed)-implementing agencies that are planning to evaluate their own SNAP-Ed interventions.

The purpose of my interview today is primarily to ask you about your experiences with perceptions of BASICS at your center. I will use what you tell us today to provide recommendations for how the BASICS program could be improved to better serve the children and families in your community and those in other communities like yours.

I expect that our discussion today will take about 30 minutes. Before I begin, do you have any questions?

Background Information

First, I would like to confirm that you are a teacher at the [NAME OF SCHOOL] and that you currently teach the INN BASICS program.
What is your current job title here at the school?
How long have you worked in this position at this school?

4. On a scale of 0–5, where 0 is not important and 5 is extremely important, how important do you think eating more fruits and vegetables should be for school age children and their families? Tell me why you feel this way.

5. On a scale of 0–5, where 0 is not important and 5 is extremely important, how important do you think choosing 1 percent or nonfat milk products should be for children and their families? Tell me why you feel this way.

Exposure and Satisfaction with Intervention Classes Targeted to the Children

Now I would like to ask you about your experiences with the eight BASICS classes taught by the INN nutrition educator to your students.

6. How many of the BASICS lessons taught by the nutritionist were you able to observe in the classroom? [FOR RESPONDENTS WHO ANSWER FEWER THAN THE MAXIMUM NUMBER OF CLASSES OFFERED] What would have made it easier for you to be in the classroom for these classes (e.g., fewer papers to correct)?

7. What do you think worked well in these in-classroom activities?

8. What changes or improvements, if any, would you suggest to the BASICS nutrition lessons? Tell me why you feel this way.

Feedback on and Teacher Use of Take-Home Materials (e.g., parent pages/informational materials and recipes)

9. Considering your available time, how much have you been able to review the BASICS take-home materials designed for parents or guardians of the children in your classroom?

□ Not looked over or read at all

□ Glanced at materials

□ Browsed through most materials

□ Read thoroughly

10. What do you think were the most helpful aspects of these take-home materials?

11. What changes or improvements, if any, would you suggest to the take-home materials?

Now, I would like to ask you about the four BASICS lessons you taught in your classroom.

12. Did you feel prepared to teach the four BASICS lessons?

13. What changes or improvements, if any, would you suggest to the four BASICS lessons?

14. Are you interested in teaching the BASICS lessons next year at your school?

15. Outside of the four lessons you taught, did you incorporate any nutrition messages, sample activities, or tools from BASICS in the classroom?

[IF NO, SKIP TO QUESTION 14]

[IF YES] How did you incorporate these messages in your classroom?

16. How often would you estimate you used the new information you received from the BASICS in your classroom?

□ A couple of times

□ Once every week

□ A few times a week

□ More than a few times a week

(a) What aspects of the BASICS content or design of the messages, sample activities or tools made it easier for you to incorporate these into your classroom activities (e.g., ease of use in the classroom setting, cultural sensitivity, age appropriateness of the materials for the target audience)?

(b) What aspects prevented you from using these tools in your classroom (e.g., lack of time, lack of money for supplies, lack of confidence)?

Exposure and Feedback on Parent/Family Classes

Now, I would like to hear your feedback on the classes that were provided for the parents and guardians of the children in your classroom by BASICS and any other recommendations you would suggest for reaching parents with the messages of this program.

17. How many of the BASICS parent/family activities that were held at your center were you able to attend? [PLEASE PROVIDE A CONCRETE NUMBER.]

18. What do you think worked well about these activities for parents and families?

19. What do you think did not work well about these activities for parents and families?

20. What changes or improvements would you suggest to better reach the parents with the messages of BASICS?

Feedback on the Program Overall and other Ways to Meet Program’s Nutrition Objectives

21. Do you have any other recommendations or suggestions for ways that the BASICS could be improved?

[ADDITIONAL QUESTION NOT ON MAIL TEACHER SURVEY] Do you have any other suggestions for how schools like yours can encourage school age children to choose more fruits and vegetables as snacks and encourage their parents to serve more fruits and vegetables at home? If so, what are they?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. We have a gift card to thank you for your time.

Questionnaire for Lead Teachers in Classrooms Receiving the Iowa Nutrition Network Building and Strengthening Iowa Community Support Program

The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) wants to know about your experience with the Building and Strengthening Iowa Community Support (BASICS) program. They have contracted with Altarum Institute to study how this program is being implemented at local schools. Please fill out the form below to provide your feedback and help improve this program for children and families in your community and those in other communities like yours.

Your response to this questionnaire will be kept private except as otherwise required by law. After we have received all of the completed questionnaires and conducted interviews with a number of sites, we will write a report for the FNS. Your name will not appear anywhere in the report. Nothing that you write will be attached to your name at any point. None of your responses will affect your job or be shared with the school administrator where you work.

Instructions
Please complete the short questionnaire on the following pages. Answer each question honestly and thoughtfully. When you have answered the questions, please mail it in the stamped self-addressed envelope provided. Please be sure to also fill out and include the enclosed contact form to receive a $10 check for completing the questionnaire.
School name
Today’s date (mm/dd/yyyy)
Your current job title
1. On a scale of 0–5, where 0 is Not Important and 5 is Extremely Important, how important do you think eating more fruits and vegetables should be for children and their families? (Please mark only one box below.)
Not at All Extremely Important Important □ □ □ □ □ □ 0 1 2 3 4 5

2. On a scale of 0–5, where 0 is Not Important and 5 is Extremely Important, how important do you think choosing 1 percent or nonfat milk products should be for children and their families? (Please mark only one box below.)

Not at All Extremely

Important Important

□ □ □ □ □ □

0 1 2 3 4 5

3. How many of the BASICS lessons taught by the nutritionist were you able to observe in the classroom?

□ None □ One to two □ Three to four □ Five to six

□ Seven to eight □ All

4. What do you think worked well in these in-classroom activities?

5. What changes or improvements, if any, would you suggest to the BASICS nutrition lessons?

6. Considering your available time, how much have you been able to review the BASICS take-home materials designed for parents or guardians of the children in your classroom?

□ Not looked over or read at all

□ Glanced at materials

□ Browsed through most materials

□ Read thoroughly

7. What do you think were the most helpful aspects of these take-home materials?

8. What changes or improvements, if any, would you suggest to the take-home materials?

9. Did you feel prepared to teach the four BASICS lessons?

10. What changes or improvements, if any, would you suggest to the four BASICS lessons?

11. Are you interested in teaching the BASICS lessons next year at your school?

12. Outside of the 4 lessons you taught: did you incorporate any nutrition messages, sample activities or tools from the BASICS in your classroom?			□ Yes □ No  GO TO Q14
12a. How did you incorporate these messages in your classroom?

13. How often would you estimate you used the new information you received from the BASICS in your classroom?		□ A couple of times □ Once every week □ A few times a week □ More than a few times a week
13a. What aspects of the BASICS content or design of the messages, sample activities or tools made it easier for you to incorporate these into your classroom activities (e.g., ease of use in the classroom setting, cultural sensitivity, age appropriateness of the materials for the target audience)?

13b. What aspects prevented you from using these tools in your classroom (e.g., lack of time, lack of money for supplies, lack of confidence)?

14. How many of the BASICS parent/family activities that were held at your center were you able to attend?	□ None □ One □ Two
15. What do you think worked well about these activities for parents and families?

16. What do you think did not work well about these activities for parents and families?

17. What changes or improvements would you suggest to better reach the parents with the messages of the BASICS?

18. Do you have any other recommendations or suggestions for ways that the BASICS could be improved?

Thank you very much for your time and input on this very important project. Please mail your private responses along with the completed short contact form to the following address: [STAFF CONTACT INFO].

SNAP-Ed Wave II: Discussion Guide for Retail Stores: Produce Managers

(Dairy Managers if Appropriate)

[Post-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time to participate in this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of the Iowa Nutrition Network (INN) Building and Strengthening Iowa Community Support for Nutrition and Physical Activity (BASICS) program that is offering information to children and their families about healthy foods to eat and the importance of being active. The BASICS program includes two social marketing initiatives conducted in grocery stores called, Pick a better snack and 1% or Less, Yes! Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping to improve the health and nutrition status of children, families, and adults.

This study will provide information on how the BASICS program works from the perspective of the people who planned the program, the program teachers, you and school district food service directors and some of the parents whose children participated. We also will use what you tell us today to provide recommendations for how BASICS can be improved to better work with stores like yours and the children and families you serve.

Any answers you provide for this study will be kept private except as otherwise required by law and your name will not be identified with any answers you provide. The estimated amount of time required to complete this interview is 15 minutes. I want to thank you for taking the time today to speak with me.

Note to interviewer: before interviewing the produce manager, contact the store manager to obtain permission and set up an interview time when the store staff is not busy. It is recommended that the Store Manager participate in the interview in addition to the Produce Manager (if the 1% or Less, Yes! campaign has been conducted in the store, include the dairy manager as well).

Before I begin, do you have any questions?

Let’s begin with some general information about your store and customers:

How long have you been the produce manager (dairy manager) of this store?

PROBE: Were you the produce manager (dairy manager) during the time of the Pick a better snack campaign taking place November through May of this year? [If 1% or Less, Yes! campaign held at store, ask the same question]

How would you describe your store in terms of size? Small, medium or large?
Would you be able to roughly break down your customers’ race and ethnicity in percentages? (Designate a percentage for each racial/ethnic category below.)

a.) American Indian or Alaska Native

b.) Asian

c.) Black or African American

d.) Hispanic or Latino

e.) Native Hawaiian or other Pacific Islander

7.) White

Approximately what percentage of your customers uses Food Assistance EBT at your store?

Next, I’d like to ask you some questions about the Pick a better snack [1% or Less, Yes!] campaign:

Does your store participate in the Pick a better snack [1% or Less, Yes!] campaign?

PROBES:

How did you find out about the Pick a better snack [1% or Less, Yes!] campaign?

Why is your store involved in the Pick a better snack [1% or Less, Yes!] campaign?
Have you seen any of the Pick a better snack [1% or Less, Yes!] campaign materials used in the social marketing campaign?

PROBES:

Bingo cards
Recipes
Posters
Shelf Talker

If you have seen the above materials, what did you think about them? (Probes: Effective? Ineffective?)
1. Has your store held fruit and vegetable [milk] snack demonstrations as part of the Pick a better snack [1% or Less, Yes!] campaign?
2. How many demonstrations has your store held? [per month and which months]
3. Approximately how many people “attend” each demonstration?
What would you say are the most useful aspects of the Pick a better snack [1% or Less, Yes!] campaign and demonstrations overall for the age groups of children and parents it is targeting?
Did you provide additional support to the demonstrations at your store by providing food for the

demonstrations?

What challenges or issues were faced in implementing the Pick a better snack [1% or Less, Yes!] campaign and demonstrations in your store?

How did you address these?

Did you need to communicate with the INN/Iowa Department of Health staff to address any of these issues? If so what did you need to communicate to them about and how were those issues addressed?
What could be done to make the Pick a better snack [1% or Less, Yes!] campaign more appealing to retail stores like yours?
Have you made any changes to your inventory or made any other changes to your store since starting with the Pick a better snack [1% or Less, Yes!] campaign? If so, what types of changes have you made? [Probe: the campaign started November 2011]

And finally, I have a few follow-up questions about the Pick a better snack [1% or Less, Yes!] campaign:

Have you increased or decreased the amount of fruits and vegetables [1% or less milk products] you carry?
Have you seen a change (avoid leading question) in sales of sampled products on the days when the Pick a better snack [1% or Less, Yes!] demonstrations took place?
How much of a role do you think that you should play in increasing the availability of healthy foods in your community? Why?
Do you have any suggestions for ways the Pick a better snack [1% or Less, Yes!] campaign could be improved?
Would you be interested in continuing the Pick a better snack [1% or Less, Yes!] campaign in your retail store (without the presence of INN)? Note: the materials are free of cost to consumers online.
How feasible would it be to incorporate the activities in your retail store on an ongoing basis?
Would you want additional help from the INN/Iowa Department of Health if it were available?
Finally, would you want the Pick a better snack [1% or Less, Yes!] campaign in your retail store next year? Why or why not?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. We have a gift card to thank you for your time.

SNAP-Ed Wave II: Group Discussion Guide for Parents/Caregivers (INN)

[Post-implementation ONLY]

Date of Discussion:
Location:
Study ID #:
Facilitator:
Note Taker:
Number of Participants:
Start Time:
End Time:

Welcome! My name is _____. I am here with my co-worker _____ from Altarum Institute. Thank you for taking the time for this group discussion. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the BASICS nutrition education program that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults.

This study will provide information on how the program in which your children participated works from the perspective of the people who planned the program, the teachers, you, and your child. The purpose of today’s group is to hear from you about your own and your child’s experiences and satisfaction with this program that recently took place at your child’s day care/school. We also will use what you tell us today provide recommendations for how BASICS can be improved to better serve the children and families in your community and those in other communities like yours.

We will be using first names only today. Everything you say will be kept private except as otherwise required by law. After we conduct several of these group discussions, we will write a report for the FNS. Your name will not appear anywhere in the report. Nothing that is said today will be attached to your name at any point. Nothing that you say will affect the services you receive through any of the programs we talk about today.

Before we begin, I would like to review a few details about our discussion:

First, your participation in today’s discussion is voluntary. You are free to leave at any time.
There are no right or wrong answers. Remember that we don’t work for the schools or with the educators, so please feel free to say whatever you think.
Also, it is okay to have ideas or opinions that are different from each other. We want to hear everyone’s point of view.
We are tape-recording this session so that we don’t miss anything important, and it will be helpful to have only one person talking at a time. If two people talk at once, we can’t understand what anyone is saying. We may remind you of this during the group discussion.

We would like everyone to participate. But you each don’t have to answer every question. You don’t have to raise your hand either. If, however, some of you are shy or we really want to know what you think about a particular question, we may ask you what you think.
We have a lot to talk about today, so don’t be surprised if at some point we interrupt the discussion and move to another topic. But don’t let us cut you off. If there is something important you want to say, let us know and you can add your thoughts before we change subjects.
Finally, we just want to emphasize what we said earlier: we will be using first names only. Everything you say is private. What you say today will not be attached to your name at any point. Nothing that you say will affect the child care you receive at this site or any other services you receive from this or any other program.

The group will last no more than 2 hours and will end no later than _______. We will not be taking a formal break, but if you need to leave for a restroom break, the bathrooms are _____________. And feel free to get snacks.

For this session, I will read a question and then listen to your responses. I also may ask follow up questions to get some more detail.

Let’s get started! I’m looking forward to hearing more about BASICS.

Do you have any questions before we begin?

Introductions/Icebreaker

Let’s go around the room for this one: Please introduce yourself, tell us how long your child has been coming to this school, and name one fun activity you like doing with your child. [MODERATOR NOTE: It is helpful to go in order of seating to allow the transcriptionist to label responses by person. Also, for note taking, you can then label Person1, Person 2, Person 3, etc. when writing comments.]

Exposure and Accessibility of SNAP-Ed Intervention for Parents/Caregivers

Please raise your hand if you know that your child has been participating in a program at this school where they learn about healthy foods and being active. [ASK THE FOLLOWING QUESTIONS FOR THOSE WHO RAISE THEIR HANDS.]

What did your children tell you about what they did in these classes or sessions?

PROBES: Food they tried? Activities they did? Games they played? Lessons they learned?

Did you see any take home materials on food and physical activity recently provided for you by the BASICS nutrition education program? [THE MODERATOR SHOULD PROMPT A RESPONSE BY SHOWING SOME SAMPLE TAKE-HOME MATERIALS USED IN THE INTERVENTION.]
What were the most helpful aspects of these take-home materials?

PROBES: What did you like about the materials? What were the least helpful aspects of these take-home materials? What didn’t you like about the materials?

Did you hear about the parent classes, family activity nights, or events that were offered at [NAME OF INTERVENTION SITE]? If yes, how did you hear about them?
Raise your hand if you went to at least one class.

If you raised your hand, what made you decide to go?
If you didn’t raise your hand, what were the reasons that you didn’t go (e.g., barriers related to timing and location, other barriers related to accessibility, level or interest or perceived need)?

Please think for a moment about what could be done to encourage more people like you to participate in these classes/family events. I will hand out a pencil and paper if you want to write down your ideas before you answer out loud.

[AFTER ABOUT 2 MINUTES, TAKE ANSWERS VIA ROUND-ROBIN QUESTIONING.]

If you went to any of these classes did you receive any handouts?

Which handouts were most helpful and why?
Which handouts were not helpful and why?

Do you think the educator who led the classes provided information in a way that was easy for the people in the class to understand?
Would you say that the educator who led the classes was a good teacher for you?
If yes, what made her a good teacher? If not, why not?

Satisfaction/Likes and Dislikes With Intervention

Tell me about the parts of the program overall—including the classes for your children, the take home materials, and any classes you may have participated in—that you liked the best and why you liked these parts.
What parts of the program did you like least and why?
What parts of the program do you think your child liked the best and why?
What parts of the program did your child like the least and why?

Perceptions of Goals and Relevancy of Intervention

We are interested in hearing more about what you thought about the purpose of the classes and whether they helped you and provided useful information to you.

What do you think BASICS was trying to teach you and your child?
How useful was the information the program offered for parents like you with children?
How well did the program suggestions and information fit with the ways that people of your racial or ethnic background live your life?
How well did the program suggestions and information fit with the challenges faced by people who do not have a lot of money?

Intervention Impacts

These next few questions are about how you think BASICS classes and materials may have helped you learn new information or other ways it may have changed things for you or your children.

What are the most important things that your child learned from this program?
What are the most important things that you learned from this program?
Now I would like to ask you a question that you probably need to think about: What is the most significant change or changes that have taken place in your household because of this program?

Factors Affecting Fruit and Vegetable Availability at Home and Ways of Addressing these Barriers

Now I would like to take a few moments to ask you about the difficulties that parents who live in your neighborhood might face in trying to buy, store, and prepare fruits and vegetables for your preschool child.

What makes it harder for you or other parents like you to buy and keep fruits and vegetables at home (e.g., cost, access, and storage)?
What makes it harder for you or other parents of young children like you to prepare and serve fruits and vegetables to your young children?
Did the information or take home materials provided to you by BASICS help you to address any of these difficulties or barriers?

For those who said yes, how was the information or materials helpful?
For those who said no, what could have been done to make the information or take-home materials more helpful for parents?

Recommendations

Would you recommend this program to friends? Why or why not?
If you could change anything about the classes or take-home materials or other aspects of the BASICS program, what would it be?
Is there anything we haven’t asked that you would like to tell us about your experience with and opinions of the BASICS program?
Before we close, I would like you to help us by giving us your ideas for other ways that schools could encourage children to eat more fruits and vegetables and encourage their parents to serve fruits and vegetables more often.

Thank you very much for participating in this discussion group today. We have learned a lot from your experiences and recommendations.

In appreciation of your time and trouble today, we have gift cards for each of you today. Before you leave, make sure to take one of gift cards and sign the form indicating you have received one of the cards. Enjoy your day.

SNAP-Ed Wave II: Discussion Guide for UKCES FCS Agents

[Pre-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of the Leap 2 Program that is offering information to children about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education models around the country and to provide recommendations for how these interventions could be improved to better serve the children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Today we will specifically be discussing the planning process and your expectations for the intervention. Once it has been implemented, we will follow up with you to find out whether the intervention met your expectations and how it might be improved. I expect that this interview will take about 40 minutes. Thank you for taking the time to speak with me.

Before I begin, do you have any questions?

Can you please describe your role in the program design, implementation and evaluation of LEAP2, specifically your role as bridge between UKCES State staff, the direct educators, and the schools in your county?
What challenges, if any, have you faced during the design and planning phases of this nutrition education program? What factors do you feel have contributed most to a successful design and planning phase (e.g., using education materials that were already developed, good communication between contributors, knowledgeable staff, establishment of strong partnerships)?

3. What lessons have you learned during this key phases of program development?

(a) What would you do differently? Why?

(b) What would you do the same? Why?

Now I would like to shift our focus to the upcoming implementation of the LEAP2 SNAP-Ed project.

Now that you are ready to transition from the planning and design phase of your project to the implementation phase, what challenges, if any, are you anticipating? Why? How do you think you will address these challenges?
Do you feel that the environment in which the intervention will take place will be able to support the intended change in behavior, knowledge, and/or attitudes? For example, do you have any sense of the school’s buy-in and/or enthusiasm about the intervention and what impact this might have on the children? Does the school offer the children healthy food options and are healthy foods otherwise available?

PROBE: Can you describe the foods offered by the school? Is there a website where the menus are posted?

Could you please describe the roles and responsibilities of the nutrition education providers who will be delivering the LEAP2 curriculum as part of this demonstration project—both overall and as they relate to this specific project?
Were these individuals recruited specifically for the purpose of delivering LEAP2 as part of this demonstration project? If so, did you have any difficulty recruiting adequate staff for LEAP2 nutrition education delivery? If so, what were the recruitment challenges/problems?
Please describe any training LEAP2 nutrition education providers receive?

Who provides the training?
How often does training take place?
Was any additional training provided because of their involvement in the LEAP2 demonstration project? Was there a training event specific to delivering LEAP2 as part of this demonstration project? If so, would you be able to provide an agenda for this training event?

Do the educators have flexibility in how they deliver the program? Or are they directed to follow the curriculum strictly as written? How will that be assessed?
Will nutrition education providers be responsible for documenting or collecting any information or data related to nutrition education delivery [PROBES: reach, dose, lessons taught]?

[IF YES] Is this information the nutrition education providers are always asked/required to collect, or is it a special requirement because of their role in the demonstration project?
[IF BECAUSE OF DEMOSTRATION PROJECT] Will the nutrition education providers be trained on how to accurately collect the desired information?

Can you describe how you will manage and supervise the direct educators that will deliver the LEAP2 Program? [Collect copies of any forms to be used.] How will fidelity to the LEAP2 program be assessed (i.e., how will you ensure that the program plan will be followed as intended)?
Please describe any quality control and monitoring efforts that will take place during implementation (e.g., nutrition education delivery, nutrition education data collection). [Collect copies of any forms to be used.]
Can you please tell me how you will collect reach and dosage of the intervention? What forms will be used? How will data be analyzed?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. As I mentioned, we will follow up and talk with you after the intervention and evaluation period are over.

SNAP-Ed Wave II: Discussion Guide for UKCES FCS Agents

[Post-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of the LEAP2 Program that is offering information to children about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education (SNAP-Ed) models around the country and to provide recommendations for how these interventions could be improved to better serve the children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Before I begin, do you have any questions?

Do you feel that the environment in which the LEAP2 intervention took place was able to support the intended change in behavior, knowledge, and/or attitudes?

For example, what was the school’s buy-in and/or enthusiasm about the LEAP2 intervention and what impact this had on the children?
Does the school offer the children healthy foods options and are healthy foods otherwise available?

What, if any, other nutrition education messages were the children in the intervention sites being exposed to during the intervention period (that you are aware of)?

Can you please describe your role as a bridge between State UKCES staff, the direct educators, and the schools? Do you think your role was integral to the program’s success? Why or why not?
Did the LEAP2 have any difficulty retaining adequate staff for the nutrition education delivery? If so, what were the recruitment challenges/problems?
Did you find the level of staff, both in terms of qualifications and total number of staff (and types of staff), adequate for optimally delivering the nutrition education intervention?
How effective were staff in delivering the intended nutrition education messages?

(a) Why do you think these staff were effective/ineffective?

(b) What could they have done differently to improve their effectiveness?

What changes, if any, were made to planned key staff involvement and what were the reasons for any such changes?
Please describe any quality control and monitoring efforts that took place during the implementation of LEAP2 (e.g., nutrition education delivery, nutrition education data collection). [Collect copies of any forms to be used.]

Lessons Learned for Improvement and Replicability

Next I’d like to talk about lessons learned during implementation of the study.

Overall, what factors were key to the success of LEAP2?
What factors hindered or limited the success of LEAP2?
Looking back over the past __ months, what lessons have you learned? What would be most valuable for another State or demonstration project to know if they were considering using this model?
In your opinion, are there any aspects of this SNAP-Ed program that would make it difficult to implement on a larger scale?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. As I mentioned, we will follow up and talk with you after the intervention and evaluation period are over.

Questionnaire for Lead Teachers in Classrooms Receiving the LEAP2 Program

The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) wants to know about your experience with the LEAP2 Program. They have contracted with Altarum Institute to study how this program is being implemented in schools. Please fill out the form below to provide your feedback and help improve this program for children and families in your community and those in other communities like yours.

Instructions
Please fill out the short questionnaire on the following pages. Answer each question honestly and thoughtfully. When you have answered the questions, please mail it in the stamped self-addressed envelope provided. Please be sure to also fill out and include the enclosed contact form to receive a $10 check for completing the questionnaire.
School name
Today’s date (mm/dd/yyyy)
Your current job title at this school
1. On a scale of 0–5, where 0 is Not Important and 5 is Extremely Important, how important do you think eating more fruits and vegetables should be for preschool children and their families? (Please mark only one box below.)
Not at All Extremely Important Important □ □ □ □ □ □ 0 1 2 3 4 5

1a. Why do you think this?

2. How many of the LEAP2 lessons taught by the educator were you able to observe in the classroom?	□ None □ One to three □ Four to six □ Seven to eight □ All
2a. What would have made it easier for you to be in the classroom for these classes?
3. Did you receive any training on the LEAP2 curriculum?	□ Yes □ No
3a. If so, who provided the training?
4. Where you able to complete the daily sticker calendar with your students?	□ Yes □ No
4a. If yes, Did you think it was an effective activity to promote fruit and vegetable consumption?
5. What do you think worked well in the LEAP2 nutrition lessons?

6. What changes or improvements, if any, would you suggest to the LEAP2 nutrition lessons?

7. Considering your available time, how much have you been able to review the LEAP2 take-home materials designed for parents or guardians of the children in your classroom?		□ Not looked over or read at all □ Glanced at materials □ Browsed through most materials □ Read thoroughly
8. What do you think were the most helpful aspects of these take-home materials?

9. What changes or improvements, if any, would you suggest to the take-home materials?

10. Did you incorporate any nutrition messages, sample activities or tools from the LEAP2 program in your classroom?			□ Yes □ No

10a. How did you incorporate these messages in your classroom?
11. How often would you estimate you use the new information you received from the LEAP2 program in your classroom?	□ A couple of times □ Once every week □ A few times a week □ More than a few times a week
11a. What aspects of the LEAP2 content or design of the messages, sample activities or tools made it easier for you to incorporate these into your classroom activities? (e.g., ease of use in the classroom setting, cultural sensitivity, age appropriateness of the materials for the target audience)
11b. What aspects prevented you from using these tools in your classroom (e.g., lack of time, lack of money for supplies, lack of confidence)?
12. Do you think the storybooks used in the LEAP2 program apply the goals of the LEAP2 program? Please explain.

13. What recommendations or suggestions do you have for ways that the LEAP2 program could be improved?

14. What changes or improvements would you suggest to better reach parents with the messages of the LEAP2 program?

Thank you very much for your time and input on this very important project. Please mail your private responses along with the completed short contact form to [STAFF CONTACT INFO].

SNAP-Ed Wave II: Group Discussion Guide for Parents/Caregivers (UKCES)

[Post-implementation ONLY]

Date of Discussion:
Location:
Study ID #:
Facilitator:
Note Taker:
Number of Participants:
Start Time:
End Time:

Welcome! My name is _____. I am here with my co-worker _____ from Altarum Institute. Thank you for taking the time for this group discussion. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the LEAP2 nutrition education program that is offering information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults.

This study will provide information on how the program in which your children participated works from the perspective of the people who planned the program, the teachers, you, and your child. The purpose of today’s group is to hear from you about your own and your child’s experiences and satisfaction with this program that recently took place at your child’s day care/school. We also will use what you tell us today provide recommendations for how LEAP2 can be improved to better serve the children and families in your community and those in other communities like yours.

Before we begin, I would like to review a few details about our discussion:

First, your participation in today’s discussion is voluntary. You are free to leave at any time.
There are no right or wrong answers. Remember that we don’t work for the schools or with the educators, so please feel free to say whatever you think.
It is okay to have ideas or opinions that are different from each other. We want to hear everyone’s point of view.
We are tape-recording this session so that we don’t miss anything important, and it will be helpful to have only one person talking at a time. If two people talk at once, we can’t understand what anyone is saying. We may remind you of this during the group discussion.

We would like everyone to participate, but you each don’t have to answer every question. You don’t have to raise your hand either. If, however, some of you are shy or we really want to know what you think about a particular question, we may ask you what you think.
We have a lot to talk about today, so don’t be surprised if at some point we interrupt the discussion and move to another topic. But don’t let us cut you off. If there is something important you want to say, let us know and you can add your thoughts before we change subjects.
Finally, we just want to emphasize what we said earlier: We will be using first names only. Everything you say is private. What you say today will not be attached to your name at any point. Nothing that you say will affect the child care you receive at this site or any other services you receive from this or any other program.

For this session, I will read a question and then listen to your responses. I also may ask follow up questions to get some more detail.

Let’s get started! I’m looking forward to hearing more about LEAP2. Do you have any questions before we begin?

Introductions/Icebreaker

Let’s go around the room for this one: Please introduce yourself, tell us how long your child has been coming to this school, and name one fun activity you like doing with your child. [MODERATOR NOTE: It is helpful to go in order of seating to allow the transcriptionist to label responses by person. Also, for note taking, you can then label Person 1, Person 2, Person 3, etc. when writing comments.]

Exposure and Accessibility of Supplemental Nutrition Assistance Program-Education Intervention for Parents/Caregivers

What did your children tell you about what they did in these classes or sessions?

PROBES: Food they tried? Activities they did? Games they played? Lessons they learned?

Did you see any take-home materials on food and physical activity recently provided for you by LEAP2? [THE MODERATOR SHOULD PROMPT A RESPONSE BY SHOWING SOME SAMPLE TAKE-HOME MATERIALS USED IN THE INTERVENTION.]
What were the most helpful aspects of these take-home materials?

PROBE: What did you like about the materials?

What were the least helpful aspects of these take-home materials?

PROBE: What didn’t you like about the materials?

Satisfaction/Likes and Dislikes With Intervention

Tell me about the parts of the program overall—including the classes for your children, the take home materials, and any classes you may have participated in—that you liked the best and why you liked these parts.
What parts of the program did you like least and why?
What parts of the program do you think your child liked the best and why?
What parts of the program did your child like the least and why?

Perceptions of Goals and Relevancy of Intervention

We are interested in hearing more about what you thought about the purpose of the classes, whether they helped you and provided useful information to you.

What do you think LEAP2 was trying to teach you and your child?
How useful was the information the program offered for parents like you with children?
How well did the program suggestions and information fit with the ways that people of your racial or ethnic background live your life?
How well did the program suggestions and information fit with the challenges faced by people who do not have a lot of money?

Intervention Impacts

These next few questions are about how you think LEAP2 classes and materials may have helped you learn new information or other ways it may have changed things for you or your children.

What are the most important things that your child learned from this program?
What are the most important things that you learned from this program?
Now I would like to ask you a question that you probably need to think about: What is the most significant change or changes that have taken place in your household because of this program?

Factors Affecting Fruit and Vegetable Availability at Home and Ways of Addressing These Barriers

What makes it harder for you or other parents like you to buy and keep fruits and vegetables at home (e.g., cost, access, and storage)?
What makes it harder for you or other parents of young children like you to prepare and serve fruits and vegetables to your young children?
Did the information or take-home materials provided to you by LEAP2 help you to address any of these difficulties or barriers?

For those who said yes, how was the information/materials helpful?
For those who said no, what could have been done to make the information or take-home materials more helpful for parents?

Recommendations

Would you recommend this program to friends? Why or why not?
If you could change anything about the classes or take home materials or other aspects of the LEAP2 program, what would it be?
Is there anything we haven’t asked that you would like to tell us about your experience with and opinions of the LEAP2 program?
Before we close, I would like you to help us by giving us your ideas for other ways that schools could encourage children to eat more fruits and vegetables and encourage their parents to serve fruits and vegetables more often.

Thank you very much for participating in this discussion group today. We have learned a lot from your experiences and recommendations.

SNAP-Ed Wave II: Discussion Guide for School Classroom Teachers (UKCES)

[Post-implementation]

State:
Interviewer:
Respondent:
Date of Interview:
Title:
School Name:
Address:
Phone:
Fax:
E-mail:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the LEAP2 Program that offers information to children and their families about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping to improve the health and nutrition status of children, families, and adults. This study will include not only outcome evaluation information but also process information on how it is being implemented and how you are evaluating the intervention. All of this will be useful to both FNS and to other Supplemental Nutrition Assistance Program-Education (SNAP-Ed)-implementing agencies that are planning to evaluate their own SNAP-Ed interventions.

The purpose of my interview today is primarily to ask you about your experiences with perceptions of the LEAP2 Program at your school. I will use what you tell us today to provide recommendations for how the LEAP2 Program could be improved to better serve the children and families in your community and those in other communities like yours.

I expect that our discussion today will take about 30 minutes. Before I begin, do you have any questions?

Background Information

First, I would like to confirm that you are a teacher who works at [SCHOOL]?
What is your current job title here at the school?
How long have you worked in this position at this school?

1. On a scale of 0–5, where 0 is not important and 5 is extremely important, how important do you think eating more fruits and vegetables should be for preschool children and their families? Why do you think this?

Exposure and Satisfaction with Intervention Classes Targeted to the Children

Now I would like to ask you about your experience with the classes that the LEAP2 program nutrition educator held here for the children in your classrooms.

2. How many of the LEAP2 lessons taught by the educator were you able to observe in the classroom? [FOR RESPONDENTS WHO ANSWER FEWER THAN THE MAXIMUM NUMBER OF CLASSES OFFERED] What would have made it easier for you to come to these classes (e.g., scheduling issues, length of class, language barriers)?

3. What do you think worked well in the LEAP2 nutrition lessons?

4. What changes or improvements, if any, would you suggest to the LEAP2 nutrition lessons? Why do you think this?

Feedback on and Teacher Use of Take-Home Materials (e.g., parent pages/informational materials and recipes)

5. Considering your available time, how much have you been able to review the LEAP2 take-home materials designed for parents or guardians of the children in your classroom?

□ Not looked over or read at all

□ Glanced at materials

□ Browsed through most materials

□ Read thoroughly

6. What do you think were the most helpful aspects of these take-home materials?

7. What changes or improvements, if any, would you suggest to the take-home materials?

8. Did you incorporate any nutrition messages, sample activities or tools from the LEAP2 program in your classroom?

[IF NO, SKIP TO QUESTION 9(b)]

[IF YES] How did you incorporate these messages in your classroom?

9. How often would you estimate you use the new information you received from the LEAP2 program in your classroom?

□ A couple of times

□ Once every week

□ A few times a week

□ More than a few times a week

(a) What aspects of the LEAP2 content or design of the messages, sample activities or tools made it easier for you to incorporate these into your classroom activities? (e.g., ease of use in the classroom setting, cultural sensitivity, age appropriateness of the materials for the target audience)

(b) What aspects prevented you from using these tools in your classroom (e.g., lack of time, lack of money for supplies, lack of confidence)?

10. Do you think the storybooks used in the LEAP2 program apply the goals of the LEAP2 program? Please Explain.

Feedback on the Program Overall and Other Ways to Meet Program’s Nutrition Objectives

11. What recommendations or suggestions do you have for ways that the LEAP2 Program could be improved?

12. What changes or improvements would you suggest to better reach the parents with the messages of the LEAP2 Program?

13. [ADDITIONAL QUESTION NOT ON MAIL TEACHER SURVEY] Do you have any other suggestions for how schools like yours can encourage preschool children to eat more fruits and vegetables at home and encourage their parents to serve more fruits and vegetables? If so, what are they?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. We have a gift card to thank you for your time.

SNAP-Ed WAVE II: Discussion Guide for Demonstration Project:

MSUE Regional/Area-Level Extension Staff

[Pre-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of the Eat Smart, Live Strong program that is offering information to older adults about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education (SNAP-Ed) models around the country and to provide recommendations for how these interventions could be improved to better serve the older adults/children and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Today we will specifically be discussing the planning process and your expectations for the intervention. Once it has been implemented, we will follow up with you to find out whether the intervention met your expectations and how it might be improved. I expect that this interview will take about 40 minutes. Thank you for taking the time to speak with me.

Before I begin, do you have any questions?

Can you please describe what your role will be in the implementation and evaluation of the Eat Smart, Live Strong program, specifically your role as bridge between State demonstration project staff, the direct educators, and the senior centers that will be implementing the program in your area?

What challenges, if any, have you faced during the planning (including center recruitment) phase of this program? What factors do you feel have contributed most to a successful design and planning phase (e.g., using education materials that were already developed, good communication between contributors, knowledgeable staff, establishment of strong partnerships)?

3. What lessons have you learned during this key phase of program planning?

(a) What would you do differently? Why?

(b) What would you do the same? Why?

Okay, now I would like to shift our focus to the upcoming implementation of your SNAP-Ed project.

4. Now that you are ready to transition from the planning phase of your project to the implementation phase, what challenges, if any, are you anticipating? Why? How do you think you will address these challenges?

5. Do you feel that the environment in which the intervention will take place will be able to support the intended change in behavior, knowledge, and/or attitudes? For example, do you have any sense of the senior centers’ buy-in and/or enthusiasm about the intervention and what impact this might have on the older adults? What, if any, other nutrition education messages have the older adults in the intervention sites recently being exposed to (that you are aware of)?

6. Did the program have any difficulty recruiting adequate staff for the nutrition education delivery? If so, what were the recruitment challenges/problems?

7. Can you describe how you will manage and supervise the 16 direct educators that will deliver the Eat Smart, Live Strong program? [Collect copies of any forms to be used.]

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. As I mentioned, we will follow up and talk with you after the intervention and evaluation period are over.

SNAP-Ed WAVE II: Discussion Guide for Demonstration Project:

MSUE Regional/Area-Level Extension Staff

[Post-implementation]

State:
Respondent/Title/Organization:
Address:
Phone:
Fax:
E-mail:
Interviewer:
Date of Interview:
Time of Interview:

Thank you for taking the time for this interview. The U.S. Department of Agriculture’s Food and Nutrition Service has contracted with Altarum Institute to conduct a study of the Eat Smart, Live Strong Program that is offering information to older adults about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research and consulting institute and our work focuses on helping improve the health and nutrition status of children, families, and adults. The purpose of the study is to evaluate several Supplemental Nutrition Assistance Program-Education (SNAP-Ed) models around the country and to provide recommendations for how these interventions could be improved to better serve the older adult and families in your community. We also will be evaluating how the intervention might be replicated in other communities.

Today we will specifically be discussing your experience with the intervention. I expect that this interview will take about 40 minutes. Thank you for taking the time to speak with me. Before I begin, do you have any questions?

First I would like to ask you a few questions about the training you and your extension staff received from the State for the implementation of Eat Smart, Live Strong.

How well, on a scale of 1–10, would you say the staff training presentations and materials were to help ensure standardized and effective implementation of the Eat Smart, Live Strong program for this demonstration project?

What do you think was most effective about the training:

To help in recruiting older adults at the targeted centers for the program?
To help in retaining program participants once they started?
To help in assure educators could implement the program in a standardized and effective manner?

How would you suggest the training of extension staff might be improved in the future in Michigan or in other States that might want to implement Eat Smart, Live Strong as a SNAP-Ed program with older adults?

Now, I would like to ask you a few questions about the senior centers where the demonstration program took place and your role as a bridge between the State agency and the centers.

For each of the centers where the Eat Smart, Live Strong program was implemented in your region/area, overall how well do you feel that the environment in which the intervention took place was able to support the intended change in behavior, knowledge, and/or attitudes?

What was the [NAME OF CENTER]’s level of buy-in and/or enthusiasm about the intervention and what impact did this have on the older adults?
How well do you think your role as a bridge between State demonstration project staff, the direct educators, and the centers worked?

What aspects were key to the program’s successful implementation?
What recommendations would you have to modifying your role and the role of other area-level extension staff to help the program implementation run successfully in Michigan and in other States that may implement the Eat Smart, Live Strong program?

I have a few questions now about the staffing of the program and your role in overseeing the direct educator’s in implementing this demonstration program.

Did the program have any difficulty retaining adequate staff for the nutrition education delivery? If so, what were the recruitment challenges/problems?
Did you find the level of staff, both in terms of qualifications and total number of staff (and types of staff), adequate for optimally delivering the nutrition education intervention? If not, what would you recommend in the future in terms of level of staff for implementing Eat Smart, Live Strong?
How effective do you think the Extension staff were in delivering the Eat Smart, Live Strong lessons and the program’s intended education messages?

(a) Why do you think these staff were effective/ineffective?

(b) What could they have done differently to improve their effectiveness?

What changes, if any, were made to planned key staff involvement, and what were the reasons for any such changes?
Please describe any quality control and monitoring efforts that you were involved in during implementation of the Eat Strong, Live Smart demonstration project (e.g., nutrition education delivery, nutrition education data collection)?

Lessons Learned for Improvement and Replicability

Next I’d like to talk about what you see as the key successes and challenges in implementation of this curriculum and program and the lessons that can be learned if this program is continued in Michigan or implemented in other states.

Overall, what factors were key to the successful implementation of this nutrition education program?
What factors hindered or limited the successful implementation of this nutrition education program?
What factors contributed to this program’s potential to improve fruit and vegetable consumption and physical activity levels of older adults who participated?
What factors limited this program’s potential to improve fruit and vegetable consumption and physical activity levels of older adults who participated?
Looking back over the past __ months, what lessons have you learned? What would be most valuable for another State or demonstration project to know if they were considering using this model?
In your opinion, are there any aspects of this SNAP-Ed program that would make it difficult to implement on a larger scale with either more classes per center or at more centers statewide?
What aspects of the program (e.g., the curriculum, the training, or implementation) should be addressed if it was to be implemented on a larger scale?

That ends my formal interview questions. Do you have any comments or recommendations that you would like to add?

Thank you very much for your time and input on this very important project. As I mentioned, we will follow up and talk with you after the intervention and evaluation period are over.

SNAP-Ed Wave II: GROUP Discussion Guide for Eat Smart, Live Strong Participants

[Post-implementation ONLY]

Date of Discussion:
Location:
Facilitator:
Number of Participants:
Start Time:
End Time:

Welcome! My name is _____. I am here with my co-worker _____. Thank you for taking the time for this group discussion. The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) has contracted with Altarum Institute to conduct a study of the Eat Smart, Live Strong program that is offering information to older adults about healthy foods to eat and the importance of being active. Altarum is a health and nutrition policy research consulting institute, and our work focuses on helping improve the health and nutrition status of children, families, and adults.

This study will provide information on how the program in which you participated works from the perspective of: the people who planned the program, the instructors, and yourself. The purpose of today’s group is to hear from you about your experiences and satisfaction with this program that recently took place at _____. We also will use what you tell us today provide recommendations for how Eat Smart, Live Strong can be improved to better serve older adults like you in your community and those in other communities like yours.

Before we begin, I would like to review a few details about our discussion:

Your participation in today’s discussion is voluntary. You are free to leave at any time.
There are no right or wrong answers. Remember that we don’t work for [NAME OF HOSTING ORGANIZATION] or with the instructors, so please feel free to say whatever you think.
It is okay to have ideas or opinions that are different from each other. We want to hear everyone’s point of view.
It would be helpful to have only one person talking at a time. We are tape recording this session so that we don’t miss anything important. If two people talk at once, we can’t understand what anyone is saying. We may remind you of this during the group discussion.

We would like everyone to participate. But you each don’t have to answer every question. You don’t have to raise your hand either. If, however, some of you are shy or we really want to know what you think about a particular question, we may ask you what you think.
We have a lot to talk about today, so don’t be surprised if at some point we interrupt the discussion and move to another topic. But don’t let us cut you off. If there is something important you want to say, let us know and you can add your thoughts before we change subjects.
Finally, we just want to emphasize what we said earlier: we will be using first names only. Everything you say is private. What you say today will not be attached to your name at any point. Nothing that you say will affect the services you receive at this site or any other services you receive from this or any other program.

The group will last no more than 2 hours. You will not get out any later than _______. We will not be taking a formal break, but if you need to leave for a restroom break, the bathrooms are _____________. And feel free to get snacks.

For this session, I will read a question and then listen to your responses. I also may ask follow up questions to get some more detail.

Let’s get started! I can’t wait to hear what you think of the Eat Smart, Live Strong program! Do you have any questions before we begin?

Introductions/Icebreaker

Let’s go around the room for this one: Please introduce yourself. Tell us about your favorite activity or food. [MODERATOR NOTE: It is helpful to go in order of seating to allow the transcriptionist to label responses by person. Also, for note taking, you can then label Person 1, Person 2, Person 3, etc. when writing comments.]

Exposure and Accessibility of Supplemental Nutrition Assistance Program-Education II Intervention for Parents/Caregivers

First I want to hear about how you heard about the program? We really want to know people find out about Eat Smart, Live Strong in your community.

Where and from who did you hear about the Eat Smart, Live strong classes that were offered at [INTERVENTION SITE]?

What did you like about the way you received information about the program?
What didn’t you like about the way you received information about the program?

Why did you decide to sign up for the Eat Smart, Live Strong classes?
Please think for a moment about what could be done to encourage more older adults like you to participate in these classes. I will hand out a pencil and paper if you want to write down your ideas before you answer out loud.

[AFTER ABOUT 2 MINUTES, TAKE ANSWERS VIA ROUND-ROBIN QUESTIONING.]

Satisfaction/Likes and Dislikes With Intervention

Did you see any of the logs and tip sheets provided for you by the Eat Smart, Live Strong program? [THE MODERATOR SHOULD PROMPT RESPONSE BY SHOWING SOME SAMPLE MATERIALS USED IN THE INTERVENTION.]
What were the most helpful aspects of these materials? Why?

PROBE: What did you like about the materials?

What were the least helpful aspects of these take home materials? Why?

PROBE: What didn’t you like about the materials?

Do you think the educator who led the classes provided information in a way that was easy for older adults in the class to understand?
Would you say that the educator who led the classes was a good instructor for you?

If yes, what made him/her a good instructor? Why?
If not a good instructor, why not?

Now I would like to ask specifically about the classes you attended.

How many of the Eat Smart, Live Strong classes were you able to attend?

If you did not attend them all, can you explain why?
If you didn’t attend them all, what would have helped you be able to attend them all?

Tell me about the parts of the program overall—including logs, other written materials, and class activities—that you liked best and why you liked these parts.
What parts of the overall program you liked least and why?
Of each of the four classes, which was your favorite class and why?
Do you think length or number of minutes of classes was just right, too long, or too short?

Perceptions of Goals and Relevancy of Intervention

We are interested in hearing more about what you thought about the purpose of the classes, whether they helped you and provided useful information to you.

What do you think the Eat Smart, Live Strong classes were trying to teach you?
How useful was the information the program offered for older adults like you?
How well did the program suggestions and information fit with the ways that people of your racial or ethnic background live your life?
How well did the program suggestions and information fit with the challenges faced by people who do not have a lot of money?

Intervention Impacts

These next few questions are about how you think Eat Smart, Live Strong classes and materials may have helped you learn new information or other ways it may have changed things for you.

What are the most important things that you learned from this program?
Now I would like to ask you a question that you probably need more time to think about: What are the most significant change or changes that have taken place (for you) because of this program?

I am passing out pieces of paper again if you want to write down your response.

[AFTER ABOUT 2 MINUTES, TAKE ANSWERS VIA ROUND-ROBIN QUESTIONING.]

OPTIONAL PROBES AS NEEDED:

Changes in food you buy?
Changes in the food you select when you are not eating at home?
Changes in your physical activity?

Factors Affecting Fruit and Vegetable Availability at Home and Ways of Addressing These Barriers

Now I would like to take a few moments to ask you about the difficulties that older adults who live in your neighborhood might face in trying to buy, store, and prepare fruits and vegetables for yourself.

What makes it harder for you or other older adults like you to buy and keep fruits and vegetables where you live (e.g., have meals cooked for at center, cost, access, storage)?
What makes it harder for you or other older adults like you to prepare and eat fruits and vegetables (e.g., have meals cooked for at center and fruits and vegetables are not appealing or available there, cost, access, storage)?
Did the information or materials provided to you by Eat Smart, Live Strong help you to address any of these difficulties or barriers to increasing the amount of fruits and vegetables you eat daily?

(a) For those who said yes, how were the information or materials helpful?

(b) For those who said no, what could have been done to make the information or materials more helpful for older adults like you?

Recommendations

Would you recommend this program to friends? Why or why not?
Is there anything we haven’t asked that you would like to tell us about your experience with and opinions of the Eat Smart, Live Strong program?
Before we close, I would like you to help us by giving us your ideas for other ways that senior centers could encourage older adults to eat more fruits and vegetables and encourage senior centers to serve fruits and vegetables more often.

Thank you very much for participating in this discussion group today. We have learned a lot from your experiences and recommendations.

In appreciation of your time and trouble today, we have gift cards for each of you today. Before you leave, please take one of these and sign the form indicating you have received one of these cards. Enjoy your day.

Secondary Data Collection Instruments

All Sites

SNAP-Ed Wave II: Data Abstraction Form for Demonstration Project-Led Evaluation [Pre-implementation]

SNAP-Ed Wave II: Data Abstraction Form for Demonstration Project Design and Implementation Plans [Pre-implementation]

SNAP-Ed Wave II: Project Resource and Expense Tracking Form for Program Administrator [Post-Implementation]
Questionnaire on Nutrition and Physical Activity Education

SNAP-Ed Wave II: Data Abstraction Form for Demonstration Project-Led Evaluation

[Pre-implementation]

DEMONSTRATION PROJECT NAME: ________________________________________________________________

State: _______________________________________________________________________________________

Data abstractor: _______________________________________________________________________________

Date of abstraction: ____________________________________________________________________________

Resources used for data abstraction:

__ DEMONSTRATION PROJECT Evaluation Plan (if available)

__ DEMONSTRATION PROJECT Evaluation Instruments

__ DEMONSTRATION PROJECT SNAP-Ed Plan

__ Other Information provided by DEMONSTRATION PROJECT Evaluation Staff

What is the research design for the impact evaluation (e.g., pre/post- survey design, only post-intervention data collection, quasi-experimental with control group)?

What is the rationale for this evaluation approach and/or what research is available to support the use of such methods?

What are the key measures or indicators used to assess the intervention’s impact?

What information is provided on reliability (internal consistency (alpha), test-retest reliability, and/or reliability across raters) and construct validity of measures?

Are the measures used scales or single-item measures?

Are pre-existing evaluation instruments being used or modified or are new instruments being developed?

Have the evaluation instruments ever been tested? If so, describe testing conducted on the instruments.

Were the same tools used during the pilot phase (if applicable)? If yes, were modifications made for any reason and what were they?

Who will be evaluated (e.g., children, parents, older adults, and/or teachers)?

What are the sampling techniques and sample size of the population being assessed?

Was a power analysis conducted? If so, provide the details including whether it indicates the sample is sufficient to detect statistically significant differences in outcomes between treatment and control/comparison groups, and whether published literature or pilot work were used as the basis for an anticipated program effect size.

What data collection techniques will be used and what is the planned timing for these techniques?

When will the pre- and post-intervention surveys be administered (e.g., will data be collected 1 week, 2 weeks, or more before intervention implementation)?

What methods are planned for increasing the likelihood that members of the target population will agree to participate in the data collection?

What follow-up techniques are planned for ensuring adequate pre- and post-survey response rates? E.g., how many times will staff [indicate which staff] prompt participants to return the surveys? What is the planned procedure for doing so?

What process measures and data collection methods, if any are being used to assess intervention effectiveness? Describe data collection methods.

What methods are planned for ensuring confidentiality of the participant responses?

What training will the data collectors be required to have before beginning data collection (e.g., classroom education and/or format and content of training by intervention evaluation staff)?

Will any quality control or monitoring take place during data collection? If so, please describe.

Describe the staffing plan for the evaluation. Which project staff or other staff will be responsible for designing and conducting your evaluation this year?

Indicate staff, the amount of time allotted, and the tasks in which they will be involved.

Staff Name	Activities in Which They Will Be Involved (Y/N)
Staff Name	Data Collection	Data Analysis	Project Implementation

SNAP-Ed Wave II: Data Abstraction Form for

Demonstration Project Design and Implementation Plans

(Primary secondary sources: demonstration project application to FNS and 2012 SNAP-Ed Plans)

[Pre-implementation]

DEMONSTRATION PROJECT:
State:

Data abstractor:
Date of abstraction:
Resources used:

TOPIC AREA 1: Formative Research and Intervention Design

Target audience(s)

Reach or intended size of intervention

Description of nutrition education intervention

Describe the overall intervention goal(s).

Describe the key education methods that are being used in the nutrition education intervention, including how this may vary for different target audiences (e.g., children versus their caregivers).

Describe each nutrition education lesson in detail using the following format. [Please copy and paste as many copies of this table as you need to capture all nutrition education messages and number them accordingly).

Lesson #1

Short title:
Detailed description of education message(s):
Specific objectives:
Intended impact/change
Materials supporting lesson

Lesson #2

Short title:
Detailed description of education message:
Specific objectives:
Intended impact/change
Materials supporting lesson

Lesson #3

Short title:
Detailed description of education message:
Specific objectives:
Intended impact/change
Materials supporting lesson

Lesson #4

Short title:
Detailed description of education message:
Specific objectives:
Intended impact/change
Materials supporting lesson

List and describe other key components of the nutrition education intervention that supports or reinforces its objectives.

Anticipated dose and intensity of each nutrition education intervention method

___ A. Direct education

Dose (# of contacts with each participant)
Intensity (# of contacts X length of contact)

___ B. Indirect education

Dose (# of contacts with each participant)
Intensity (# of contacts X length of contact)

____ C. Social marketing

Dose (# of contacts with each participant)
Intensity (# of contacts X length of contact)

____ D. Other

Dose (# of contacts with each participant)
Intensity (# of contacts X length of contact)

Nutrition education materials

____ A. Materials developed by the U.S. Department of Agriculture’s Food and Nutrition Service (FNS)

Please list the title, the source, and how to locate the source.

If FNS materials were modified, how and why?

____ B. Materials developed by other State Supplemental Nutrition Assistance Program-Education programs

Please list the title, the source, and how to locate the source.

If the existing materials were modified, how and why?

____ C. Materials developed by other public nutrition educations programs

Please list the title, the source, and how to locate the source.

If the existing materials were modified, how and why?

____ D. Materials developed by private agencies

Please list the title, the source, and how to locate the source.

If the existing materials were modified, how and why?

____ E. Materials developed by project

Please list the title, the source, and how to locate the source.

What was the justification for the development of these materials?

____ F. Other

Please list the title, the source, and how to locate the source.

Theoretical underpinning for nutrition education

Evidence that suggests that the intervention will be successful (e.g., pilot project results, previously tested instruments)

Key players in the design of the intervention

Who were the key players from the implementing agency?
Were there any partnerships with other public or private organizations that were key to the design and implementation plan of the intervention?
If so, how were these partnerships formed?

TOPIC AREA 2: Operational Steps Involved in Intervention Implementation
Other key players?

Management and oversight structure

Who are the program administrators and coordinators?
Who is responsible for quality control and monitoring the nutrition education delivery?

2-2. Nutrition education provider(s)

How many direct educators will be involved in the intervention?
What are their qualifications?

-Level of education

-Specialized training

-Years of experience delivering nutrition or health education

-Experience working with this target population

Plans for training of nutrition education providers (Describe the frequency and duration of training, training agenda and method, etc.)

Who will do the direct training?
What topics will be covered in the training
What is the training outline/agenda?
What format will the training be conducted
Qualifications of trainer(s):

-Level of education

-Specialized education

-Years of experience in nutrition or health education

-Experience working with this target population

Recruitment of intervention sites/participants

How were individual intervention sites selected to participate in the intervention (specifically for this FNS evaluation component)?
How will individual classrooms be selected to participate in the intervention [for the INN (INN) and UKCES only]?

c. How will teachers be engaged in direct training, if at all [for INN and UKCES only]?

SNAP-Ed Wave II: Project Resource and Expense Tracking Form for Program Administrator

[Post-implementation]

This data collection form will be used to summarize information about actual resources used for and expenses related to your SNAP-Ed WAVE II intervention. In Section 1, we are requesting information that is specific to the planning and design of your project. In Section 2, we are requesting cost related data specific to the implementation of your project. In Section 3, we are requesting information that is specific only to the evaluation (Demonstration Project-led assessment) component of your intervention.

SECTION 1. Planning and design

In the following tables, please provide the requested information as it relates to the planning and design of your project. Please do not include resources or expenses related to the implementation or evaluation of your project.

Summarize staff costs (human capital) for the planning and design of your SNAP-Ed WAVE II intervention.

At the administrative, coordination, oversight, and trainer levels

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

At the nutrition educator level (per intervention site), if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

IT/technical staff, if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Other

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Please provide the following information for ACTUAL expenditures related to the planning and design of your SNAP-Ed WAVE II intervention only (NOT FOR IIMPLEMENTATION OR EVALUATION).

Expenses	(a) Non-Federal Public Funds		(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Expenses	Cash	In-Kind Donations	(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Salary/benefits
Contracts/grants agreements
Noncapital equipment/ supplies
Materials
Travel
Administrative
Building/space
Maintenance
Equipment and other capital expenditures
TOTAL Direct Costs
Indirect costs
TOTAL Costs

SECTION 2. Implementation

In the following tables, please provide the requested information as it relates to the implementation of your project. Please do not include resources or expenses related to your planning and design or evaluation.

Summarize staff costs (human capital) for the implementation of your SNAP-Ed WAVE II project.

At the administrative, coordination, oversight level, and trainer levels

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

At the nutrition educator level (per intervention site), if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

IT/technical staff, if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Other

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Describe the actual costs other than staff costs (physical capital) required to implement project.

Space
Audiovisual
Computer/software
Other

Please provide the following information for actual expenditures related to the implementation of your SNAP-Ed WAVE II intervention only (NOT FOR EVALUATION).

Expenses	(a) Non-Federal Public Funds		(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Expenses	Cash	In-Kind Donations	(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Salary/benefits
Contracts/grants agreements
Noncapital equipment/ supplies
Materials
Travel
Administrative
Building/space
Maintenance
Equipment and other capital expenditures
TOTAL Direct Costs
Indirect costs
TOTAL Costs

SECTION 3. Evaluation

In the following tables, please provide the requested information as it relates to the evaluation of your SNAP-Ed WAVE II project.

Summarize actual staff costs (human capital) used for your evaluation.

At the administrative, coordination, and oversight levels

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

At the evaluator level, if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

IT/technical staff, if applicable

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Other

Title of position	Brief description of responsibilities	FTEs	Average salary for this position	Salary range for this position

Describe the actual physical capital required to evaluate this project.

Space
Audiovisual
Computer/software
Other

Please provide the following information for actual expenditures related to the evaluation of your SNAP-Ed WAVE II intervention only (NOT FOR IMPLEMENTATION).

Expenses	(a) Non-Federal Public Funds		(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Expenses	Cash	In-Kind Donations	(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Salary/benefits
Contracts/grants agreements
Noncapital equipment/ supplies
Materials
Travel
Administrative
Building/space
Maintenance
Equipment and other capital expenditures
TOTAL Direct Costs
Indirect costs
TOTAL Costs

SECTION 4. Total Expenditures

In the following table, please provide the requested information as it relates to the TOTAL cost of your SNAP-Ed WAVE II project.

Provide the total expenditures for the SNAP-Ed WAVE II project (sum of 1.2, 2.3, and 3.3).

Expenses	(a) Non-Federal Public Funds		(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Expenses	Cash	In-Kind Donations	(b) Non-Federal, Non-cash	(c) Total Non-Federal Funds (a+b)	(d) Federal Funds	Total Funds (c+d)
Salary/benefits
Contracts/grants agreements
Noncapital equipment/ supplies
Materials
Travel
Administrative
Building/space
Maintenance
Equipment and other capital expenditures
TOTAL Direct Costs
Indirect costs
TOTAL Costs

[CUSTOMIZE NAME OF INTERVENTION]

Questionnaire on Nutrition and Physical Activity Education

We are interested in learning what activities have been or are currently being conducted in your [senior center or school] to educate [older adults/children, parents and/or staff] about nutrition and physical activity.

Name of [Senior Center or School]: _______________________________________________________________________________

Name of Person Completing Survey: _______________________________________________________________________________

Date Survey Completed: _______________________________________________________________________________

During the past 12 months, have you worked with any agency/organization (e.g., County Health Department, Cooperative Extension, Dairy Council, Community Organization, U.S. Department of Agriculture Fresh Fruit and Vegetable Program) to implement any nutrition education activities for your students or seniors?

_____ Yes _____ No

If yes, please complete the following chart:

Organization/Program	Description of Nutrition Education	Dates

During the past 12 months, have you provided nutrition education to [older adults/ children or parents]?

_____ Yes _____ No

If so, please identify in the table below which curricula was used. Describe how the curriculum was used, who taught the classes, and the type of audience reached. If you have used any curriculum that is not listed in the chart,

complete the “Other” row listing the curriculum name, how you used it, who taught the classes and what audience was reached.

Nutrition Education Curriculum Used at Your [Senior Center/School] With the [Older Adults/Children]

NAME OF CURRICULUM	During what time period did you use this curriculum?	What type of organization and staff teach/taught the classes?	What audience is/was reached (e.g., note ages of persons’ reached, type of staff reached, and/or whether parents were directly reached with the education)?

* From [DATES], we would like you to refrain from providing any direct nutrition education to the [older adults/children and parents of the children in the classrooms] where the [NAME OF INTERVENTION] will take place. Can you do this?

_______Yes _______ No

Thank you!

Please fax or e-mail back the completed form to [STAFF CONTACT INFORMATION].

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Anna Pierson
File Modified	0000-00-00
File Created	2021-01-31

Process Evaluation

Eval of SNAP Nutrition Edu Practices Study - Wave II

APPENDIX B. Data Collection Instruments for Process Evalua SLO 3.1.11