Team Nutrition Cooks - State, Local, and Tribal Government

Attachment F – Discussion Guide for In-Depth Interviews with Program Operators

OMB BURDEN STATEMENT: According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-0524. The time required to complete this information collection is estimated to average 30 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information.

Research Objective: Conduct in-depth interviews (IDI) with Team Nutrition Cooks! program operators to identify implementation barriers and facilitators (Section II), assess usability and design of the program materials (Section III), and gather feedback on potential indicators of educational effectiveness for the target audience (Section IV).

NOTES TO REVIEWER:

This discussion guide is not a script and, therefore, will not be read verbatim. The moderator will use these questions as a roadmap and probe as needed to maintain the natural flow of conversation. Question probes are italicized.

Interviewer instructions are highlighted in yellow.

Materials are highlighted in blue.

Concepts of interest are highlighted in green.

Session Overview: Total time – 30 minutes

SECTION I: Introduction and Warmup (3 min.)

The interviewer will explain the purpose of the research, present the ground rules, and allow the participant to ask any questions.

SECTION II: Program Implementation (10 min.)

The purpose of this section is to find out how participants implemented the program, the challenges they faced (if any), what worked well, and what could be improved.

SECTION III: Program Materials (10 min.)

The purpose of this section is to gauge participants’ reactions to the content, design, and usability of the program materials (activity guides, videos, and family handouts).

SECTION IV: Program Impact (5 min.)

The purpose of this section is to gather observations of any potential change in students’ knowledge, attitudes, or behaviors related to food or cooking and gauge perceptions around the program’s potential for impact.

SECTION V: Conclusion (2 min.)

The interviewer wraps up discussion and ensures that all questions are answered and all comments have been heard.

S ECTION I: Introduction and Warmup (3 min.)

[Interviewer introduces self and reviews ground rules with the participant]

• Thank you for meeting with me today and for your willingness to be involved in our study. The purpose of today’s interview is to get your thoughts and reactions to the Team Nutrition Cooks! (TNC) program.

• My name is __________ and I work for [company name]. We are a private research firm, and we are helping the U.S. Department of Agriculture (USDA)/Food and Nutrition Service (FNS) test and improve the Team Nutrition Cooks! program.

Before we begin, I want to say a couple of things:

• Your participation is voluntary and you have the right to withdraw from the study at any time. If I ask any questions you do not wish to answer, you do not have to provide a response.

• There are no wrong answers here—we just want to know what you think. I did not make any of the Team Nutrition Cooks! materials, so it is OK for you to be totally honest. Your responses will not hurt my feelings.

• We are also interested in learning about how you implemented the program and any challenges you faced. However, we are not here to evaluate you or make any judgments about your program. Your feedback and experiences will only help us improve the Team Nutrition Cooks! program.

• If it’s OK with you, I’d like to audio-record our conversation. The recordings will only be used to confirm our notes and allow us to revisit this conversation. Additional project staff may hear the tapes at a later date. However, your name and personal information will be removed from any transcripts and will not be used in any of our reports. May I start recording now?

• We will have about 30 minutes for our discussion.

• Please turn your cell phone off or switch to silent mode.

• Do you have any questions before we begin?

Okay, great! Why don’t we start with you telling me a little bit about yourself?

• How long have you been working at [name of program]?

• What brought you here? –OR– What inspired you to start working with children?

Thank you. I am pleased to meet you and I appreciate your participation!

S ECTION II: Program Implementation (10 min.)

To get us started I would like to hear a little about how you and your team implemented the Team Nutrition Cooks! program.

1. Can you walk me through the process of how you prepared for and then delivered the program?

   1. How did you prep for the lessons? [probe on shopping and any challenges faced during prep]

   2. How much time was spent teaching each lesson?

      1. [Probe on number of hours or if they had to spread over more than one day]

   3. How many students and staff participated?

      1. How was that ratio? What would be ideal?

   4. Was there a particular day or time dedicated to Team Nutrition Cooks!?

      1. How did it integrate with your other lessons or activities?

   5. Where did you deliver the lessons?

      1. [If not a full kitchen] – Why was that location selected?

         1. [probe for any specific utilities, resources, or equipment that were needed in the facility]

   6. How were the students involved in the lesson?

      1. [probe on whether some students only did parts of the recipe or process or if they worked as groups]

      2. Which parts of the process did the children enjoy the most? ...and the least?

2. How did you use the Team Nutrition Cooks! resources in the delivery of the lesson? (e.g., activity guide, skills video)

   1. [Probe for timing of use] – When did you first read the activity guide?

      1. Did you try out the recipe at home?

      2. Did you use the guide while teaching the lesson?

      3. When did you use the skills video?

   2. What was the most helpful material in delivering the lessons?

      1. [probe for what resource—video or guides—they used the most and why]

   3. How did you customize the lesson for your students?

      1. [probe for specific examples/anecdotes]

      2. Did you supplement with other resources or materials?

3. Overall, what challenges did you face in implementing the Team Nutrition Cooks! program at your site? [probe for structural or logistical barriers, and site- or community-specific issues]

   1. How were these challenges overcome or how did you adapt?

   2. What could be changed to make the program more flexible or easier to implement?

OK. Before we jump into specifics about the content and design of the materials, I’d like to hear some general feedback about the TNC lessons.

Set out all 5 Activity Guides for participant to view (materials found in Attachment M: Stimuli for IDIs – Team Nutrition Cooks! Activity Guides)

4. In your opinion, which lessons/recipes were most successful? And why?

   1. [If not offered by the participant probe on the following areas]:

      1. Equipment/tool access, safety, and ease of use

      2. Taste of ingredients or recipe

      3. Skill or recipe complexity

      4. Ingredient access or ease of use

      5. Fun factor

      6. Repeatability or memorability

5. Were there any lessons/recipes that were not successful? Which ones and why?

[probe on sub-items listed above in #4 as needed]

Great! Thank you for being open to share your insights and experiences.

S ECTION III: Program Materials (10 min.)

Now, I’d like to dive into some specifics about the design and format of the program materials—specifically, the activity guides and skills videos. I would like us to focus on just one lesson and walk through it section by section.

[Interviewer uses pages 1-26 of the Applesauce Activity Guide (found in Attachment M: Stimuli for IDIs – Team Nutrition Cooks! Activity Guides) to facilitate discussion.]

Just as a reminder, we are using the [Applesauce recipe] as an example. Since the activity guides are very similar in how they are designed, I’d like you to think broadly and share your thoughts about format of the materials in general, not just this one.

1. What did you like about the activity guides? [Probe for reasons why.]

   1. What would you change about them?

2. What did you not like about the activity guides? [Probe for reasons why.]

3. [Usability] What specific parts or sections of the activity guide were most useful? And why?

   1. Were the activity guides easy to read?

      • [If used during the lesson] – What would make them easier to use while teaching?

   2. Were the instructions easy to follow? Do you like the tone or how things are explained?

      • Was anything missing from the instructions?

   3. Were there any words that were unclear or confusing?

      • Any words that might be helpful to have pronunciations for? (e.g., quinoa, vinaigrette)

   4. Anything else that would make the activity guide easier to use?

      • Would it be helpful to have all of the lessons bound together into a single program guide?

4. [Appeal] Did you find the materials to be visually appealing? [Probe for reasons why/why not.]

   1. What visual elements did you like or dislike?

   2. How do you feel about the photos?

   3. Is there anything you would change about the design?

Now, I’d like to transition and talk a bit about the skills videos. From your recollection of them, please tell me…

5. What did you like about the skills videos? [Probe for reasons why.]

6. What did you dislike about the skills videos? [Probe for reasons why.]

7. [Usability] How helpful were the skills videos in delivering each lesson?

   1. Did you feel they were appropriate for children ages 8–12?

   2. Were the students able to understand the instructions from the video? [probe on pace, timing of actor delivery, visuals shown, etc.]

      1. Were there elements that could be better explained through text or visuals?

   3. What else would make the videos more useful for you?

8. [Appeal] What did you think about the actors or imagery used in the videos?

   1. What did the children/students think of the videos?

      1. Any specific reactions you can recall?

   2. Were the scenes relatable to you or your students?

   3. How did you feel about the length of the videos?

      1. Did the videos feel like fun (for you and the students)? What might make them more engaging?

S ECTION IV: Program Impact (5 min.)

Now, I’d like to spend our remaining time together talking about anything you observed about the students over the course of the program.

1. [KAB] What do you think the children took away from the Team Nutrition Cooks! program?

   1. [Probe on changes in attitudes, knowledge or behavior.]

      1. What do you think they learned? –or– What was new to them?

      2. Have you observed any changes in the way they talk or act related to food?

   2. [Capture Y/N responses to the following questions]

      1. Have they been more willing to try new foods?

      2. Have they expressed interest in cooking at home or trying recipes with their families?

      3. Have you heard or seen them applying information from the Team Nutrition Cooks! lessons (e.g., discussing what they learned when served their usual snacks)?

2. [Skill acquisition] Have you observed a change in the students’ confidence or abilities over the course of the program? If so, please explain.

   1. [Probe on comfort level in food preparation, cooking techniques or use of equipment]

3. Did you find the program to be engaging for the children? [Probe for why/why not.]

   1. Did it seem like they enjoyed eating the foods they had prepared?

   2. Did it seem like they enjoyed the cooking/preparation activities?

   3. Do you think your students would be able to replicate these recipes at home if the right tools and ingredients were made available to them? Why or why not?

To wrap up, I’d like to ask for three final thoughts from you…

• Would you want to use this program again in the future? Why or why not?

• Would you recommend Team Nutrition Cooks! to other schools or afterschool programs? Why or why not?

  • What kind of programs would you recommend it for?

  • What would they need to have in order to be successful? [Probe for potential barriers they would anticipate in other schools/programs—language, cost of supplies/ingredients, tools/equipment, facilities]

• Did implementing these lessons make you more interested or excited about healthy eating and cooking, either in your own life or in doing more of these types of activities with the children?

S ECTION V: Conclusion (2 min.)

Thank you very much for participating in this interview. I appreciate you sharing your time and valuable feedback. Is there anything that you would like to share that you didn’t have the chance to share yet?

10