State, Local and Tribal Agencies (Respondents)

OMB Control No: 0584-0606 Expiration Date: 03//31/2019 Expiration Date: 03/31/2019

Appendix H6. Menu Planning Survey and Menu Materials Pretest Protocols

INTERVIEWER: ENSURE YOU HAVE REVIEWED COMPLETED SURVEY QUESTIONNAIRE(S) PRIOR TO INTERVIEW AND HIGHLIGHTED ANY ISSUES YOU WANT TO FOLLOW-UP ON.

INTRODUCTION AND CONSENT SECTIONS NOT REQUIRED AS MENU TESTING WILL FORM PART OF THE SPONSOR AND SITE SUPERVISOR QUANITATIVE SURVEY QUESTIONANRIE TESTING.

{INTRODUCTION (5 mins)

Hello, my name is ______________ and I work for Westat, a survey research company in Rockville, Maryland. Thank you for taking the time to participate in this research study.

Westat is working with the Food and Nutrition Service, U.S. Department of Agriculture, to carry out a study about the Summer Meals Programs. Menu planning is of course an important aspect of the Summer Meals Program. Our study aims to find out more about the challenges experienced in planning and providing meals and snacks to the children who attend the summer program by asking those who sponsor or supervise summer meals sites to take part in a survey and provide information about the menus and recipes that are used.

It is important to try out the survey questions and other materials we are planning to use, with the help of people such as yourself, to ensure that the survey questions and what we are asking people do make sense.

Thank you for agreeing to take part in this study today, and for completing the survey.

Today we will talk about your experience completing the survey and how you answered the survey questions and also your understanding of the instructions to provide menu and recipe information.There are no right or wrong answers. Our purpose is not to compile information on you. Instead, your interview along with those of other site sponsors and supervisors will show us how to make improvements to the study materials.

INFORMED CONSENT

Before we get started, there are a few things I should mention.

• This is a research study, and your participation is voluntary. If you prefer not to answer any questions just say so and we’ll go on to the next one. It’s also okay if you change your mind after starting and would rather not participate.

• All your answers, everything you say, will be kept confidential. We will not use your name or other identifying information in any reports, though we may include quotes you provide in our reports. The thoughts and opinions that you share during our discussion will be used for research purposes only. We will combine your responses with those from other site supervisors. They will not be used as an evaluation of your work, your staff’s work or <PROGRAM / SITE>.

• There are no direct benefits to you or <PROGRAM / SITE> for participating in this study.

• This discussion will take about 60 minutes.

• We will also need to audio record our conversation. This helps me so I can listen to what you are saying and won’t have to take a lot of detailed notes while you are talking. It will also help when we write up a summary of this interview.

• Only project staff will have access to the recording and other project materials. These materials will be destroyed once we have completed the project

• (Finally, some of the project staff from {Westat} are {listening to/ here today observing} our discussion to learn if there are things that might need to be changed to improve the survey questionnaire.)

CONSENT FORM. The consent form you signed previously contains all of the things I just told you about your rights in this interview. Let me know if you have any questions.

[HAVE R SIGN TWO CONSENT FORMS, KEEP ONE AND RETURN ONE TO I’ER.]

TURN ON RECORDER. The date and time is ____________. Now that the recorder is running, let me ask again, is it okay with you if we record this interview? }

INTERVIEWER: Bold red indicates probes you can read aloud as written. Non-bold red indicates points you may need/want to probe on.

ADDITIONAL UPSCRIPTED PROBES. PROBE ON THE FOLLOWING OBSERVATIONS OR BEHAVIORS NOTED DURING THE INTERVIEW:

• PROBE ON ITEMS WHERE RESPONSES ARE LEFT BLANK.

• PROBE ON ITEMS WHERE R SEEMS UNCOMFORTABLE SHARING INFORMATION OR CONFUSED.

• FOR ANY NON-VERBAL REACTIONS, ASK: Tell me what you’re thinking here.

• IF R SELECTED MORE THAN ONE RESPONSE OPTION AT “CHECK ONE ONLY,” ASK FOR THE ONE OPTION THEY WOULD CHOOSE FROM THE LIST.

• PROBE ON “OTHER SPECIFY” IF RESPONSE ALREADY LISTED OR DESCRIPTION NOT CLEAR.

• NOTE IF R’S RESPONSES INDICATE FATIGUE (R CHECKED ONLY THE FIRST ITEMS IN A LONG LIST OR APPEARED TO BE CHECKING FEWER ITEMS IN LONG LISTS AS THE SURVEY PROGRESSED).

INTERVIEWER: Indicate if site supervisor or sponsor and name of site.

Site Supervisor: Yes/No

Site Sponsor: Yes/No

Before we get started, can you tell me if you are the sponsor or site supervisor for more than one site?

__Yes IF YES: How many programs or sites do you sponsor/supervise? ___

__No

{And may I just check whether you have a copy of the questionnaire and other information you provided in front of you? }

INTERVIEWER: SPONSOR/SITE SUPERVISOR HAS BEEN SENT THE MENU SURVEY IN ADVANCE TO COMPLETE. YOU WILL EMAIL OTHER DOCUMENTS OR SHARE SCREENS IMMEDIATELY PRIOIR TO COGNITIVE INTERVIEW.

MENU SURVEY INITIAL CONTACT LETTER (3MINS)

Lets start by taking a look at the letter we will send initially to those selected to complete the menu survey.

Please take a few moments to familiarize yourself with the letter.

What’s your overall impression of the letter?

What are you being asked to do?

What, if anything, is confusing or unclear?

IF NEEDED: Was it clear to you or not, how should you go about completing the survey?

What about how to return the menus and recipes?

Do you think you would take part in the study if you received this letter? What, if anything, would motivate you to respond?

MENU SURVEY FOLLOW-UP REPORT (5 MINS)

Now let’s look at the Menu Survey Follow-Up Report. This will be sent after site sponsors or supervisors provide their menus and recipes for the selected weeks.

Please take a few moments to familarize yourself with the letter and sample follow-up questions.

What’s your overall impression of the letter?

What are you being asked to do?

What, if anything, is confusing or unclear?

Was it clear to you that this is asking for information that was missing in the initial menu submission?

Were the questions under the “additional details” column clear?

Do you have any other comments about this letter?

INTERVIEWER: MAKE SURE YOU KNOW IF THE R UNDERSTANDS THAT THEY ARE BEING ASKED TO PROVIDE MISSING INFORMATION ONLY IN THE FOLLOW-UP REPORT.

Menu Planning Survey

Now I would like to move on to ask about your experiences with the survey questionnaire (Menu Planning Survey).

INSTRUCT RESPONDENT TO SKIP THE SURVEY INTRODUCTION.

Lets first consider questions 1 through 6 about providing the meals or snacks, the meal patterns provided, and the planning tools and resources you use.

GENERAL PROBES

Again, if you’d like, you can take a moment to re-familiarize yourself with question 1 through 6.

How did you find completing these questions?

Are there any questions that were unclear to you or that you are uncertain about? Which and why?

IF NEEDED: Were there any specific words or phrases that were unclear to you or that you were uncertain about? (What words or phrases, if any, were unclear to you?)

Were there any questions where the answer options were incomplete or did not allow you to answer fully about <SITE NAME>? NOTE WHERE “OTHER SPECIFY” WAS USED.

IF R SPONSORS/SUPERVISES MORE THAN ONE SITE: How certain were you about which program or site you were to report on? How did you decide which site to answer about?

Over what time were you thinking when you answered these questions? This summer or longer ago?

SPECIFIC PROBES MAY NOT BE ADMINISTERED DEPENDING ON TIME

SPECIFIC PROBES WHERE NEEDED / IF NOT MENTIONED IN RESPONSE TO PREVIOUS PROBES

Q2b: IF RESPONSE: How easy or difficult was it to answer Q2b? IF NEEDED: Tell me more about why it was easy/difficult.

Q3: What local or cultural practices did you consider when you read the answer options to this question?

Q4: Over what time period were you thinking of when you answered this question?

Were you thinking only of the resources you have used for <SITE NAME>?

Q5: IF YES: Tell me a bit more about what other tools and resources you need to plan summer meals?

Q6: In your own words, what is this question asking?

Q6a: How easy or difficult was it to answer Q6a? IF NEEDED: Tell me more about why it was easy/difficult.

What does the term “user-friendly” mean to you?

IF OTHER: Tell me a bit more about your other challenges?

Now lets consider questions 7 through 10 about the types of foods used to prepare summer meals and how often they are served.

GENERAL PROBES

Again, if you’d like, you can take a moment to re-familiarize yourself with question 7 through 10.

How did you find completing these questions?

Are there any questions that were unclear to you or that you are uncertain about? Which and why?

IF NEEDED: Were there any specific words or phrases that were unclear to you or that you were uncertain about? (What words or phrases, if any, were unclear to you?)

Were there any questions where the answer options were incomplete or did not allow you to answer fully about <SITE NAME>? NOTE WHERE “OTHER SPECIFY” WAS USED.

Over what time were you thinking when you answered these questions? This summer or longer ago?

Q7 and Q7a: Was the definition of “local foods” clear to you, or not?

How easy or difficult was it to answer Q7 and Q7a? IF NEEDED: Tell me more about why it was easy/difficult.

Q9: Are the terms “USDA Foods” and “Commodity Foods” ones that you are familiar with or not? IF NEEDED: What do the terms mean to you? Are they the same or different?

Q9a: Are there any USDA foods that are missing from the list?

Q10: IF SIGNIFICANT CHALLENGE OR CHALLENGE: Tell me more about how THE CHALLENGES YOU FACE WITH <RESPONSE ITEM>?

Overall impressions of the survey questionnaire. (2 mins)

What were your overall impressions of the survey? Explain.

Closing and Incentive (3 mins)

{Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP TAPE RECORDER.}

Public reporting burden for this collection of information is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Policy Support, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302 ATTN: PRA (0584-0606). Do not return the completed form to this address.