Pretest Memo

A Pretest Memo for Parent Questionnaire - Appendix L SFSP parent survey cognitive testing memo 6-10-2011 (2).docx

Evaluation of the Summer Food Service Program Enhancement Demonstrations

Backpack Program Parent Questionnaire

Expert Interview Protocol: ENGLISH

Backpack Program Parent Questionnaire

Expert Interview Protocol: SPANISH

Pretest Memo

OMB: 0584-0560

⚠️ Notice: This form may be outdated. More recent filings and information on OMB 0584-0560 can be found here:

2012-02-14 - No material or nonsubstantive change to a currently approved collection
2011-09-30 - No material or nonsubstantive change to a currently approved collection

Document [docx]

Download: docx | pdf

Shape1

Date:	February 22, 2011

To:	Chan Chanhatasilpa and Hoke Wilson

From:	Sujata Dixit-Joshi and Lynn Elinson

Subject:	Parent Questionnaire for Backpack Program: Results of Interviews With Fort Worth SFSP Backpack Program Administrators

Background

Westat revised the Parent Questionnaire based on feedback received from FNS on February 2, 2011. Since many of the other questions have been tested or validated for other surveys, we extracted 20 questions for cognitive testing specific to the SFSP Backpack program. The Westat translation team (2 experts) translated these 20 questions into Spanish. A senior Westat Methodologist developed the cognitive testing protocol to pre-test the questionnaires with 4-5 parents of mixed race/ethnicity and gender. The English and Spanish questionnaires used for cognitive interviews are included in Appendix A.

Sample

Westat staff contacted Arkansas and Montgomery County Schools to recruit parents who participate in the school year Backpack program. However, because of the short recruitment and data collection timeframe combined with difficulties reaching individuals due to heavy snow in the area, we were unable to make contact with any parents. We contacted the Fort Worth, Texas providers who had implemented the Summer Backpack Program in 2010 and scheduled interviews with them. We succeeded in completing five interviews (3 English and 2 Spanish) with these expert providers. All interviews were conducted over the telephone.

Methodology

On February 9 and 10, 2011 Westat conducted 5 interviews with administrators of summer backpack programs in the Fort Worth, TX area. Three interviews focused on the English-language questionnaire; two asked about the Spanish-language questionnaire. The questionnaire was emailed to the respondents prior to the interview; this provided them with the opportunity to review the questionnaire before the interview. During the interview, we asked them their overall impression of the survey and specific comments on each question, based on their experience with the summer backpack program and working with parents similar to those who will eventually receive the questionnaire. All interviews followed the established protocol (Appendix B). All interviews were conducted over the phone, lasted an hour or less, and respondents received $40 for their time (some of them chose to donate the incentive to their programs).

Results

Comments on specific items included in cognitive interviewing are included in Appendix C. Item specific recommendations are provided, along with a final decision for each item. This decision reflects FNS input. Overall, the following comments were noted:

All thought this was a good survey questionnaire. One mentioned that it collects information her program would find useful because they often wonder what happens after the backpack goes home. Another said the questions were “relatively straightforward.”
One thought the questionnaire was too long. “The shorter the better” for parents in her program.
All said it was a questionnaire parents in the program would be able to answer.
Of the four who were asked about questionnaire organization, all said it was good.
All said the words and phrases were fine, “simple,” “direct.”
Of those who were asked, respondents felt it would be difficult for parents with multiple children in the backpack program to answer some of the questions for each individual child.
Both Spanish-language respondents said the translation is good.
Spanish-language respondents said program participants often refer to the English-language name of the program. One said the translation was clear, however.
One Spanish-language respondent said parents often use the English word “backpack” but the translation is fine.
One English- and one Spanish-language respondent (both from the same program) noted that the foods are actually provided in plastic bags, not backpacks.

Revisions Based on Findings from Cognitive Interviews

Question specific comments and recommendations for the 20 questions that were included in the cognitive interviews are included in Appendix C. Changes that were made as a result of the interviews are as follows:

Throughout the backpack questionnaires, reference to backpack meals has been changed to indicate either “backpack” or “backpack foods” rather than “backpack meals.”
Question # 3 has been revised to ask how many backpacks the child brought home (in June, July, and August) rather than asking how many weeks the child brought home.
Question #11: response option “kitchen counter” has been changed to “counter or table” and a new response option, “kid’s room” has been added for the backpack program questionnaires.
Question # 12: response option: cost has been added for the backpack program questionnaires.
Question # 15 has been changed to ask if they perceived the meal to be healthy to the backpack and meal delivery questionnaires.

Decision based on subsequent input from FNS

For questions on participation in the SFSP, retain current version of enumerating and obtaining data for all children in the household. These questions are unchanged.

Public reporting burden for this collection of information is estimated to average (insert time) hours [or minutes] per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Services, Office of Research and Analysis, 3101 Park Center Drive, Room 1014, Alexandria, VA 22302 ATTN: PRA (0584-xxxx*). Do not return the completed form to this address.

APPENDIX A:

ENGLISH AND SPANISH QUESTIONNAIRE FOR CONGITIVE TESTING

BACKPACK PROGRAM PARENT QUESTIONNAIRE ENGLISH VERSION

INTRODUCTION

NOTE: Interviews will be conducted with primary care giver or other adult who can answer questions about children in the household.

Participation in Summer Food Service Program and Demonstration Projects

1. For this survey, household means your family and other people who live with you and with whom you share food and food expenses). Please tell me the first name and age of everyone in your household who attended the SFSP.

Name	Age (years)

I am going to ask you a few questions about the summer program that each of them just attended. [IF NEEDED, From DATE to DATE.]

FOR EACH PERSON LISTED IN Q1, CYCLE THROUGH Q2a THROUGH Q10c

2a.Is [NAME] still attending the program now? YES – skip to Q3a

2b. Did {NAME} attend the program the entire time or did {he/she} drop out before it was over?

Name of CHILD	Is currently attending	Attended entire program	Attended but dropped out before the end	DK	Refused
	AUTO FILLED IN Q. 3

IN Q3a-Q3c, IF INTERVIEWS ARE DONE BEFORE PROGRAM IS OVER, USE “WILL.” OTHERWISE USE “DID.

3a. How many weeks {did/will} (NAME) attend the (NAME OF PROGRAM) in June?

3b. How many weeks {did/will} {he/she} attend in July?

3c. How many weeks in August?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	JUNE	JULY	AUGUST
	ENTER NUMBER OF WEEKS; 77 if REFUSED, 99 if DK

While {NAME} was in the {PROGRAM}, did {he/she} ever bring home a backpack with meals for the weekend?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	YES	NO	REFUSED	DK
	GO TO Q. 5	GO TO Q. 6	GO TO Q. 7	GO TO Q. 7

How often did {NAME} bring home a backpack? Would you say every week, most weeks, some weeks, or only one or two weeks? SKIP TO Q7

What are the reasons {NAME} did not bring home a backpack with meals for the weekend? (CHECK ALL THAT APPLY)

SFSP Meal Description, Consumption, Sharing and Wastage

Now please think about the most recent backpack {NAME} received. What foods were in that {backpack/meal delivery}?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Name of PERSON	MILK	FRUIT	JUICE	VEGETABLES	BREAD/ GRAINS	MEAT	MEAT ALTERNATE

8a. FOR EACH FOOD IN Q7, Did {NAME} {eat/drink} any of the {FOOD}? IF YES, ASK Q8b

8b. Did {he/she} {eat/drink} all of the {FOOD}?

Food	Yes	No	Refused	DK
Drank Milk
Ate Fruit
Drank Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

FOR EACH FOOD IN Q8a=NO or Q8b=NO, ASK Q9

9b. Did other children in the household who did not attend a summer program, adults in the household, friends, or someone else [SPECIFY] {eat/drink} the {FOOD}?

		Who was (NAME OF FOOD) shared with?
FOOD	Share Y/N	Other children who also get backpack	Other children who don’t get backpack	Adults in the household	Friends	Pet	Other, Specify	Refused	DK
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

10a. What foods were left over from {NAME}’s most recent backpack?

10b. What was the reason for not eating {FOOD}?

10c. What did you do with the leftover {FOOD}?

FOOD	WHY WAS (FOOD NOT EATEN)	WHAT WAS DONE WITH FOOD?^*
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

^*Codes for What was done with food

Thrown away 1

Returned to center 2

Given away to friend/neighbor 3

Saved for later 4

Other, specify 5

REFUSED 77

DK 99

CYCLE THROUGH ALL HH PROGRAM PARTICIPANTS THEN MOVE TO Q11

IN Q11, USE “EVERYONE’S” IF MORE THAN ONE PROGRAM PARTICIPANT IN THE HH. OTHERWISE USE “THE.”

11. Now please think about all the food that was provided in {the/everyone’s} backpacks. Where did you store or keep {FOOD} at home?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

FOOD	Fridge	Pantry	Kitchen counter	Backpack	Other, specify	Refused	DK
Milk					TEST TO SEE WHAT COMES OUT
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

Parent Satisfaction with SFSP Demonstration and Meals

Now I am going to ask you a few questions about {PROGRAM} and your impression of the meals included in the backpacks.

12. What made you decide to send your children to (NAME OF PROGRAM) this summer?

Backpack meals 1

Activities ……………………………………………………………………………2

Center timing …………………………………………………………………… 3

Near home (or work) …………………………………………………………………… 4

Always send them there for the summer …………………………………………………………………… 5

Other, specify: __________________________ …………………………………………………6

REFUSED 77

DK 99

13. What do your children like about the (NAME OF PROGRAM)?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Activities offered 1

Foods provided ……………………………………………………………………………2

Engaged program staff …………………………………………………………………… 3

Close to home (or work) …………………………………………………………………… 4

Convenient timing …………………………………………………………………… 5

All of their friends attend this program …………… ……………………………………………… 6

Other, specify: __________________________ …………………………………………………7

REFUSED 77

DK 99

14. How did you find out that (NAME OF PROGRAM) has a weekend backpack meals program?

Flyer 1

Brochure ……………………………………………………………………………2

Newsletter ……………………………………………………………………………3

Word of mouth ……………………………………………………………………………4

Other, specify: __________________________ …………………………………………………5

REFUSED 77

DK 99

15. How would you describe the meals that are provided in the (NAME OF PROGRAM) backpack? Would you say the meals are healthy and balanced, somewhat healthy and balanced, or not at all healthy and balanced?

Very healthy and balanced 1

Somewhat healthy and balanced 2

Not at all healthy and balanced 3

REFUSED 77

DON'T KNOW 99

Next, please tell me if you agree strongly, somewhat agree, slightly agree or don’t agree at all with each statement I read.

16. The backpack meals include a variety of foods. Would you say you…

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

17. The backpack meals are convenient to eat.

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

18. Children in my household like the foods provided in the backpack.

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

Perception of Change in Food Expenditure

Now I am going to ask you a couple of questions about the money you spend on food during the school year and summer.

50. Compared with the amount of money you spend on food each month during the school year, would you say you spend:

The same amount on food in the summer months 1

Thank you for taking the time to speak with me today about the Backpack Program Questionnaire. My name is {Martha/Cristina/Stephanie}. I work at Westat, a company that conducts survey research on a variety of topics for lots of different clients. This particular project is for the U.S.D.A. Food and Nutrition Service, as part of an effort to evaluate programs that provide food in children’s backpacks. Your participation today will help us improve the questions on the survey.

Your particular survey answers are not the focus of interest. Instead, I will be asking you about things such as how well you understand what the questions are asking, and whether or not certain words and response categories are clear and appropriate. Your input will help us to correct any problems or make improvements before launching the survey at a later date.

This is a research project, and your participation is voluntary. If you prefer not to answer certain questions, just tell me and I’ll go on to the next one. You may stop the interview at any time.
The information you provide today will be private and your name or other personal facts that would identify you will not be used when we discuss, or write about this study, except as otherwise required by law.
The interview will take about one hour and we’ll give you $40 for your assistance today.
If it’s alright with you, I’d like to tape record our interview, so that I don’t have to take a lot of notes while we’re talking, and can still get an accurate record of what you tell me. The recording will be stored on a secure network drive, only project staff will have access to the information you share with us, and we’ll destroy the tape after our project is finished.
IF OBSERVERS: NAME, a colleague of mine on the project, is also listening in on the conversation today.

START RECORDER.

I’d like to start by asking some general questions about the survey, then we’ll work through it item-by-item. And as a little background, this survey will be conducted with the primary care giver or other adult who can answer questions about children in the household who are participating in the summer backpack meal program. It is part of a larger survey that also includes questions about participation in other nutrition assistance programs and the characteristics of the household. It will be conducted over the phone.

So, with that in mind, what are your overall impressions of the Backpack Program Parent Questionnaire?

Is this a survey that, in your opinion, a primary care giver of a child participating in a summer backpack meal program like yours would be able to complete? (Ask for elaboration of yes/no answers.)

What did you think about the way the sections of the survey were organized?

What did you think about the words and phrases that were used in the survey?

What words, phrases or concepts do you think were out-of-place or inappropriate?

Participation in Summer Food Service Program and Demonstration Projects

1. For this survey, household means your family and other people who live with you and with whom you share food and food expenses. Please tell me the first name and age of everyone in your household who attended in the SFSP.

Name	Age (years)

What did you think of the definition of “household” in Question 1?

Based on this definition of household, do you think parents would know who to include?

What other comments, if any, do you have about Question 1?

I am going to ask you a few questions about the summer program that each of them just attended. [IF NEEDED, From DATE to DATE.]

FOR EACH PERSON LISTED IN Q1, CYCLE THROUGH Q2a THROUGH Q10c

2a. Is [NAME] still attending the program now? YES – skip to Q3a

2b. Did {NAME} attend the program the entire time or did {he/she} drop out before it was over?

Name of CHILD	Is currently attending	Attended entire program	Attended but dropped out before the end	DK	Refused
	AUTO FILLED IN Q. 3

IN Q3a-Q3c, IF INTERVIEWS ARE DONE BEFORE PROGRAM IS OVER, USE “WILL.” OTHERWISE USE “DID.

What comment, if any, do you have about question 2a and 2b?

3a. How many weeks {did/will} (NAME) attend the (NAME OF PROGRAM) in June?

3b. How many weeks {did/will} {he/she} attend in July?

3c. How many weeks in August?

For Question 3, is “number of weeks” typically the way program participants would think about their length of participation?

What other comments, if any, do you have about Question 3?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	JUNE	JULY	AUGUST
	ENTER NUMBER OF WEEKS; 77 if REFUSED, 99 if DK

While {NAME} was in the {PROGRAM}, did {he/she} ever bring home a backpack with meals for the weekend?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	YES	NO	REFUSED	DK
	GO TO Q. 5	GO TO Q. 6	GO TO Q. 7	GO TO Q. 7

What comments, if any, do you have about Question 4?

How often did {NAME} bring home a backpack? Would you say every week, most weeks, some weeks, or only one or two weeks? SKIP TO Q7

What comments, if any, do you have about Question 5?

What are the reasons {NAME} did not bring home a backpack with meals for the weekend? (CHECK ALL THAT APPLY)

What reasons, if any, should be added to the options in the table for question 6?

Are there any reasons that shouldn’t be there or that would not be applicable?

SFSP Meal Description, Consumption, Sharing and Wastage

Now please think about the most recent backpack {NAME} received. What foods were in that {backpack/meal delivery}?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Name of PERSON	MILK	FRUIT	JUICE	VEGETABLES	BREAD/ GRAINS	MEAT	MEAT ALTERNATE

What foods, if any, should be added to the table in question 7? Are there any foods that shouldn’t be there or that would not be applicable?

Do you think parents will respond with more specific food categories, such as “banana” rather than fruit?

What other comments, if any, do you have about Question 7?

8a. FOR EACH FOOD IN Q7, Did {NAME} {eat/drink} any of the {FOOD}? IF YES, ASK Q8b

8b. Did {he/she} {eat/drink} all of the {FOOD}?

Food	Yes	No	Refused	DK
Drank Milk
Ate Fruit
Drank Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

What comments, if any, do you have about questions 8a and 8b?

FOR EACH FOOD IN Q8a=NO or Q8b=NO, ASK Q9

9b. Did other children in the household who did not attend a summer program, adults in the household, friends, or someone else [SPECIFY] {eat/drink} the {FOOD}?

		Who was (NAME OF FOOD) shared with?
FOOD	Share Y/N	Other children who also get backpack	Other children who don’t get backpack	Adults in the household	Friends	Pet	Other, Specify	Refused	DK
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

What comments, if any, do you have about question 9b?

10a. What foods were left over from {NAME}’s most recent backpack?

10b. What was the reason for not eating {FOOD}?

10c. What did you do with the leftover {FOOD}?

FOOD	WHY WAS (FOOD NOT EATEN)	WHAT WAS DONE WITH FOOD?^*
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

^*Codes for What was done with food

Thrown away 1

Returned to center 2

Given away to friend/neighbor 3

Saved for later 4

Other, specify 5

REFUSED 77

DK 99

What kinds of answers do you think parents might give for questions 10b and 10c?

How do you think parents, particularly of children who may not have eaten the food, will react to this question?

CYCLE THROUGH ALL HH PROGRAM PARTICIPANTS THEN MOVE TO Q11

IN Q11, USE “EVERYONE’S” IF MORE THAN ONE PROGRAM PARTICIPANT IN THE HH. OTHERWISE USE “THE.”

11. Now please think about all the food that was provided in {the/everyone’s} backpacks. Where did you store or keep {FOOD} at home?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

FOOD	Fridge	Pantry	Kitchen counter	Backpack	Other, specify	Refused	DK
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

Tell me in your own words what this question is asking. IF NEEDED, is it about what was done with food from the most recent backpack or what is done with the backpack food on a regular basis?

Describe for me the types of food provided in your program? Is there significant variability across the weeks?

What locations for food storage, if any, should be added to the options in table for question 11? Are there any that should not be there or that are not applicable?

Parent Satisfaction with SFSP Demonstration and Meals

Now I am going to ask you a few questions about {PROGRAM} and your impression of the meals included in the backpacks.

12. What made you decide to send your children to (NAME OF PROGRAM) this summer?

Backpack meals 1

Activities ……………………………………………………………………………2

Center timing …………………………………………………………………… 3

Near home (or work) …………………………………………………………………… 4

Always send them there for the summer …………………………………………………………………… 5

Other, specify: __________________________ …………………………………………………6

REFUSED 77

DK 99

What comments, if any, do you have about question 12?

What reasons, if any, for sending the child to the summer program should be added or deleted from this list?

13. What do your children like about the (NAME OF PROGRAM)?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Activities offered 1

Foods provided ……………………………………………………………………………2

Engaged program staff …………………………………………………………………… 3

Close to home (or work) …………………………………………………………………… 4

Convenient timing …………………………………………………………………… 5

All of their friends attend this program …………… ……………………………………………… 6

Other, specify: __________________________ …………………………………………………7

REFUSED 77

DK 99

What comments, if any, do you have about question 13?

What options, if any, should be added to the list in question 13? Are there any options that shouldn’t be there or that would not be applicable?

14. How did you find out that (NAME OF PROGRAM) has a weekend backpack meals program?

Flyer 1

Brochure ……………………………………………………………………………2

Newsletter ……………………………………………………………………………3

Word of mouth ……………………………………………………………………………4

Other, specify: __________________________ …………………………………………………5

REFUSED 77

DK 99

What comments, if any, do you have about question 14?

What ways of finding out about the meal program, if any, should be added to the options in question 14?

Very healthy and balanced 1

Somewhat healthy and balanced 2

Not at all healthy and balanced 3

REFUSED 77

DON'T KNOW 99

What comments, if any, do you have about question 15?

In your mind, what does “healthy and balanced” mean? What is the difference, if any, between the two?

Next, please tell me if you agree strongly, somewhat agree, slightly agree or don’t agree at all with each statement I read.

16. The backpack meals include a variety of foods. Would you say you…

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

Tell me in your own words what this question is asking. IF NEEDED, Is it asking about the variety of food within each backpack, or the variety of food across backpacks?

17. The backpack meals are convenient to eat.

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

18. Children in my household like the foods provided in the backpack.

Strongly agree 1

Somewhat agree 2

Slightly agree, or 3

Don’t agree at all 4

REFUSED 77

DK 99

What comments, if any, do you have about question 17 or 18?

Perception of Change in Food Expenditure

Now I am going to ask you a couple of questions about the money you spend on food during the school year and summer.

50. Compared with the amount of money you spend on food each month during the school year, would you say you spend:

The same amount on food in the summer months 1

Name of PERSON WHO BROUGHT A BACKPACK HOME (AUTOFILL FROM # 4, IF YES)

Every week

Most weeks

Some weeks

Only one or two weeks

Refused

This is a research project, and your participation is voluntary. If you prefer not to answer certain questions, just tell me and I’ll go on to the next one. You may stop the interview at any time.
The information you provide today will be private and your name or other personal facts that would identify you will not be used when we discuss, or write about this study, except as otherwise required by law.
The interview will take about one hour and we’ll give you $40 for your assistance today.
If it’s alright with you, I’d like to tape record our interview, so that I don’t have to take a lot of notes while we’re talking, and can still get an accurate record of what you tell me. The recording will be stored on a secure network drive, only project staff will have access to the information you share with us, and we’ll destroy the tape after our project is finished.
IF OBSERVERS: NAME, a colleague of mine on the project, is also listening in on the conversation today.

START RECORDER.

So, with that in mind, what are your overall impressions of the Backpack Program Parent Questionnaire?

What did you think about the way the sections of the survey were organized?

What did you think about the words and phrases that were used in the survey?

What words, phrases or concepts do you think were out-of-place or inappropriate?

What do you think about the translation of the name of the program (Summer Food Service Program)? How do participants in your program refer to it?

What do you think of the word used for “backpack”?

What other general comments do you have about the translation?

Participation in Summer Food Service Program and Demonstration Projects

Name	Age (years)

What did you think of the definition of “household” in Question 1?

Based on this definition of household, do you think parents would know who to include?

What other comments, if any, do you have about Question 1?

Le voy a hacer algunas preguntas acerca del programa de verano al que cada una de ellas asistió. [IF NEEDED, From DATE to DATE.]

FOR EACH PERSON LISTED IN Q1, CYCLE THROUGH Q2a THROUGH Q10c

2a. ¿Asiste [NAME] al programa actualmente? YES – skip to Q3a

2b. ¿Asistió {NAME} al programa durante todo el tiempo, o se retiró del programa antes de que terminara?

Name of CHILD	Is currently attending	Attended entire program	Attended but dropped out before the end	DK	Refused
	AUTO FILLED IN Q. 3

IN Q3a-Q3c, IF INTERVIEWS ARE DONE BEFORE PROGRAM IS OVER, USE “WILL.” OTHERWISE USE “DID.

What comment, if any, do you have about question 2a and 2b?

3a. ¿Por cuántas semanas {asistió/asistirá} (NAME) al programa (NAME OF PROGRAM) en junio?

3b. ¿Por cuántas semanas {asistió/asistirá} {él/ella} en julio?

3c. ¿Por cuántas semanas en agosto?

For Question 3, is “number of weeks” typically the way program participants would think about their length of participation?

What other comments, if any, do you have about Question 3?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	JUNE	JULY	AUGUST
	ENTER NUMBER OF WEEKS; 77 if REFUSED, 99 if DK

Mientras {NAME} estaba en el programa {PROGRAM}, ¿alguna vez llevó {él/ella} a casa una mochila con comidas para el fin de semana?

Name of PERSON WHO ATTENDED PROGRAM (AUTO FILL FROM # 2)	YES	NO	REFUSED	DK
	GO TO Q. 5	GO TO Q. 6	GO TO Q. 7	GO TO Q. 7

¿Con qué frecuencia llevó {NAME} la mochila a casa? ¿Diría que todas las semanas, la mayoría de semanas, algunas semanas o solo una o dos semanas? SKIP TO Q7

What comments, if any, do you have about Question 5?

¿Cuáles fueron las razones por las que {NAME} no llevó una mochila a casa con comidas para el fin de semana? (CHECK ALL THAT APPLY)

What reasons, if any, should be added to the options in the table for question 6?

Are there any reasons that shouldn’t be there or that would not be applicable?

SFSP Meal Description, Consumption, Sharing and Wastage

Ahora por favor piense acerca de la mochila más reciente que {NAME} recibió. ¿Qué alimentos había en esa {mochila/entrega de alimentos}?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Name of PERSON	MILK	FRUIT	JUICE	VEGETABLES	BREAD/ GRAINS	MEAT	MEAT ALTERNATE

What foods, if any, should be added to the table in question 7? Are there any foods that shouldn’t be there or that would not be applicable?

Do you think parents will respond with more specific food categories, such as “banana” rather than fruit?

What other comments, if any, do you have about Question 7?

8a. FOR EACH FOOD IN Q7, ¿{Comió/Tomó} {NAME} algo de {FOOD}? IF YES, ASK Q8b

8b. ¿{Comió/Tomó} {él/ella} todo(s)/toda(s) el/los/la(s) {FOOD}?

Food	Yes	No	Refused	DK
Drank Milk
Ate Fruit
Drank Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

FOR EACH FOOD IN Q8a=NO or Q8b=NO, ASK Q9

What comments, if any, do you have about questions 8a and 8b?

9b. ¿{Comieron/Tomaron} el(los)/la(s) {FOOD} otros niños del hogar que no asistieron al programa de verano, adultos del hogar, amigos o alguien más [SPECIFY]?

		Who was (NAME OF FOOD) shared with?
FOOD	Share Y/N	Other children who also get backpack	Other children who don’t get backpack	Adults in the household	Friends	Pet	Other, Specify	Refused	DK
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

What comments, if any, do you have about question 9b?

10a. ¿Qué alimentos sobraron de la mochila más reciente de {NAME}?

10b. ¿Cuál fue la razón por la que no se comieron el(los)/la(s) {FOOD}?

10c. ¿Qué hizo usted con el(los)/la(s) {FOOD} que sobró(sobraron)?

FOOD	WHY WAS (FOOD NOT EATEN)	WHAT WAS DONE WITH FOOD?^*
Milk
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

^*Codes for What was done with food

Thrown away 1

Returned to center 2

Given away to friend/neighbor 3

Saved for later 4

Other, specify 5

REFUSED 77

DK 99

What kinds of answers do you think parents might give for questions 10b and 10c?

How do you think parents, particularly of children who may not have eaten the food, will react to this question?

CYCLE THROUGH ALL HH PROGRAM PARTICIPANTS THEN MOVE TO Q11

IN Q11, USE “EVERYONE’S” IF MORE THAN ONE PROGRAM PARTICIPANT IN THE HH. OTHERWISE USE “THE.”

11. Ahora por favor piense acerca de todos los alimentos que se proporcionaron en {las mochilas/las mochilas de cada uno}. ¿Donde guardó o almacenó el(los)/la(s) {FOOD} en la casa?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

FOOD	Fridge	Pantry	Kitchen counter	Backpack	Other, specify	Refused	DK
Milk					TEST TO SEE WHAT COMES OUT
Fruit
Fruit Juice
Vegetable
Bread/Grains
Meat
Meat Alternate

Tell me in your own words what this question is asking. IF NEEDED, is it about what was done with food from the most recent backpack or what is done with the backpack food on a regular basis?

Describe for me the types of food provided in your program? Is there significant variability across the weeks?

What locations for food storage, if any, should be added to the options in table for question 11? Are there any that should not be there or that are not applicable?

Parent Satisfaction with SFSP Demonstration and Meals

Ahora le voy a hacer algunas preguntas acerca del programa {PROGRAM} y de su opinión sobre las comidas que se incluyen en la mochila.

12. ¿Por qué razón decidió que sus hijos asistieran al programa (NAME OF PROGRAM) este verano?

Backpack meals 1

Activities ……………………………………………………………………………2

Center timing …………………………………………………………………… 3

Near home (or work) …………………………………………………………………… 4

Always send them there for the summer …………………………………………………………………… 5

Other, specify: __________________________ …………………………………………………6

REFUSED 77

DK 99

What comments, if any, do you have about question 12?

What reasons, if any, for sending the child to the summer program should be added or deleted from this list?

13. ¿Qué les gusta a sus hijos del programa (NAME OF PROGRAM)?

INTERVIEWER: DO NOT READ, CHECK ALL THAT APPLY

Activities offered 1

Foods provided ……………………………………………………………………………2

Engaged program staff …………………………………………………………………… 3

Close to home (or work) …………………………………………………………………… 4

Convenient timing …………………………………………………………………… 5

All of their friends attend this program …………… ……………………………………………… 6

Other, specify: __________________________ …………………………………………………7

REFUSED 77

DK 99

What comments, if any, do you have about question 13?

What options, if any, should be added to the list in question 13? Are there any options that shouldn’t be there or that would not be applicable?

14. ¿Cómo se enteró de que el programa (NAME OF PROGRAM) tiene un programa de mochilas con comidas para el fin de semana?

Flyer 1

Brochure ……………………………………………………………………………2

Newsletter ……………………………………………………………………………3

Word of mouth ……………………………………………………………………………4

Other, specify: __________________________ …………………………………………………5

REFUSED 77

DK 99

What comments, if any, do you have about question 14?

hat ways of finding out about the meal program, if any, should be added to the options in question 14?

Muy saludables y balanceadas 1

Un poco saludables y balanceadas 2

Nada saludables ni balanceadas 3

REFUSED 77

DON'T KNOW 99

Ahora por favor dígame si usted está muy de acuerdo, algo de acuerdo, muy poco de acuerdo o en desacuerdo con las siguientes afirmaciones.

What comments, if any, do you have about question 15?

In your mind, what does “healthy and balanced” mean? What is the difference, if any, between the two?

16. Las comidas de la mochila incluyen una variedad de alimentos. ¿Diría que está …

Muy de acuerdo 1

Algo de acuerdo 2

Muy poco de acuerdo 3

En desacuerdo 4

REFUSED 77

DK 99

Tell me in your own words what this question is asking. IF NEEDED, Is it asking about the variety of food within each backpack, or the variety of food across backpacks?

17. Las comidas de la mochila son fáciles de comer.

Muy de acuerdo 1

Algo de acuerdo 2

Muy poco de acuerdo 3

En desacuerdo 4

REFUSED 77

DK 99

18. A los niños de mi hogar les gustan los alimentos de la mochila.

Muy de acuerdo 1

Algo de acuerdo 2

Muy poco de acuerdo 3

En desacuerdo 4

REFUSED 77

DK 99

What comments, if any, do you have about question 17 or 18?

Perception of Change in Food Expenditure

Ahora le voy a hacer un par de preguntas acerca del dinero que gasta en alimentos durante el año escolar y el verano.

50. En comparación con la cantidad de dinero que gastó en alimentos cada mes durante el año escolar, ¿diría que usted gasta:

La misma cantidad en alimentos que durante los meses de verano 1

Más en alimentos durante los meses de verano 2

Menos en alimentos durante los meses de verano 3

REFUSED 77

DK 99

What comments, if any, do you have about question 50?

70. ¿Es el ingreso anual de su hogar de todas las fuentes:

Menos de $25,000 ……………………………………………………………………………..1

If yes, ask

Menos de 20,000 ……………………………………………………………………………….2

If yes, ask

Menos de 15,000………………………………………………………………………………..3

If yes, ask …

Menos de 10,000………………………………………………………………………………..4

If NO to LESS THAN $25,000, ask ..

Menos de 35,000………………………………………………………………………………..5

Menos de 50,000………………………………………………………………………………..6

Menos de 75,000………………………………………………………………………………..7

75,000 o más …..……………………………………………………………………………..8

Refused …………………………………………………………………………..……77

Don’t know…………………………………………………………………………………..….99

Gracias por tomarse el tiempo de hablar conmigo hoy. ¿Tiene alguna pregunta?

How comfortable do you think parents of children who participate in the backpack food program will be answering the income question? What do you think their reaction will be?

What other comments, if any, do you have about question 70.

That’s the end of the survey questions. I just have a few more questions for you today.

IF NEEDED: What questions, if any, do you think a parents of a child participating in the summer meal program would struggle with?

Do you think there were any questions missing that would be important to ask in regard to participation of the program, parent satisfaction with the program, and use of the food provided?

What other comments or feedback would you like to share with me that we haven’t already talked about or discussed?

Thank you very much for your time and your feedback today.

[Turn off recorder. Get contact information to send incentive.]

APPENIDIX C

Q1. For this survey, household means your family and other people who live with you and with whom you share food and food expenses. Please tell me the first name and age of everyone in your household who attended in the SFSP.

Respondents felt parents would understand the definition of household, saying it seems “straightforward” and “makes sense.” One respondent pointed out that it is very similar to household definitions parents would see when applying for other assistance programs would see.
One respondent felt “attended” was misleading because, for example, in her case, the backpack program is run in conjunction with the summer camp program. Children “attend” the summer camp but “participate” in the backpack program.

Recommendation: Leave item as is. However, this is an item that may need to be tailored depending on the nature of the specific programs being evaluated.

Decision: Implement the recommendation.

Q2a. Is [NAME] still attending the program now? YES – skip to Q3a

Q2b. Did {NAME} attend the program the entire time or did {he/she} drop out before it was over?

Respondents felt the question was fine as is. One, whose backpack program is also tied to a summer camp, said there are children in the backpack program who don’t attend the summer camp and therefore have “spotty attendance” picking up their backpacks. Another explained that some parents stop picking up the backpacks because their kids won’t eat the foods in it.

Recommendation: Leave item as is.

Decision: Implement the recommendation.

3a. How many weeks {did/will} (NAME) attend the (NAME OF PROGRAM) in June?

3b. How many weeks {did/will} {he/she} attend in July?

3c. How many weeks in August?

Respondents had no comments about this question.

Recommendation: Leave as is.

Decision: Implement the recommendation. Add a follow-up question to capture if the child attended the program most weekdays, some weekdays, or only on Fridays.

Q4. While {NAME} was in the {PROGRAM}, did {he/she} ever bring home a backpack with meals for the weekend?

Most respondents thought the question was fine.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q5. How often did {NAME} bring home a backpack? Would you say every week, most weeks, some weeks, or only one or two weeks? SKIP TO Q7

Two respondents said it was fine to ask about weeks.
Three seemed to think asking about the number of backpacks would be easier for parents to answer. As one respondent said, “ask them how many times did you take a backpack home?”
One respondent also pointed out that the month of June is tricky because her summer camp doesn’t start until the third week of that month, and the backpack program doesn’t start until another week or more after that.

Recommendation: Consider asking about the number of backpacks in each month rather than the number of weeks.

Decision: Implement the recommendation.

Q6. What are the reasons {NAME} did not bring home a backpack with meals for the weekend? (CHECK ALL THAT APPLY) Forgot to bring it/left it at the SFSP site; don’t like food in backpack; friends tease him/her if s/he brings backpack home; other specify.

Most respondents thought the question was fine and the response categories comprehensive. As one put it, “these would be the top 5 reasons.”
One respondent pointed out that the reasons are all from the child’s perspective. In her program, parents are sometimes with their kids at the time the backpacks are distributed and will refuse them, saying the food would go to waste.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q7. Now please think about the most recent backpack {NAME} received. What foods were in that {backpack/meal delivery}?

Respondents for the most part thought the question was fine. They mentioned yogurt, string cheese, “kool-aid sticks” (that come as liquid wrapped in plastic and can be frozen for a popsicle treat) as items that didn’t seem to fit in any of the existing categories.
All respondents thought parents would answer with specific foods (e.g., corn, apple, or apple juice), brand names (e.g., Cheerios) or food packaging (e.g., breakfast meal or lunch meal).
One respondent didn’t know what a “meat alternate” would be and another reported that meat isn’t provided in their backpacks.

Recommendation: If the broad categories are retained, interviewers will have to be thoroughly trained how to categorize respondents’ specific answers. Alternatively, the categories could be tailored to the foods offered by the program being evaluated.

Decision: Ask questions at the food category level.

Q8a. FOR EACH FOOD IN Q7, Did {NAME} {eat/drink} any of the {FOOD}? IF YES, ASK Q8b

Q8b. Did {he/she} {eat/drink} all of the {FOOD}?

Respondents thought the question was fine.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q9b. Did other children in the household who did not attend a summer program, adults in the household, friends, or someone else [SPECIFY] {eat/drink} the {FOOD}?

Most respondents thought the question was fine.
One respondent raised the possibility of social desirability bias associated with the question. He felt parents may be hesitant to admit other people are eating the backpack food out of concern that the backpacks would be taken away.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q10a. What foods were left over from {NAME}’s most recent backpack?

Q10b. What was the reason for not eating {FOOD}?

Q10c. What did you do with the leftover {FOOD}? Thrown away; returned to center; given away to friend/neighbor; saved for later; other specify

Respondents thought the questions were fine.
Most felt the main reason for not eating the food would be that the child didn’t like it. One respondent said that some parents have complained the food is “old or spoiled.”
Most felt the main things parents do with the uneaten food is throw it away or give it away. One respondent said they would never return it to the center.
Most felt parents would have no problem answering the question. One respondent said parents might not be truthful because they wouldn’t want to seem wasteful or ungrateful. Another respondent took the opposite view, saying respondent didn’t pay for the food so they wouldn’t feel wasteful throwing it away. Further, he felt they would want to help out the program by describing the foods their children did and didn’t like.

Recommendation: Create precoded response list for Q10b that includes “didn’t like it” and “food was spoiled.”

Decision: Implement the recommendation.

Q11. Now please think about all the food that was provided in {the/everyone’s} backpacks. Where did you store or keep {FOOD} at home? Fridge; pantry; kitchen counter; backpack; other specify

Respondents understood the question was asking about the routine for the backpack across weeks, not just for the most recent week.
Two respondents suggested adding “kid’s room” because they felt some of the foods, such as the snack items, might end up there.
One respondents pointed out that “kitchen counter” is too specific; people might put the food on the dining room table, for example.
This same respondent felt parents might be inclined to answer for where they think the food “should” be stored (e.g., not admit to leaving string cheese out on the counter) rather than where it actually was stored.
Another respondent felt parents might answer more specifically such as “in a plastic container.”

Recommendation: Revised “kitchen counter” to “counter or table” and add “kid’s room” to the list of response options. Leave the item open-ended for pretesting with respondents but for the evaluation, consider reading out the preset response options to orient respondents to the desired level of specificity.

Decision: Implement the recommendation.

Q12. What made you decide to send your children to (NAME OF PROGRAM) this summer? Backpack meals; activities; center timing; near home (or work); always send them there for the summer; other specify

One respondent said her program doesn’t publicize the backpack meals.
Another respondent suggested “teachers and staff”, “cost of program,” “benefits of the program” and “location of the program” as possible additions to the list of response options.

Recommendation: Response list may need to be tailored to the program being evaluated.

Decision: Implement the recommendation.

Q13. What do your children like about the (NAME OF PROGRAM)? Activities offered; foods provided; engaged program staff; close to home (or work); convenient timing; all of their friends attend this program; other specify

Respondents thought this question was fine.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q14. How did you find out that (NAME OF PROGRAM) has a weekend backpack meals program? Flyer; brochure; newsletter; word of mouth; other specify

Most respondents suggested adding “internet,” “website,” or “electronic means” such as Facebook or Twitter to the list of ways to find out about the program.
The respondent who said her program doesn’t advertise the backpack meals said parents find out about it “when they actually get the backpack.”

Recommendation: This list may need to be tailored to the programs being evaluated. Delete “meals” from the question (see comments at Q15).

Decision: Implement the recommendation.

Q15. How would you describe the meals that are provided in the (NAME OF PROGRAM) backpack? Would you say the meals are healthy and balanced, somewhat healthy and balanced, or not at all healthy and balanced?

Most respondents agreed that “healthy” and “balanced” are not the same things.
One respondent felt parents wouldn’t necessarily be knowledgeable about whether the backpack foods were healthy and balanced. Another thought they would focus on “healthy” and ignore “balanced.”
One respondent pointed out that the backpack food is usually not in the form of “meals.”

Recommendation: Consider asking separate questions about “healthy” and “balanced” or asking only about one or the other of them. Revised “meals” to “food” for consistency across the questionnaire.

Decision: Implement the recommendation on replacing “meals” with “food”. Also, revise the question to focus on “healthy” and delete reference to “balanced.”

Q16. The backpack meals include a variety of foods. Would you say you strongly agree; somewhat agree; slightly agree; don’t agree at all

Respondents all understood the question was asking about foods across all backpacks, not just for the most recent backpack. To reinforce that notion, one respondent suggested adding the word “generally” to the question.

Recommendation: Revise “backpack meals” to “backpacks.” Consider rewording the question as “The backpacks generally include a variety of foods. Would you say you…”

Decision: Implement the recommendation.

Q17. The backpack meals are convenient to eat. Strongly agree; somewhat agree; slightly agree; don’t agree at all

Respondents thought the question was fine.

Recommendation: Revise to read “The backpack foods are convenient to eat.”

Decision: Implement the recommendation.

Q18. Children in my household like the foods provided in the backpack. Strongly agree; somewhat agree; slightly agree; don’t agree at all

Respondents thought the question was fine. One suggested adding the word “generally” here, too.

Recommendation: Leave as is.

Decision: Implement the recommendation.

Q50. Compared with the amount of money you spend on food each month during the school year, would you say you spend the same amount on food in the summer months; more on food in the summer months; less on food in the summer months.

Respondents thought the question was fine. One felt there wouldn’t be much variability in answers because of the stereotype that food costs more in the summer because the kids aren’t in school and so are eating parents “out of house and home.”
The response options in the Spanish translation may be somewhat unclear.

Recommendation: Leave the English as is. To make the Spanish-language options more specific, add the word “dinero” (“money”) as below.

La misma cantidad de dinero en alimentos que durante los meses de verano

Más dinero en alimentos durante los meses de verano

Menos dinero en alimentos durante los meses de verano

Decision: Implement the recommendation.

Q70. Income question

Only one respondent thought parents wouldn’t have trouble answering the income question, saying they’re used to providing it when applying for assistance programs.
All other respondents thought parent would want to know why they were being asked this question or would refuse to answer out of fear their benefits would be affected.

Recommendation: Leave as is.

Decision: Implement the recommendation.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Martha Stapleton
File Modified	0000-00-00
File Created	2021-02-01