Appendix Y Summary of Pretest Findings and Changes

This memorandum summarizes the pretests completed for the Farm to School Census and Comprehensive Review. The study team pretested both the 2019 Farm to School Census (Census) and Distributor Interview instruments during May 2018 and this memorandum reports the key findings from the pretests of both instruments.

The purpose of the pretest was to ensure the instruments were clear and understandable to respondents. Specific pretest objectives included:

Identifying problems related to communicating intent or meaning of questions
Determining whether respondents could provide the information requested
Identifying problems with introductions, instructions, or explanations
Assessing the time needed to complete the questionnaire and other respondent burden issues

Recruitment and Data Collection Methods for the Distributor Interview Pretest

For purposes of pretesting the data collection instruments, the United States Department of Agriculture (USDA) Food and Nutrition Services (FNS) Project Officer identified and provided the study team with a list of 14 distributors as potential respondents for the Distributor Interview. The list included distributors that came from seven southeastern states.

The Abt study team called potential participants from the list. These initial calls were meant to introduce the Farm to School study and request the respondent’s participation in the Distributor Interview. The Abt study team then scheduled calls with three distributors.

The Abt study team reached out to the three participating distributors via email. In this email, team members confirmed meeting times, reiterated background information described in the initial call and provided a two-page document with the interview questions. By including the document with the meeting invitation, the study team ensured interviewees would know what to expect during the call and could prepare information at their discretion. The confirmation email and interview question document appear in Appendix B.

After completing the interview during the scheduled time, the study team asked the participants to respond to 10 debrief questions, regarding the length, content, and structure of the interview.

Findings for the Distributor Interview Pretest
1. Distributor Interview Protocol

Study team members interviewed three broadline distributors: two from Tennessee and one from Mississippi. The team conducted these three interviews from May 9-16, 2018.

Duration

Two of the interviews took 38 minutes to administer, and the third took 25 minutes. Each interview was immediately followed by debriefing questions. The debriefing questions took an additional 20 to 30 minutes to complete. Please see details regarding the Distributor Interviewees below:

Table 1. Distributor Interviewee Details

Respondent ID	Location	Size	Distributor Type	Time to complete pretest
D001	TN	$150M	Broadline	38 mins
D003	TN	N/A	Broadline	25 mins
D004	MS	$800M	Broadline	38 mins

General Findings and Recommendations

Each of the participating distributors was very engaged throughout the interview. Each noted they had very busy schedules, and they needed to specifically block time for the call. As the debrief questions were near completion, one respondent commented he needed to end the call soon because he had a meeting, though he was able to complete the debrief.

Each of the interviewees was in the contracts division of their respective companies. They held the titles: Contract Sales Manager, Contract and Bid Specialist, and Contract Manager. They each completed the calls individually, but reported they either consulted with a colleague to gather information prior to the interview or that there was someone in a different role/department (procurement department, sales department, produce manager) that could better answer some of the questions.

One interviewee reported that getting the questions in advance allowed them to prepare for the interview by finding the answers to some of the more specific questions in the interview. Another interviewee expressed concern at the beginning of the interview that, after reviewing the questions, she thought she would not be able to provide adequate answers to many of the questions because she had not had time to consult with colleagues about what information was appropriate to share. When asked if the study team should have given more time between the confirmation email and the interview, she responded it was plenty of time (almost a week), but she had taken some time off from work. These findings point to the importance of providing the questions in advance, and interviewees need a few days to review the interview document in order to prepare for the call.

One respondent noted that receiving the initial notification from FNS would have made him feel more sure of the legitimacy of the study and the importance of their participation. Suggested text for this notification email can be found in Appendix A. Following the initial email from FNS, we suggest the Abt study team directly calls the distributor to determine a date and time for the interview, to be followed by a confirmation email with a meeting invitation and an abbreviated set of questions, as seen in Appendix B.

Throughout the three interviews, each of the respondents indicated on various questions they were not the authority for their company on the given topic. As such, Abt will work with company contacts to ensure the team is correctly identifying the person at each distributor with the most familiarity with purchasing, procurement and sourcing for schools. The study team suggests, during the initial call to outline the main topics interviewers will discuss and ask if they need to talk with someone else, or if they need to get the information from someone else prior to the interview.

Question-by-Question Findings and Recommendations

See Table 1 beginning on the next page which highlights the questions on the Distributor Interview where pretest participants had difficulty and/or provided suggestions for changes.

Table 2. Item-level Findings and Recommendations for the Distributor Interview Protocol

Question	Responses/Findings/Observations	Revisions Made Based on Pretest Findings
2. Please describe in detail the way, or ways, the word “local” is defined geographically for procuring and marketing local food products and local food producers to school districts.	The responses to this question varied. D#1: They only define local for produce and that included the state or within a 250 radius of their location. D#2: They define by states—within their state or sometimes within two of the bordering states. D#3: They don’t have an official definition—they use what the schools provide (if given). Sometimes it is just GA, but mostly it’s a 50 mile radius and could go up to 100 miles.	Expand this question with specifics. “In terms of state boundaries, how is “local” defined for procuring and marketing local food products and producers? In terms of radius around your location, how do you define “local”? Is there another way that your company defines “local”? Do you define “local” differently for different products/Districts?”
6. What types of solicitations or procurement methods do school districts use to purchase local foods from your company?	This question seemed to involve similar information to question 4. D#2: Fixed delivery fee contract. MS schools send out the solicitation that is good for two years, with three, one year options. You have to guarantee a price for 180 days, after that, the market changes. We have good working relationships, so if the state office is pushing something special for farm to table, we can get it for them even if it wasn’t included in the original solicitation. D#3: Invitation to bid.	Question 6 should be moved to directly follow question 4.
8. Does your company see value in providing local products to school districts? Why or why not? Have you seen a ROI from your local foods program?	The answers to this question often seemed to come through earlier in the Interview—questions 5 and 7.	Question was eliminated and incorporated into the prompts in the Q. 5 questions.
10. Do you feel that providing local items gives your company a competitive advantage with school districts? Do you promote or advertise this capability? If so, how?	This questions seemed somewhat repetitive of some of the questions in 5e.	Question changed to number 7 with final Interview Protocol.
11. Does your company have any goals regarding what percentage of local products it makes available to school districts?	The answers to this question seemed to echo what was said to questions 5e and 11.	Question changed to number 8 with final Interview Protocol.
12. In order to supply local foods to school districts, have you had to establish new supply chains to connect with local producers? If YES, please describe the process of establishing new supply chains. What is the process for sourcing local foods for the districts you serve?	This question seemed difficult to answer for two respondents, without the help of their procurement department. D#1: No, we don’t see customers asking for it. D#2: That’s a procurement question. D#3: I get my pricing from my produce manager. For produce, if it is bid-related, I would ask him for help on that.	Scheduling script highlights that more than one person is invited to attend the interview if there are areas where the Respondent may not have familiarity. The abbreviated list of questions sent with the confirmation email depicts the breadth of questions to be asked.
13. Do you have staff specifically dedicated to working with local vendors to procure local foods on the supply side? Do you spread out local sales across multiple sales representatives (e.g., schools, healthcare, restaurants, etc.) or do you have one designated local foods representative?	D#1: No. D#2: We have a guy in our procurement department, but he’s not just local. D#3: I don’t know.	Scheduling script highlights that more than one person is invited to attend the interview if there are areas where the Respondent may not have familiarity. The abbreviated list of questions sent with the confirmation email depicts the breadth of questions to be asked.
14. Describe your general vendor requirements. Do these differ for local vendors (e.g., food safety, growing practices, packaging, storage, etc.?)	None of the respondents seemed to be particularly sure of this question. It seemed this is not part of what they cover in their roles.	Question was eliminated in final Interview Protocol.
15. Do you often change vendors or recruit new vendors? Where/how do you find local vendors?	Two of the respondents provided answers about why they would change vendors, rather than how. D#2: We buy based on the season and what the farmer is offering. Some work through a co-op, which makes paperwork easier. You go farmer to farmer depending on what they are growing. D#3: Yes, it could be local. It could be price-related or availability of a product—it could be a lot of factors.	Change the question to, “Describe the process of how you find local vendors. Where do you find them?”
16. What are the biggest factors that determine whether or not you would bring in a local product for a school customer? From your perspective, what barriers exist?	Two of the respondents had already referenced the answer to this question earlier in the interview. (Question 8) D#3: It would just be the way to distribute it. In a big county like Shelby, it is just not feasible—transporting is the barrier.	Question was re-worded and incorporated in the series of questions in Q5-Q5e.
18. What are the top 5 local food items, by volume, your company sold to school districts in 2018? Please be as specific as possible. For example: sliced apples, whole apples, chicken drumsticks, bags of lettuce, fish.	One respondent did not know the answer to this question, the other two respondents gave 2 and 3 examples. D#1: Tomatoes and apples for fresh produce; bottled water and hamburger buns our the largest local products, by volume. D#2: The top two would be watermelons and sweet potatoes. I don’t know after that. D#3. I don’t know.	Ask for 3 examples instead of 5.
19. What are the top 5 local food items, by dollar sales, your company sold to school districts in 2018? Please be as specific as possible.	For the two respondents that answered this question, the answers were the same as number 18.	First ask, “Are the top 3 local food items by dollar sales different from the top 5 local food items by volume?”
21 Based on your definition of local, has the number of local producers you work with increased, decreased, or remained the same over the past five years?	D#1: Remained about the same. D#2: It’s increased with more emphasis given to farm to table. D#3. I don’t know.	Changed time period to include the last ten years.
26. Have you been approached by school districts for reports of volume or price of locally sourced products? Sometimes these are called “velocity reports.” How do you track velocity information, if you provide these reports to school districts?	Respondents noted that while they have gotten requests for velocity reports from schools, they are not specifically looking for information about local products, but rather using it for a discount from some of the larger vendors.	In the text of this question, be more explicit about the types of velocity reports we are asking about.
29. Please estimate the total number of school districts with whom you work. Do you have an estimate of how many of these are urban, suburban and rural? Total number of states?	One distributor asked for a definition of urban/rural. All respondents seemed to be estimating the percent of rural/suburban/urban. D#1: 50 districts, 2 large urban, about 10 suburban with the rest being rural. All in TN. D#2: We have five in MS, five regions…probably 350 different locations. Totally a guess—most of our schools are rural. I would say 65% rural and the rest are metro-type areas. We service AL, but we don’t offer fresh or local products to them. D#3: 20 districts. Half urban, half rural. Hard to say. Four states: TN, AK, MS and MO. Head Starts in MS too.	Prepare prompts to use with definitions of rural/suburban/urban. Provide specifics about this question in the question document that is sent in the invitation email.
30. How many local individual producers, such as farmers, ranchers, or fishers?	None of the respondents had an exact answer for this question. Two had guesses. D#1: Two farmers that I know if, but procurement may know more. No ranchers (too expensive) or fishers. D#2: Don’t know, about 10. It’s all row crop farmers, no ranchers, no fishers. D#3. I don’t know.	Revised with table and specification of actual vs. estimate in final Interview Protocol.
31. How many local producer cooperatives (may be called “food hubs”), including farmer, rancher, or fisher cooperatives?	Two respondents did not know the answer to this—in both cases, stated one, maybe two. One asked for a definition of cooperatives.	Revised the language for clarity and is Q.27. in final Interview Protocol.
32. How many local processors that process local foods? If your company processes local foods, please include yourself in this number.	This seemed to be a difficult question to answer for the respondents. D#1: None that I know of—the agriculture here is different from other states. D#2: No local. D#3. I don’t know.	Eliminate the question.

Recruitment and Data Collection Methods for the Census Pretest

For purposes of pretesting the data collection instruments, USDA Food and Nutrition Services (FNS) identified a list of 13 school food authorities (SFAs) as potential respondents for the Census Interview. The list included SFAs who came from FNS’s southwestern, southeastern, and northeastern regions.¹

The Abt study team e-mailed six potential participants from the list. These initial e-mails were meant to introduce the Farm to School Census and request the respondent’s participation in the Census pre-test interview. One week later, the Abt study team sent out Census hard copies to these respondents and followed up the mailing with a telephone call to confirm delivery and schedule a debrief interview to discuss issues and challenges they may have encountered when completing the Census. Debrief interviews were then scheduled with the same six SFAs.

The Abt study team reached out to the six participating SFAs via email and telephone to ensure return delivery of their completed Census Interviews in ample time for their debrief interviews. The six Census interviews were returned to Abt and reviewed by the study team prior to each debrief interview. As part of the pretest communications, SFAs were instructed to keep a copy of their completed interview and make note of questions that were unclear, not easily understandable, hard to recall, and/or difficult to answer accurately. They were also asked to estimate the time needed to complete the Census interview and note any suggestions they had for improving the instrument.

During the follow-up calls, participants were asked to respond to ten debrief questions, regarding the length, content, and structure of the interview.

Each of the participating respondents was very engaged throughout the interview. The debriefs were scheduled for 30 minute slots, and each of the six directors participated in the debrief at the scheduled time. The debriefs lasted about 20-25 minutes.

Each interviewee was a child nutrition director, or had a similar title, in the school district. Each director completed the call individually, but most reported they consulted with at least one colleague to gather information prior to completing the pre-test Census. None of the respondents had trouble obtaining the information asked as part of the pre-test Census.

Findings from the Census Pretest
1. Census Interview Protocol

Study team members interviewed six SFAs: one each from New Hampshire, Connecticut, New York, Texas, Arkansas, and Oklahoma. The participating SFAs were all public school districts and varied in size, as noted in Table 2 below. The study team pretested the Census Interview instrument with the six respondents over a three week period from May 9-May 25, 2018.

Duration

The time needed by SFAs to complete the Census ranged from as little as 23 minutes to as much as 2 hours. This time included both the gathering of information as well as completing the Census. This amount of time appeared to be consistent regardless of the size of the SFA. Because the study team conducted the pretest with SFAs that conduct many farm to school activities, the completion time will be much faster for many of the SFAs that do fewer activities. Based on this pretest and completion in prior years when the Census was conducted, the team estimates the overall burden will remain at 30 minutes for the Census.

Table 2. Duration of pretest and size of SFA

Respondent ID	Student population of SFA	Time to complete Census pretest	Questionnaire path
1001	2,263	1 hour 39 minutes	Farm to school participant
1002	3,322	23 minutes	Farm to school participant
1004	33,345	58 minutes	Farm to school participant
1006	160,253	30 minutes	Farm to school participant
1007	8,400	2 hours*	Farm to school participant
1008	14,658	56 minutes	Farm to school participant

*respondent 1007 reported “1 hour” during the debrief, but 2 hours is reported on the pretest instrument. She gathered information as she completed the pretest and counted all of that time as part of the pretest completion.

General Findings and Recommendations

Overall, the pretest respondents stated that the questions were easy to understand, easy to answer and appropriate for the Census. The financial data seemed to be the main area where input was needed from other departments or other colleagues. However, each SFA director found that they could obtain that data from their finance, procurement or own office.

Two of the respondents mentioned that much of the data requested was easily accessible because of recently completing a grant report. Others kept local purchase data in a spreadsheet so it was easily available.

All of the pre-test respondents answered the all of the school garden questions. They all seemed very involved and/or knowledgeable about the school gardens in their districts. Their experiences may not be reflective of all farm to school Census participants since the pre-test respondents were chosen because of their Farm to School participation.

Each respondent liked knowing ahead of time what information would be needed to complete the Census. They felt that knowing this information ahead of time would help them coordinate with others to gather the correct data. Having the worksheet available or a list of the questions ahead of time would help facilitate the gathering of information. Because the questionnaire is online, and the SFAs will not be able to share physical paper pages, the study team will include what data will be needed to complete the Census. in the recruitment materials.

Most recommendations from the pretest resulted in simple text changes to questions. These are meant to enhance the question options or add clarity to what is being asked.

Question-by-Question Findings and Recommendations

See Table 3 beginning on the next page which highlights the questions on the Census where several pretest participants had difficulty and/or provided suggestions for changes.

Table 3. Item-level Findings and Recommendations for the 2019 Census Interview Protocol

Question	Responses/Findings/Observations	Recommendations and Revisions made based on Pretest
4. What percent of meals are made from “scratch”?	Only part of the meal is made from scratch. Usually make something from scratch and then have convenience foods as side items.	Changed to “What percent of your SFA’s recipes are made from scratch?”
13. Are any of the following considered to be challenges in procuring local products or reasons why your district does not purchase local products?	Respondent thought these were two separate ideas.	Split out the option Lack of Staff time or Interest in preparing local foods in the Staff/Kitchen category.
22. How does your SFA handle food preparation? (Please check all that apply.)	Respondent wrote in “Each school has its own kitchen”	Added “School based kitchens”
33. How does your SFA track the number of schools that had edible school gardens during the 2016-2017 school year?	Multiple respondents wrote in “Survey” for the questions that ask how things are tracked.	Added “Survey” as a response option.
36. For the 2016-2017 school year, please indicate if your district procured local foods from any of the following sources. (Please choose all that apply.)	Respondent concern that not everyone knows what a food hub is.	Added a hover over definition for “food hub.”
37. Please indicate if your district or any schools in your district purchased any of the following foods as local products – IN ANY FORM – from local growers/producers/processors/manufacturers/ distributors during the 2016-2017 school year or would like to in the future.	Regarding the “I don’t know” option, does it mean I don’t know the answer or I don’t know if I want to purchase locally?	Expand upon the “I don’t know” option to clarify what is being asked.
39. On average, about how frequently does your SFA’s meals or snacks include at least one locally sourced food item from the categories below?	Some SFAs can only get local produce seasonally. Local may get missed in the question text, so bold it to make it more obvious.	Change “Occasionally” to “Occasionally/Seasonally” as a response option. Bold the word “Local” so it is not overlooked.
“Local” definition	is “local” produced locally or sold locally? One respondent buys from a local milk distributor, but milk comes from around the region. She considered that “local.”	Be aware that all SFAs may have a different definition of local and it is so diverse, it is unable to be defined uniformly.

Appendix A. FNS Email Notification to Distributors

Dear [Distributor Name]:

Congratulations! You’ve been recommended to participate in the 2019 Farm to School Distributor Interview, which is part of the United States Department of Agriculture (USDA) Food and Nutrition Service (FNS) Farm to School Census and Comprehensive Review. This three-year study will review and describe the multiple facets of USDA’s Farm to School program, and general growth of farm to school efforts across the country.

The USDA FNS has selected Abt Associates, an independent research company, to conduct the 2019 Farm to School Distributor Interviews.

You have been nominated to participate as one of a selective group of 20 food distributors to share your industry knowledge and experience regarding the processes and challenges of using local supply chains, and the economic scope and impacts of local food purchases. Your input, provided through a 60-minute telephone interview, will provide critical feedback that will help to inform FNS policy decisions.

You will be asked to provide information regarding [INSERT NAME OF COMPANY]’s approximate volume and cost of local food purchasing and procurement, in addition to open-ended questions about the procurement process.

A member of the our study team will contact you via telephone in the next week or so to provide more detail about the study, answer any questions you might have, and to schedule your interview.

We greatly appreciate your support of this study and look forward to speaking with you soon. For additional information about the Distributor Interview, please review the “Frequently Asked Questions”.

If you have any questions, please feel free to contact our project team at toll-free 866-778-1316 from 9 AM to 9 PM ET or e-mail [email protected]. If you have any questions regarding the authorization for this project, you may contact the FNS Project Officer by telephone at 703-457-7741 or via email at [email protected].

Thank you in advance for your support and assistance.

Sincerely,

Maria Boyle Cindy Long

Project Director Deputy Administrator

Farm to School Census and Comprehensive Review

Child Nutrition Programs

Abt Associates USDA Food and Nutrition Service

Appendix B. Confirmation Email to Distributors with Abbreviated Questions

SUBJECT: Farm to School Distributor Interview

Dear [DISTRIBUTOR NAME],

As we discussed on the phone, you were recommended to participate in the 2019 Farm to School Distributor Interview, which is part of the United States Department of Agriculture (USDA) Food and Nutrition Service (FNS) Farm to School Census and Comprehensive Review. This three-year study will review and describe the multiple facets of USDA’s Farm to School program, and general growth of farm to school efforts across the country. The USDA FNS selected Abt Associates (the research company for whom I work), to manage the 2019 Farm to School Distributor Interviews.

You are one of a selective group of food distributors who has been asked to share your industry knowledge and experience regarding the processes and challenges of using local supply chains and the effect of local food purchases. Through this research, we hope to gather critical feedback to inform future Farm to School FNS policy decisions.

I attached a list of general questions we will cover during the interview. These are broad in scope and we anticipate some may not be applicable to you or your company, or you may not be able to answer some—you are welcome to invite other colleagues to the interview if they have experience in areas you do not.

We greatly appreciate your support of this study and look forward to speaking with you on [SCHEDULED DATE AND TIME].

Best,

The Farm to School Study Team, Abt Associates

ATTACHMENT: 2019 FARM TO SCHOOL DISTRIBUTOR INTERVIEW TOPICS

DEFINITION OF LOCAL

Distributors and school districts may define the word “local” when referring to “local foods” in different ways. We are interested in how your company thinks of the word “local” when it comes to the foods you provide to school districts.

Please describe the way “local” is defined geographically for procuring and marketing local food products and local food producers to school districts.
Do you provide any foods to school districts from local producers?
Do you get requests from school districts for local foods? If so, do school districts have defined local preferences?

REQUESTS FOR LOCAL FOODS & CONTRACTS

4. How frequently do you get specific requests for local foods from the school districts?

5. This study is interested in what happens when your company receives a request from a school district for local foods. What local foods have typically been asked for by school districts? How have you been able to meet these requests?

6. What types of solicitations or procurement methods do school districts use to purchase local foods from your company?

VALUE IN PROVIDING LOCAL & MARKETING

7. Do you market or promote your company’s ability to sell local products to school districts?

8. Does your company see value in providing local products to school districts?

9. Have you noticed changes in how farmers grow or process products?

10. Do you feel that providing local items gives your company a competitive advantage with school districts?

11. Does your company have any goals regarding what percentage of local products it makes available to school districts?

SUPPLY CHAIN & TOP 5 FOODS

12. In order to supply local foods to school districts, have you had to establish new supply chains to connect with local producers?

13. What are the biggest factors that determine whether or not you would bring in a local product for a school customer?

14. What are the top 5 local food items, by volume, your company sold to school districts in 2018?

15. Would you say the cost of these local items is more, less, or the same, as the non-local commercially purchased alternative?

CHANGES OVER TIME

The Healthy, Hunger-Free Kids Act of 2010 authorized the USDA to establish a farm to school program to assist eligible entities, through grants and technical assistance, in implementing farm to school programs that improve access to local foods in schools. For school districts, farm to school activities may include purchasing locally-produced food for school meal programs, taking students to farms, bringing farmers into classrooms, and cultivating school gardens.

16. Based on your definition of local, has the number of local producers you work with increased, decreased, or remained the same over the past ten years?

17. From your perspective, what are the top 3 factors driving that change?

18. Based on your definition of local, has the volume of local sales to school districts increased, decreased, or remained the same over the past ten years?

REPORTING

The following questions are asked to better understand what information on local products is being requested by school districts.

19. Have you been approached by school districts for reports of volume or price of locally sourced products?

DEMOGRAPHICS

20. Please estimate the total number of school districts with whom you work.

21. How many local individual producers, such as farmers, ranchers, or fishers do you work with?

22. Thinking about your company’s definition of “local,” what is the estimated percentage of your total school district sales that were local food in 2018?

Appendix C. Pretest Version of Distributor Interview

P retest Version of Distributor Interview

Farm to School Census and Comprehensive Review

April, 2018

2019 FARM TO SCHOOL DISTRIBUTOR INTERVIEW - PRETEST

INTRO: Hello, my name is and I am calling on behalf of the USDA (United States Department of Agriculture) Food and Nutrition Service (FNS). I’m with Abt Associates, a firm that has been hired by USDA/FNS to conduct research with distributors of school food to learn more about the supply chain for local foods in school meals and whether or not farm to school has influenced distributor practices and operations.

[READ FIRST CONTACT ONLY]

SCREEN_SFA: First, does your organization distribute food to schools, school districts, or school food authorities?

 YES  CONTINUE

 NO  THANK AND TERMINATE

 DON’T KNOW  “May I speak with someone who would know?”

SCREEN_R: Next, I want to be sure I’m speaking with the person who is most knowledgeable about local products purchased by schools and school districts. Is that you?

 YES  EMAIL

 NO  “May I please speak with that person?” COLLECT NEW R INFORMATION. GO TO NEW_R

NEW_R: Hello, my name is _____ and I am calling from Abt Associates on behalf of the USDA (United States Department of Agriculture) Food and Nutrition Service (FNS). Abt Associates has been hired to conduct research with distributors of school foods to learn more about the supply chain for local foods in school meals and whether or not farm to school (Note to interviewer, you may define further if needed.) has influenced distributor practices and operations. I was just speaking with _____ about this study, who suggested that you were the best person to speak with on this topic. Would you have the time to discuss now?

IF NO: I would like to send you an email with more information about the study and schedule another time to talk.

May I please have your email address?

COLLECT EMAIL:

Thank you. I will call you again to schedule a time for the interview.  TERMINATE

EMAIL: Recently we sent you an email describing the study. Have you received the email?

 YES  SKIP TO Y_EMAIL

 NO  CONTINUE TO N_EMAIL

 DON’T KNOW  CONTINUE TO N_EMAIL

N_EMAIL: I will resend the email immediately following this call. May I confirm your email address as
? COLLECT/CORRECT EMAIL IF NEEDED

Thank you. I will call you again to schedule a time for the interview.  TERMINATE

Y_EMAIL: As the email stated, you have been selected by the United States Department of Agriculture’s Food and Nutrition Service (USDA/FNS) and the Farm to School Advisory Panel to participate in the 2019 Farm to School Distributor Interview. This interview is being conducted with a group of only 20 food distributors to capture your experience with the process and challenges associated with providing local foods to schools. The interview will ask questions about local foods you’ve supplied to school districts, and last approximately 60 minutes. When is the best time to complete this interview?

 NOW  CONTINUE

 SCHEDULE APPOINTMENT AND TERMINATE

Before we begin, I am required to let you know that all information gathered from food distributors is for research purposes only and will be kept private to the full extent allowed by law. Findings will be made public, but responses will be grouped with those of other study participants, and no individual food distributors will be identified. Participation in this study will not affect any reimbursements, credits, or foods your company receives through USDA programs.

[IF ASKED: Once complete, the report will be posted on the website for the FNS Office of Policy Support. I can give you that web address if you like. IF YES: https://www.fns.usda.gov/ops/research-and-analysis]

Permission to Record: In order to ensure that we accurately capture the points raised during this interview, we would like to digitally record this conversation. Please note that the interviews will remain private. Your identity and any information attributable to you will not be released to anyone outside of the research team and the recording of your interview will be deleted at the end of the study, after all data have been analyzed. May I start recording now?

 PERMISSION GRANTED  START RECORING NOW

 PERMISSION DENIED  ”That’s okay, I can continue without recording.”

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx (expires xxx). The time required to complete this information collection is estimated to average 60 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. This information collection is conducted under the authority of 7 U.S.C. 427.

Distributor Company Name
Interview Date
Time Start
Time End

I’d like to confirm your contact information to make sure what we have is accurate:

Distributor Company Name
R’s Name
R’s Title
R’s Phone
R’s Email
R’s Location Address
R’s Overall Job Duties

The focus of this interview is on sales, and potential shifts in demand for local products to school districts. For this interview, when I say “school districts” I mean districts, schools, or School Food Authorities (SFAs). I understand that you may also do business with other companies and institutions. For this interview, please focus only on your relationship with school districts unless otherwise specified.

DEFINITION OF LOCAL

2. Please describe in detail the way, or ways, the word “local” is defined geographically for procuring and marketing local food products and local food producers to school districts. (RQ Obj 1a)

[PROBE: examples include: same city or county; produced within a 50 mile radius of your warehouse, a specific school district, or jurisdiction; within a 100 mile radius; within a 200 mile radius; produced within the State; produced within a Region]

[PROBE IF R ANSWERS “REGION”: How do you define Region? Do you code products by State or ZIP?]

[PROBE: How did your company arrive at this definition of local? [IF NECESSARY: Does your company have one definition, have one definition for “school districts,” or does it vary by client?]

3. Do you provide any foods to school districts from local producers? [IF NECESSARY: by “local” we mean any way you, your company and/or your school district clients define local] (RQ Obj 1 d,f)

 YES  SKIP TO “LOCAL” (AFTER THE BOX BELOW)

 NO  CONTINUE TO Q3a BELOW

QUESTIONS FOR DISTRIBUTORS WHO DO NOT PROVIDE FOODS FROM LOCAL PRODUCERS:

3a. Do you get requests from school districts for local foods? If so, do school districts have defined local preferences? (RQ 1d)

3b. Please describe the reasons why you have not included local producers in your supply chain for school districts. (RQ 1d)

THANK YOU AND END INTERVIEW

LOCAL: For the remainder of this interview, when I say “local foods” I’m referring to the definition you just provided. [READ ANSWER IN QUESTION 2 ABOVE] INTERVIEWER: USE QUESTION 2 ANSWER TO DEFINE “LOCAL” THROUGHOUT.

REQUESTS FOR LOCAL FOODS & CONTRACTS

4. How frequently do you get specific requests for local foods from the school districts? (RQ Obj 1h)

4a. In what ways are these requests for local foods communicated by school districts? [PROBES: Is this done formally, through an RFQ (request for quote) or more informally (e.g., market basket purchases), or both? Are these requests codified in the school district’s procurement contract language?] (RQ Obj 1h)

4b. What types of requests (Note to interviewer, this question focuses on whether the request is codified in the procurement language) related to local foods do you receive? (RQ Obj 1f)

5. This study is interested in what happens when your company receives a request from a school district for local foods. For the next questions, please think about how your company has responded to requests for local foods, or how you would respond to requests for local foods.

5a. What local foods have typically been asked for by school districts? How have you been able to meet these requests? [PROBES: Fruits, vegetables, meat, seafood, grains, etc.? whole or processed product? (RQ Obj1 h)

5b. For which products (is it easiest/would it be easiest) to source locally? Why? (RQ Obj1 h)

5c. For which products (is it most challenging/would it be most challenging) to source locally? For those that are most challenging, what are the biggest challenges? (RQ Obj1 j)

5d. IF THEY HAVE NOT RECEIVED REQUESTS FOR LOCAL FOODS: If you were to respond requests for local foods, what business practices would you need to implement to fulfill these requests? [Changes may include: seeking out more local producers; providing different information to potential buyers; developing a local order/availability guide; offering reports on local expenditures; answering different RFPs/IFPs; or any other changes.] (RQ Obj1 d)

5e. IF THEY HAVE RECEIVED REQUESTS FOR LOCAL FOODS: Have requests for local foods from school districts led to changes in overall company practices? [IF NECESSARY: For example, does your company now seek out more local producers? Did you hire new staff? Are you now providing different information to your potential buyers? Do you answer different RFPs/IFPs than before? Did you develop local order/availability guides? Do you offer local expenditure reports? Are there any other changes in how you engage with school districts or local producers based on local food requests?] (RQ Obj1 d)

6. What types of solicitations or procurement methods do school districts use to purchase local foods from your company? [PROMPT IF NECESSARY: Such as: invitation for bid, request for proposals, information solicitation, or micro-purchase?] (RQ Obj1 j)

6a. [For interviewer] For contract types mentioned, please check the following:

Fixed-price contracts
Fixed-price with economic price adjustment contracts
Cost-reimbursable contracts (no fixed fee)
Cost-reimbursable with fixed-fee contracts
Forward contracts

VALUE IN PROVIDING LOCAL & MARKETING

7. Do you market or promote your company’s ability to sell “local” products to school districts? IF YES: please describe how. Is this part of a bigger initiative towards providing/promoting local foods at your company? IF YES: please describe. [E.g., do you offer educational programs or farmer visits?] (RQ Obj1 d)

8. Does your company see value in providing local products to school districts? Why or why not? [Probe: e.g., increased sales/customers?] Have you seen a ROI from your local foods program?

9. Have you noticed changes in how farmers grow or process products? (RQ Obj1 b)

10. Do you feel that providing local items gives your company a competitive advantage with school districts? Do you promote or advertise this capability? If so, how? (RQ Obj1 f)

11. Does your company have any goals regarding what percentage of local products it makes available to school districts? (RQ Obj1 k)
IF YES: Please describe these goals.

SUPPLY CHAIN & TOP 5 FOODS

12. In order to supply local foods to school districts, have you had to establish new supply chains to connect with local producers? (RQ Obj1 j)

 YES  CONTINUE TO QUESTION 12a.

 NO  SKIP TO QUESTION 12b.

 REFUSED  SKIP TO QUESTION 12b.

 DON’T KNOW  SKIP TO QUESTION 12b.

12a. If YES, please describe the process of establishing new supply chains. (RQ Obj1 j)

12b. What is the process for sourcing local foods for the school districts you serve? (RQ Obj1 d, g)

13. Do you have staff specifically dedicated to working with local vendors to procure local foods on the supply side? Do you spread out local sales across multiple sales representatives (e.g., schools, healthcare, restaurants, etc.) or do you have one designated local foods representative? (RQ Obj1 d)

14. Describe your general vendor requirements. Do these differ for local vendors (e.g., food safety, growing practices, packaging, storage, etc.?)

15. Do you often change vendors or recruit new vendors? Where/how do you find local vendors? (RQ Obj1 d, j)

16. What are the biggest factors that determine whether or not you would bring in a local product for a school customer? From your perspective, what barriers exist?

17. What would motivate your company to provide more local products to schools? [PROBE: Think about factors that would encourage you to find new local vendors, or expand your current supply chains if school districts showed more interest in purchasing local foods.]

18. What are the top 5 local food items, by volume, your company sold to school districts in 2018? Please be as specific as possible. For example: sliced apples, whole apples, chicken drumsticks, bags of lettuce, fish. (RQ Obj1 h)

19. What are the top 5 local food items, by dollar sales, your company sold to school districts in 2018? Please be as specific as possible. (RQ Obj1 h)

20. Would you say the cost of these local items is more, less, or the same, as the non-local commercially purchased alternative? (RQ Obj1 h)

CHANGES OVER TIME

21. Based on your definition of local, has the number of local producers you work with increased, decreased, or remained the same over the past five years? (RQ Obj1 d)

 INCREASED

 DECREASED

 REMAINED THE SAME  SKIP TO QUESTION 23

 DON’T KNOW  SKIP TO QUESTION 23

22. From your perspective, what are the top 3 factors driving that change?

23. Based on your definition of local, has the volume of local sales to school districts increased, decreased, or remained the same over the past two years? (RQ Obj1 h)

 INCREASED

 DECREASED

 REMAINED THE SAME  SKIP TO QUESTION 25

 DON’T KNOW  SKIP TO QUESTION 25

24. What are the top 3 factors driving that change?

25. How do you think farm to school efforts may have affected purchasing trends in school food? [Probe for changes in grade, aesthetics, value-added of foods purchased.] (RQ Obj2 h)

REPORTING

FNS is interested in collecting more accurate and hard data on local food purchases by school districts nation-wide in future studies. The following questions are asked so we can better understand what information on local products is being requested by and reported to school districts, as well as whether this information is tracked by your organization.

26. Have you been approached by school districts for reports of volume or price of locally sourced products? Sometimes these are called “velocity reports.” How do you track velocity information, if you provide these reports to school districts? (RQ Obj1 g)

 YES  CONTINUE TO QUESTION 27

 NO  SKIP TO QUESTION 28

 REFUSED  SKIP TO QUESTION 29

 DON’T KNOW  SKIP TO QUESTION 29

QUESTIONS FOR DISTRIBUTORS WHO HAVE RECEIVED REPORT REQUESTS FROM SCHOOL DISTRICTS

27. When these types of reports are requested by school districts, are you able to provide them? [IF NECESSARY: Reports on the volume and/or price of locally-sourced products.] [IF NO PROBE:
Why not?]

SKIP TO QUESTION 29

QUESTIONS FOR DISTRIBUTORS WHO HAVE NOT RECEIVED REPORTING REQUESTS FROM SCHOOL DISTRICTS

28. What type of information or reporting do you provide for school districts on locally sourced products? [PROBE: Please describe in detail the elements of these reports. Examples could include volume, list of local items, and price.] [PROBE: Even if you do not regularly provide information on locally sourced products to school districts, what information would you be able to provide if requested?]

DEMOGRAPHICS

29. Please estimate the total number of school districts [IF NECESSARY: also include schools and School Food Authorities] with whom you work. (RQ Obj1 g)

29a. TOTAL NUMBER OF school districts: . Do you have an estimate of how many of these are urban, suburban and rural?

29b. TOTAL NUMBER OF States: ___________________

Thinking about your definition of “local,” how many of the following do you directly source from?

30. How many local individual producers, such as farmers, ranchers, or fishers? (RQ Obj1 b)

NUMBER OF Farmers:  DON’T KNOW

NUMBER OF Ranchers:  DON’T KNOW

NUMBER OF Fishers:  DON’T KNOW

31. How many local producer cooperatives (may be called “food hubs”), including farmer, rancher, or fisher cooperatives?

NUMBER OF COOPERATIVES:  DON’T KNOW

32. How many local processors that process local foods? If your company processes local foods, please include yourself in this number.

NUMBER OF PROCESSORS:  DON’T KNOW

33. What was your company’s estimated total sales for all customers in 2018? Please include all customers, and not just school districts. (RQ Obj1 b)

TOTAL SALES ($)

34. What percentage of your company’s total sales was accounted for by sales to school districts in 2018? (RQ Obj1 b)

PERCENTAGE TO school districts (%)

35. Thinking about your company’s definition of “local,” what is the estimated percentage of your total school district sales that were local food in 2018? [IF NECESSARY: For this question, the term “local” is defined as ANSWERS TO QUESTION 4] (RQ Obj1 b)

PERCENTAGE LOCAL SALES (%)

36. Is there anything else you would like to tell us about farm to school or local food sourcing? We are particularly interested in:

36a. From your perspective, what action(s) could schools and/or school districts take to make it easier to fulfill their local (product) requests?

36b. What action(s) could farms or farm associations do to build better business relationships with distributors?

36c. What resources, if any, could your company (or the industry) use to better meet local product requests?

CLOSING

Thank you for completing the Distributor Interview!

IF NECESSARY: If you have any additional questions or comments, please feel free to contact our project team, toll-free, at XXX-XXX-XXXX or by email at [email protected].

Appendix D. Distributor Interview Pretest Debrief Form

FARM TO SCHOOL DISTRIBUTOR INTERVIEW DEBRIEF QUESTIONS

Distributor
Date of Interview
Name of Respondent
Title of Respondent
Contact Information
Start time of Interview
End time of Interview
Significant notes regarding Interview

During the interview, make notes on the interview guide regarding delivery, comprehension and response issues for reference during the de-briefing:

Delivery Issues	R asks for Q repeat
Delivery Issues	R ask for Q clarification

Comprehension Issues	R seems to misunderstand what is being asked

Response Issues	R had difficulty providing a response
	R’s answer doesn’t seem to fit the Q
	R declines to answer Q

How Responded	R based answer on her/his own knowledge
	R doesn’t know answer/would have to ask someone else
	R based answer on specific data/report

Reactions	R verbally reacted to a specific Q
Reactions	R had non-verbal reactions to Q (such as a long pause)

De-briefing Questions

Thank you for completing the pre-test interview. Now, I am going to ask you some questions about the interview.

Please tell me what your overall impression of the questions is (use relevant probes from notes taken during interview).

What did you think about the order of the questions?

(If there were clarification requests by R). You asked for [note what R asked] for Question xx. Can you tell me more about what you were thinking or how you think the question may be improved?

(If applicable). I noticed you hesitated at Question xx, could you say more about what made you hesitate?

(If R seemed confused). What was it about Question xx that was confusing? What information did you think was being asked? How could the question be improved so it is clearer about the information we sought?

(If R could not answer). For Question xx, you said you couldn’t answer it. Could you please say more about that?

(If R refused to answer). For Question xx, you did not answer it. Would you mind letting us know why you refused to answer? (Note if proprietary information)

Which questions were the easiest to answer? What made them so?

Which questions were more difficult to answer? What made them so?

Is there anything you would like to add you feel would improve the interview process overall?

Appendix E. Pretest Verison of Census

2019 FARM TO SCHOOL CENSUS PRE-TEST QUESTIONNAIRE

Welcome to the 2019 Farm to School Census Pre-Test!

We know this is a busy time of year, and we deeply appreciate your time.

The 2019 Farm to School Census is being conducted by the United States Department of Agriculture (USDA), Food and Nutrition Service (FNS) and is being administered by Abt Associates. This Census will provide updated information about how school food authorities (SFAs) conduct farm to school activities, such as purchasing locally-produced food for school meal programs, taking students to farms, bringing farmers into classrooms, teaching students culinary skills, adopting food and agricultural curricula, and cultivating edible school gardens.

Before you begin, a few important reminders:

Please complete the questionnaire even if you think you do not participate in farm to school, as farm to school activities are widely defined and vary based on school district participation.

Each school district or SFA must complete a separate Farm to School Census questionnaire. If you have multiple schools in your school district, only complete ONE questionnaire for the entire school district.

The questionnaire should average 30 minutes if you are involved with farm to school and only 5 minutes if you are not. Some questions may require that you coordinate with other school staff or partners.

A few questions ask for the total value of food purchases during school year 2016-2017, and how much (in dollars) of those purchases were locally-produced. We recommend you obtain your purchase records before starting the Census.

The questionnaire does not include any personal information. Participation in this study will not affect any reimbursements, credits, participation in or foods received through USDA programs.

If you need assistance completing the questionnaire, please call a toll free help line 1-866-778-1316 or email [email protected].

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is xxxx-xxxx (expires xxx). The time required to complete this information collection is estimated to average 30 minutes per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. This information collection is conducted under the authority of 7 U.S.C. 427.

ID: [ID#]
Start time:	End time:

School district/school identification.

Email address of respondent:

SFA (Please use complete name without abbreviations.):

State or territory:

5 digit zip code for SFA mailing address:

ID number assigned to your SFA by your State Agency (If known.):

Farm to school activities generally center around the procurement of local or regional foods and/or food, agriculture, or nutrition-based educational activities. These activities would include but are not limited to:

Serving or promoting local food products in the cafeteria (meals and snacks), in the classroom, or elsewhere at school (For example, School Breakfast Program (SBP), the National School Lunch Program (NSLP), the Fresh Fruit and Vegetable Program (FFVP), afterschool snacks or suppers through both the Child and Adult Care Food Program (CACFP) and NSLP and through food distribution programs like USDA Foods and USDA Department of Defense (DoD) Fresh Fruit and Vegetable Program (USDA DoD Fresh))

Conducting educational activities related to local foods, including bringing farmers and producers to the schools, taste testing, integration of food local items, and including agricultural lessons in classroom curricula, and culinary education focused on local foods

Taking field trips to farms, farmers' markets, or food processing facilities

Educational sessions for parents and community members

Creating, tending, or harvesting from school gardens (growing edible fruits, vegetables, and herbs), conducting lessons in and about the school gardens

Based on the definition above, did your SFA participate in farm to school activities during the 2016-2017 school year? (Please choose one.)

Yes
No, but started activities in 2017-2018 school year
No, but plan to start activities in the future
No activities currently and no plans for the future
We used to do farm to school activities but no longer do so

SCHOOL INFORMATION

How many meals does your SFA serve per week?

Don’t know

What percent of meals are made from “scratch”?

Don’t know

What percent of your SFA is on free or reduced lunch?

Don’t know

To the best of your knowledge, approximately how many schools in your SFA had salad bars during the 2016-2017 school year?

Don’t know

If the answer to Question 7 >=1, go to Question 8. If the answer to Question 7 is “0”, go to Question 9.

How many of the salad bars serve local products?

Don’t know

How long has your SFA been conducting farm to school activities?

Less than 3 years
3-5 years
6-10 years
More than 10 years
I don’t know

In which programs does your SFA participate? (Please choose all that apply.)

School Breakfast Program
National School Lunch Program
Fresh Fruit and Vegetable Program
CACFP (e.g., in a Pre-K setting)
CACFP At-Risk Afterschool (snacks and/or supper)
Summer meals (i.e., meals in the Summer Food Service Program, in Seamless Summer, or in the NLSP under accredited summer school programs)
None of these

If the answer to Question 2 is “We used to do farm to school activities but no longer do so”, please proceed to this question:

Why are you no longer participating in farm to school activities? (Please choose all that apply.)

Lack funds to operate/conduct farm to school activities
No staff available to lead farm to school activities
No interest in farm to school activities
Don’t see the benefits of farm to school activities
Don’t have buy-in from district management
Lack of administrative support
Unable to find vendors, farmers, and /or producers that provide local foods
The cost of purchasing local products is too high
Other reason: (please specify)
I don’t know

PLEASE SKIP TO QUESTION 13 ON PAGE 9

If the answer to Question 2 is “No, but started activities in 2017-2018 school year”, please proceed to these questions:

What activities are you starting this school year (2017-2018)? (Please choose all that apply.)

FOOD

Serving locally produced foods in the cafeteria as part of school lunch or breakfast
- Serving locally produced foods as a snack (in the classroom, sold a la carte, as fundraisers, etc.)
- Serving locally produced foods or providing farm to school activities as part of afterschool programs
- Serving locally produced foods as part of the Fresh Fruit and Vegetable Program
- Serving locally produced foods as part of USDA DoD Fresh
- Serving products from school-based or district-based gardens/farms in any school meal, including summer
- Holding taste tests/cooking demonstrations of locally-produced or garden-grown foods in the cafeteria, classroom, or other school-related setting
- Working with local food producers to develop specific food products using local foods

EDUCATION

Implementing strategies to encourage student selection and consumption of locally produced foods (e.g., product placement, food prompts, creative signage, etc.)
Using cafeteria food coaches to promote the consumption of local foods (e.g. adults or students in the cafeteria encouraging kids to eat healthy/local foods)
Using USDA Team Nutrition materials (such as The Great Garden Detective Adventure or Dig In!) as part of taste testing or educational activities
Conducting educational edible school garden as part of a school curriculum
Conducting educational edible school garden as part of a summer program
Conducting educational edible school garden as part of an afterschool program
Conducting student field trips to farms, farmers’ markets, producers, processors
Having farmer(s) visit the cafeteria, classroom or other school-related setting
Integrating farm to school activities (such as gardening and local foods education) into Pre-Kindergarten curriculum
Providing training to school food service staff on farm to school or school gardens

PROMOTION

Promoting local efforts through themed or branded promotions (e.g. Harvest of the Month, Local Day, Taste Washington Day, etc.)
Promoting locally produced foods at school in general (e.g. via cafeteria signs, posters, newsletters, etc.)
Generating media coverage of local foods being used in schools (e.g. press interviews or other activities that resulted in local coverage)
Hosting farm to school related family and community events (e.g. invite parents to lunch, corn shucking contests, etc.)
Celebrating National Farm to School Month (October 2017)
Hosting special local foods events in CACFP and SFSP

OUTCOMES

Evaluating the impact of farm to school activities (e.g., measuring changes in food waste, student acceptance of local items, changes in participation rates)

OTHER ACTIVITIES

Other: (please specify)
Nothing
I don’t know

Which of the following do you feel has resulted from your participation in farm to school activities? (Please choose all that apply.)

Reduced food waste
Lower school meal program costs
Greater acceptance of items served in school meals
Increased participation in school meals
Greater community support for school meals
Cooking more food from “scratch,” like peeling and cutting up fruits and vegetables, measuring out raw ingredients, adding seasonings
Increased kitchen staff satisfaction
Better quality foods
Increased student knowledge about local and healthful foods
Increased student knowledge about how to grow food
Increased student consumption of local foods
Increased student engagement in other food activities outside of the school (e.g., community service and food access projects)
Increased student experiential learning opportunities
Increased professional skill building or job training on the part of students
Other: (please specify)

Nothing

I don’t know

Are any of the following considered to be challenges in procuring local products or reasons why your district does not purchase local products? (Please choose all that apply.)

VENDORS

Local producers aren’t bidding on Invitations for Bids (IFBs) or Requests for Proposals (RFPs)
Limited availability of local items due to seasonality
Difficult to coordinate procurement of local items with non-local items/Local items not available from primary vendors
Local items are not identified or marked as local by distributor/vendor
Vendors of local items don't offer a broad range of products
Lack of availability of precut/processed products
Difficult to find local producers, suppliers, and distributors
Local items are not identified or marked by distributor/vendor

Don’t always receive ordered items
Inconsistent on-time deliveries
Difficult to get product that meets quality requirements & other specifications (e.g., size)
Producers unable to meet food safety requirements (e.g. Good Agricultural Practices)
Local vendors unable to meet delivery needs
Producers unable to supply requested volume of local items

PRICE AND PURCHASING

Local products are more expensive than conventional products
Unstable product prices
School/district payment procedures do not align with farmers' needs

STAFF/ KITCHEN

Lack of kitchen equipment to process/prepare local foods

Lack of skilled/trained staff to prepare local foods
Lack of staff time or interest in preparing local foods
Unclear on how to apply the geographic preference option
Unclear on how to apply to write specifications targeting local foods

OTHER CHALLENGES

Other: (please specify)

How are you tracking the benefits and/or challenges of your new farm to school activities?

Spreadsheets
Reports from teachers/staff
We don’t formally track
Other: (please specify)

How does your district define "local" for the majority of its school food procurement? (Please choose one.)

Produced within a 20 mile radius END SURVEY
Produced within a 50 mile radius END SURVEY
Produced within a 100 mile radius END SURVEY
Produced within a 200 mile radius END SURVEY
Same county END SURVEY
Produced within the State END SURVEY
Produced within the region PROCEED TO QUESTION 16
Other: (please specify) END SURVEY
Our definition of local varies dependent upon the product SKIP TO QUESTION 17

We don’t have a set definition for local END SURVEY

Don’t know END SURVEY

What do you mean by “region”?

Other: (please specify) END SURVEY

For your most common definition of local, what radius does that cover?

Produced within a 20 mile radius
Produced within a 50 mile radius
Produced within a 100 mile radius
Produced within a 200 mile radius

END SURVEY END TIME: ____________________

Thank you for completing the 2019 Farm to School Census Pre-Test! For more information on starting or expanding farm to school activities, please visit the FNS Office of Community Food Systems’ website at https://www.fns.usda.gov/farmtoschool/farm-school

Questions or comments? Please contact Abt Associates at [email protected].

If the answer to Question 2 is “No, but plan to start activities in the future”, please skip to these questions:

What activities are you planning to start in the future? (Please choose all that apply.)

FOOD

Serving locally produced foods in the cafeteria as part of school lunch or breakfast
- Serving locally produced foods as a snack (in the classroom, sold a la carte, as fundraisers, etc.)
- Serving locally produced foods or providing farm to school activities as part of afterschool programs
- Serving locally produced foods as part of the Fresh Fruit and Vegetable Program
- Serving locally produced foods as part of USDA DoD Fresh
- Serving products from school-based or district-based gardens/farms in any school meal, including summer
- Holding taste tests/cooking demonstrations of locally-produced or garden-grown foods in the cafeteria, classroom, or other school-related setting
- Working with local food producers to develop specific food products using local foods

EDUCATION

Implementing strategies to encourage student selection and consumption of locally produced foods (e.g., product placement, food prompts, creative signage, etc.)
Using cafeteria food coaches to promote the consumption of local foods (e.g. adults or students in the cafeteria encouraging kids to eat healthy/local foods)
Using USDA Team Nutrition materials (such as The Great Garden Detective Adventure or Dig In!) as part of taste testing or educational activities
Conducting educational edible school garden as part of a school curriculum
Conducting educational edible school garden as part of a summer program
Conducting educational edible school garden as part of an afterschool program
Conducting student field trips to farms, farmers’ markets, producers, processors
Having farmer(s) visit the cafeteria, classroom or other school-related setting
Integrating farm to school activities (such as gardening and local foods education) into Pre-Kindergarten curriculum
Providing training to school food service staff on farm to school or school gardens

PROMOTION

Promoting local efforts through themed or branded promotions (e.g. Harvest of the Month, Local Day, Taste Washington Day, etc.)
Promoting locally produced foods at school in general (e.g. via cafeteria signs, posters, newsletters, etc.)
Generating media coverage of local foods being used in schools (e.g. press interviews or other activities that resulted in local coverage)
Hosting farm to school related family and community events (e.g. invite parents to lunch, corn shucking contests, etc.)
Celebrating National Farm to School Month (October 2018)
Hosting special local foods events in CACFP and SFSP

OUTCOMES

Evaluating the impact of farm to school activities (e.g., measuring changes in food waste, student acceptance of local items, changes in participation rates)

OTHER ACTIVITIES

Other: (please specify)
Nothing
I don’t know

When do you plan to start conducting farm to school efforts?

Later this school year (2017- 2018)
Next school year, 2018-2019
The following school year, 2019-2020
A few years from now, past 2019-2020 year

Not sure when we will start

END SURVEY END TIME:_________________

Thank you for completing the 2019 Farm to School Census Pre-Test! For more information on starting or expanding farm to school activities, please visit the Office of Community Food Systems’ website at https://www.fns.usda.gov/farmtoschool/farm-school

Questions or comments? Please contact Abt Associates at [email protected].

If the answer to Question 2 is “No activities currently and no plans for the future”, please skip to these questions:

Why does your SFA choose not to participate in and have no plans to participate in farm to school activities? (Please choose all that apply.)

VENDORS

Local producers aren’t bidding on Invitations for Bids (IFBs) or Requests for Proposals (RFPs)
Limited availability of local items due to seasonality
Difficult to coordinate procurement of local items with non-local items
Local items not available from primary vendors
Local items are not identified or marked as local by distributor/vendor
Vendors of local items don't offer a broad range of products
Lack of availability of precut/processed products

Difficult to find local producers, suppliers, and distributors

Local items are not identified or marked by distributor/vendor

Don’t always receive ordered items
Inconsistent on-time deliveries
Difficult to get product that meets quality requirements & other specifications (e.g., size)
Producers unable to meet food safety requirements (e.g. Good Agricultural Practices)
Local vendors unable to meet delivery needs
Producers unable to supply requested volume of local items

PRICE AND PURCHASING

Lack funds to operate/conduct farm to school activities
Lack of administrative support
Don’t have buy-in from district management
Local products are more expensive than conventional products
Unstable product prices
School/district payment procedures do not align with farmers' needs

STAFF/ KITCHEN

No staff available to lead farm to school activities
Lack of kitchen equipment to process/prepare local foods

Lack of skilled/trained staff to prepare local foods
Lack of staff time or interest in preparing local foods
Unclear on how to apply the geographic preference option.
Unclear on how to apply to write specifications targeting local foods

OTHER CHALLENGES

No interest in farm to school activities
Don’t see the benefits of farm to school activities
Other reason: (please specify)

I don’t know

END SURVEY END TIME:_________________

Questions or comments? Please contact Abt Associates at [email protected].

If the answer to Question 2 is “Yes”, please skip to these questions:

How does your SFA handle food service?

Self-operated
Food Service Management Company (FSMC)
Vended meals
Other (please specify)

How does your SFA handle food preparation? (Please check all that apply.)

Central kitchen
Regional kitchens
Receiving (Satellite) kitchen
Production kitchens
Other (please specify)

During the 2016-2017 school year, what grades participated in farm to school activities? Please keep in mind that farm to school activities include gardens, food and agricultural education, and procurement; you don’t have to directly participate in a garden of some sort. (Please choose all that apply.)

Pre-K
K through 5^th grade
6^th grade to 8^th grade
9^th grade to 12^th grade
I don’t know

How does your district define "local" as it relates to procuring school food? (Please choose one.)

Produced within a 20 mile radius SKIP TO QUESTION 27
Produced within a 50 mile radius SKIP TO QUESTION 27
Produced within a 100 mile radius SKIP TO QUESTION 27
Produced within a 200 mile radius SKIP TO QUESTION 27
Same county SKIP TO QUESTION 27
Produced within the State SKIP TO QUESTION 27
Produced within the region PROCEED TO QUESTION 25

Other: (please specify) SKIP TO QUESTION 27
Our definition of local varies dependent upon the product SKIP TO QUESTION 26
We don’t have a set definition for local SKIP TO QUESTION 27
I don’t know SKIP TO QUESTION 27

What do you mean by “region”?

Other: (please specify) SKIP TO QUESTION 27

For your most common definition of local, what radius does that cover?

Produced within a 20 mile radius
Produced within a 50 mile radius
Produced within a 100 mile radius
Produced within a 200 mile radius

Which of the following do you feel has resulted from your participation in farm to school activities? (Please choose all that apply.)

Reduced food waste
Lower school meal program costs
Greater acceptance of items served in school meals
Increased participation in school meals
Greater community support for school meals
Cooking more food from “scratch,” like peeling and cutting up fruits and vegetables, measuring out raw ingredients, adding seasonings
Better quality foods
Other: (please specify)
Nothing

I don’t know

How are you tracking the benefits of your new farm to school activities?

Spreadsheet
Reports from teachers/staff
We don’t formally track
Other (please specify)

To the best of your knowledge, approximately how many schools within the SFA participated in any farm to school activities during the 2016-2017 school year?

To the best of your knowledge, approximately how many of the schools participating in farm to school activities during the 2016-2017 school year are serving local food of any kind?

To the best of your knowledge, approximately how many of the schools participating in farm to school activities during the 2016-2017 school year are providing food, nutrition, and agricultural education?

To the best of your knowledge, approximately how many schools in the SFA had edible school gardens during the 2016-2017 school year? Some states may track these data.

Don’t know

How does your SFA track the number of schools that had edible school gardens during the 2016-2017 school year?

Spreadsheet
Reports from teachers/staff
We don’t formally track
Other (please specify)

If the answer to Question 32 >=1, go to Question 34. If the answer to Question 32 is “0”, skip to Question 35.

How did schools use the harvest from the school gardens? (Please choose all that apply.)

Donated or sold to the cafeteria for inclusion in school meals programs
Donated or sold it to outside entities, like at a farmers’ market
Sent it home with the students
Served it in the classrooms or gardens as part of classroom or garden-based educational activities
Something else (specify)

I don’t know

Please indicate whether your district or any schools in your district used local products IN ANY FORM (i.e., fresh, frozen, dried, minimally processed fluid milk, and more) for any of the following federal nutrition programs during the 2016-2017 school year. (Please choose all that apply.)

School Breakfast Program
National School Lunch Program
Fresh Fruit and Vegetable Program
CACFP (e.g., in a Pre-K setting)
CACFP At-Risk Afterschool (snacks and/or supper)
Summer meals (i.e., meals in the Summer Food Service Program or through the Seamless Summer Option)
None of these

I don’t know

For the 2016-2017 school year, please indicate if your district procured local foods from any of the following sources. (Please choose all that apply.)

Individual food producers (i.e. farmers, ranchers, fishers), including but not limited to a Community Supported Agriculture (CSA) model
Cooperatives of farmers, ranchers or fishers
Farmers’ markets or roadside stands
School garden/farm
Community garden
Through a cooperative agreement with other SFAs
Distributors
Food buying cooperative (of school districts or other buyers)
Food hub
Grocery stores
USDA DoD Fresh Program vendors
USDA Foods
None of these
I don’t know

Please indicate if your district or any schools in your district purchased any of the following foods as local products – IN ANY FORM -- from local growers/producers/processers/manufacturers/distributors during the 2016-2017 school year or would like to in the future. (Please choose one answer per row.)

	Yes, currently purchase as local products	No, and have no plans to purchase these as local products	No, but would like to purchase as a local product in the future	I don’t know
Fruit
Vegetables
Fluid milk
Other dairy
Meat
Poultry
Eggs
Seafood
Plant-based protein items such as beans, seeds, nuts
Grains and flour
Bakery products
Herbs
Other product type: (please specify)

Please list the top 5 SPECIFIC food items (e.g. apples, chicken drumsticks) your school district purchased as local products in 2016-2017, based on TOTAL DOLLARS SPENT. (e.g. apples, chicken drumsticks, etc.)

	Name of item	Dollars Spent
Food item #1:
Food item #2:
Food item #3:
Food item #4:
Food item #5:

On average, about how frequently does your SFA’s meals or snacks include at least one locally sourced food item from the categories below?

	Daily	A few times per week	Weekly	A few times per month	Monthly	Occasionally	Never
Fruit
Vegetables
Fluid milk
Other dairy
Meat
Poultry
Eggs
Seafood
Plant-based protein items such as beans, seeds, nuts
Grains and flour
Bakery products
Herbs
Other product type: (please specify)

The following questions ask how much money you spent on food and local food during the 2016-2017 school year. Please give your best approximation and do not include the value of USDA Foods or USDA DoD Fresh (entitlement).

For the 2016-2017 school year, what were your approximate total food costs (in dollars)?

TOTAL food costs (Please round to the nearest dollar. Do not count USDA Foods/USDA DoD Fresh.):

What source did you use for your answer to the previous question (For the 2016-2017 school year, what were your approximate total food costs)?

Procurement records, including bid documents
Financial records/receipts
Budget
Estimated

For the 2016-2017 school year, about how much of your total food purchases (in dollars) were spent on LOCAL foods INCLUDING fluid milk? (If you're not sure, a rough estimate is perfectly acceptable.)

Total cost of local food, including fluid milk (Please round to the nearest dollar. Do not count USDA Foods/USDA DoD Fresh.):

What source did you use for your answer to the previous question (For the 2016-2017 school year, what were your approximate total LOCAL food costs, including fluid milk)?

Procurement records, including bid documents

Financial records/receipts
Budget
Estimated

For the 2016-2017 school year, about how much did you spend on local fluid milk (in dollars)? (If you're not sure, a rough estimate is acceptable.)

Costs of Local Fluid Milk (Please round to the nearest dollar.):

What source did you use for your answer to the previous question (For the 2017-2018 school year, how much did you spend on local fluid milk)?

Procurement records, including bid documents
Financial records/receipts
Budget
Estimated

For the 2016-2017 school year, what were your approximate local food costs (in dollars) for USDA DoD Fresh ONLY? (Do not count USDA Foods.)

Costs of Local food for USDA DoD Fresh ONLY (Please round to the nearest dollar.):

What source did you use for your answer to the previous question (For the 2016-2017 school year, what were your approximate local food costs (in dollars) for USDA DoD Fresh ONLY?)

Financial records/receipts
Budget
Fresh Fruits and Vegetables Order Receipt System (FFAVORS)
Estimated

To the best of your knowledge, please choose the activities that your district or schools engaged in during school year 2016-2017. (Please choose all that apply.)

FOOD

Serving locally produced foods in the cafeteria as part of school lunch or breakfast
- Serving locally produced foods as a snack (in the classroom, sold a la carte, as fundraisers, etc.)
- Serving locally produced foods or providing farm to school activities as part of afterschool programs
- Serving locally produced foods as part of the Fresh Fruit and Vegetable Program
- Serving locally produced foods as part of USDA DoD Fresh
- Serving products from school-based or district-based gardens/farms in any school meal, including summer
- Holding taste tests/cooking demonstrations of locally-produced or garden-grown foods in the cafeteria, classroom, or other school-related setting
- Working with local food producers to develop specific food products using local foods

EDUCATION

Implementing strategies to encourage student selection and consumption of locally produced foods (e.g., product placement, food prompts, creative signage, etc.)
Using cafeteria food coaches to promote the consumption of local foods (e.g. adults or students in the cafeteria encouraging kids to eat healthy/local foods)
Using USDA Team Nutrition materials (such as The Great Garden Detective Adventure or Dig In!) as part of taste testing or educational activities
Conducting educational edible school garden as part of a school curriculum
Conducting educational edible school garden as part of a summer program
Conducting educational edible school garden as part of an afterschool program
Conducting student field trips to farms, farmers’ markets, producers, processors
Having farmer(s) visit the cafeteria, classroom or other school-related setting
Integrating farm to school activities (such as gardening and local foods education) into Pre-Kindergarten curriculum
Providing training to school food service staff on farm to school or school gardens

PROMOTION

Promoting local efforts through themed or branded promotions (e.g. Harvest of the Month, Local Day, Taste Washington Day, etc.)
Promoting locally produced foods at school in general (e.g. via cafeteria signs, posters, newsletters, etc.)
Generating media coverage of local foods being used in schools (e.g. press interviews or other activities that resulted in local coverage)
Hosting farm to school related family and community events (e.g. invite parents to lunch, corn shucking contests, etc.)
Celebrating National Farm to School Month (October 2017)
Hosting special local foods events in CACFP and SFSP

OUTCOMES

Evaluating the impact of farm to school activities (e.g., measuring changes in food waste, student acceptance of local items, changes in participation rates)

OTHER ACTIVITIES

Other: (please specify)
Nothing
I don’t know

Have you encountered any of the following problems in procuring local products? (Please choose all that apply.)

VENDORS

Local producers aren’t bidding on Invitations for Bids (IFBs) or Requests for Proposals (RFPs)
Limited availability of local items due to seasonality
Difficult to coordinate procurement of local items with non-local items
Local items not available from primary vendors
Local items are not identified or marked as local by distributor/vendor
Vendors of local items don't offer a broad range of products
Lack of availability of precut/processed products
Difficult to find local producers, suppliers, and distributors
Local items are not identified or marked by distributor/vendor

Don’t always receive ordered items
Inconsistent on-time deliveries
Difficult to get product that meets quality requirements & other specifications (e.g., size)
Producers unable to meet food safety requirements (e.g. Good Agricultural Practices)
Local vendors unable to meet delivery needs
Producers unable to supply requested volume of local items

PRICE AND PURCHASING

Local products are more expensive than conventional products
Unstable product prices
School/district payment procedures do not align with farmers' needs

STAFF/ KITCHEN

Lack of kitchen equipment to process/prepare local foods

Lack of skilled/trained staff to prepare local foods
Lack of staff time or interest in preparing local foods
Unclear on how to apply the geographic preference option
Unclear on how to apply to write specifications targeting local foods

OTHER CHALLENGES

Other: (please specify)

Which distributors do you currently use to procure local food? (Please write in all that apply.)

Other 1: (please specify)
Other 2: (please specify)
Other 3: (please specify)
Other 4 (please specify)

Do you ask your vendors or distributors for reports that include any of the following items? (Please choose all that apply.)

Local order/availability guide PROCEED TO QUESTION 52
Origin of item PROCEED TO QUESTION 52
Volume of all local foods purchased/procured by that vendor or distributor PROCEED TO QUESTION 52
Price of all local foods purchased/procured by that vendor or distributor PROCEED TO QUESTION 52
Something else 1 (please specify PROCEED TO QUESTION 52
Something else 2 (please specify) PROCEED TO QUESTION 52
I have not asked for any of these SKIP TO QUESTION 54

Are the vendors or distributors able to provide the reports requested above?

Yes
No
I don’t know

How often do you request (or have you requested) this information?

Weekly
Monthly
Every few months
Yearly
Sporadically

I don’t know

How many staff, either full or part-time, are dedicated to farm to school efforts within your SFA/school/district? This includes purchasing, procurement, educational activities and gardens. Please only include paid staff, including any contracted and grant-funded staff. (Please include nutrition specialists, garden educators, AmeriCorps members, teachers, and any others who facilitate farm to school activities.)

a. Number of full time staff dedicated to farm to school efforts (Full time is defined as at least 40 hours in one week)
b. Number of part time staff who use some portion of their time for farm to school (Part time is defined as less than 40 hours in one week)

If the answers to Questions 54a or 54b >=1, proceed to Question 55. If the answer to Question 54a is “0” and the answer to Question 54b is “0”, skip to Question 57.

How are the above positions and farm to school activities funded? (Please choose all that apply.)

USDA Farm to School Grant
Other federal grants (e.g. CDC, USDA)
Local or state government grants
Private foundation grants (national or local) (please specify)
Corporate partnerships and donations
School/District funding such as PTA or a school/district foundation
Food and Nutrition Service funds
Individual donations
In-kind contributions
From the non-profit food-service account
Other (please specify):

I don’t know

What are the primary farm to school activities that farm to school staff members concentrate on? (Please choose all that apply.)

School garden growing and harvesting
Field trips to farms, orchards or processing facilities, including planning and/or bringing farmers to school for classroom visits
Menu development
Nutrition education
Conducting procurement for local products
Local product promotion
Sourcing new potential vendors
Developing new supply chains
Conducting trainings for staff on topics like gardening, local procurement, food safety
Something else (please specify)

I don’t know

Does your SFA have a farm to school network, task force or advisory board that promotes or assists with implementing farm to school activities?

I don’t know

From the list below, please select which, if any, of the following policies are currently in place at your SFA to support farm to school? (Please select all that apply.)

Wellness policies that support farm to school
Procurement policies that support the purchasing of local foods
Policies that support fundraising for farm to school activities
Budget allocations dedicated to farm to school efforts
No policies are in place to support farm to school activities
Don’t know

Other (please specify)

Is there other information on farm to school activities that was not asked for that you think we need to know?

Please share a local food procurement success story. We want to know about best practices or hear about amazing stories that feature partnerships or highlight community engagement. (Optional)

End time: _____________________

Questions or comments? Please contact Abt Associates at [email protected]

Appendix F. Census Pretest Debrief Form

Farm to School Census Pretest Debrief Form

Respondent name: __________________ Title:_______________________________

Phone number: _____________________ SFA: _______________________________

Debriefer name: ____________________ Date of debriefing: ___________________

Explain that you have a few general questions and allow them to provide additional comments as you progress through the questions.

How long did it take to complete? (If parts of the survey were completed by another staff member make note of the time they spent as well.)
What was your general impression: Was it easy/difficult? Were the questions reasonable/unreasonable?

3. What did you think about the order in which the questions were asked? Could this be improved? (Since survey will be programmed, we are not concerned about difficulties with the skip patterns.)

4. Were the questions clear? Were any of the items hard to answer or recall? Did you have access to the information? Did you have to look up information in records? If so, how long did that take?

5. Were there any words, terms, or definitions that you did not understand?

6. Any suggestions for improving any of the instructions, questions or response options?

Did you have to speak with other staff to answer a question? Who? Why? Did that pose any problems?
Is there anything you think we should have asked but didn’t?

Explain that this survey will be administered via web, not paper and part of what we are trying to determine is if the respondent foresees any issues with this being conducted that way.

If you were taking this as a web survey, how feasible would it be to have different staff complete it?
If you were sent this as a web survey, how willing would you be to complete it? Why?

Thank respondent for all the help.

1 https://www.fns.usda.gov/fns-regional-offices

Pretest Memorandum Farm to School Census and Comprehensive Review ▌pg. iv

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Abt Single-Sided Body Template
Author	Susan Bartlett
File Modified	0000-00-00
File Created	2021-01-20