Supporting Statement - Part B
Farm to School Census
OMB No. 0536-XXXX
B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection has been conducted previously, include the actual response rate achieved during the last collection.
The Farm to School Census questionnaire (see Attachment K) will collect information from food service directors in all public school district School Food Authorities (SFAs) participating in the National School Lunch Program on purchases of locally produced food for school meal programs, as well as other educational activities related to local foods.
The respondent universe for the Farm to School Census questionnaire includes all public school district SFA food service directors in the 50 States and the District of Columbia, numbering 13,629. The expected response rate, defined as the proportion of the universe that complete a questionnaire, is 77 percent. This expectation is based on results of the North Carolina Farm to School Survey, which used a questionnaire of similar length, and used a similar recruitment methodology.a
A sample of 100 non-respondents, out of an expected universe of 3,135 non-respondents, will be followed up by telephone in order to analyze non-response. The expected response rate for the follow-up calls, defined as the proportion of the follow-up sample that complete at least an abridged set of questions by phone, is 100 percent.
State Child Nutrition Directors will be asked for assistance in forwarding a memorandum to food service directors encouraging participation in the data collection (see Attachments L and M). Also, in order to construct a frame of SFAs for mapping SFAs with local purchasing programs and for follow-up and analysis of non-response, State Child Nutrition Directors will be asked to provide an electronic list of business contact information for all School Food Authorities in each State. The respondent universe for these requests includes all State Child Nutrition Directors from the 50 States and the District of Columbia, numbering 51 in total. The expected response rate for the request to forward the invitation to participate in the Census is 100 percent. The expected response rate for the request for business contact information for food service directors is 78 percent.
Respondent / Instrument |
Universe |
Sample |
Expected response rate |
State Child Nutrition Directors from 50 states and District of Columbia –
|
51 |
51 |
100%
78% |
School Food Authority Food Service Directors
|
13,629 3,135 |
13,629 100 |
77% 100% |
2. Describe the procedures for the collection of information including:
• statistical methodology for stratification and sample selection,
• estimation procedure,
• degree of accuracy needed for the purpose described in the justification,
• unusual problems requiring specialized sampling procedures
Sample selection
The Farm to School Census is a universe survey of public school district SFAs. All Food Service Directors in public school district SFAs in the 50 States and the District of Columbia will receive an invitation to participate in the survey.
A stratified sample of 100 non-respondents will be contacted by telephone and asked to respond to an abridged set of questions (see Attachment N). Four strata will be based on classification of SFAs by total enrollment in the SFA, the main variable expected to be associated with non-response. Data for total enrollment will be taken from the Common Core of Data, a data file compiled annually by the U.S. Department of Education’s National Center of Education Statistics and attached to the list frame (as described in further detail below) before the sample of non-respondents is drawn. The enrollment levels for the strata are the same used in the 2011 School Food Purchase Survey conducted by USDA/FNS:
Under 1,000;
1,000 – 4,999;
5,000 – 24,999;
25,000 and above.
The sample will be allocated to each stratum in proportion to the number of non-responding SFAs in the stratum.
In order to determine which Food Service Directors are non-respondents, completed questionnaires will be compared to a list of SFAs, developed from a combination of data from three sources:
Lists of public SFAs requested from State Child Nutrition Directors.
A list of public SFAs that submitted income verification reports to USDA’s Food and Nutrition Service (FNS).
A list of public school districts from the U.S. Department of Education’s National Center for Education Statistics Common Core of Data (CCD) for school year 2009-10.
Each of these lists is expected to be incomplete individually, but expected to be very nearly complete in combination. Some States may not be able to provide a list of SFAs in a timely fashion. The FNS list of SFAs does not include SFAs that were not required to submit verification reports, i.e. those with no Free or Reduced Price Meals (FARM), and some SFAs providing universal free meals under Provision 2, which requires income verification only during base years. The latest available list of public school districts from the CCD may not be completely up to date, and does not indicate which districts serve as SFA for smaller school districts. To create the best list possible, we plan the following procedure:
In States that are able to provide a list of SFAs, that list will be compared with data from the other two sources to reconcile any differences, and then assumed to be the best available list. If the State Child Nutrition Director is unresponsive to the request for a list, ERS will create a draft list of public SFAs for each nonresponsive state based on information from both FNS and CCD lists. The draft list will be revised through discussion with the State Child Nutrition Director to create the final list frame for these states. These States will be asked to assist with specific reconciliations of the FNS-CCD list, such as identifying any districts from the CCD that serve as SFAs to smaller school districts. Both the CCD records and FNS records have data on enrollment levels for use in stratification.
The list frame resulted from this study will be used to match to geocode boundary data for use in developing the map of SFAs by percent of food expenditures going to locally produced food. It will also be used to draw a sample for telephone follow-up of nonrespondents.
Recruitment
The Farm to School Census questionnaire (see Attachment K) will be disseminated electronically to all public school district School Food Authority (SFA) food service directors as part of an invitation to participate from the Food and Nutrition Service (FNS) Child Nutrition Division, through State Child Nutrition Directors (see Attachments L and M).
This invitation will be sent via the PartnerWeb system, a communication network that allows memoranda from the Food and Nutrition Service to be transmitted electronically to State Child Nutrition Directors. This is the only channel that will be used for initial contact with Food Service Directors and can be used even if State Child Nutrition Directors are unable to provide a list of SFAs. The USDA/FNS Regional Office Farm to School liaisons have agreed to assist in confirming that invitation emails are sent from each State through the FNS PartnerWeb system.
The request forwarded to SFA Food Service Directors will include a link to the questionnaire in SurveyMonkey (a survey design and collection software) as well as an attached version of the questionnaire that can be filled out by hand and returned by fax (see Attachment O). The request will also provide a telephone number respondents can call for assistance completing the questionnaire on-line.
The invitation letters and reminders have been worded to encourage participation by BIE and tribal schools based on comments from staff at the Arizona Department of Education.
Estimation, Mapping and Weighting
Three data products are planned:
A web-based map of all responding school districts showing access to locally produced foods through the school meal programs, categorized by percent of food expenditures that are sourced locally
Estimates of State-level prevalence of purchasing locally produced foods for school meal programs
Estimate of State-level profiles of local purchasing programs, i.e., the proportion of districts in categories based on responses to questions about local purchasing, including:
Age groups targeted for Farm to School activities
Definition of “local” as it relates to food procurement
Supply channels for local foods
Food categories sourced locally and those desired for the future
Top 5 specific foods source locally based on value
Frequency of offering locally sourced foods
Problems in procuring local products
Web-based maps will be generated using geocode boundary coordinates of school districts, available from the U.S. Census Bureau. Because SFAs can include more than one school district, district-level boundary data will be adjusted to create SFA boundaries for multi-district SFAs. Available data on SFAs from the Food and Nutrition Service is only available for SFAs that submit income verification reports and is thus not a complete list of SFAs. Therefore, in order to obtain a more complete list, we will ask State Child Nutrition Directors to submit a list of SFAs with business contact information for SFA Food Service Directors in each State. For States unable to provide this list, the FNS income verification report list will be used in combination with data from the Common Core of Data.
This list will be matched to geocode boundary coordinates from the U.S. Census Bureau and characteristics such as enrollment and percent of students receiving free and reduced price meals, from the Common Core of Data, a data file compiled annually by the U.S. Department of Education’s National Center of Education Statistics. Geocode boundary coordinates will be used to map SFAs purchasing locally produced foods.
To estimate State-level prevalence and profiles, responding SFAs will be assigned weights to account for nonresponse. Non-response weights will be developed based on a model of the probability of response as a function of SFA characteristics including enrollment and percent of meals that are free or reduced price. Responses from the non-respondent phone sample will be used to test whether SFAs that are not sourcing locally produced food are less likely to respond, as part of the analysis of non-response.
Degree of accuracy needed
The results of the Farm to School Census will be used as a baseline to assess the geographic distribution of SFAs already purchasing locally produced food for school meal programs in order to target technical assistance from USDA’s Farm to School program.
One product for this purpose will be a map of SFAs categorized by percent of total food purchases that are locally produced, with districts not reporting indicated as blank. For this purpose a coverage of 75 percent will be adequate if non-response is random.
The telephone follow-up of non-responders will provide an assessment of nonresponse bias. A minimum detectable difference of 10 percentage points (with 80 percent certainty) will be adequate to test the hypothesis that non-respondents have different rates of sourcing locally produced food than responders.
Similarly, for comparing the State-level prevalence to the national average (21 percent, based on the 2009-10 School Food Purchase Survey), a 75 percent response rate is adequate if non-response is random. The final estimation error will be a function of the response rate; a response rate of 75 percent would result in a margin of error of ¼ of the minimum detectable difference from the telephone sample of non-responders.
3. Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The recruitment methodology for the Farm to School Census has been designed based on the 2008 North Carolina Farm to School Survey, which had a response rate of 78 percent.
In order to maximize response rates, the questionnaire is structured to minimize burden. The second question, after a request for the name of the School Food Authority for identification, is a screener which briefly outlines several farm to school activities (local procurement for school meals, educational activities related to local foods, edible school gardens) and asks whether the SFA participates in any of these activities. Food service directors who answer affirmatively will be directed to questions about their farm to school activities while other respondents will be directed to an abridged set of questions. Food Service Directors will receive 3 reminder emails. Food Service Directors that have not responded after 4 weeks will receive a reminder telephone call.
In order to collect accurate and reliable information, question phrasing is based on questions already used successfully in the Minnesota Farm to School Survey, the School Food Purchase Survey, and the Special Nutrition Program Operation Survey.
In addition 8 food service directors were asked to review the questionnaire and provide feedback on the time necessary to respond and to identify any confusing questions. Their feedback was incorporated in the questionnaire in order to minimize the burden to respondents. In particular, a question on purchases of locally produced food as a percent of total food expenditures was clarified to indicate that only an estimate is requested.
To assess the potential for non-response bias, a sample of non-respondents will be followed up by telephone in order to analyze non-response. Food service directors contacted by phone will be asked to complete an abridged set of questions, focusing on whether the SFA purchases locally produced food for school meal programs. These responses will be used to determine whether SFAs that do not respond are less likely to be purchasing locally produced food.
Adequacy of information collection for intended purposes
The adequacy of the information collection for the purpose of targeting technical assistance will be dependent on minimizing non-response bias. If non-response bias is sufficiently small, a response rate of 75 percent will provide adequate coverage for the map of local procurement and for identifying State, and subgroups within States (such as rural or urban, large or small, high-poverty or low poverty), that have much lower prevalence of local purchasing than the national average.
The School Food Purchase Survey III found that the national average prevalence for local purchasing was 21 percent. If we assume that respondents to the Census have a prevalence similar to the national average, the top priority would be to detect a difference as large as 10 percentage points between responders and non-responders, with 80 percent certainty.
Out of the stratified sample of 100 non-respondents, 100 are expected to cooperate with a telephone request for an abridged set of questions. The resulting sample size will result in a minimum detectable difference of 8 percentage points, for alpha = 0.05, and 80 percent power.
If the difference is found to be above 8 percentage points, suggesting non-response bias, this will be noted as a caution in reporting results.
4. Describe any tests of procedures or methods to be undertaken.
The recruitment procedure has already been tested in the North Carolina Farm to School Survey. The questionnaire uses question phrasing that has already been tested in the Minnesota Farm to School Survey, the Special Nutrition Program Operation survey, and the School Food Purchase Survey.
The State information request and questionnaire also underwent cognitive testing in a sample of 6 State Child Nutrition Directors and 8 SFA Food Service Directors, respectively (see Attachment J). We asked State Child Nutrition Directors from Montana, Texas, West Virginia, Arizona, Michigan, and Florida about their ability to provide contact information for SFA food service directors electronically, and how much lead time it would require. We also asked about how they forward memos sent from FNS through PartnerWeb to assess our plan for using PartnerWeb to disseminate the invitation to participate in the Census. Finally, we asked whether food service directors in their States would have difficulty completing the questionnaire on-line or would be unwilling to respond if answers were not confidential.
The questionnaire was reviewed by 8 SFA Food Service Directors from Great Falls MT; Tempe, AZ; Deweyville Unified School District, TX; Clint Unified School District, TX; Richardson Unified School District, TX Grand Rapids, MI; Benzie Central School District, MI; and Fayette County WV. We asked about time required to complete the questionnaire, whether any questions would be difficult to answer, whether the questions required consultation with other staff, whether non-confidentiality could be a barrier to response, and whether there were other questions that should be added.
Results from the cognitive testing and actions taken in response are summarized below:
The plan to disseminate the invitation to participate in the survey through the FNS PartnerWeb email system is likely to work smoothly, as State Child Nutrition Directors already use this system for transmittal of information from FNS.
The invitation should be worded carefully to address concerns about confidentiality.
The invitation has been revised to clarify that personal information is not included, and questions that may have compliance implications have been removed.
For respondents that have a Farm to School program, the questionnaire is likely to take 15 to 30 minutes, consistent with our estimated average of 20 minutes from similar surveys.
Questions about value of food purchases should be designed to request an estimate rather than exact figures.
The questionnaire was modified accordingly.
The questionnaire should make clear what is meant by “local” food, since many districts purchase some items locally because those are the products that meet their specifications, especially in the case of milk.
The questionnaire has been revised to clarify that “local” is based on criteria set by the district, both in terms of distance and other requirements.
Some districts, like one of the cognitive testing respondents, may have technical difficulties using the on-line questionnaire, and the data collection plan must include a method of allowing respondents to complete the questionnaire by an alternative method, such as fax.
Respondents will be given instructions to print out an attached version of the questionnaire to be completed by hand and submitted by fax if they have difficulty completing the questionnaire online. Respondents will also be given a telephone number to call for assistance.
5. Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Statistical reviewer:
Michael Jacobsen, National Agricultural Statistics Service
Cooperators designing the surveys and managing collection, and collaborating in data analysis:
Anupama Joshi, ([email protected], (847) 917-7292) and Amy Paxton, ([email protected] (757) 561-6511) assisted in the design of the questionnaire and methodology under a cooperative agreement with Occidental College.
A consulting firm yet to be determined will assist data collection by entering data from questionnaires received by fax, contacting a sample of non-respondents to administer an abridged questionnaire, staffing a help line to assist respondents in completing the questionnaire on-line, and conducting reminder phone calls.
All other data collection, management, and analysis activities will be conducted by ERS in-house.
ERS project officer:
Katherine Ralston
202-694-5463
a Paxton, A, Ammerman A, Hobbs SH, Fleming P. “Farm to School: addressing obesity and school nutrition through local food systems.” Poster presentation: American Public Health Association, October 27, 2008; San Diego, CA
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | HancDa |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |