SUPPORTING STATEMENT - PART B for
OMB Control Number 0584-NEW:
SFA Survey II on School Food Supply Chain Disruption and Student Participation
September 15, 2022
Sarah Reinhardt
Social Science Policy Analyst
Office of Policy Support
USDA, Food and Nutrition Service
1320 Braddock Place
Alexandria, Virginia 22314
TABLE OF CONTENTS
B1. Respondent Universe and Sampling Methods 1
B2. Procedures for the Collection of Information 3
B3. Methods to Maximize the Response Rates and to Deal with Nonresponse 5
B4. Test of Procedures or Methods to be Undertaken 7
B5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 10
TABLES
Table B1. Summary of Respondent Universe and Expected Response Rates 2
Table B2. Individuals consulted on statistical aspects of study design 10
Appendix A. Section 28 of the Richard B. Russell National School Lunch Act and Section 305 of the Healthy, Hunger-Free Kids Act of 2010
Appendix B. SFA Survey II on School Food Supply Chain Disruption and Student Participation
Appendix C. Survey Support Email
Appendix D. Reminder Email
Appendix E. Thank You Email
Appendix F. Pretest
Appendix G. Burden Table
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
The respondent universe for this collection includes the 56 State Agency (SA) Child Nutrition (CN) directors that administer the CN Programs in the 50 States, District of Columbia, Guam, Puerto Rico, and U.S. Virgin Islands as well as the 19,050 school food authorities (SFAs), public and private, that administer the CN Programs at the local level. No sampling or weighting is required as the collection will be a census of both SAs and SFAs. The survey will utilize existing SFA contact information obtained for the SFA Survey I on School Food Supply Chain Disruption. This information is critical to helping FNS better understand, and provide tools and resources to address, the localized concerns and challenges school districts nationwide are facing in their efforts to feed children healthy and nutritious meals this school year.
Based on previous experience with State-level collections, the timeliness of the survey, and the minimal amount of burden involved on the part of State agencies for this collection, a 100 percent response rate is expected for the state-level distribution of the survey support email (Appendix C). Additionally, because school food vendor, cost, and labor issues are urgent challenges affecting school districts nationwide and this short survey presents an opportunity for SFAs to communicate their challenges and frustrations directly to FNS, FNS expects all 19,050 SFAs to respond.
Table B1. Summary of Respondent Universe and Expected Response Rates
Respondent |
Universe |
Target completed cases |
Expected response rate |
|
State CN Agencies |
56 |
56 |
100% |
|
SFAs |
19,050 |
19,050 |
100% |
|
Total |
19,106 |
19,106 |
100% |
|
The approach to achieving high response rates builds on prior FNS survey experience as well as FNS’s relationships with its partners at the State and local levels. Throughout the pandemic, FNS, including the regional offices, has worked hand in hand with SAs to address new and emergent issues affecting the provision of meals to children. Based on what we are hearing from the regional offices and States’ expressed needs for support, we believe that State and local program operators will welcome this opportunity to share school food vendor, cost, and labor challenges and experiences with FNS. The respondents that participated in the pretest of the instrument indicated that SFAs will welcome the opportunity to share their unique experiences this school year. In addition, this survey builds on the success of a previous survey assessing supply chain issues facing SFAs in SY 2021-2022.
FNS will build support for this survey through engaging with regional and State partners to promote awareness and the importance of responding. Regional and State offices have consistently been in contact with the SFAs throughout the pandemic to discuss specific supply chain concerns and administer the aforementioned survey, meaning many respondents will already be familiar with the survey process and topics. Because we know that SFAs consistently look to their SAs for guidance related to the CN Programs, FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix C) to let SFAs know that the survey is coming and to express their support for the collection. FNS intends to collect data over a 4-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) each week. FNS expects each SFA to receive 2 reminder emails, on average.
To further promote high response rates, the web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later. FNS has also created a dedicated email address through which SFAs may request assistance or ask questions. These tools will help SFAs to complete the survey whenever it is convenient for them to do so and will help FNS to track and respond to SFA requests for assistance in a timely manner. Additionally, States and SFAs will be informed that this a mandatory collection under the provisions of the National School Lunch Act.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection;
Estimation procedure;
Degree of accuracy needed for the purpose described in the justification;
Unusual problems requiring specialized sampling procedures; and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
Because this is a census and a 100 percent response rate is expected, there is no need for sampling, weighting or nonresponse adjustments. The prior SFA Survey on School Food Supply Chain Disruption yielded a 60% response rate, and no differences between respondents and nonrespondents were identified.
FNS will utilize SFA contact information obtained from the SFA Survey I on School Food Supply Chain Disruptions for the 56 State CN agencies that administer the CN Programs at the state level in the 50 States, District of Columbia, Guam, Puerto Rico, and U.S. Virgin Islands. FNS will first contact SAs with a Survey Support Email (Appendix C) to be distributed to the SFAs in their respective States letting SFAs know that the survey is coming and to express their support for the collection. Shortly thereafter, FNS will email the SFA Survey on Supply Chain Disruption (Appendix B) to each SFA via the Qualtrics Survey Software platform.
FNS intends to collect data over a 4-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) from experienced survey support personnel at FNS each week. FNS expects each SFA to receive 2 reminder emails, on average. Upon completion of the survey, FNS will send each SFA a thank you email (Appendix E). Respondents may also email the survey team at FNS to request help in completing their survey or with technical issues.
Statistical Methodology for Stratification and Sample Selection. Because the collection will be administered via a census of State CN agencies and SFAs, there will be no sample stratification or selection.
Estimation Procedure. Because data will be collected from a census of State CN agencies and SFAs, no estimation procedures are necessary for this collection.
Degree of Accuracy Needed for the Purpose Described in the Justification: Because data will be collected from a census of State CN agencies and SFAs, this question is not relevant to this submission.
Unusual Problems Requiring Specialized Sampling Procedures: No specialized sampling procedures are planned.
Use of Periodic (Less Frequent than Annual) Data Collection Cycles to Reduce Burden. The data collection procedures will be conducted once. Concern regarding the periodicity of data collection cycles is not applicable.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.
The study is expected to achieve a 100 percent response rate. This means that the data collected will represent the entire universe of State CN Directors and SFAs, and rather than providing estimates to answer the research questions, we will be able to provide actual population totals. Thus, these data will provide reliable information on the extent and impacts of current and emerging challenges in school food that represent the full population.
Achieving the specified response rate involves contacting the States, securing their support for the study, and then contacting SFAs and offering support and completion reminders. The study team will use the following methods to maximize participation and reduce nonresponse:
FNS will inform States and SFAs that their participation in this collection is mandatory per the provisions of the National School Lunch Act.
FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix C) to let SFAs know that the survey is coming and to express their support for the collection.
The web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later, allowing respondents to complete the survey at their convenience.
SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) each week. FNS expects each SFA to receive 2 reminder emails, on average.
Recruiting materials were carefully developed to emphasize the following points, which may resonate with respondents:
State agency and SFA participation in the collection are vital to providing FNS with information we need to support school districts during this difficult time.
This is an opportunity to communicate the specific issues each SFA is experiencing directly to FNS.
This survey will assist FNS and its partners to enhance the toolbox for school nutrition professionals working hard to make sure students have reliable access to healthy meals.
The survey was crafted with input from SFAs and will only take 20 minutes to complete, on average.
FNS created a dedicated email address through which SFAs may request assistance or ask questions to enable FNS to track and respond to SFA requests for assistance in a timely manner. Staff will be readily available to clarify survey questions and work with participants to resolve technical issues, such as difficulty logging on or advancing past pages. Personalized assistance bolsters the perceived legitimacy of the collection and will encourage respondents to persist in completing the survey.
If the response rate is lower than expected after reminder emails have been sent, FNS will compare the characteristics of the non-responding SFAs to those of all SFAs to determine if there are patterns in non-response across key characteristics such as SFA size, geographic location (State or region), or urbanicity (urban vs. rural). If there are identifiable patterns in nonresponse, FNS will consider providing more targeted outreach to the non-responding SFAs to encourage their participation. If necessary, FNS will also prepare suitable nonresponse adjustment weights to be used for estimates and will prepare suitable estimates of standard errors on critical outcomes prior to presenting results of the survey.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
The study team pretested the SFA Survey on Supply Chain Disruption with nine SFAs in August and September of 2022. Through the regional offices, FNS asked State agencies to recommend SFAs for the pretest with consideration for a set of identified criteria (described below). SFA respondents were selected based on their availability to participate within the pre-test timeframe and to ensure variation in participation across characteristics such as SFA size in terms of student enrollment, geographic location, urbanicity, percentage of students certified for free or reduced-price meals, and use of a food service management company, in order to gather different perspectives on the survey.
FNS scheduled a 30-minute debriefing conversation with each of the nine participants and sent each a link to complete the web survey via the Qualtrics Survey Software platform. In the invitation email, FNS requested feedback from the pretest SFAs on question wording, response options, and burden. Prior to each debriefing phone call, FNS sent the respondent a copy of his or her completed survey responses to facilitate the discussion. The pretested version of the survey and the debriefing protocol are available in Appendix F Pretest participant names and titles are available in Question 8 of Supporting Statement Part A.
Pretest participants reported spending from 15-30 minutes on the survey, with an average time of 19 minutes. Because several response options were added to the survey based on participant feedback, we believe that a 20 minute estimate for survey completion is accurate. All participants provided positive feedback on the overall content and structure of the survey, and suggested that the survey would be well-received by SFAs given its timeliness and ease of completion. Participants did not identify any questions that were particularly burdensome or time consuming to respond to, though several recommended expanding the survey introduction to let respondents know that they may need to reference records from previous years, including labor costs, food costs, and student participation, during the survey.
Although participants overwhelmingly agreed that the response options provided in the survey were relevant and appropriate, about half indicated that the list of response options was too lengthy for several questions. In response to this feedback, FNS limited the number of unique response options per question to no more than 15, creating separate questions with fewer response options as needed. Several participants also indicated that it would be useful to provide definitions of some less commonly used terms, which will be addressed through the Qualtrics survey platform, and recommended providing more direct reference points (e.g., “in a typical pre-pandemic year”) for questions asking respondents to compare across time periods. In response to these suggestions and other more specific participant feedback, FNS made the following changes to the survey:
Question 2
Added details to clarify what household applications meant.
Question 3
Provided more information to help clarify the meaning of “vended meals company.”
Question 4
Provided a reference point with respect to “high food costs” and “high labor costs” (compared with a typical pre-pandemic school year).
Clarified “insufficient bids for food service contracts” by rephrasing as “low number of bids for food service contracts.”
Added response option to address pretest feedback: “Receiving items that are damaged or unusable due to distributor issues.”
Deleted response option: “Difficulty maintaining compliance with meal pattern requirements,” as it is named as an impact in Question 8.
Deleted response options related to local foods and scratch cooking, which are now included as part of skip patterns later in the survey, to reduce number of response options.
Question 5
New question added to direct respondents to question about USDA Foods or skip past it.
Question 6 (previously Question 5)
Added response option “Difficulty buying as much food as we would like due to lack of storage space.”
Deleted response option “Difficulty maintaining compliance with meal pattern requirements,” as it is named as an impact in Question 8.
Question 7 (previously Question 6)
Simplified question language to avoid confusion: “How are the challenges your SFA is experiencing impacting school meal operations?”
Provided examples to clarify response option based on pretest participant feedback: “confusion from students or parents/guardians regarding shift to standard program operations, including changes to meal service, payment, or paperwork.”
Added “Staffing challenges due to changes in meal preparation or service (e.g., from grab-and-go to cafeteria meals).”
Question 8 (previously Question 7)
Consolidated “Difficulty maintaining routine program documentation” with “Inability to fulfill other job requirements” and provided more clarity.
Provided more clarity on “Inability to offer enough reimbursable meals to participating children due to lack of food or service materials” and “Changes in meal preparation methods (e.g., scratch cooking to grab-and-go meals).”
Added “Increased competition from nonprogram (competitive) or off-campus food sales”
Added “Difficulty retaining enough staff with adequate skills and training”
Questions 9-10 (previously Question 8)
Split into Question 9 (purchasing strategies) and Question 10 (other strategies)
Deleted response options: “Partnering with restaurants, catering companies, or community partners…” due to confusion, the fact this could be considered cooperative purchasing, and skepticism that this was happening frequently.
Question 9 (purchasing strategies)
Provided
more clarity on “Purchasing foods directly from grocery
stores or warehouses retail outlets” and
other minor changes.
Added “Increasing communication with vendors, distributors, or manufacturers to identify available products.”
Added “Increasing use of local vendors and/or working with multiple vendors.”
Consolidated “Requesting shorter bids and/or making more frequent orders.”
Added “Increasing local food purchases” and “decreasing local food purchases” as strategies.
Added “Planning further ahead and/or placing orders further in advance.”
Question 11 (previously Question 9)
Provided more clarity on “Used state funding to cover the cost of reduced-priced meals.”
Provided more clarity on “Communicating more with parents/guardians about program changes (e.g., changes to meal service, payment, or applications).”
Added “Increasing communication and/or training with staff about program changes.”
Question 13 (previously Question 11)
Added “Changes in meal preparation or service (e.g., from scratch cooking to ready-to-eat meals).”
Added “Substitutions for unavailable items are more expensive.”
Consolidated “Vendors, distributors, or manufacturers increased costs.”
Provided more clarity on “Using own transportation to transport food more frequently.”
Deleted “My SFA procures high quality food products, which have become more expensive” because it doesn’t give us meaningful information beyond overall food cost increase; “My SFA procures local food, which has become more expensive” because it is included elsewhere now.
Question 15 (previously Question 13)
Added “Wage increases implemented by SFA to hire and retain staff” to distinguish from “Wage increases due to factors beyond SFA control (e.g., state minimum wage increase).”
Question 17 (previously Question 15)
Added “Decrease in enrollment;” “Meals are no longer free for all students.”
Question 21 (previously Question 19)
Consolidated “Federal, state, or local procurement regulations (please specify.)”
Question 22, 23, 24, and 25 were added to provide skip pattern for local foods and scratch cooking and reduce response options for previous questions.
Question 27 (previously Question 21)
Added “Missed deadline to apply for funds;” “Unclear on the intended use or recipient of funds.”
Question 29 (previously Question 23)
Added “Kitchen equipment repair or maintenance.”
There were a number of recommendations made by pretest participants that FNS declined to incorporate into the survey, all of which related to Question 7. The rationale for declining these recommendations is provided below:
Question 7 (previously Question 6)
Declined to incorporate challenge named by one pretest participant (difficulty contacting parents to refund surplus school food credits from past year) because it did not seem to be a pressing concern, or one shared by others.
Several pretest participants indicated that the survey would be issued too early in the school year for SFAs to have reliable information about unpaid school meal debt. However, this was left as an option because other pretest participants confirmed that there would be information about unpaid school meal debt. However, when survey results are collected, we should interpret findings from this particular response with this consideration in mind.
Pretest participants were divided on whether “leveraging state contracts” was clear, but agreed that adding “piggybacking” did not improve clarity. It was left as-is, assuming those who use this option will recognize it.
Pretest participants agreed ranges were better than blanks, and in some cases proposed narrower ranges. This recommendation was not adopted because providing narrower ranges would not change how FNS would utilize this information.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
Table B5.1 lists staff consulted on statistical aspects of the design. The same staff will be responsible for collecting and analyzing the study data.
Table B2. Individuals consulted on statistical aspects of study design
Staff |
Title |
Phone |
|
Sarah Reinhardt, FNS |
Social Science Policy Analyst |
|
|
Maggie Applebaum, FNS |
Assistant Deputy Administrator, Child Nutrition Programs |
703-305-2578 |
|
Barbara Murphy, FNS |
Director, Child Nutrition Research and Analysis Division |
703-305-2532 |
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Mathematica Standard Report Template |
| Author | MATHEMATICA |
| File Modified | 0000-00-00 |
| File Created | 2022-10-07 |