Supporting Statement Part B Corrected

SUPPORTING STATEMENT - PART B for

OMB Control Number 0584-NEW:

2023 Pulse Survey: Operational Challenges in Child Nutrition Programs

September 2023

Sarah Reinhardt

Social Science Policy Analyst

Office of Policy Support

USDA, Food and Nutrition Service

3101 Park Center Drive

Alexandria, Virginia 22302

[email protected]

Table of Contents

B.1 Respondent Universe and Sampling Methods 3

B.2 Procedures for the Collection of Information 5

B.3 Methods to Maximize the Response Rates and to Deal with Nonresponse 6

B.4 Test of Procedures or Methods to be Undertaken 8

B.5 Individuals Consulted on Statistical Aspects & Individuals Collecting and/or Analyzing Data 10

Tables

Table B1. Summary of Respondent Universe and Expected Response Rates

Table B2. Individuals Consulted on Statistical Aspects of Study Design

Appendices

Appendix A. Section 28 of the Richard B. Russell National School Lunch Act and Section 305 of the Healthy, Hunger-Free Kids Act of 2010

Appendix B. 2023 Pulse Survey: Operational Challenges in Child Nutrition Programs

Appendix C. Sample Survey Support Email

Appendix D. Sample Reminder Email

Appendix E. Sample Thank You Email

Appendix F. Sample Pretest

Appendix G. Burden Table

Appendix H. Federal Register Notice

Appendix I. Comment from Texas Department of Agriculture

Appendix J. Food and Nutrition Service Response to Comment

Appendix K. NASS Comments and FNS Response

B.1 Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The respondent universe for this collection includes the 56 State Agency (SA) Child Nutrition (CN) directors that administer the CN Programs in the 50 States, District of Columbia, American Samoa, Guam, Northern Mariana Islands, Puerto Rico, and U.S. Virgin Islands and the 19,050 school food authorities (SFAs), public and private, that administer the CN Programs at the local level. No sampling or weighting is required as the collection will be a census of SFAs. The survey will utilize existing SFA contact information obtained for the SFA Survey II on School Food Supply Chain Disruption. Information collected through this survey is critical to helping FNS better understand, and provide tools and resources to address, the localized concerns and challenges school districts nationwide are facing in their efforts to feed children healthy and nutritious meals this school year.

Expected Response Rates

Based on previous experience with State-level collections, the timeliness of the survey, and the minimal amount of burden involved on the part of State agencies for this collection, a 100 percent response rate is expected for the state-level distribution of the survey support email (Appendix C). Additionally, because it is assumed the topics addressed in this survey are of great concern to SAs and SFAs and the survey provides an opportunity to communicate pressing challenges and frustrations directly to FNS, FNS expects all 19,050 SFAs to respond.

Table B1. Summary of Respondent Universe and Expected Response Rates

Respondent		Universe	Target completed cases	Expected response rate
State CN Agencies	56		56	100%
SFAs	19,050		19,050	100%
Total	19,106		19,106	100%

The approach to achieving high response rates builds on prior FNS survey experience as well as FNS’s relationships with its partners at the State and local levels. Throughout the duration of the COVID-19 public health emergency, FNS, including the regional offices, worked hand in hand with SAs to address new and emergent issues affecting the provision of meals to children. Based on what we are hearing from the regional offices and States’ expressed needs for continued support, we believe that local program operators will welcome this opportunity to share their challenges and experiences with FNS. This survey builds on the success of two previous surveys assessing supply chain issues facing SFAs in SY 2022-2023 and earlier.

FNS will build support for this survey through engaging with regional and State partners to promote awareness and the importance of responding. Regional and State offices were consistently in contact with the SFAs throughout the COVID-19 public health emergency to discuss specific supply chain concerns and administer the aforementioned survey, meaning many respondents will already be familiar with the survey process and topics. Because we know that SFAs consistently look to their SAs for guidance related to the CN Programs, FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix C) to let SFAs know that the survey is coming and to express their support for the collection. FNS intends to collect data over a 6-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) each week. FNS expects each SFA to receive 2 reminder emails, on average.

To further promote high response rates, the web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later. FNS has also created a dedicated email address through which SFAs may request assistance or ask questions. These tools will help SFAs to complete the survey whenever it is convenient for them to do so and will help FNS to track and respond to SFA questions or requests for assistance in a timely manner. Additionally, SFAs will be informed that this a mandatory collection under the provisions of the National School Lunch Act.

B.2 Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Because this is a census and a 100 percent response rate is expected, there is no need for sampling, weighting or nonresponse adjustments. The prior SFA Survey II on School Food Supply Chain Disruption yielded a 75% response rate, and no differences between respondents and nonrespondents were identified.

FNS will first contact SAs to provide information about the survey and to provide a Survey Support Email (Appendix C) to be distributed to the SFAs in their respective States letting SFAs know that the survey is coming and to express their support for the collection. Shortly thereafter, FNS will email the SFA Survey on Supply Chain Disruption (Appendix B) to each SFA, using contact information obtained from the SFA Survey II on School Food Supply Chain Disruptions, via the Qualtrics Survey Software platform.

FNS intends to collect data over a 6-week period. SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) from experienced survey support personnel each week. FNS expects each SFA to receive 2 reminder emails, on average. Upon completion of the survey, FNS will send each SFA a thank you email (Appendix E). Respondents may also email the survey team at FNS to request help in completing their survey or with technical issues.

Statistical Methodology for Stratification and Sample Selection. Because the collection will be administered via a census of SFAs, there will be no sample stratification or selection.

Estimation Procedure. Because data will be collected from a census of SFAs, no estimation procedures are necessary for this collection.

Degree of Accuracy Needed for the Purpose Described in the Justification: Because data will be collected from a census of SFAs, this question is not relevant to this submission.

Unusual Problems Requiring Specialized Sampling Procedures: No specialized sampling procedures are planned.

Use of Periodic (Less Frequent than Annual) Data Collection Cycles to Reduce Burden. The data collection procedures will be conducted once. Concern regarding the periodicity of data collection cycles is not applicable.

B.3 Methods to Maximize the Response Rates and to Deal with Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

The study is expected to achieve a 100 percent response rate. This means that the data collected will represent the entire universe of SFAs, and rather than providing estimates to answer the research questions, we will be able to provide actual population totals. Thus, these data will provide reliable information on the extent and impacts of current and emerging challenges in school food that represent the full population.

Achieving the specified response rate involves contacting the States, securing their support for the study, and then contacting SFAs and offering support and completion reminders. The study team will use the following methods to maximize participation and reduce nonresponse:

• FNS will inform States and SFAs that their participation in this collection is mandatory per the provisions of the National School Lunch Act.

• FNS will request State agency assistance to build support for the study among SFAs by asking all SAs to distribute the Survey Support Email (Appendix C) to let SFAs know that the survey is coming and to express their support for the collection.

• The web survey will allow respondents to save and exit the survey at any point, and then return to access and complete the survey later, allowing respondents to complete the survey at their convenience.

• SFAs that have not yet completed their surveys will receive a reminder email (Appendix D) each week. FNS expects each SFA to receive 2 reminder emails, on average.

• Recruiting materials were carefully developed to emphasize the following points, which may resonate with respondents:

• State agency and SFA participation in the collection are vital to providing FNS with information we need to support school districts during this difficult time.

• This is an opportunity to communicate the specific issues each SFA is experiencing directly to FNS.

• This survey will assist FNS and its partners to enhance the toolbox for school nutrition professionals working hard to make sure students have reliable access to healthy meals.

• The survey was crafted with input from SFAs and will only take 20 minutes to complete, on average.

• FNS created a dedicated email address through which SFAs may request assistance or ask questions to enable FNS to track and respond to SFA requests for assistance in a timely manner. Staff will be readily available to clarify survey questions and work with participants to resolve technical issues, such as difficulty logging on or advancing past pages. Personalized assistance bolsters the perceived legitimacy of the collection and will encourage respondents to persist in completing the survey.

If the response rate is lower than expected after reminder emails have been sent, FNS will compare the characteristics of the non-responding SFAs to those of all SFAs to determine if there are patterns in non-response across key characteristics such as SFA size, geographic location (State or region), or urbanicity (rural vs. nonrural). If there are identifiable patterns in nonresponse, FNS will consider providing more targeted outreach to the non-responding SFAs to encourage their participation. If necessary, FNS will also prepare suitable nonresponse adjustment weights to be used for estimates and will prepare suitable estimates of standard errors on critical outcomes prior to presenting results of the survey.

B.4 Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

The 2023 Pulse Survey: Operational Challenges in Child Nutrition Programs is provided in Appendix B and largely reflects the SFA Survey II on Supply Chain Disruption administered in 2022. The study team pretested the SFA Survey II on Supply Chain Disruption with nine SFAs in August and September of 2022. Through the regional offices, FNS asked State agencies to recommend SFAs for the pretest with consideration for a set of identified criteria (described below). SFA respondents were selected based on their availability to participate within the pre-test timeframe and to ensure variation in participation across characteristics such as SFA size in terms of student enrollment, geographic location, urbanicity, percentage of students certified for free or reduced-price meals, and use of a food service management company, in order to gather different perspectives on the survey.

FNS scheduled a 30-minute debriefing conversation with each of the nine participants and sent each a link to complete the web survey via the Qualtrics Survey Software platform. In the invitation email, FNS requested feedback from the pretest SFAs on question wording, response options, and burden. Prior to each debriefing phone call, FNS sent the respondent a copy of his or her completed survey responses to facilitate the discussion. With permission, the names, titles, and contact information of pretest participants are listed in Table 1 below. The pretested version of the survey and the debriefing protocol are available in Appendix F.

Table 1. Pretest Participants

Name	Title	State	Email
Becky Hardin	Cafeteria Director, Rosary Catholic School	Oklahoma	[email protected]
Brigette Clark	District Clerk, Geraldine Public Schools	Montana	[email protected]
Dawn Smith	Director of Food and Nutrition, School District of Beloit	Wisconsin	[email protected]
Dean Gallegos	Director, Food Service, Rio Rancho Public Schools	New Mexico	[email protected]
Debby Webster	Nutrition Services Director, Rainier School District	Oregon	[email protected]
Jacob Wood	Business Administator, Essex North Supervisory Union	Vermont	[email protected]
Joe Urban	Director of Food and Nutrition Services, Greenville County Schools	South Carolina	[email protected]
Kristen Osborn	School Nutrition Director, Gunnison Watershed School District	Colorado	[email protected]
Sylvana Bryan	School Nutrition Director, Pittsfield Public Schools	Massachusetts	[email protected]

Pretest participants reported spending from 15-30 minutes on the survey, with an average time of 19 minutes. Because several response options were added to the survey based on participant feedback, we believe that a 20-minute estimate for survey completion is accurate. All participants provided positive feedback on the overall content and structure of the survey and suggested that the survey would be well-received by SFAs given its timeliness and ease of completion. Participants did not identify any questions that were particularly burdensome or time consuming to respond to, though several recommended expanding the survey introduction to let respondents know that they may need to reference records from previous years, including labor costs, food costs, and student participation, during the survey.

Although participants overwhelmingly agreed that the response options provided in the survey were relevant and appropriate, about half indicated that the list of response options was too lengthy for several questions. In response to this feedback, FNS limited the number of unique response options per question to no more than 15, creating separate questions with fewer response options as needed. Several participants also indicated that it would be useful to provide definitions of some less commonly used terms, which will be addressed through the Qualtrics survey platform, and recommended providing more direct reference points (e.g., “in a typical pre-pandemic year”) for questions asking respondents to compare across time periods.

B.5 Individuals Consulted on Statistical Aspects & Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Table B5.1 lists USDA staff consulted on statistical aspects of the design. Survey revisions and data collections will be completed by a qualified contractor (to be determined) with appropriate experience in data collection and statistical analysis.

Table B2. Individuals consulted on statistical aspects of study design

Staff	Title	Phone	Email
Sarah Reinhardt, FNS	Social Science Policy Analyst		[email protected]
Maggie Applebaum, FNS	Assistant Deputy Administrator, Child Nutrition Programs	703-305-2578	[email protected]
Barbara Murphy, FNS	Director, Child Nutrition Research and Analysis Division	703-305-2532	[email protected]
NASS	Peter Quan, Sampling and Frames Development Section	202-720-5269	[email protected]