Appendix K_NASS Comments and FNS Response

APPENDIX K

NASS COMMENTS AND FNS RESPONSE

2023 Pulse Survey

Operational Challenges in Child Nutrition Programs

SFDS Review

BACKGROUND

The United States Department of Agriculture (USDA) Food and Nutrition Service (FNS) administers federal level Child Nutrition (CN) level programs. State Agencies (SAs) administers local level CN programs via school food authorities (SFAs) and other local entities.

One of FNS’s roles is to collect relevant CN information to “inform current and future policy decisions that support effective implementation of CN programs.” One recent survey suggested many SFSs faced supply chain issues (which includes high costs, etc.), and staffing shortages during the pandemic and faced issues getting operation processes in place after the pandemic.

FNS will be conducting the 2023 Pulse Survey to ascertain “current operational challenges or emerging topics in child nutrition programs, including those related to school food procurement, program administration, and program finances.”

SAMPLE PLAN

Sampling Frame, Sample Size Determination

The 2023 Pulse Survey (universe) sampling frame (N=19,120) comprises SAs Directors in all US States, District of Columbia, Guam, Puerto Rico, and U.S. Virgin Islands, (N1=56); SFA Directors (N2=14); and SFAs (N3=19,050). The sampling frame will be sampled at a 100% rate.

The total sampling frame should be (N=19,106). The number of SFAs is (N=19,050) and the number of SA Directors is (N=56). The latter will be asked to help distribute a survey support email but will not be asked to complete any surveys. While SFAs are discussed in the clearance package, as they are the focus of the survey, any communication with SFAs is expected to happen through SFA Directors (i.e., SFAs themselves are not an independent group of respondents).

Of the 19,050 SFAs: 3,401 are private not-for-profit businesses and 15,649 are public.

Study Design

Nine pretests of the universe (one SA Director, seven SFA Directors and one SFA) were used to guide the final version of the survey.

There were nine SFA Directors that served as pretest participants from the universe (see Table 1. Pretest Participants in Supporting Statement B).

The nine pre-tests were administered to ascertain information from different SFA segments:

1. Public and private not-for-profit SFAs.

2. Geographic location.

3. Urbanicity.

4. Use of food management companies.

5. SFA size by student enrollment, and

6. SFA participation rates in free and reduced-price meals.

DATA COLLECTION

Questionnaire:

This a mandatory survey to collect nationally representative information on current operational challenges and assess the scope, reach, and variation in challenges. The challenges fall into the categories of food procurement, program administration, and program finances. The survey has 31 questions and is to take approximately 20 minutes to complete. Numeric calculations of food costs, labor costs, and student participation can be estimated if data are not readily available. All SAs and SFAs must complete the survey by January 1, 2024.

Methods to Maximize Response Rate:

Based on prior experience, high response rates are expected. The census will be a mandatory reporting. Support for engagement will be fostered through State awareness, familiarity with past data collection, and the assistance of State agencies in distributing the Survey Support Email. Flexibility to complete all or part of the questionnaire at any given time is provided to assist in improving completion. A dedicated email is provided to answer any queries related to questionnaire completion. In addition, reminders and follow-up thank you messages will support engagement. If response rate is low in identifiable patterns, targeted outreach to address issues will be used.

COMMENTS AND SUGGESTIONS

Overview

1. Please review document for potential inconsistent information. A couple of examples follow:

a. Are the values in these two statements supposed to match each other:

i. Supporting Statements Part A - A12 states: “With this submission, there are 19,120 respondents, 95,329 responses, and 8,846.81 burden hours.”

The survey burden table and associated text has been updated to reflect consistent and accurate respondent numbers. The text now reads: “With this submission, there are 19,106 respondents, 95,306 responses, and 8,833.30 burden hours.”

ii. Supporting Statements Part A - A15 states: “This submission is a new information collection request as a result of program changes and will add 8,869 hours of burden to OMB’s inventory attributable to 95,539 total annual responses.”

The survey burden table and associated text has been updated to reflect consistent and accurate respondent numbers. The text now reads: “This submission is a new information collection request as a result of program changes and will add 8,833.30 hours of burden to OMB’s inventory attributable to 95.306 responses.”

b. Is the following FNS statement correct? i. FNS requires survey results by January 15, 2024, but is requesting approval to collect data through February 28, 2024.

Yes. We have provided a broader time window for data collection in the event of unexpected delays in survey administration or survey response.

Sampling Frame, Sample Size Determination

1. The sampling frame definition seems to be missing two SA Director entities.

a. SFDS assumes these entities are Northern Marianas and American Samoa.

This is correct.

2. It is unclear if there are only 14 SFA Directors or only 14 SFA Directors were included in the sampling frame.

The total sampling frame should be (N=19,106). The number of SFAs is (N=19,050), meaning 19,050 SFA Directors are in the sampling frame, and the number of SA Directors is (N=56).

3. Table B1 Summary of Respondent Universe and Expected Response Rates total seems to exclude SFA Directors.

a. Is this correct?

While SFAs are discussed in the clearance package, as they are the focus of the survey, any communication with SFAs is expected to happen through SFA Directors (i.e., SFAs themselves are not an independent group of respondents). Therefore, the entirety of the respondent universe (19,050 SFAs represented by 19,050 SFA Directors and 56 SA directors) is included in Table B1.

4. Supporting Statements Part A, states the survey will be administered to SAs Directors and SFAs: it is not clear if it will be administered to the 14 SFA Directors.

The supporting documentation and burden estimates have been revised to reflect that SA Directors will not receive the survey and are not considered survey respondents. SA Directors will only be contacted to provide contact information for SFAs. The survey will be administered to 19,050 SFAs represented by their individual SFA Directors.

5. For OMB burden calculations, SA Directors and SFA Directors do not include the survey, pre-survey letter, etc., so it is unclear if they are part of the universe or not.

See above.

6. In Supporting Statements Part A, consider defining the universe and using the term universe in place SA Directors, SAs, SFA Directors, SFAs.

It is expected that the confusion surrounding these various respondent groups will be made more clear with the revisions in the supporting documents and the answers in this document.

7. The most recent SFA Survey II on School Food Supply Chain Disruption suggests there was “no differences between respondents and nonrespondents.”

a. This statement supports a sample is a viable alternative to a complete enumeration in the event FNS faces resource limitations.

This is an important acknowledgment and will be under consideration by FNS in the event that resources become limited.

Study Design

1. SFDS understands FNS expects a 100% response rate and FNS is prepared for the unlikely event FNS does not obtain a 100% response rate.

Yes. We are prepared to weight the survey as needed to ensure that the respondent population does not differ significantly from and is representative of the universe of respondents.

2. There was no mention of plant-based meals in the questionnaire. There is a Fresh Fruit and Veggie Program, but physicians are concerned that there is no plant-based meal option as a standard for school meals.

This concern is heard and understood. However, because the focus of this particular survey is on the ways in which supply chain disruptions and other challenges are interrupting school food service, it is not the most appropriate place for gathering information about plant-based meals. FNS will seriously consider incorporating additional questions about plant-based meals in appropriate survey instruments, including instruments being developed in the next year.

0. a. There were 114 comments from members of the Physicians Committee for Responsible Medicine (Regulations.gov) regarding the absence of plant-based meal option.

See above.

DATA COLLECTION

1. Consider adding the Sample Pretest timeline to the project schedule table.

The supporting documentation has been revised to make it clear that the previous edition of this survey underwent pretesting in 2022, and few revisions have been made meriting additional pretesting.