Ssb Sfps Iv

School Food Purchase Study IV (SFPS IV)

OMB# 0584-0471

OMB Supporting Statement
Part B

May 12, 2020

Project Officer: Jinee Burdg

1320 Braddock Place

Alexandria, VA 22314

Phone: (703) 305-2744

Email: [email protected]

Contents

B.1 Respondent Universe and Sampling Methods 1

B.2 Procedures for the Collection of Information 4

B.3 Methods to Maximize Response Rates and to Deal with Non-Response 9

B.4 Tests of Procedures or Methods to Be Undertaken. 13

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data . 15

TABLES

B.1 Respondent Universe, Samples, and Expected Response Rates 4

B.2 Statistical Precision and Minimum Detectable Differences 9

B.5 Individuals Consulted on Design, Data Collection, or Analysis 16

APPENDICES

A. State Government Materials

A1. FNS Notification to State Directors

A2. Notification to States of Sampled SFAs

A3. Study Brochure

A4. State Email to Sampled SFAs

A5. Letters of Support

A6. Request to SDAs to Submit USDA Foods Data

A7. Overview of USDA Foods Data

A8. Reminder Email

A9. Telephone Script to Collect USDA Foods Data

B. Local Government Materials

B1. FedEx Letter to Sampled SFAs

B2. Frequently Asked Questions

B3. Study Overview Webinar Invitation

B4. Study Overview Webinar

B5. Email with Link to Survey

B6. Telephone Script to Complete Survey

B7. Survey of Food Purchase Practices

B8. Purchase Data Webinar Invitation

B9. Purchase Data Webinar

B10. Email to Submit Quarterly Data

B11. Food Purchase Data Checklist

B12. Telephone Script to Collect Quarterly Data

B13. Clarification and Confirmation Email

B14. Telephone Script for Quarterly Data Clarifications and Confirmation

B15. Quarterly Program Data

C. For-Profit Business Materials

C1. FNS Notification to Vendors

D. Other Materials

D1. Richard B. Russell National School Lunch Act (section 28(c))

D2. IRB Approval Letter

D3. Pretest Findings

D4. Confidentiality Agreement

D5. Study Website (screenshot)

E. NASS Comments

F. Public Comments

F1. Public Comment 1

F2. Public Comment 2

F3. Public Comment 3

F4. Public Comment 4

F5. Response to Public Comment 1

F6. Response to Public Comment 2

F7. Response to Public Comment 3

F8. Response to Public Comment 4

G. Burden Table

COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1 Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

The fourth School Food Purchase Study (SFPS-IV) involves data collection from two respondent groups: (1) school food authorities (SFAs) and (2) State Distributing Agencies (SDAs).

The universe of SFAs includes all 15,190 public SFAs in the contiguous 48 States and the District of Columbia, excluding SFAs that serve only charter schools. FNS set precision requirements of ±5 percent for national and ±10 percent for subgroup estimates. Our proposed sample design that achieves these targets is similar to that of SFPS-III but accounts for design effects due to nonresponse adjustments and oversampling SFAs that use FSMCs to achieve the ±10 percent precision target for this subgroup. About 20 percent of SFAs use FSMCs, but this subgroup needs to comprise 25 percent of the sample to reach the ±10 percent precision target.¹ The sample will be stratified by whether an SFA uses an FSMC, and SFAs that use FSMCs will be oversampled. A sample size of 420 SFAs will achieve the precision targets of ±5 percent for national estimates and ±10 percent for subgroup estimates, as discussed in Section B.2.² To reduce SFA burden and produce national estimates of food purchases, the sample of SFAs will be divided into four equally sized groups and asked to provide food purchase data for one quarter of School Year (SY) 2021-2022. The results will be aggregated to produce annual estimates of food purchases.

Our SFA frame will be based on the most recently available FNS-742 Verification Summary Report data that we will obtain from FNS. This data file contains the list of SFAs currently participating in the NSLP and the SBP, as well as SFA-level information such as type of school and number of schools and students. We will supplement this frame with the Common Core of Data (CCD) “Local Education Agency (School District) Universe Survey” from the National Center for Education Statistics (NCES) that contains additional information about districts, including urbanicity. We plan to stratify by whether an SFA uses a FSMC and to oversample FSMC SFAs to achieve the ±10 percent precision target for this subgroup. We will identify SFAs that use FSMCs via data from the FNS-640 Administrative Review Report Form and/or the most recently complete Farm to School Census Survey. Within each FSMC stratum, we will use a probability proportionate to size (PPS) sample, with enrollment as the measure of size. Use of simple random sampling would place equal weight on all SFAs and yield many small SFAs. Use of PPS sampling is statistically efficient and ensures that large SFAs are included in the sample.

We will divide the SFA sample into four equal-sized quarters as was done in the previous three School Food Purchase studies. Each quarter, we will collect three months of data from one quarter of the sample. We will weight these four quarterly samples to be one-fourth each of the full estimate, and put the quarters back together to provide estimates for the full year. This approach has the advantage of imposing a low burden on the respondents and ensures that FNS gains an estimate of food purchases that is not biased by any seasonal effects. Keeping the burden low is paramount to achieving a high response rate.

The universe for SDAs includes all 48 contiguous states and the District of Columbia. SDAs will be included based on the SFAs selected. No statistical sampling will be used to select SDAs.

Expected Response Rates

The expected response rate for SFAs is 67 percent, which is the response rate that was achieved for SFPS-III. However, 67 percent is a conservative response rate due to the technological changes over the past decade that have made it less burdensome for SFAs to provide the requested data. In addition, we will follow an established process for contacting SFAs, beginning with notifying key stakeholders about the study through well-established FNS communication channels used for other studies, then offering a user-friendly study website and technical assistance, and conducting nonresponse follow-up via email reminders and telephone calls. We will also obtain letters of support from two key national professional associations that are prominent in the school nutrition community: the School Nutrition Association and the American Commodity Distribution Association. To allow for nonresponse, a total of 630 SFAs will be sampled with a replacement sample of 10 percent as backup, in case the response rate is lower than expected. All SDAs are expected to respond to the data collection activities.

The estimated size of the respondent universe for the study is presented in Table B.1.

Table B.1. Respondent Universe, Samples, and Expected Response Rates

Respondent	Universe	Initial Sample	Target Completed Cases	Response Rate
SFAs	15,190	630	422	67%
FSMC SFAs	3,038	158	106	67%
Non-FSMC SFAs	12,152	472	316	67%
SDAs	49	49	49	100%

Note: The universe of 15,190 SFAs is from the FNS-742 Verification Summary Report, school year 2018-2019. The universe may be slightly smaller because the estimate includes charter school SFAs, which will not be eligible for the study.

B.2 Procedures for the collection of information:

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

Statistical methodology for stratification and sample selection

The 630 SFAs will be drawn using FSMC status (yes or no) as an explicit stratum. A total of 158 SFAs (one-quarter) will be drawn within the FSMC stratum, and 472 SFAs in the non-FSMC stratum, to guarantee 25 percent of the sample in the FSMC stratum. Within the two explicit stratum, the sample will be drawn using systematic sampling with the 10 USDA Farm Production Regions and size (enrollment) to determine the sort order. This will implement implicit stratification and allow for some control of sample sizes by FNS region (and also by size). The sample will be a Probability Proportionate to Size (PPS) sample by size. The sample of 630 SFAs will be divided into four quarters respecting the explicit and implicit stratification (in the FSMC stratum, there will be 40, 40, 39, and 39 SFAs for each quarterly sample). A further reserve sample of 10 percent will be selected, again respecting the explicit and implicit stratification (and PPS by size design).

Estimation procedures

In general, we need weights in the analysis to (1) reflect the differential probabilities of selection and (2) compensate for survey nonresponse. We will design the weights to produce nationally-representative estimates of the SFA population. The base weight for a sampled SFA is equal to the reciprocal of the probability of selecting the SFA from the sampling frame. Next, we will adjust the SFA base weight for nonresponse. We will make nonresponse adjustments within groups of SFAs referred to as “weighting classes” that are internally homogeneous with respect to response propensity. To construct the weighting classes, we will conduct a search analysis for cells that are homogeneous in response propensity. We plan to check for extreme weights (which increases variance) and do trimming if necessary. We will carry out poststratification to adjust the SFA weights to frame-based control totals, to reduce variability. Poststratification cells will be based on factors such as urbanicity, SFA size, percent of children eligible for free or reduced-price meals.

To adjust for nonresponse bias, we will make nonresponse adjustments to offset differences in response propensity. This will prevent systematic differences in response propensity that may cause biases in the estimates. We will use a cell-based approach where weighting cells that are heterogeneous in response propensity are developed based on an analysis of response propensity, with weighting adjustments equal to the inverse of weighted response rates within the selected weighting cells.³

We will use jackknife replicate weights to estimate variance for estimation and analysis purposes. Replication weights can be utilized now by many commercially available software packages (SAS, WesVar, STATA) to provide consistent variance estimates for a wide variety of population totals, means, percentiles, ratios, model parameter coefficients, etc. We will form the jackknife by systematically deleting small sets of sample units for each replicate, and reweighting the remaining sample units. Each set of replicate weights will be put through the weighting algorithms so that the replicate weights reflect nonresponse adjustments and poststratification (raking) adjustments, as is standard practice.

Degree of accuracy needed for the purpose described in the justification

FNS requires that the survey sample design will result in a nationally representative sample of SFAs. Additionally, FNS requires the sample design will achieve precision levels of +/-5 percent at the national level and +/-10 percent at the subgroup level. There are two potential sources of design effects (increase in variance) that we need to account for when determining the required sample size. First, the nonresponse weighting adjustments will induce a small design effect. We assume a four percent increase in variance,⁴ which should be offset in slightly higher sample sizes. Another sample design issue is the need to oversample SFAs that use FSMCs. Assuming proportional sampling, any subgroup which is at least one-quarter of the universe will have a sample size of at least 100, achieving the ±10 percent precision target. Assuming that 20 percent of SFAs use FSMCs, the total sample size would need to be 404 in order to achieve the 10 percent precision target for FSMC SFAs.⁵

To offset the various design effects and the need to oversample SFAs that use FSMCs, we propose a sample size of 420 SFAs. For sample percentages based on binary (0-1) questionnaire items, the maximum standard error occurs with a sample percentage of 50 percent.⁶ Assuming the design effects noted above, a sample size of 420 is sufficient to achieve the half-width of ±5 percent and a sample size of 100 is sufficient to achieve the half-width of ±10 percent for subgroups. Percentages that are aggregations of SFA-level percentages (for example, the percentage of food acquisitions that are vegetables by weight), will have lower standard errors because the population standard deviation is smaller for this type of percentage.⁷ Thus, the sample size of 420 based on the simple binary sample percentage of 50 percent is comfortably conservative. All other sample percentages will have lower standard errors and higher precision.

Additionally, FNS requires that the sample design will also support comparisons among subgroups of SFAs. Table B2 provides Minimum Detectable Effect Sizes (MDESs) and Minimum Detectable Differences (MDDs) for a sample size of 420 SFAs (which, after accounting for the design effects is an effective sample size of 400). The sample is split between two subgroups of size 210 and 210 (half the sample as against the complement), and two subgroups of size 105 and 315 (one quarter of the sample as against the complement).⁸ The latter would be similar to the split expected when comparing SFAs that use FSMCs to those that do not. The MDES is the minimum difference in mean percentages that can be detected as a fraction of the null population mean.⁹ MDESs of less than 20 percent are considered acceptable. The MDD is the percentage point difference corresponding to the MDES.

One challenge of calculating MDDs for the study is that there are numerous foods items, groups, and subgroups which may be compared. Therefore, we present a range of population mean percentages from 5 percent to 20 percent. Food groups of interest for comparison over time fall within these ranges. For example, in SFPS-III, vegetables accounted for 9 percent of all food acquisitions by SFAs and milk accounted for 22 percent of all food acquisitions. MDD calculations require an assumption about the size of population standard deviation. We assume that the standard deviation is 25 percent of the mean (a low to moderate amount of dispersion) and 50 percent of the mean (a moderate to high amount of dispersion). ¹⁰ Table B2 shows that for a mean percentage of 10 percent and a moderate/high amount of dispersion, the MDES is 14.6 percent, which in turn translates to an MDD of 1.46 percent. While the MDESs are higher for the 75/25 split assuming a moderate/high amount of dispersion, all of the MDESs are below the acceptable threshold of 20 percent. Disaggregated food subgroups or individual food items (such as low fat milk or whole grains) with population percentages of less than 5 percent would have even smaller MDDs than those presented here. Therefore, we believe that the corresponding MDDs are acceptable.

Table B2. Statistical Precision and Minimum Detectable Differences

Population percent	Low/Moderate Std. Dev. (Std. Dev. = 25% of Mean)				Moderate/High Std. Dev. (Std. Dev. = 50% of Mean)
	50/50 split		75/25 split		50/50 split		75/25 split
	MDES	MDD	MDES	MDD	MDES	MDD	MDES	MDD
5	7.00%	0.35%	8.09%	0.40%	14.01%	0.70%	16.17%	0.81%
10	7.00%	0.70%	8.09%	0.81%	14.01%	1.40%	16.17%	1.62%
20	7.00%	1.40%	8.09%	1.62%	14.01%	2.80%	16.17%	3.23%

Unusual problems requiring specialized sampling procedures

We do not anticipate any unusual problems requiring use of specialized sampling procedures.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden

This is a one-time study; concern regarding the periodicity of data collection cycles is not applicable.

B.3 Describe methods to maximize response rates and to deal with issues of nonresponse. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield "reliable" data that can be generalized to the universe studied.

Overall response rate projections are presented in Section B.1. We will use a wide array of strategies to maximize participation and reduce nonresponse for all data collection activities:

• We will obtain letters of support from the School Nutrition Association (SNA) and the American Commodity Distribution Association (ACDA), which will be included in a FedEx package providing information about the study (Appendix A5). SNA and ACDA are national, nonprofit professional organizations that include many SFAs, industry representatives, and State agencies as members. The letters will be worded to emphasize the importance of participation and how the information from the study will help SFAs to be more cost efficient in efforts to meet meal requirements.

• FNS will send initial informational emails to the State agencies, and large FSMCs and vendors, and encourage them to participate in the study (Appendices A1 and C1). Following the FNS email and notification by the study team, State Child Nutrition (CN) agencies will send an informational email to the sampled SFAs in their State (Appendix A4). The email will provide an overview of the study and urge SFAs to participate in the study.

• Current SFA contact information will be used for all initial correspondence and updated as needed throughout the data collection period to facilitate communication between the SFA and the study team. The initial email from the study team to State agencies informing them of their sampled SFAs will ask the State Directors to review and provide updates to SFA contact information (Appendix A2).

• All study participants will have access to resources from the study website (Appendix D5) and the study’s Technical Assistance Center (TAC). A toll-free number and study email address will be provided so that SFA directors can receive assistance. The TAC will be available from 9am to 4pm EST and the message line will be available 24/7, with responses provided the following day.

• We plan to conduct an initial (live) Study Overview Webinar (Appendix B4) and quarterly Purchase Data Webinars (Appendix B9) and make these available to SFAs through the study website (Appendix D5). The recordings provide SFAs with the flexibility to view the webinars as well as other study information available on the study website at their convenience.

• The study team will send email reminders (Appendix A8) as needed to SFA directors who have not completed the Survey of Food Purchase Practices or provided the school food purchase data. Similarly, the study team will send email reminders (Appendix A8) as needed to SDAs who have not submitted the SFA Level USDA Foods Data

• We will offer a $300 incentive to SFAs who complete the Survey of Food Purchase Practices and submit complete food purchase data for one quarter. SFAs will be encouraged to use the incentives towards the cost of attending a conference or similar professional development opportunity.

• Before considering SDAs or SFAs as nonrespondents, the final contact with all nonrespondents will be attempted via telephone (Appendices A9, B6, and B12). We will schedule calls at different times of the day and on different days of the week to increase the chances of reaching nonresponding SFAs.

• If necessary, State agency staff will follow up with nonresponding SFAs and encourage participation.

Plans to Boost Response Rates and Address Nonresponse

Despite our efforts, it is possible that the response rate will fall below 80 percent. We will conduct nonresponse bias analysis if the response rate is less than 80 percent. The two main goals of the nonresponse bias analysis are to assess and document the impact nonresponse may have on estimates derived from a survey, and to inform the construction of sampling weights that will be effective in attenuating potential nonresponse biases.

First, we will summarize response rates for subgroups of SFAs defined by important characteristics. SFA characteristics will be derived from the FNS-742 sample frame and district-level variables from the Common Core of Data (CCD) LEA universe file and will include SFA enrollment size class, poverty status based on percentage of students eligible for free/reduced price meals, FNS region, urbanicity, minority enrollment based on the percentage of students who are minority, and the percentage of students approved as directly certified. We will test if there is a statistically significant association between the weighted and unweighted response rates and each of the specified characteristics (i.e., differences of response rates across levels of the characteristic).

Second, we will compare characteristics of respondents to nonrespondents to determine if there is a potential for bias. Specifically, we will compare the base-weighted distributions of the respondents and nonrespondents for the same categories of SFA characteristics that we use above for summarizing response rates. Comparison of the base-weighted distributions of responding SFAs with the corresponding base-weighted distributions of the complete sample will provide an indication of the potential impact of nonresponse on the survey-based estimates. We will test the hypothesis that the base-weighted distribution of the respondent sample is the same as the distribution of the total sample for a given characteristic.

Third, we will generate preliminary nonresponse weighting adjustments. We will use search algorithms to identify subsets in which predicted probabilities of response are similar, based on characteristics available for both respondents and nonrespondents. We will use a CHAID analysis (Chi-square Automatic Interaction Detector) to identify subgroups of SFAs in which the predicted probabilities of response are similar. The same SFA characteristics used above will be used as potential predictor variables in the CHAID analysis. The output from the CHAID analysis will be a tree diagram that defines the final cells used in the nonresponse adjustment. The nonresponse adjustments will be weighted reciprocals of response rates within the cells. These nonresponse adjustments are designed to eliminate the response rate differences.

Fourth, we will compare the nonresponse-adjusted weighted distribution of SFA characteristics in the respondent sample to the base-weighted distribution of SFA characteristics in the full sample. The objective here is to see whether the nonresponse adjustments reduce or eliminate biases identified in the second step. We will also compare the nonresponse-adjusted weighted estimates of survey items to the base-weighted estimates of survey items in the response sample. The objective is to determine the effect of nonresponse adjustments on survey estimates.

B.4 Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Pretesting of study instruments was conducted with four SFA Directors; separate OMB approval was not required. The burden associated with pretesting is included in the burden estimates for this data collection. Because there were minimal changes to the data collection instruments used in SFPS-III, the pretest focused on the new and revised items in the survey, terminology to reference data sources (e.g., vendor summaries, velocity reports, etc.), available data formats (e.g., Excel workbooks, PDF, etc.), and the feasibility of obtaining the data from SFAs electronically.

The four SFA Directors who participated in the pretest varied with regards to SFA size and level of free and reduced price students, as well as in their use of buying cooperatives and FSMCs. Pretest participants were emailed draft study materials in advance and asked to attend a telephone debrief session. Specifically, the study team sent a two-part draft of the survey and a list of the data elements requested for food purchase data. During the telephone debrief, a semi-structured pretest protocol was used to elicit their feedback , and respondents were provided an opportunity to provide general comments about the data collection materials. SFAs could also provide any comments in writing following the telephone debrief. The discussions with SFAs elicited their input on who would be the correct respondent for their district, how they would gather the necessary information so that we could include this in the instructions, clarification on items in the survey, and the timeline and burden to obtain the data from SFAs. The findings from the pretesting are included in Appendix D3. In summary, the following changes were made to the instruments and data collection plan as a result of the pre-testing:

• Merged the two-part survey into one Survey of Food Purchase Practices that includes all questions, and changed the timing of data collection. The original data collection plan called for SFAs to complete a short web-based “Initial Vendor List” at the beginning of the quarter for which they were sampled, and provide detailed food purchase data and complete the web-based Survey of Food Purchase Practices after the quarter ended. There was concern by pretest respondents about the burden of completing the longer Survey of Food Purchase Practices at the same time they are compiling food purchase data. The revised approach will deploy a single instrument, the Survey of Food Purchase Practices, to all SFAs at the beginning of data collection. This will reduce the number of data collection activities for SFAs, and spread out their burden over a longer period of time. It will also maximize the data collection window for obtaining responses to the survey. In addition, combining the two data collection instruments will reduce the number of respondents who complete one but not the second data collection activity.

• Added a PDF copy of the Survey of Food Purchase Practices as an attachment to the email to SFAs requesting completion of the survey (Appendix B5). SFAs recommended including this so that they can review the full questions and gather the necessary information prior to launching the web survey.

• Revised the terminology of several questions. These changes to a small number of survey items used in the prior study were warranted due to the changes in food purchase practices in the past decade.

Statistical methods remained unchanged after pre-testing.

In addition, we plan to conduct a field test with three SFAs, to ensure that our data collection methodology is able to capture the school food operations and procurement practices. Planned study procedures will be administered to collect the data from invited SFAs. In addition to obtaining feedback from the SFAs on the process of providing study information, we will also review the purchase data and the invoices submitted by SFAs.

B.5 Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

Table B5 presents a summary of individuals consulted on study design, data collection, and/or analysis. The information will be collected and analyzed by Westat and Agralytica. The sampling and weighting procedures were developed by Lou Rizzo and are consistent with those used in SFPS-III (OMB Control Number 0584-0471, Discontinued 03/21/2012). The sampling plans were reviewed by Adam Chu, Laurie May, Joseph Gasper, and Melisa Rothstein at Westat. The study plan, including the data collection approach was also reviewed by Sujata Dixit-Joshi and Thea Zimmerman at Westat. Three subject matter experts also reviewed the study plan: Dr. Keith Rushing, Director of Applied Research Division at the Institute of Child Nutrition; Mr. Chris Facha, USDA Foods Program Administrator, Oregon Department of Education; and Mr. Bertrand Weber, Director of Culinary and Nutrition Services, Minneapolis County Public Schools, MN. The statistical procedures included in this information request have also been reviewed by Dana Butler with the USDA National Agricultual Statistics Service (NASS) (telephone: 202-690-8627). Comments from NASS are included in Appendix E, and responses are incorporated in this supporting statement.

Table B5. Individuals Consulted on Design, Data Collection, or Analysis

Name	Title	Telephone Number	Email
Laurie May	Vice President and Associate Director, Westat	301-517-4076	[email protected]
Joseph Gasper	Associate Director, Westat	240-314-2470	[email protected]
Melissa Rothstein	Project Director, Westat	301-315-5975	[email protected]
Adam Chu	Associate Director and Statistician, Westat	301-251-4326	[email protected]
Lou Rizzo	Sampling Statistician, Westat	301-294-4486	[email protected]
Sujata Dixit-Joshi	Deputy Project Director, Westat	240-314-2442	[email protected]
Thea Zimmerman	Nutritionist and Senior Study Director, Westat	240-314-2413	[email protected]
Salli Diakova	Vice President, Agralytica	703-739-9090	[email protected]
Keith Rushing	Director of Applied Research Division, Institute of Child Nutrition	601-543-9866	[email protected]
Chris Facha	USDA Foods Program Administrator, Oregon Department of Education	503-947-5896	[email protected]
Bertrand Weber	Director of Culinary and Nutrition Services, Minneapolis County Public Schools, MN	612-668-2821	[email protected]
Stephanie Bruce	Nutrition Service Director, Palm Springs Unified School District, CA	760-322-4117	[email protected]
Lynn Petrowski	Director of Food Services, Hanover Schools, MA	7810878-5450	[email protected]
Jinee Burdg	Policy Analyst, FNS USDA	703-305-2744	[email protected]
NASS	Mathematical Statistician	202-690-8627	[email protected]

1 The School Nutrition Program Operations Study (SNOPS) study found that in SY 2012-13, 22 percent of SFAs used FSMCs. The School Nutrition and Meal Cost Study found that in SY 2014-15, 20 percent of SFAs used FSMCs.

2 The sample sizes are stated in terms of numbers of participating SFAs (that is, the target completed sample sizes). The sizes of the sample selected will be expanded to allow for nonparticipation due to ineligibility or noncooperation.

3See for example Valliant, R., Dever, J. A., Kreuter, F. (2014), Practical tools for designing and weighting survey samples. Springer (www.springer.com), Section 13.4.

4If the response rate is 67 percent, the mean nonresponse weighting adjustment will be 1.5. One formula for the increase in variance from weighting adjustments is the Kish 1+CV² formula, where CV is the coefficient of variation in the weights. If we assume a CV in the weighting adjustments of 0.2 (or a standard deviation of 0.3, if the mean is 1.5), then 1+ CV² will be 1.04. This is fairly small, but the previous cycle report indicated very limited differences in response propensity across subgroups, so we believe it will be accurate.

5For some subgroups that are less than 25 percent of the population, strata can be defined to meet the precision requirement. For example, enrollment size strata can by defined based on quartiles, such that each stratum has 25 percent enrollment. Poverty status can also be handled in this way (e.g., terciles of percent of students approved for free and reduced price meals).

6For simple random samples, the standard error is p*(1-p)/n, where p is the binary percentage and n is the sample size, and this is maximized when p equals 50 percent.

7The Bernoulli variance is p*(1-p), reflecting all of the probability distribution at 0 and 1. Each SFA is either 0 or 1. For percentages at the SFA level (e.g., 10%, 15%, 20% over different SFAs), the probability distribution falls on percentages rather than the two endpoints 0 and 1, and the resultant standard deviation will be smaller.

8MDES and MDD calculations are based on an effective sample size of 400 after design effects are taken into account.

9MDES calculations assume a two-sided test with significance level 95 percent, with 80 percent power.

10 In a normal distribution, 68 percent of the sample falls within one standard deviation and 95 percent within two standard deviations. For a mean percentage of 10 percent, a standard deviation that was 50 percent of the mean would indicate that 68 percent of the sample would fall between 5 percent and 15 percent.