Appendix I.2 FNS Response to National Agricultural Statistics Service Comments

Appendix i.2

fns response to National Agricultural statistics service comments

Food and Nutrition Service

Revision to OMB # 0584-0607, School Meals Operations Study (SMO)

Doug Kilburg

NASS Reviewer

Most of the study was well documented and justified. Most of the suggested comments below are for enhanced documentation for mostly clarity purposes but may require some adjustments to the plans if the criteria aren’t met.

Part A:

A1: no comments

A2: We at NASS have found early reminders sent at short intervals are the most effective in soliciting improved response rates.

FNS Response: We generally agree. We proposed the SMO recruitment and reminder strategy, including the intervals for follow-up and the approaches used, based on past experience with surveys of State Child Nutrition Directors and School Food Authorities.

A3-A18: no comments

Part B:

B1: The proposed sampling strategy seems possible but will require some weighting strategy adjustments to make it feasible. Clearly outline how the weighting will be handled if the minimum number of reports are not met within a stratum and the reserve sample is utilized. Typically, sample quotas are only used in a nonprobability-based setting where sample weighting isn’t implemented.

FNS Response: The following information was added to B1 to clarify how weighting will be handled if the minimum number of reports are not met within a stratum and the reserve sample is utilized:

• The final sampling weights will then be calculated as the product of the initial sampling weight (the inverse of the probability of selection for the augmented sample) and the release adjustment (the number of cases selected for the augmented sample in each stratum divided by the number of released cases in that stratum).

B2: The documentation states, “The web survey of State CN Directors and the disaggregated administrative data collection will be conducted with a census of State CN directors. Because this study involves a census of SAs and a 100 percent response rate is expected, there is no need for sampling, weighting or nonresponse adjustments at the state level.” I’m not sure how a 100% response rate can be guaranteed from a survey of state CN directors. Even with statutory reporting, unless a penalty is enforced for not reporting. If not, it would be feasible to have a back-up plan in place for nonresponse weighting adjustments, if a 100% response rate is not obtained.

I did see a comment that telephone follow ups will be made with directors who do not complete the survey; however, a telephone follow-up is not likely to result in a 100% response rate and would necessitate a nonresponse weighting adjustment plan. It should be relatively easy to implement using the strata design.

FNS Response: In the CNOPS-II study, SMO’s predecessor, as well as in the base period SMO data collection, all SAs completed the surveys. During the base period SMO data collection, the team conducted email reminders and telephone follow-ups, as planned, and we plan to again use those approaches to help reach a 100 percent response rate. In addition, when developing the SMO-II and SMO-III SA surveys included in this ICR, changes were made to the surveys based on pretest feedback, as well as feedback regarding the base period SA survey, to lower respondent burden and ease survey completion.

B3. What are the characteristics of the respondents that you plan to assess for non-respondent bias across the frame, sample and respondents? You reference them, but never list them.

FNS Response: The following information was added to B3 to clarify the SFA characteristics that will be used to assess the risk for non-response bias:

• These characteristics include variables available on the FNS-742 file, such as size measures (number of schools and number of students), percent of students with free or reduced-price lunch, and geographic characteristics. We will also include variables about the associated school district from the CCD file, such as charter school status, number of each type of school in the district (elementary, middle, and/or high) and grades offered in the district.

B4. Great responses to the results of the cognitive testing.

B5. No comments