Appendix 27.a NASS Comments

Supporting Statement for OMB Clearance for the Study of Non-Response to the School Meals Application Verification Process

Appendix 27a

NASS Comments

NASS Review of OMB Documents on Non-Response to the School Meals Application Verification Process

Review was conducted by Alison Black – Mathematical Statistician

June 29, 2017

I. Respondent Universe and Sampling Methods

The purpose of this study is clearly stated. All terminology and concepts related directly to this study are thoroughly defined and explained. The argument for the usefulness of the results is also well written.

There is error associated in the sample selection of convenience method described in the docket. For example, selecting districts based on geography and having the necessary data in a central location. Although as stated this is not a nationally representative study and as long as these procedures are clearly stated in the publication they seem reasonable.

All of the numbers in Table B 1.1 are clearly described and calculations are shown throughout the document except for how the Target Completed Cases counts were initially determined. It would be helpful to show/describe the calculations for how these numbers were derived for the resulting confidence intervals for the number of districts and households within each sample as was done in the footnote for the Initial Sample.

What will happen if 20 of the 25 selected districts can’t participate? Will additional districts be selected and if so, how?

II. Data Collection Procedures

The data collection procedures outlined in this document all seem appropriate. The different plans for contacting and working with directors as opposed to households is clearly identified. Although it wasn’t clear whether the $25 Visa Gift card will be mailed with the household questionnaire to be used if the household completed the survey or not OR will it be mailed upon receiving a completed questionnaire.

III. Maximizing Response Rates

The bulleted plans for maximizing response rates and data quality are described in detail. The plan to send in-person interviews to district and household non-responders will most likely be a useful follow up method.

When contacting participants by phone, if a voicemail will be left or if the participant is busy and prefers to call back, the wording that will be left in a voicemail or as instructions to call back should be documented so all members of the interviewing team will use the same wording and include the same information in their messages.

IV. Test of Procedures

The pretest procedures were well documented and appear to be designed appropriately to collect the necessary information. No other comments for this section.

V. Statistical Methods for Summary

The non-response analysis is well described. There was mention of confidence intervals for analysis across districts and household. There could be more information on the type of results that will be tabulated related to the quantitative side of the study. This could be brief if it will just be basic frequencies or cross tabulations, but if there are any plans for more complex analysis or summaries, it should be documented.