OMB Part B_Final

0584-NEW SUPPORTING STATEMENT PART B FOR

Jinee Burdg

Office of Policy Support

Food and Nutrition Service

US Department of Agriculture

3101 Park Center Drive

Alexandria, VA 22302

Phone: 703- 305-2744

E-mail: [email protected]

5/25/17

Table of Contents

Part Page

B Collections of Information Employing Statistical Methods 1

B.1 Respondent Universe and Sampling Methods 1

B.2 Procedures for the Collection of Information 2

B.3 Methods to Maximize Response Rates 4

B.4 Test of Procedures or Methods to be Undertaken 4

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 5

Appendices

Appendix B: Recruitment Materials

B-1. Initial Study Notification E-Letter to all FNS Regional CN Directors

B-2. Initial Study Notification E-Letter to all FNS Regional SNAP Directors

B-3. Initial Study Notification E-Letter to all State CN Directors

B-4. Initial Study Notification Letter to all State SNAP Directors

B-5. State Selection Notification E-Letter From FNS to FNS Regional Directors

B-6. State Selection Notification E-Letter From FNS to 4 Selected CN State Directors

B-7. State Selection Notification E-Letter from FNS to 4 Selected SNAP State Directors

B-8. SFA Study Notification E-Letter

B-9. School Study Notification E-Letter

B-10. Contact Guide_previsit State CN

B-11. Contact Guide_onsite interviews

B-12. Study FAQs

Appendix C: Interview Protocols

C-1. State Child Nutrition Director Pre-Test Protocol

C-2. State Child Nutrition Director Pre-Test Protocol

C-3. State Child Nutrition Key staff Pre-Test Protocol

C-4. State SNAP Director Pre-Test Protocol

C-5. SFA Pre-Test Protocol

C-6. School Pre-Test Protocol

C-7. State Child Nutrition Guide - Pre-Visit

C-8. State Child Nutrition Director Guide

C-9. SNAP Director Guide

C-10. State-level Key Staff Guide-FNS 10

C-11. State-level Key Staff Guide- FNS 742 & FNS 834

C-12. School Food Authority Guide

C-13. School Food Manager Guide

Appendix D: Summary of Public Comments

D-5. National Agricultural Statistics Service (NASS) Comments

Part B

Collections of Information Employing

Statistical Methods

B.1 Respondent Universe and Sampling Methods

Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.

Respondent Universe

The respondent universe for this 4 State qualitative case study includes all schools that administer the National School Lunch Program. Of the four States that will be selected, the sample will contain 4 State Child Nutrition Directors, 20 State Child Nutrition key staff, 4 State SNAP Directors, 2 State SNAP key staff, 47 School Food Authority (SFA) Directors and 120 School Cafeteria Managers. The expected response rate is 4 State Child Nutrition Directors, 20 State Child Nutrition key staff, 4 State SNAP Directors, 2 State SNAP key staff, 40 School Food Authority (SFA) Directors and 120 School Cafeteria Managers.

Sampling Methods

States, SFAs, and schools/sites will be selected to yield diversity in order to provide the richest descriptive information for FNS. The following criteria will be used to select the States: State size, geography (i.e., representing different FNS regions), use and type of management information system (MIS), type of direct certification systems and performance, and FNS assessment of State reporting performance for Child Nutrition Programs.

First, an electronic study notification letter will be sent to all States participating in the National School Lunch Program, informing them of the study. Once the selection of four States is finalized, an electronic letter informing those States of their selection with an invitation to participate will be sent. SFAs in each selected State will represent a mix of urbanicity and size. Form FNS-742 data from School Year (SY) 2015-16 will be used for each State to classify SFAs based on key factors such as public/private status, number of schools, level of free and reduced-price certifications, participation in alternate provisions, and others. In addition, FNS-742 data will also be used to stratify SFAs into size (small versus large based on number of enrolled students, where SFAs with less than 500 students are considered small). National Center for Education Statistics Common Core of Data will be used to stratify SFAs into urbanicity (urban vs. not urban). To the extent information is available, we will also consider diversity in Point of Sale (POS) systems used by SFAs.

Within each selected SFA, we will work with the SFA to select three schools to participate in the evaluation, for a total of 30 schools within each selected State. For most SFAs, one elementary school, one middle school, and one high school will be selected. If there are SFAs with fewer than three schools, we will select additional schools from larger SFAs in the State. We will maintain constant contact during each step of school selection and communication with the relevant State agency and SFAs to obtain their recommendations, as needed.

B.2 Procedures for the Collection of Information

Describe the procedures for the collection of information including:

• Statistical methodology for stratification and sample selection,

• Estimation procedure,

• Degree of accuracy needed for the purpose described in the justification,

• Unusual problems requiring specialized sampling procedures, and

• Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

In order to elicit qualitative responses at each level of program administration to effectively assess and evaluate how the FNS-10, FNS-742 and FNS-834 forms are completed and submitted, the following steps will occur:

1) Regional and State SNAP and CN Directors will receive an email notifying them of the study, and informing them that four States will be selected to participate (see Appendices B-1 through B-4).

2) Respondents in the four selected States will receive an electronic letter informing them of their selection in the study, (see Appendices B-5 through B-9) and a Frequently Asked Questions (FAQ) document to help answer any questions they have about the study or their participation (see Appendix B-12). All qualitative data collection except for the State CN key staff will be in-depth interviews. State CN key staff will participate in a focus group by State CN agency. There will be no more than three participants in each key staff focus group.

3) Respondents will be contacted to schedule the on-site interview or focus group (see Appendices B-10 and B-11).

4) Prior to conducting either the in-depth interview or focus group, participants will sign an informed consent form found at the start of each interview guide (Appendices C8 through C13 ).

5) In-depth interviews with State CN, SNAP Directors, and School Food Managers varies and will not exceed one hour each of their burden is clearly identified in the excel spreadsheet A-4. In-depth interviews with SFA Directors and Child Nutrition key staff will not exceed 1.5 hours. All interviews and focus groups will occur under the direction of a professionally trained researcher. A discussion guide will be used throughout the duration of each session (Appendices C-8 through C-13).

All focus group and in-depth interviews for this information collection will be audio-recorded, and verbatim notes will be compiled for each group. For the analysis, an iterative approach will used, first analyzing the rich and detailed information we collect during on-site qualitative interviews. Analysis will include review of data elements with most frequent errors by each of the three forms and any variations among school and SFA size. We will also analyze methods of data collection and aggregation by form as well as school and SFA size in addition to most frequent method and modes of data transmission for each form. Most frequent sources of 30- and 90-day meal count volatility on the FNS-10 form as well as type and frequency of data quality control measures will be explored.

B.3 Methods to Maximize Response Rates and to Deal with Issues of Nonresponse

Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.

Notification from the Food and Nutrition Service through FNS regional offices to the four State CN will help maximize response rates. Similar notification will be made to SNAP agencies. In addition, we will also enlist the support of the State CN Director in communicating the importance of participation to State CN key staff within their agencies and to the SFA Directors. An electronic study notification letter will be sent to all respondents as well as a follow up phone call to schedule the interview. We expect to replace some of the SFA Directors initially selected to accommodate scheduling conflicts which is reflected in the sampling universe of 47 SFA Directors and 40 SFA Directors in the expected sample.

B.4 Test of Procedures or Methods to be Undertaken

Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.

Pre-test interviews were conducted with each of the six interview guides (Appendix C-1 through C-7). Each guide was tested with one relevant staff member to ensure that the respondents interpreted the questions as intended and could easily respond, and that the trained interviewer could easily administer the instruments. Trained interviewers administered the full in-depth interview guide, observed and documented any issues that arose for both respondents and interviewers, and discussed any points of difficulty with respondents. Findings and recommendations from the cognitive testing were used to refine the guides and question wording.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.

The proposed protocol and discussion guide were developed and reviewed extensively by FNS and Westat staff. FNS staff and Westat will participate in the analysis of the data, as well as development of reports. In addition, Brent Farley, Mathematical Statistician with the National Agricultural Statistical Service’s Summary, Estimation, and Disclosure Methodology Branch reviewed Part A and Part B of this OMB Clearance Package (Appendix D-5).

Name	Affiliation	Telephone number	e-mail
Melissa Rothstein	Westat	301-315-5975	[email protected]
Laurie May	Westat	301-517-8068	[email protected]
Mustafa Karakus	Westat	301-294-2874	[email protected]
Allison Roeser	Westat	301-279-4537	[email protected]
Lindsay Giesen	Westat	978-443-3602	[email protected]
Jinee Burdg	USDA/FNS	703- 305-2744	[email protected]
Brent Farley	NASS	202-690-8122	[email protected]