SUPPORTING STATEMENT PART B FOR
Assessing the Child Nutrition State Administrative Expense Allocation Formula
Jinee Burdg
Office of Policy Support
Food and Nutrition Service
US Department of Agriculture
3101 Park Center Drive
Alexandria, VA 22302
Phone: 703- 305-2744
E-mail: [email protected]
9/26/2017
Part Page
B Collections of Information Employing Statistical Methods 1
B.1 Respondent Universe and Sampling Methods 1
B.2 Procedures for the Collection of Information 4
B.3 Methods to Maximize Response Rates 6
B.4 Test of Procedures or Methods to be Undertaken 7
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 9
Appendices
Appendix A: Study Materials
A-1. Richard B. Russell National School Lunch Act (Section 28(c))
A-2. Child Nutrition Act of 1966 (Section 7)
A-3. 7 CFR Part 235.4
A-4. Westat IRB Approval Letter
A-5. Statement of Confidentiality and Nondisclosure
A-6. Burden Table
Appendix B: Recruitment Materials
B-1. State Selection Notification E-Letter From FNS to FNS Regional Directors
B-2. State Selection Notification E-Letter From FNS to State Directors in 12 Selected States
B-3. Contact Guide: Scheduling Previsit Telephone Interviews
B-4. Email to State Directors: Scheduling Site Visits
B-5. Email to Key Staff: Scheduling Site Visits
B-6. Email to State Directors/ Key Staff: Follow-up
B-7. Email to State Directors/ Key Staff: Thank you
Appendix C: Interview Protocols
C-1. Previsit Telephone Interview Guide
C-2. Site Visit Interview Guide
C-3. Pretest: Previsit Telephone Interview Guide
C-4. Pretest: Site Visit Interview Guide
Appendix D: Summary of Public Comments
D-1. Public Comment from Anonymous
D-2. Public Comment from Wyoming Department of Education
D-3. Public Comment from Minnesota State Department
D-4. Public Comment from American Commodity Distribution Association
D-5. Public Comment from the Office of the State Superintendent of Education (Washington DC)
D-6. Public Comment from School Nutrition Association
D-7. Public Comment from New Jersey Department of Agriculture, Division of Food and Nutrition
D-8. Public Comment from FL Department of Agriculture and Consumer Services, Division of Food, Nutrition and Wellness
D-9. Public Comment from North Carolina Department of Public Instruction, School Nutrition Services
D-10. Public Comment from Hunger Free Vermont
D-11. Public Comment from Texas Department of Agriculture
D-12. Public Comment from California Department of Education, Nutrition Services Division
Appendix E: Response to Public Comments
E-1. FNS Response to Wyoming Department of Education
E-2. FNS Response to Minnesota State Department
E-3. FNS Response to American Commodity Distribution Association
E-4. FNS Response to the Office of the State Superintendent of Education (Washington DC)
E-5. FNS Response to School Nutrition Association
E-6. FNS Response to New Jersey Department of Agriculture, Division of Food and Nutrition
E-7. FNS Response to FL Department of Agriculture and Consumer Services, Division of Food, Nutrition and Wellness
E-8. FNS Response to North Carolina Department of Public Instruction, School Nutrition Services
E-9. FNS Response to Hunger Free Vermont
E-10. FNS Response to Texas Department of Agriculture
E-11. FNS Response to California Department of Education, Nutrition Services Division
Appendix F: Comment from NASS
F-1. NASS Comment – FNS- Assessing the Child Nutrition State Administrative Expense Allocation Formula – July 13, 2017
Describe (including a numerical estimate) the potential respondent universe and any sampling or other respondent selection method to be used. Data on the number of entities (e.g., establishments, State and local government units, households, or persons) in the universe covered by the collection and in the corresponding sample are to be provided in tabular form for the universe as a whole and for each of the strata in the proposed sample. Indicate expected response rates for the collection as a whole. If the collection had been conducted previously, include the actual response rate achieved during the last collection.
FNS allocates State Administrative Expense (SAE) funds to 54 States, districts and territories—including the 50 States, the District of Columbia (DC), Guam, the Virgin Islands and Puerto Rico1--for the operation of five Child Nutrition Programs: the National School Lunch Program (NSLP), School Breakfast Program (SBP), Child and Adult Care Food Program (CACFP), Special Milk Program (SMP) and the Food Distribution Program for schools (FDP). In these States, there are 83 State agencies (SAs) that receive SAE funds. The respondent universe for this study includes the directors of these 83 SAs and their key staff involved in administering SAE allocations. In 31 States there is one agency administering all of these Child Nutrition Programs; 17 States have two agencies, and the remaining six States have three agencies. Each of these SAs has one director. There are approximately three key staff with duties relevant to SAE per SA, or 249 key staff across all 54 States. The universe of eligible respondents is the sum of all SA directors and their key staff, or approximately 332.
This study uses qualitative methodologies (i.e., key informant interviews) to address the research questions, and there will be no attempt to make statistical inferences about the population as a whole or to generate statistical estimates from the data. Therefore formal statistical sampling techniques are not required to select the States and individual respondents included in the study. Because States do vary, however, with respect to SAE funds usage history, program size, location, administrative structure and operations, and other factors, we will take a systematic approach to State selection so that the group of 12 States to be included in the study reasonably reflects the diversity and variation of programs nationwide. In the 12 States, we will interview the State Director and key staff for all State agencies that receive SAE funds in the selected State. In many States there is more than one State agency that receives SAE funds. All of the agencies in the selected States will be included in the sample. Therefore, we estimate there will be a total of 22 State agencies in the selected States, resulting in interviews of 22 State Directors and 66 key staff. We will use existing information about the distribution of the following characteristics to select a sample of States that achieves a balanced representation across these factors:
Program size. We will use current data on the initial SAE allocation amounts to identify each State as large (top third of States), medium (middle third of States), or small (bottom third of States), with an ultimate goal of roughly equal representation across the three categories.
Historical SAE Funds Usage. Using SAE data for the prior 10 years, we will identify States based on frequency of receipt of reallocated funds; return or recovery of funds vs. full use of funds; and funds transfers within agencies in a State.
State contribution levels. We will calculate each State’s contribution as a percentage of their statutorily required amount, to identify States that contribute significantly more than required.
State administrative structure. We will identify States with more than one agency operating the programs covered by SAE within the State. When there are multiple State agencies within a State, we will also look at whether the FDP is operated in a different agency than the School Meal Programs. This will allow us to explore processes, efficiencies and impacts in various scenarios related to FDP, given the potential challenges some States face with funding for FDP State administration.
Geography. We will ensure that there is at least one State from each of the seven FNS regional offices. This will help ensure we capture the full range of State interface with regional offices in gathering process data from the sampled States.
Once the selection of the 12 States is finalized, an electronic notification letter to Directors of all SAs that receive SAE funds in the 12 States will be sent. State Directors will be asked during the subsequent previsit telephone interview to identify the key staff within their agencies with responsibility for SAE.
We expect all State Directors and identified key staff to participate in the study interviews, with no selected respondents refusing participation. Based on FNS experience with other studies, State Directors in Child Nutrition agencies typically cooperate with such requests, especially when the request is made by the FNS regional office and with sufficient advance notice, as we are planning to do. For key staff, we anticipate the State Director will take into account staff availability as well as expertise when identifying staff for interviews. In addition, the State Director will support the interviews in communications with key staff and will provide time during the work day to participate. Finally, the research team will contact respondents to schedule the on-site interviews well in advance of the visits, and maintain flexibility to accommodate the availability of respondents and help ensure their participation.
Describe the procedures for the collection of information including:
Statistical methodology for stratification and sample selection,
Degree of accuracy needed for the purpose described in the justification,
Unusual problems requiring specialized sampling procedures, and
Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
B2.1. Statistical methodology for stratification and sample selection
As noted in Section B1, this study relies on qualitative data collection to answer the research questions. No statistical estimates will be generated and no statistical inferences to any population will be made. Statistical methods are not required to draw the sample, and there are no formal strata.
B2.2. Estimation procedure
No estimates are being produced, therefore no estimation procedures are proposed.
B2.3. Degree of accuracy needed for the purpose described in the justification
No statistical estimates are being produced, therefore no specific degree of statistical accuracy or margin of error is assumed.
B2.4. Unusual problems requiring specialized sampling procedures
None. A systematic approach, described in Section B1, to ensuring reasonable diversity and variation across key program characteristics relevant to the study goals was employed to select the sample of States.
B2.5 Any use of periodic (less frequent than annual) data collection cycles to reduce burden.
This collection is periodic (less frequent than annual). It is a qualitative study, implemented cross-sectionally. There will not be repeated or follow-on data collections. Our data collection sequence is presented below.
In order to elicit qualitative responses at each level of program administration to effectively assess and evaluate the current SAE formula, the following steps will occur:
1) FNS Regional Directors and State Agency Directors in the 12 selected States will receive an email notifying them of the study, and informing them of the 12 States that have been selected to participate (Appendices B-1 and B-2).
2) Respondents will be contacted to schedule the interviews (Appendices B-3, B-4, B-5).
3) Prior to conducting all in-person interviews, respondents will review and sign an informed consent form found at the start of the site visit interview guide (Appendix C-2). Prior to conducting all telephone interviews, the interviewer will read to respondents the same components of informed consent, and obtain each person’s verbal consent to participate and to be audio recorded (Appendix C-1).
4) Pre-visit interviews with State Agency Directors will not exceed 45 minutes. In-depth interviews with State Agency Directors and Key Staff will not exceed two hours. All interviews will occur under the direction of a professionally trained researcher. A discussion guide will be used throughout the duration of each session (Appendices C-1 and C-2).
All in-depth interviews for this information collection will be audio-recorded, and verbatim notes will be compiled for each group. Research team members will analyze notes and code to answer the study research questions.
Describe methods to maximize response rates and to deal with issues of non-response. The accuracy and reliability of information collected must be shown to be adequate for intended uses. For collections based on sampling, a special justification must be provided for any collection that will not yield “reliable” data that can be generalized to the universe studied.
This study does not rely on statistical sampling and will not generate estimates intended to be generalized to the population. However, the approach to selecting States for participation in the study will maximize variation and provide a cross-section of State Directors whose agencies receive SAE funds. The study will provide FNS with analysis of historical data on SAE funds usage for all States, and experiences from a broad cross-section of State agencies on State processes, barriers and challenges to managing SAE funds. This information may be used by FNS to help inform future policy and/or regulations to improve the SAE allocation process.
Full participation by the key informants from the 12 selected States can be expected. While this study is voluntary, under Section 28(c) of the Richard B. Russell National School Lunch Act (42 U.S.C. 1769i), States participating in the Child Nutrition Programs must cooperate in the conduct of studies and evaluations, and they typically do agree to participate. The study team notification from FNS to the FNS regional offices and to the State Agencies will help maximize response rates (Appendix B-1 and B-2). Notifications will be sent well in advance of the timeframe for on-site visits. If a State Director raises concerns about participating in the study, we will ask the FNS regional office director to call the State Director to answer questions, mitigate concerns and gain support for participation. If scheduling or timing of the study is of concern, the FNS regional director will convey the flexibility in scheduling to accommodate availability of key informants. If necessary, the State Director will be able to delegate responsibility for the State Director interviews to a senior staff person in the agency (e.g., Deputy Director). In addition, we will also enlist the support of the State Directors in communicating the importance of participation to key staff within their agencies. For key staff, we anticipate the State Director will take into account staff availability as well as expertise when identifying staff for interviews. The short length of the interviews—from 45 minutes to 2 hours—will also help facilitate participation. Because the interviews will be conducted in person, followup will be minimized and take place via email to reduce respondent burden. Study team staff will remain flexible and be able to accommodate interviewee requests for schedule changes or accommodate other issues they may have, in order to ensure data are collected from each interviewee.
Describe any tests of procedures or methods to be undertaken. Testing is encouraged as an effective means of refining collections of information to minimize burden and improve utility. Tests must be approved if they call for answers to identical questions from 10 or more respondents. A proposed test or set of tests may be submitted for approval separately or in combination with the main collection of information.
Cognitive tests of the interview guides (Appendices C-3 and C-4) were conducted with key respondents from relevant agencies – the Department of Education and the Department of Human Services – in Arkansas. The previsit and on-site interview guides were tested as small group interviews with a total of 5 respondents in the State of Arkansas: two staff from the State Department of Education, and three from the State Department of Human Services. Arkansas was selected because multiple State agencies administer the Child Nutrition Programs, and it provided an opportunity to test the on-site interview guide with respondents who oversee different programs. In addition, historical SAE data for Arkansas show instances of recovery of SAE funds as well as receipt of additional funds through reallocation, both topics of focus in the interview guides. Interviews ranged from 45-90 minutes.
The tests were conducted to ensure that the respondents interpreted the questions as intended and could easily respond, and the interviewer could easily administer the instruments. A trained interviewer administered both guides, observed and documented any issues that arose for both respondents and interviewers, and discussed any points of difficulty with respondents. Findings and recommendations from the cognitive testing were used to refine the guides. Participants provided useful feedback on tweaking the wording for several questions for clarity and common terminology used by State agencies. For the pre-visit interview guide, respondents suggested the research team obtain and review SAE State Plans from the FNS Regional Offices in advance of conducting the interviews, instead of waiting to go on-site to the State agency, to better prepare the interviewer. In addition, respondents recommended modifying questions to more fully and directly describe the SAE responsibilities of the staff requested for the on-site interviews. For the general guide, respondent feedback resulted in changes to questions and probes to ensure the interviews are better able to elicit specific information about the State process for budgeting and spending; and to change the timeframe from 10 years to 5 years when asking for examples of prior funds usage in the State. Pre-testing also provided information on the time needed to conduct the interviews with respondents. As a result, we increased the time for the in-person interviews from 90 minutes to 2 hours.
Provide the name and telephone number of individuals consulted on statistical aspects of the design and the name of the agency unit, contractor(s), grantee(s), or other person(s) who will actually collect and/or analyze the information for the agency.
The proposed protocol and discussion guide were developed and reviewed extensively by FNS and Westat staff identified below. FNS staff and Westat will participate in the analysis of the data, as well as development of reports. In addition, Sarah Goodale, Mathematical Statistician with the National Agricultural Statistical Service’s Summary, Estimation, and Disclosure Methodology Branch, reviewed Part A and Part B of this OMB Clearance Package (Appendix F-1).
Name |
Affiliation |
Telephone number |
|
Melissa Rothstein |
Westat |
301-315-5975 |
|
Laurie May |
Westat |
301-517-8068 |
|
Lindsay Giesen |
Westat |
978-443-3602 |
|
Jinee Burdg |
USDA/FNS |
703- 305-2744 |
|
Sarah Goodale |
NASS |
202-690-8122 |
|
Ted Macaluso |
Westat |
571-214-9658 |
1 For the purposes of this document, the term “State” is used to refer to all 54 entities.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Linda Allers |
| File Modified | 0000-00-00 |
| File Created | 2021-01-22 |