ART Generic Clearance Memo Final

Memorandum

Date: 24 October 2018

To: Stephanie Tatham, OMB Desk Officer

Through: Christina Sandberg, Food and Nutrition Service, Information Collection Clearance Officer, Planning & Regulatory Affairs

Ruth Brown, United States Department of Agriculture, Office of the Chief

Information Office, Department Clearance Officer

From: Conor McGovern

Food and Nutrition Service, Social Science Research Analyst, Office of Policy Support

Re: Under Approved Generic OMB Clearance No. 0584-0524 Request for Approval for the Child Nutrition Formative Research on Administrative Review and Training Grants

The Food and Nutrition Service (FNS) of the United States Department of Agriculture (USDA) is requesting approval for formative research under Approved Generic OMB Clearance No. 0584-0524 Generic Clearance to Conduct Formative Research.

This request is to acquire clearance to conduct formative research with the 20 Administrative Review and Training (ART) grant recipients to assist FNS with identifying effective design and implementation approaches for future ART grants. This research will identify grantees’ experiences, expectations, challenges and lessons learned in order to inform the development and success of future ART grant funding activities.

The following information is provided for your review:

1. Title of the Project: Child Nutrition Formative Research on Administrative Review and Training Grants

2. Control Number: 0584-0524, Expires 09/30/2019

3. Public Affected by this Project:

State and Local Agency Employees

• 20 State Agencies

• 60 School Food Authorities

See section 7, Project Purpose, Methodology & Formative Research Design, for a description of the number of participants for each audience and the research methodology for this effort.

4. Number of Respondents:

Table 4.1 – Recruitment
Audience	# of Participants
State Government Staff	20
School Food Authority Staff	60
Total	80*
*FNS estimates 80 unique respondents. This includes State Agency staff in the 20 ART Grant States. It also includes 3 School Food Authority (SFA) staff per Grant State, for a total of 60 SFA staff. FNS expects that all of the staff will participate in the research activities.

Table 4.2 – Research Activities by Audience

Target Audience	Research Activity	# of Participants
State Government Staff	State Agency Interview	20
	Follow-up E-mail	10*
	State Agency Follow-up Interview	10*
School Food Authority Staff	SFA Interview	60

*The number of unique respondents for this research effort is 80. Out of the 20

State Government Staff participating in the State agency interview, 10 of them

will be selected for the follow-up interviews. These 10 are already counted as

part of the 20 State agency participants.

5. Time Needed Per Response:

Table 5.1 - Time Needed per Initial Recruitment*
Target Audience	Time (minutes)	Time (hours)
State Government Staff	20	0.33
School Food Authority Staff	20	0.33
*The time is an average response per respondent.

Table 5.2 - Time Needed for Research Activities by Audience

Target Audience	Research Activity	Time (minutes)	Time (hours)
State Government Staff	State Agency Interview	60	1
	Follow-up E-mail	20	0.33
	State Agency Follow-up Interview	60	1
School Food Authority Staff	SFA Interview	60	1

6. Total Burden Hours on Public:

Table 6.1 – Estimated Annual Burden Hours
Instrument	Original sample size	Estimated number of respondents	Frequency of response	Total annual responses	Hours per response	Estimated annual burden (hours)
Respondent: State Government
Appendix D: Email Request to Participate	20	20	1	20	0.33	6.6
Appendix A: State Agency Interview	20	20	1	20	1.00	20.00
Appendix E: Follow-Up Email	10	10	1	10	0.33	3.3
Appendix B: State Agency Follow-Up Interview	10	10	1	10	1.00	10.00
Respondent: School Food Authorities (SFAs)
Appendix F: Email Request to Participate	60	60	1	60	0.33	20
Appendix C: SFA Interview	60	60	1	60	1	60.00
Total	80	80	2.25	180	0.67	119.90

Total burden hours on public: 119.90 hours

7. Project Purpose, Methodology, and Formative Research Design:

Background

Federal legislation in 2004 and 2010 amended the Richard B. Russell National School Lunch Act to require that state agencies conduct additional reviews of selected School Food Authorities (SFAs); to provide annual funding for states to be used for oversight and training of SFAs, with a focus on SFAs that demonstrated a high level of, or a high risk for, administrative error; and required the implementation of a more robust and unified accountability system.

After the changes from the 2010 legislation, via the Healthy, Hunger-Free Kids Act of 2010 (HHFKA), the Food and Nutrition Service (FNS) established a new process for SFA reviews, referred to as the Administrative Review (AR). The new AR process implemented a number of changes:

• New supplemental forms that include the Off-site Assessment and Risk Assessment Tools.

• More comprehensive scope of review; still includes the National School Lunch Program (NSLP), but adds the School Breakfast Program (SBP) and other federal School Nutrition (SN) programs.

• Increased focus on the overall financial health of the school food authority’s (SFA’s) Nonprofit School Food Service account and addition of a Resource Management section to the AR.

• Elimination of federally mandated follow-up review thresholds.

• Collaboration of state agencies and SFAs to complete the AR, to ensure program integrity and increase overall compliance.

State agencies (SAs) must review all SFAs at least once during the 3-year review cycle.

To assist SAs in meeting these new AR requirements, the Food and Nutrition Service (FNS) funded Administrative Review and Training (ART) Grants available to SAs for administrative oversight and training (approved under OMB Control No. 0584-0512, Uniform Grant Application for Non-Entitlement Discretionary Grants, expires 03/31/2019). The ART Grant funds can serve a number of purposes, but three are pertinent to this project: (1) To identify, review, monitor, and train SFAs and school sites that have demonstrated a high level of, or a high risk for, administering the National School Lunch Program (NSLP) or School Breakfast Program (SBP) incorrectly; (2) To fund state and local technical solutions to streamline AR processes; and (3) To leverage FNS-approved technical assistance to effectively manage ART Grant implementation. FNS has awarded ART Grants to 30 states since fiscal year (FY) 2009, and 20 states are expected to close out their grants in time for inclusion in this study.

Within FNS’s guidelines, the strategies that ART grantees select for reducing administrative error vary widely. For example, some states implemented direct ART Grant–funded services and trainings for SFAs, which varied in duration and intensity; others implemented updates to technologies that affect SFAs statewide. As a result, the number and types of SFAs that ultimately are affected by these ART Grants vary, as well.

This formative research will improve FNS understanding of individual state implementation strategies, their effectiveness on the AR process and associated error rates, and information describing challenges and lessons learned when implementing ART Grant interventions. The study team will examine all ART grants completed and closed at the end of FY 2017. The study will exclude planning grants unless they resulted in a subsequent implementation grant because planning grant amounts are very small; as a result, the activities they fund are small in scale. Their potential impact has been measured against the cost and burden that it would be placed upon their recipients as a result of the interview. It has been determined the planning grants should be excluded.

Purpose

The study’s data collection activities can be grouped into two broad categories (1) SFA interviews and (2) Initial and Follow-up SA level staff interviews. The specific interviews and data collection activities included for each respondent type are described below.

To accomplish study objectives, the study team will conduct phone interviews with SA and SFA staff for each of the 20 States in the study. Topics of discussion for the interviews include: intended intervention outcomes, effectiveness of the intervention, quality of technical assistance (if provided), how the intervention impacted workflow, changes following implementation of the grant and sustainability of intervention changes after the grant ended and policies that may influence the Administrative Review process. In addition, follow-up phone interviews with SA level staff in ten states will collect additional detailed information to further clarify and inform the research. The follow-up interviews will have a more detailed set of questions than the initial interview and will focus particularly on relationships with vendors, sustainability of the grant activities following completion, challenges and lessons learned, and process management during the grant period.

Methodology/Research Design

This research will utilize interviews conducted with 20 State Agencies who received an ART Grant. Each State Agency will identify 3 SFAs for interviews. A subset of 10 State Agencies will be selected for more in-depth follow-up interviews after the SFA interviews in order to answer a subset of questions related to vendor management, sustainability of grant activities, and process management and provide a richer set of information by diving deeper into grants with particularly innovative designs or outsized impact. Interviewers will be trained beforehand on the interview protocol and probes. All interviews will be audio-recorded (with respondent permission) and transcribed.

Design/Sampling Procedures

The Formative Research on ART Grants does not require a sampling plan, as the universe includes all 20 States that received an ART grant. All State Agencies who have received and completed ART Grants by the end of FY 2017 will be invited to participate in the study. State Agencies shall be directed ahead of their interview to be prepared to recommend five SFAs for interviews in their interview and to consider who would be the best bet to contact as the other two will serve as alternates. Candidates for State Agency follow-up interviews will be selected based on the potential to do a deeper dive on the most interesting outcomes of projects funded through ART grants as assessed through answers to the initial interviews with the State Agency and SFAs.

Recruitment and Consent

For State Agency staff interviews, it is expected that all of the staff approached will agree to participate. One member of the interview team will take responsibility for working with the primary contact person to handle the scheduling and logistics, e.g., identifying appropriate interview respondents. Interview appointments will be confirmed via e-mail.

For School Food Authority interviews, it is expected that all of the staff approached will agree to participate. One member of the interview team will take responsibility for working with the primary contact person to handle the scheduling and logistics. Interview appointments will be confirmed via e-mail.

Compensation

Payment or gifts will not be provided to respondents.

Data Analysis

All interviews will be audio-recorded (with respondent permission) and transcribed. This text will become the data for qualitative analysis. Codes, representing new insights and relevant participant experiences and opinions, will be identified and entered into the NVivo software package in order to organize themes. Findings will be considered descriptive and directional, but not definitive. The interviews are not intended to make statistical inferences.

Outcomes/Findings

Input gathered from the target audiences through the research will inform the direction of future Administrative Review & Training Grants. Research summary findings may be published either electronically or in print, but such documents will not include information that personally identifies any of the research participants.

8. Confidentiality:

Personally-identifiable information such as names and position titles will be collected for purposes of identification in research only and will not be reported to the public in any summary report. Any information reported in summary reports will be presented only in generalities, either by State name or by geographical and demographic generalities (e.g. “a small rural school district in Illinois”) rather than by identifying the interview’s respondent. No public-use dataset or transcripts will be available as there is no way to provide interview transcripts that would be both useful to the public and sufficiently redacted to protect the identities of respondents.

Abt Associates complies with the Privacy Act of 1974, Health Insurance Portability, and Accountability Act of 1996 (HIPAA), and the E-Government Act of 2002, including Title III: Federal Information Security Management Act (FISMA). To maintain compliance with regulations, Abt Associates has a comprehensive security program, which is supported by staff information technology and data security experts who work with project teams to protect the client’s and project’s interests.

Interviews will be recorded for reference purposes by the researchers only. On page 1 of the interview protocols (Appendices A and B), subjects are asked if they will consent to recording. If they do not consent, the interviewers will inform them that they may take additional time and pauses to record and verify answers.

System of Record FNS-8 – FNS Studies and Reports, published in the Federal Register on 4/25/1991 at 56 FR 19078, covers personal information collected under this study and identifies safeguards for the information collected.

Federal Costs: $999,991

9. Research Tools/Instruments:

• Appendix A: State Agency Interview Protocol

• Appendix B: State Agency Follow-up Interview Protocol

• Appendix C: School Food Authority Interview Protocol

• Appendix D: Study Notification from Regional Offices to State Agencies

• Appendix E: Follow-up E-mail from Study Team to Selected State Agencies

• Appendix F: Study Notification from State Agencies to SFAs