072812 CE Option Evaluation Part A_Rev

This is a new information collection request. Two school-based programs form part of a safety net against childhood hunger overseen by USDA’s Food and Nutrition Service (FNS) – the National School Lunch Program (NSLP) and School Breakfast Program (SBP). Each State administers these programs through Local Educational Agencies (LEAs)¹ and ensures they meet Federal guidelines. Students can be certified for free or reduced-price (FRP) meals through annual household applications or by being matched to lists from other Federal means-tested programs (direct certification). Recognizing the burden of household applications, the Healthy, Hunger Free Kids Act (HHFKA) of 2010 made the Community Eligibility Option (CE Option) available to LEAs and schools in high poverty areas such that families are no longer required to submit applications for FRP meals. Under the CE Option, LEAs offer free meals to all students regardless of income.

NSLP and SBP facilitate access to foods that meet the government’s nutritional standards for at least 1 of 3 meals each day that school is in session. More than 90 percent of students in the U.S. attend a school that offers NSLP and/or SBP. Students in households with family incomes at or below 130 percent of the Federal poverty line (FPL) are eligible for free meals at school, while those in households with family incomes between 130 percent and 185 percent of FPL are eligible for reduced price meals. Last year over 31 million children participated in the NSLP and over 11 million children participated in the SBP. Over 65 percent of lunches and 80 percent of breakfasts served in schools were for children certified for FRP meals.²^,³

Direct certification determines children’s eligibility for free meals by matching student enrollment records with administrative records from the Supplemental Nutrition Assistance Program (SNAP), Temporary Assistance for Needy Families (TANF), or the Food Distribution Program on Indian Reservations (FDPIR). Children can be certified as eligible for free meals based on lists maintained by LEAs or other agencies, including homeless, runaway and migrant youth, and foster children. Children certified for free meals by any of these means are “identified students”.⁴ Nationally, direct certification identifies 36 percent of children approved for free or reduced-price meals.⁵

LEAs participating in NSLP and SBP receive reimbursements for qualifying meals. Under conventional procedures, LEAs count reimbursable meals served to children certified for FRP meals and children paying the LEA’s full price. LEAs submit monthly claims for reimbursement, based on meal counts and rates set by USDA, to their State Child Nutrition Agencies. To reduce local-level administrative burdens that accompany completing and processing the household applications, USDA has established 3 provisions as alternatives to the conventional requirements for annual determination of FRP meal eligibility. All 3 Options let LEAs conduct FRP meals certification less than annually.⁶ Families still must apply every few years, however, and this presents a challenge for families, schools, and LEAs out of practice with the application process.

Recognizing this problem, Congress passed Section 104(a) of the Healthy, Hunger Free Kids Act (the Act) of 2010. This Section of the Act provides the CE Option for LEAs and schools as an alternative process for FRP eligibility and reimbursement. The CE Option can be used in an LEA, a subset of schools in an LEA, or an individual school if the ratio of identified students to total enrollment is 40 percent or more in the year prior to adopting the CE Option. This criterion is known as the “Identified Student Percentage” (ISP). In order for all schools in a LEA to be eligible for the CE Option, the ISP of the LEA must be 40 percent or more. LEAs and schools using the CE Option must agree to serve both free breakfasts and lunches to all students, and to cover all costs in excess of Federal reimbursements with non-Federal funds. The CE Option has two anticipated advantages: (1) it decreases administrative burden and errors by not requiring regular applications to establish eligibility and, (2) it increases the potential for student participation in meal programs by expanding access to free meals. To understand how the CE Option is implemented, incentives and barriers for LEAs and schools, and impacts on LEAs and schools, Congress mandated that FNS conduct an evaluation of this Option.

Ten States with the most potential for covering the highest number of qualifying LEAs and schools were invited by FNS to apply for the Community Eligibility Option. Applications were reviewed by FNS and four States were selected using a defined set of criteria developed for the selection process. One State opted out after the selection. The selected three States (Illinois, Kentucky, and Michigan) began implementing the Option in the 2011-2012 school year (Year 1 States). All States were encouraged to apply for the Option for School Year 2012-2013. Based on a review of the applications FNS selected four more States (District of Columbia, New York, Ohio, and West Virginia) to implement the Option in school year 2012-13 (Year 2 States). FNS will select four more States to implement the Option in school year 2013-14 (Year 3 States). The CE Option will be available nationwide to all eligible LEAs and schools in the 2014-2015 school year.

Evaluation Objectives

The research objectives of the CE Option Evaluation were specified in the Act (Section 104(a)(1)(F)(xii)). The overall objectives are to obtain a better understanding of: 1) the acceptability of the CE Option to LEAs; 2) the incentives and barriers for LEAs adopting the CE Option; 3) operational issues that State agencies encounter in administrating this Option; and, 4) implications and impacts of choosing this Option.

This study addresses questions related to implementation and impacts separately. The Implementation Study will address the following three overarching research objectives: (1) estimate the number of eligible LEAs and schools that do not choose the CE Option, (2) assess the barriers to participation in the CE Option in non-participating but eligible LEAs and schools, and (3) describe the LEAs and schools participating in the CE Option. To fulfill these objectives, we will directly address eight research questions at the State-level and 16 research questions at the LEA-level (Appendix A). The Impact Study will measure the effect of the CE Option through the examination of two confirmatory outcomes⁷: 1) How did average daily participation in the NSLP change because of the CE Option (compared to what participation would have been in the absence of the Option)?; and, 2) How did Federal revenues per reimbursable meal change because of the CE Option (compared to revenues that would be generated from the conventional meal reimbursement structure)?

In addition, several exploratory outcomes⁸ will be examined to address foodservice staffing and costs, program integrity, and the nutritional quality of meals offered. These questions can be found in Appendix B.

A.2 Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Various data collection activities will be conducted for the Implementation Study and the Impact Study to address the research objectives (see Section A.1). Data will be collected by trained project staff solely for the Congressionally-mandated CE Option Evaluation. All study instruments can be found in Appendices C and D.

Implementation Study Data Collection Activities

The Implementation Study includes three main activities.

Local Education Authorities (LEA) Foodservice Director Web Survey. We will survey 1400 LEAs via a web-based survey (Appendices C_6, C_7, C_8, D_1, D_2, D_3) to collect information on LEA/school characteristics and environments as well as the incentives and barriers to adopting and implementing the CE Option. The primary respondent of the survey will be the LEA foodservice director. Three distinct groups of LEAs will be identified and each group will receive a similar survey with slight differences to address the various research questions. The LEA groups are as follows:

Eligible Participating (EP) LEAs (Appendices C_8, D_3). In each participating State, an LEA is eligible for the CE Option if it has at least one school with an ISP of 40 percent or more and is participating if it has one or more schools within its LEA that took up the CE Option. Web surveys (30 minutes) will be collected between October 2012 and December 2012 (Year 1 States) and in January 2013 to March 2013 (Year 2 States). Year 3 States will not receive the web-based survey. Questions include assessing reasons for adoption of the CE Option and perceived incentives and advantages to participation.
Eligible Non-Participating (EN) LEAs (Appendices C_6, D_1). In each participating State, this is an LEA that is eligible for the CE Option but has opted not to participate. A sample of LEAs will be asked to complete a web survey (28 minutes) between October 2012 and December 2012. Questions include reasons for lack of adoption of the CE Option and perceived barriers that made the Option unappealing.
Near Eligible (NE) LEAs (Appendices C_7, D_2). In each participating State, this is an LEA with at least one school with an ISP of 30 to <40 percent. These LEAs are not eligible for the CE Option, but are close to being eligible. A sample of NE LEAs will be asked to complete the web survey (24 minutes) between October 2012 and December 2012. Questions include perceived incentives and barriers to the CE Option and likelihood of participation.

The questions similar across all surveys include: 1) contact information; 2) LEA/school characteristics and environment; 3) process information; 4) communications about the CE Option; and 5) decision-making process around adoption and implementation of the CE Option. State Child Nutrition (CN) directors of the Year 1 and Year 2 States will be notified when the web survey will occur. The LEA foodservice directors will be sent advance letters and a recruiting package (Appendices E and F). The sample will include all EP LEAs and up to 950 other LEAs, divided proportionately between EN and NE LEAs.

Semi-Structured Interview for State Child Nutrition (CN) Directors (Appendix C_21). Child Nutrition (CN) directors from the 11 States will be contacted to conduct a semi-structured telephone interview (45 minutes) about implementing the CE Option in Year 1 (n=3), Year 2 (n=4) and Year 3 (n=4). State CN directors in the Year 1 and 2 States will be interviewed in July 2012 and again in May to July 2013 and State CN directors in the Year 3 States will be interviewed only during the May to July 2013 time period. Topics for these interviews include the identification and notification of eligible LEAs, questions and concerns raised by LEAs, implementation and management of the CE Option, barriers to participation, and potential program improvements.

The senior study team member assigned to the State will email the State CN director to arrange a time to conduct the telephone interview. The advance letter and recruitment materials can be found in Appendix E.

State Education Agency (SEA) Survey (Appendix C_20). We will conduct a short telephone survey (20 minutes) with SEAs in July 2012 to obtain information on the ways in which they use FRP data in their funding formulas for educational programs (e.g., Title I), as well as how States would handle the distribution of such funding if free and reduced-price data was unavailable for schools and districts electing the CE Option. We will ask all 50 States and the District of Columbia to complete the survey. The advance letter to SEAs can be found in Appendix E_6.

Impact Study Data Collection Activities

Data collection for the Impact Study includes web surveys and several on-site data collection activities. The on-site data collection (Menu Survey, Administrative Cost Interview Guide and Preparation Forms, Meal and Cashier Observation Form, Meal Counting and Claiming Forms, and Certification Record Abstraction Form) will take place during two time periods: October to December 2012 and January to March 2013. On-site activities will be divided between the two time periods for Year 1 States, and all on-site activities will be conducted during the January to March 2013 time period for Year 2 States.

LEA Survey on Participation, Enrollment, Attendance, and Revenue (PEAR) (Appendices C_16, C_17). The LEA PEAR Survey is a web-based survey designed to collect data from 240 LEAs (120 participating in the CE Option and 120 non-participating) during January to March 2013 to address research questions regarding impacts on participation, foodservice revenues, and presence of the SBP. For most of the research questions, the LEAs will provide data from their administrative records, including NSLP and SBP meal counts; revenues from student payments, a la carte food sales, and State and local funds; and, enrollment and daily attendance. Data will be collected for four school years: 2009-2010 through 2012-2013. The format of this survey will be similar to the web-based systems that LEAs in many States use to submit claims for reimbursement. As an alternative to the web survey, LEAs will be able to complete a paper version of the survey form. Also, when practical, participation and revenue data will be obtained from State Agencies (C_17). Our calculations of respondent burden assume that 80 percent of LEAs will complete the short version of the PEAR. We will obtain the necessary participation and revenue data from State Agencies for these LEAs. We will send each sampled LEA an advance e-mail a week before the survey is to begin (Appendix E_11, E_12).

Pre-Visit LEA Foodservice Director Questionnaire (Appendix C_18). The Pre-Visit LEA Foodservice Director Questionnaire will facilitate the LEA on-site interviews and data collection by collecting preliminary information. It will be administered via a 20-minute computer assisted telephone interview (CATI) to LEAs (n=106) prior to the on-site visit. Information will be collected on LEA foodservice structure; person(s) responsible for various LEA-level foodservice tasks; meal claiming process; and availability of certification records.

Pre-Visit School Information Questionnaire (Appendix C_19). The Pre-Visit School Information Questionnaire will collect preliminary information to facilitate the on-site data collection at the schools where menu surveys and observations will take place (n=156). The interview will be administered as a 15-minute CATI . For Year 1 schools, the instrument will be administered in the fall of 2012; for all other schools it will be administered in the early months of 2013. Information on school foodservice structure; details about meal service and foods available; type of meal counting system in place; and person responsible for foodservice tasks will be obtained.

Administrative Cost Interview Guide and Preparation Forms (Appendices C_1, C_2). In-person interviews in 106 LEAs and up to 318 schools will be conducted with LEA foodservice directors, school foodservice managers, and school administrators to gather data on the staff hours and costs of administrative activities affected by the CE Option. The sample will include 53 participating and 53 non-participating LEAs. Three schools will be selected within each LEA where school-level data are needed. Costs will be measured for activities such as verifying FRP meals applications; direct certification; managing student payment accounts; and preparing and submitting claims for reimbursement. The data collection effort will start with an advance letter (Appendices E_7- E_10).

Menu Survey (Appendices C_4, C_14, C_15). The Menu Survey data collection will be completed in 156 schools - three from each of 52 LEAs (26 participating LEAs, and 26 eligible non-participating LEAs). The Menu Survey consists of a set of forms that school cafeteria managers will record information into during a specific five-day “target week.” The forms contained in the Menu Survey booklet will allow for recording of: 1) daily meal counts of reimbursable meals at breakfast and lunch; 2) all food items served as part of reimbursable lunch and reimbursable breakfast with common foods preprinted on the form (to decrease respondent burden of writing every item); 3) recipes for all foods prepared by combining two or more ingredients; and, 4) ingredients included on reimbursable food bars, such as salad bars, deli bars, entree bars, and condiment bars. Included with the Menu Survey book will be a Menu Survey Instruction booklet.

Meal and Cashier Observation Form (Appendix C_11). The Meal and Cashier Observation Form will be completed in the same sample of schools that participate in the menu survey (n=156) and will be used by field interviewers to record the reimbursable foods included in meals taken by a random sample of students for both breakfast and lunch and to record the corresponding determination of meal classification (reimbursable or not) made by the cashier for each meal observed. We expect to conduct 60 observations (of individual student and cashier transactions) for lunch and 40 observations at breakfast at each of the schools in the sample.

Meal Counting and Claiming Forms. The Meal Counting (n=156 schools) (Appendices C_9, C_10) and Claiming (n=52 LEAs) Forms (Appendices C_12, C_13) will collect data to determine errors at the school and LEA levels in counting, recording, and claiming reimbursement for school meals. The forms will be completed by the field interviewer during the school and LEA visit with records and information provided by the school’s cafeteria manager and by the LEA foodservice director.

Certification Record Abstraction Form (Appendix C_5). To measure administrative certification errors, data collectors will review eligibility documents maintained by LEAs, including approved and denied applications for FRP meals and direct certification lists. Data will be collected to determine whether the available documentation supports the LEA’s determination for a sample of 38 approved students and 12 denied students in each school. These data will be collected in-person in the 106 LEAs and 318 schools selected for the administrative cost interviews. A Certification Record Abstraction Form will be completed for each school. Where it is not feasible for data collectors to take copies of applications back to the central research office for processing, the application data will be copied onto an Application Data Form (Appendix C_3).

A.3 Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

The study strives to comply with the E-Government Act of 2002 (P. L. 107-347) by using web-based surveys (at www.opinionport.com) and CATI. By including programmed skip patterns, consistency and data range checks, this technology reduces data entry error that often necessitate callbacks to respondents to clarify the responses recorded by an interviewer using pencil and paper to conduct an interview. (Paper versions of the web surveys can be found in Appendices C and D. Screen shots of the web surveys can be found in Appendices G and H.) Out of 3,574 respondents, 2430 (95 percent of web surveys and 68 percent of total respondents) will respond electronically. Note that this counts each data collection in each LEA/school as a response.

A.4 Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.

Efforts to identify duplication included a review of FNS reporting requirements, State administrative agency reporting requirements, and special studies by government and private agencies. It was concluded that no existing data sources provide all the data needed to answer the study’s research questions. If State agencies already collect the data being asked of the LEAs, the data will be collected from the State if reported accurately and in entirety, to reduce the burden on the LEAs and reduce duplication. For some evaluation activities, LEAs can report data they already compile for other purposes.

A.5 If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

Information being requested or required has been held to the minimum required for the intended use. Although smaller school districts and LEAs are involved in this data collection effort, they deliver the same program benefits and perform the same function as any other school district or LEA. Thus, they maintain the same kinds of information on file. Healthy, Hunger Free Kids Act of 2010 makes participation in evaluations of school nutrition programs such as the Community Eligibility Option Evaluation mandatory. Out of 1400 LEAs in this study, 539 are considered small entities.

A.6 Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

All data collection for the proposed study will be conducted once for each respondent during the 2012-2013 School Year, except for the 7 (Year 1 and Year 2 States) State CN Agency interviews, which will be conducted once in 2012 and once in 2013. Repeating the State CN Agency interviews will allow assessment of facilitators, barriers, and activities at different times of program implementation. Without this effort, which has been planned to address the research questions with the minimum possible burden, FNS will not have the data necessary to address the questions posed by Congress and provide the information needed to ensure effective nationwide implementation of the Option in 2014-2015.

A.7 Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical surveys, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no other special circumstances; information collection is consistent with 5 CFR 1320.5.

A.8 If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

An announcement was published in the Federal Register (Volume 76, No. 229, Page 73582) on November 29, 2011, and specified a 60-day period for comment ending on January 30, 2012. No comments were received in response to the FRN.

Consultations Outside the Agency

FNS has consulted with the following State Child Nutrition Directors on the availability of data: Chris Schmitt, Illinois State Board of Education (217-782-2491); Mary Ann Chartrand, Michigan Department of Education (517-373-4013); and Denise Hagan, Kentucky Department of Education (502-564-5625). Matthew Gregg (202-720-3388), Statistician in the Methods Branch of the National Agricultural Statistics Service reviewed the study design and data collection plans. A copy of the memo with comments from Matthew Gregg is provided in Appendix L.

A.9 Explain any decision to provide any payment or gift to respondents, other than reenumeration of contractors or grantees.

No payments will be made to respondents.

A.10 Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

A system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on March 31, 2000, Volume 65, Number 63, and is located on pages 17251-17252 discusses the terms of protections that will be provided to respondents. Participants in this study will be subject to safeguards as provided by the Privacy Act of 1974 (5 USC 552a), which requires the safeguarding of individuals against invasion of privacy. The Privacy Act also provides for the confidential treatment of records maintained by a Federal agency according to either the individual’s name or some other identifier.

The individuals at the State- and LEA-level participating in this study will be assured that the information they provide will not be released in a form that identifies them. Per contract, Abt will deliver raw datafiles and data analytic files to USDA as well as a restricted use datafile. However, no identifying information will be attached to any reports or data supplied to USDA or any other researchers. For data collected through the State-level surveys, the State CN Directors are publicly known, but the individual respondents will not be identified by name. Per SORN FNS-8, the identifiers for study participants will be securely maintained until they are no longer needed but not more than two years after the end of the study, at which point they will be destroyed.

Procedures to Ensure Confidentiality

The contractor has extensive experience in data collection efforts requiring strict procedures for maintaining the privacy, security, and integrity of data. The following data handling and reporting procedures will be employed to maintain the privacy of survey participants. All project staff will be required to sign a confidentiality and nondisclosure agreement (Appendix I). In this agreement, staff pledges to maintain the privacy of all information collected from the respondents and will not disclose it to anyone other than authorized representatives of the study, except as otherwise required by law.

In the central office, documents containing respondent information are kept in locked file cabinets. At the close of the data retention period under the study contract, such documents are shredded.
Data gathered from the interviews will be combined into master respondent files. Immediately after each file is created, the respondent will be assigned a unique identification number. Any identifying information will be removed from the survey data and replaced with the identification number.
Any respondent-identifying information will be contained in a master list to be created and protected in secure storage, to which only a limited number of project staff pledged to maintain privacy will have access.

Additional procedures will ensure the safeguards and security of electronic data during the data collection and processing period. Standard backup procedures will be implemented for the central office computer system to protect project data from user error or disk or other system failure. Backups and inactive files will be maintained on tape or compact disks. The system servers will be maintained inside a secure locked area accessible only to authorized systems personnel. Files will be accessible only by authorized personnel who have been provided project logons and passwords. Access to any of the study files (active, backup, or inactive) on any network multi-user system will be under the central control of the database manager. The database manager will ensure that the appropriate network partitions used in the study are appropriately protected (by password access, decryption, or protected or hidden directory partitioning) from access by unauthorized users. All organizations using data on study participants will maintain security, virus, and firewall technology to monitor for any unauthorized access attempts and any other security breaches.

Institutional Review Board

Abt Associates maintains its own Institutional Review Board (IRB), which conducts prospective reviews of proposed research and monitors continuing research for the purpose of safeguarding research participants’ rights and welfare. All Abt research involving interactions or interventions with human subjects is within the purview of the Abt IRB, which is responsible for ensuring that the organization’s research: 1) meets only the highest ethical standards; and 2) receives fair, timely, and collegial review by an external panel. Abt Associates’ IRB currently holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP). The FWA covers all federally supported or conducted research involving human subjects. Study materials and protocols for the Community Eligibility Option Evaluation have been submitted to Abt’s IRB, and because the study does not involve human subjects (any student data will be used at the aggregate level, and interviews/surveys with professional stakeholders will elicit descriptive, non-sensitive information), the determination was made that the study will be exempt from IRB review (Appendix J). Should there be any changes made to the study protocols to involve human subjects, all study materials and instruments for the Community Eligibility Option Evaluation will be submitted to Abt’s IRB for review.

A.11 Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

There are no personally sensitive questions contained in the data collection instruments. Information on salaries for LEA and school personnel will be collected for the purposes of measuring impacts of the Option on the costs of administrative activities and staff hours. Salary information from State and LEA employees is often public information. Respondents will be allowed to provide salary ranges rather than individual salaries. Failure to collect salary information will result in an inability to address one of the Congressionally-mandated questions about impacts of the Option on cost.

A.12 Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

Exhibit A.1 shows sample sizes, estimated burden, and estimated annualized cost of respondent burden for each part of the data collection. Respondent burden attributed to callbacks is included in the respondent burden estimate. Estimated response times are based on response times obtained through pre-testing of study instruments. Annualized cost of respondent burden is the product of each type of respondent’s annual burden and average hourly wage rate. As shown in the exhibit, the total estimated burden across all data collection components is 2,383.1 hours and the estimated cost to respondents for the hour burden is $54,297.45.

Exhibit A.1. Estimates of Respondent Burden and Annualized Cost to Respondents

Affected Public		Data Collection Activity		Respondents	Estimated Number of Respondents	Frequency of Response	Total Annual Responses	Average Burden Hours per Response ^a	Total Annual Burden Estimate (Hours)	Average Hourly Wage Rate ^b	Total Annualized Cost of Respondent Burden
State, Local, and Tribal Governments	State	State Child Nutrition (CN) Director Interview (C_21)	Nonresponse	State Child Nutrition (CN) Director	0	1	0	0.083	0	$39.36	$0
		State Child Nutrition (CN) Director Interview (C_21)	Respondent	State Child Nutrition (CN) Director	11	1.64 (1 to 2)	18	0.950	17.1	$39.36	$673
		State Education Agency (SEA) Survey (C_20)	Pretest	State Education Agency (SEA) Director	8	1	8	0.333	2.664	$41.84	$111
			Nonresponse		6	1	6	0.083	0.498	$41.84	$21
			Respondent		45	1	45	0.433	19.485	$41.84	$815
	Local and Tribal	LEA Foodservice Director Web Survey for Participating LEAs (C_8)	Pretest Full Version	Public LEA Foodservice Director	3	1	3	0.533	1.599	$25.11	$40
			Pretest Subset of Questions		6	1	6	0.369	2.214	$25.11	$56
			Nonresponse		81	1	81	0.083	6.723	$25.11	$169
			Respondent		325	1	325	0.650	211.25	$25.11	$5,304
		LEA Foodservice Director Web Survey: Eligible Non-Participating LEAs (C_6)	Pretest Full Version	Public LEA Foodservice Director	3	1	3	0.533	1.599	$25.11	$40
			Pretest Subset of Questions		6	1	6	0.067	0.402	$25.11	$10
			Nonresponse		86	1	86	0.083	7.138	$25.11	$179
			Respondent		343	1	343	0.617	211.634	$25.11	$5,314
		LEA Foodservice Director Implementation Survey for Near Eligible LEAs (C_7)	Pretest	Public LEA Foodservice Director	3	1	3	0.433	1.299	$25.11	$33
			Nonresponse		86	1	86	0.083	7.138	$25.11	$179
			Respondent		343	1	343	0.550	188.65	$25.11	$4,737
		LEA Survey on Participation, Enrollment, Attendance, and Revenues (PEAR)—full version (C_16)	Pretest	Public LEA Foodservice Director	6	1	6	3.917	23.502	$25.11	$590
			Nonresponse		12	1	12	0.083	0.996	$25.11	$25
			Respondent		48	1	48	2.92	140.16	$25.11	$3,519
		LEA Survey on Participation, Enrollment, Attendance, and Revenues (PEAR)—short version (C_17)	Nonresponse	Public LEA Foodservice Director	48	1	48	0.083	3.984	$25.11	$100
			Respondent	Public LEA Foodservice Director	192	1	192	1.25	240	$25.11	$6,026
		Pre-visit LEA Foodservice Director Questionnaire (C_18)	Pretest	Public LEA Foodservice Director	9	1	9	0.333	2.997	$25.11	$75
			Nonresponse		0	1	0	0.083	0	$25.11	$0
			Respondent		106	1	106	0.417	44.202	$25.11	$1,110
		Pre-visit School Information Questionnaire (C_19)	Pretest	Public School Cafeteria Manager	3	1	3	0.250	0.75	$17.22	$13
			Nonresponse		38	1	38	0.083	3.154	$17.22	$54
			Respondent		156	1	156	0.25	39	$17.22	$672
State, Local, and Tribal Governments	Local and Tribal	Menu Survey (self-administered) (C_15) ^c	Pretest	Public School Cafeteria Manager	3	1	3	0.5	1.5	$17.22	$26
			Nonresponse		38	1	38	0.083	3.154	$17.22	$54
			Respondent		156	5	780	0.75	585	$17.22	$10,074
		Meal & Cashier Observation Form (C_11)	Pretest	Public School Cafeteria Manager	3	1	3	0.333	0.999	$17.22	$17
			Nonresponse		38	1	38	0.083	3.154	$17.22	$54
			Respondent		156	1	156	0.25	39	$17.22	$672
		Certification Record Abstraction Form (C_5)	Pretest	Public LEA Foodservice Director	3	1	3	0.500	1.5	$25.11	$38
			Nonresponse		0	1	0	0.083	0	$25.11	$0
			Respondent		106	3	318	0.167	53.106	$25.11	$1333
		Application Data Form (C_3) ^d	Pretest	File Clerk	3	1	3	0.05	0.15	$14.69	$2
			Nonresponse		0	1	0	0.083	0	$14.69	$0
			Respondent		53	3	159	0.333	52.947	$14.69	$778
		Administrative Cost Interview: Self-administered Questionnaire (including staff roster) (C_2)	Pretest	Public LEA Foodservice Director	3	1	3	1.333	3.999	$25.11	$100
			Nonresponse		0	1	0	0.083	0	$25.11	$0
			Respondent		106	1	106	0.75	79.5	$25.11	$1,996
		Administrative Cost Interview: Field Questionnaire (C_1)	Pretest	Public LEA Foodservice Director	3	1	3	1.500	4.5	$25.11	$113
			Nonresponse		0	1	0	0.083	0	$25.11	$0
			Respondent		106	1	106	0.75	79.5	$25.11	$1996
			Pretest	Public School Cafeteria Manager	3	1	3	0.500	1.5	$17.22	$26
			Nonresponse		0	1	0	0.083	0	$17.22	$0
			Respondent		318	1	318	0.167	53.106	$17.22	$914
			Pretest	Public School Administrator	3	1	3	0.750	2.25	$47.68	$107
			Nonresponse		0	1	0	0.083	0	$47.68	$0
			Respondent		100	1	100	0.333	33.3	$47.68	$1588
		LEA Meal Counting and Claiming Review Form (C_12, C_13)	Pretest	Public LEA Foodservice Director	3	1	3	1.000	3	$25.11	$75
			Nonresponse		12	1	12	0.083	0.996	$25.11	$25
			Respondent		52	1	52	1	52	$25.11	$1,306
		School Meal Count Verification Form (C_9, C_10)	Pretest	Public School Cafeteria Manager	3	1	3	0.500	1.5	$17.22	$26
			Nonresponse		38	1	38	0.083	3.154	$17.22	$54
			Respondent		156	1	156	0.5	78	$17.22	$1,343
SUBTOTAL					3,438	1.27603	4,387	0.528	2,314.950		$52,586
Not-for-Profit Institutions		LEA Foodservice Director Web Survey for Participating LEAs (D_3)	Nonresponse	Private LEA Foodservice Director	9	1	9	0.083	0.747	$25.11	$19
			Respondent	Private LEA Foodservice Director	35	1	35	0.650	22.75	$25.11	$571
		LEA Foodservice Director Web Survey: Eligible Non-Participating LEAs (D_1)	Nonresponse	Private LEA Foodservice Director	9	1	9	0.083	0.747	$25.11	$19
			Respondent	Private LEA Foodservice Director	37	1	37	0.617	22.829	$25.11	$573
		LEA Foodservice Director Implementation Survey for Near Eligible LEAs (D_2)	Nonresponse	Private LEA Foodservice Director	9	1	9	0.083	0.747	$25.11	$19
			Respondent	Private LEA Foodservice Director	37	1	37	0.550	20.35	$25.11	$510
SUBTOTAL					136	1	136	0.501	68.17		$1,712
GRAND TOTAL					3,574	1.2655	4,523	0.527	2383.1		$54,297.45

^a The burden hours reflect the data collection activity or survey as well as the burden associated with related correspondence (advance letters, study overview, emails, etc.).

C6 includes E1, plus one of E7-10, plus either E11 or E12.
C7 includes E1, plus one of E7-10, plus either E11 or E12.
C8 includes E1, plus one of E7-10, plus either E11 or E12.
C18 includes E5.
C20 includes E1 and E6.
C21 includes E1, either E2 or E3, plus E13 and E14.
D1 includes F1, F2, and F3.
D2 includes F1, F2, and F3.
D3 includes F1, F2, and F3.

^b Equal to Total Annual Burden Estimate times Average Hourly Wage Rate. SEA Financial Officer rate based on Financial Managers, State Government; State CN Director rate based on General and Operations Managers, State Government: School Administrator, based on rate for this occupation in Elementary and Secondary Education; LEA Foodservice Director rate based on Food Service Managers; School Cafeteria Manager Rate based on Chefs/Head Cooks, Elementary and Secondary Education; File Clerk rate based on rate for this occupation in Elementary and Secondary Education.. Average hourly Wage rates are taken from Bureau of Labor Statistics, Wages by Occupation, May 2010.

^c Includes time for review of instructions (C_14) and use of ancillary materials such as the Daily Reminder Card (C_4).

^d While we will complete an Application Data Form for each application sampled, the burden for the LEA is only to pull the records so there is one response per school.

A.13 Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.

There are no capital/start-up or ongoing operation/maintenance costs associated with this information collection.

A.14 Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The annualized government costs include the costs associated with the contractor conducting the project and the salary of the assigned FNS project officer. The cost to the Federal government for all tasks is $4,743,782.00 over the course of 30 months (September 30, 2011 through March 31, 2014), for an annualized cost of $1,897,512.80. This information collection also assumes that a total of 120 hours of Federal employee time for a GS-14, step 10 Senior Program Analyst serving as the FNS project officer at $65.53 per hour for a total of $7,863.60. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2011.)

A.15 Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.

This is a new information collection request as a result of program changes and will add 2316.4 hours of burden to OMB’s inventory.

A.16 For collections of information whose results are planned to be published, outline plans for tabulation and publication.

Analysis of data for the Implementation Study will use information obtained through the semi-structured State interview and the more close-ended web-based surveys administered to LEAs to address the research objectives outlined in A.1. Analyses will be mainly descriptive in nature and will include means, medians, standard deviations, frequency distributions and cross tabulations of key outcome measures. The analyses, when appropriate, will include significance tests for subgroups for key outcome measures using t-statistics and chi-squared statistics. In order to answer the research questions for the Impact Study three approaches will be used. The analysis of impacts on participation and revenues will use a comparative interrupted time series approach. For the impact on the presence of the SBP, the analysis will use a

pretest-posttest comparison group approach. For the other impacts, we will conduct a post-test only comparative analysis. All impact analyses will use a comparison group matched to the treatment group using a multivariate propensity score model.

Exhibit A.2 presents the study schedule. The contractor will prepare a Final Interim Report due November 23, 2012 to form the basis of FNS’ Report to Congress that is due on December 31, 2012 and will prepare a Final Report due October 1, 2013 to form the basis of FNS’ Report to Congress that is due on December 31, 2013.

Exhibit A.2. Data Collection and Reporting Schedule

Data Collection and Analysis Activity	Time Schedule	Key Reports
Assemble Sampling Frame, Select Sample, and Recruit LEAs	3/1/12 – 11/15/12	Sample Recruitment Summary Memorandum/List of Selected LEAs (12/30/12)
Select and Train Data Collectors	6/1/12 – 1/15/13	Data Collection Training Materials (9/15/12) Memorandum – Completion of Data Collection Training Winter Training (1/15/13)
Data Collection	10/1/12 – 4/30/13	Bi-weekly Data Collection Reports
Create Analytic Database	12/1/12 – 5/30/13	Memorandum – Completion of Raw and Analytic Data Files (6/15/13)
Analyze Study Data	5/1/12 – 6/30/13	Draft Analytic Data Tables (6/15/13)
Prepare Study Reports: Interim Report	6/1/12 – 11/30/12	Draft Interim Report (8/30/12) Revised Interim Report (10/12/12) Final Interim Report with Executive Summary (11/23/12)
Prepare Study Reports: Final Report	6/1/13 – 10/1/13	Draft Final Report (7/30/13) Revised Final Report (9/3/13) Final Report with Executive Summary (10/1/13)
Prepare and Submit Data Files	5/1/12 – 1/30/14	Data Files and Documentation: Interim Report Analytic Files (12/30/12) Data Files and Documentation: Restricted Use and Public Use Files (1/30/14)
Presentation Materials	10/19/12 – 9/17/13	Interim Report (10/26/12) Final Report (9/17/13)
Community Eligibility Option LEA Tool	10/30/13 – 1/30/14	CE Option Tool (1/30/14)

A.17 If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

All data collection instruments for the Food and Nutrition Service Evaluation of the Community Eligibility Option will display the OMB approval number and expiration date.

A.18 Explain each exception to the certification statement identified in Item 19 "Certification for Paperwork Reduction Act."

There are no exceptions to the Certification for Paperwork Reduction Act (5 CFR 1320.9) for this study.

1 The Healthy Hunger-Free Kids Act of 2010 uses the term LEA to refer to the entities administering eligibility for the NSLP and SBP. Most LEAs are public school districts but others are independent public charter schools or private schools. NSLP legislation and regulations also refer to the entities administering the NSLP and SBP as school food authorities (SFAs). For simplicity we use the term LEA in this document.

2 http://www.fns.usda.gov/pd/slsummar.htm

3 http://www.fns.usda.gov/cnd/Breakfast/AboutBFast/SBPFactSheet.pdf

4 Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis, “Direct Certification in the National School Lunch Program: State Implementation Progress School Year 2009-2010” by Dennis Ranalli, Janis Johnston, Edward Harper, Rosemary O’Connell, Jay Hirschman, Quinn Moore, Ira Nichols-Barrer, and Nancy Cole. Report CN-10-DC. Alexandria, VA. October 2010.

5 U.S. Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis, Analysis of Verification Summary Data, School Year 2008-2009 by Dennis Ranalli, Edward Harper, and Jay Hirschman, Alexandria, VA, March 2011.

6 http://www.fns.usda.gov/cnd/governance/prov-1-2-3/Prov1_2_3_FactSheet.htm

7 A confirmatory outcome is one that – if it shows a difference due to the intervention – can be used to judge the overall effectiveness or “success” of the intervention.

8 Exploratory outcomes are those that FNS, the research community, or the public would not consider central to judging the overall effectiveness of the CE Option. Exploratory outcomes are an important part of telling the story of how the intervention might have achieved its outcomes, how outcomes vary by key subgroup, or what are the side-effects of the intervention.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Title	Abt Single-Sided Body Template
Author	Missy Robinson
File Modified	0000-00-00
File Created	2021-01-30