QRS CN Reducing Burden OMB Generic Clearance Final

Memorandum

Date: April 3, 2018

To: Steph Tatham, OMB Desk Officer, Office of Information and Regulatory Affairs, Office of Management and Budget

Through: Ruth Brown, Desk Officer, United States Department of Agriculture, Office of the Chief Information Officer

From: Christina Sandberg, Information Collection Clearance Officer,

Food and Nutrition Service, Planning & Regulatory Affairs

Re: Generic OMB Clearance No. 0584-0613 – FNS Generic Clearance for Special Nutrition Programs Quick Response Surveys

The Food and Nutrition Service (FNS) of the United States Department of Agriculture (USDA) is requesting approval for two Quick Response Surveys (QRS) under Approved Generic OMB Clearance No. 0584-0613.

Currently, FNS conducts lengthy, large, and complex studies on broad topics about each Special Nutrition Program (SNP). This generic information collection submission is to collect data quickly and in a timely manner from State Agencies (SAs) and School Food Authorities (SFAs). Two surveys, one of State Child Nutrition (CN) directors and one of SFA officials, will be conducted under the QRS mechanism to elicit information to develop recommendations to reduce administrative and reporting burden in CN programs. FNS has contracted with 2M Research (2M), and its Subcontractor, Mathematica Policy Research (Mathematica) (hereafter the research team), to conduct this study. The following information, based on the Part A and B supporting statements of the SNP QRS Umbrella Clearance package, is provided for your review:

1. Title of the Project: Child Nutrition Reducing Burden Study: State Agency Survey and School Food Authority Survey

2. Control Number: 0584-0613, Expires 09/30/2019

3. Necessity of Information Collection:

This is a request for clearance of a generic information collection (IC) under 0584-0613 that will allow FNS to quickly collect information from local administrators of SNPs, including the National School Lunch Program (NSLP) and the School Breakfast Program (SBP). This is the first QRS collection under the SNP QRS Umbrella Clearance package (OMB #0584-0613 Special Nutrition Programs Quick Response Surveys; Expiration date 9/30/19).

This request is also necessary to collect data which will be used for a Congressionally-mandated report: House Report 114-531 which is included as Appendix A.12. In the report it states the following:

For Nutrition Programs Administration, the Committee provides $168,524,000. This funding level includes $1,000,000 for FNS to contract for an independent study to identify the best means of efficiently consolidating Child Nutrition Program reporting requirements for school food authorities and state agencies. The Committee expects the study to be completed no later than 18 months from the date of enactment of this Act [which is considered to be May 4, 2017¹]. (House Report 114-531)

FNS will identify the relevant authorizing statutes for data collections submitted under this generic clearance, including this generic information collection submission to OMB. In most cases, the following authorizing statutes apply:

• The Healthy, Hunger-Free Kids Act of 2010 mandates under Section 305 (P.L. 111-296) that programs under its authorization cooperate with USDA program research and evaluation activities.

States, State educational agencies, local educational agencies, schools, institutions, facilities, and contractors participating in programs authorized under this Act and the Child Nutrition Act of 1966 (42 U.S.C 1771 et seq.) shall cooperate with officials and contractors acting on behalf of the Secretary, in the conduct of evaluations and studies under those Acts.

4. Purpose and Use of the Information:

The purpose of the QRS Child Nutrition Reducing Burden Study, which is being conducted concurrently with the Child Nutrition Reducing Burden Analysis Study, is to collect data from SAs and SFAs quickly and in a timely manner to help identify the best ways to efficiently consolidate Child Nutrition Program reporting requirements in an effort to reduce burden, as Congressionally-mandated by House Report 114-431 (see Appendix A.12). Two surveys are planned under this QRS mechanism. When combined with other information gathered from the accompanying CN Reducing Burden Analysis Study, the QRS will provide newly collected survey data to inform work group discussions and elicit information to develop recommendations for reducing administrative and reporting burden in CN programs. This information collection is voluntary.

Types of Information. The type of information collected from SAs and local entities under their purview will generally include perceptions and satisfaction with CN administrative and reporting burden. In order to capture all potential areas of challenges in the Child Nutrition Programs being studied, the approach must be broadened to encompass interpretations of guidance and policy and perceived quality of training, which will assist with providing potential reasons why FNS reports may be burdensome or difficult to complete. Information will also be collected to understand the variations in how States administer the Child Nutrition Programs, including methods of distilling policy and guidance to local operators. The study will also examine how SFAs operate the Programs to determine where challenges reside that then result in reporting burden issues. FNS plans to use this information to identify areas of possible improvement in an effort to reduce burden, as Congressionally-mandated by House Report 114-431 (see Appendix A.12). This approach will help identify actionable areas to meaningfully consolidate program requirements while maintaining efficient and effective program operations and oversight, reducing overall burden.

Information Collection Method. Recruitment and invitation packet materials used for information collection are provided in Appendix A. First, the FNS study liaison will notify FNS’ Regional Offices (ROs) of the study (using a template provided by the study team). The ROs will then inform SAs (using a template provided by the study team) of the study and encourage participation, and ROs will notify the SAs which SFAs have been selected to participate in the study. Using a template, the SAs will reach out to the selected SFAs to make them aware of the upcoming study and stress the importance of participation in the study. Once SAs have made an initial outreach to SFAs, an invitation packet will be sent by email to all the SAs and the selected SFAs.² The packet will contain background information on the study, answers to Frequently Asked Questions, and a unique link for accessing the survey and help systems. The invitation letter will stress the importance of participation in the study, the unique nature of this study, and the opportunity for the SFA to contribute to recommendations that will help them perform their duties in more efficient ways. Multimode follow-up using telephone and email reminders (up to two reminders per respondent) may also be used to enhance the response to the surveys.

Purpose of the Information Collection. This study will include two 20-minute web surveys: (1) the Survey of State Child Nutrition Directors (i.e., SAs) and (2) the Survey of School Food Authorities (i.e., SFAs). These surveys will be designed to identify the greatest sources of burden faced by SAs and SFAs that result from administrative requirements (including reporting), and to elicit recommendations for consolidating CN program burden. To our knowledge, SA and SFA CN staff have not previously been surveyed about the range of administrative burden associated with regulatory requirements, federal guidance, and reporting requirements. Therefore, new survey questions were developed for this information collection.

Frequency of Information Collection. These surveys represent a one-time data collection to occur once in a 12-month period.

Information Sharing. All information collected will be compiled in aggregated form in electronic databases. This information may be shared with relevant FNS employees and contractors who are participating in other studies involving the same SNPs and/or respondents to reduce the burden of duplicate information collections. This information will not be shared with the general public.

5. Use of Information Technology:

FNS is committed to complying with the E-Government Act, 2002 to promote the use of technology. The QRS CN information collection will use online surveys, with invitations to respond to the survey requested via email. When necessary to encourage a response, telephone and email follow-up will be used to generate reminders to provide the requested survey information. One hundred percent of survey responses are expected be submitted electronically, an estimated 1,053 responses across both surveys. Overall there are 4,602 estimated responses covering the recruitment and follow-up of the surveyed SAs and SFAs and the pretest sample; 4,006 coming from respondents and 596 coming from nonrespondents (see Appendix C). Of these 4,602 responses, the 1,053 responses noted above will be electronic submissions of the survey (about 23 percent of the responses). The remaining responses include email and phone responses, which are not considered electronic.

6. Efforts to Identify Duplication:

To the best of our knowledge there is no similar information collection. Every effort will be made to avoid duplication. These two surveys represent a one-time data collection effort so that FNS may provide the Congressionally mandated report by the required deadline (see Section 3). FNS has reviewed USDA reporting requirements and state administrative agency requirements, HHS/Department of Education requirements, etc. and has determined that there is no other similar data collection in progress or planned. FNS solely administers and monitors the school meals programs

7. Impacts Small Businesses or Other Small Entities:

Information being requested or required has been held to the minimum required for the intended use. Although smaller SFAs are involved in this data collection effort, they deliver the same program benefits and perform the same function as any other SFA. FNS estimates that 2 percent of SFA respondents are small entities. Out of the total number of SFA respondents for this study (1,000), FNS estimates that about 20 of them will be small entities.

8. Consequences of Less Frequent Collection:

SNP QRS data collection is essential to fill the information gaps that remain from the traditional long-term studies that FNS conducts on the SNPs. This specific survey effort is a one-time data collection. This QRS is motivated by a time-critical information need that must be filled to adequately monitor CN program administrative and reporting burden and to allow FNS to report findings to Congress within the Congressionally mandated time period. More details on the Congressionally-mandated report are provided in Section 3.

9. Special Circumstances:

There are no special circumstances. The collection of information is conducted in a manner consistent with the guidelines in 5 CFR 1320.5

10. Payment or Gift to Respondents:

The SA and SFA administrators providing information for the surveys will not receive an incentive payment or gift.

11. Assurance of Confidentiality:

All information gathered from SAs and SFAs is for research purposes only and will be kept private to the full extent allowed by law.

FNS published a system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register on April 25, 1991, volume 56, pages 19078-19080, that discusses the terms of protections that will be provided to respondents. FNS and its Contractors will comply with the requirements of the Privacy Act of 1974.

The research team will safeguard privacy of the information collected as follows:

• Only the interviewer and members of the research team will see the responses.

• Any computer files with contact or other personally identifiable information that identifies a respondent will be protected by a password and stored on a secure network. These data files will be maintained separately from respondent survey responses.

• In computer files used to analyze data and prepare study reports, names and other identifying information will be replaced with a code number.

• Computer files that contain names or other identifying information about a respondent will be destroyed after the study ends.

12. Justification for Sensitive Questions:

This information collection is to obtain quick and timely customer satisfaction information from SAs and SFAs on perceived administrative and reporting burden related to FNS school meal programs, and does not contain questions of a sensitive nature.

13. Estimate of Hour Burden Including Annualized Hourly Costs:

With this generic information collection submission, there are a total of 1,350 respondents. This includes 53 SA and 1,000 SFA respondents for the survey, along with 5 SA and 12 SFA respondents for the pretest. This also includes a total of 280 non-respondents, which includes those from both the main surveys and the pretests. Overall, there are 4,602 expected responses among respondents and nonrespondents, and approximately 528 burden hours. This clearance package represents the burden estimates for a maximum of two (one of SAs and one of SFAs) one-time survey information collections to be conducted over a 1-year period. Annual burden estimates for SA and SFA collections are included in Table 1 and Appendix C. The burden hour and response estimates are based on one information request to SAs and one to SFAs, with two reminders annually.

In summary:

Estimated Number of Annual Targeted Respondents: 1,321

Estimated Number of Annual Responses per Respondent: 1

Estimated Number of Annual Responses: 4,602

Estimated Time per Response: 0.13 hours

Estimated Total Annual Burden on Respondents: 527.55

Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

Table 2 provides estimates of the annual costs for the scenario presented in Table 1 and Appendix C. The estimated respondent cost to SA directors and SFA officials is based on the U.S. Department of Labor, Bureau of Labor Statistics, May 2016 National Occupational and Wage Statistics (found at https://www.bls.gov/oes/current/oes_nat.htm#11-0000), Occupational Group 11-9051 for SFA directors and Occupational Group 11-9000 for SA directors. The hourly rate for SA directors is $47.49, and for SFA officials it is $29.59.

Table 1. Estimated Annual Burden for Information Collection and Nonresponse for CN QRS Surveys*

*Some totals may not add up due to rounding. Total sample size (E21) does not equal total of respondents (F21) and non-respondents (K21) since some sample members are counted as respondents and non-respondents (i.e., for the survey and for the pretest).

**Non-response time represents time spent opening and reading email, or listening to voicemail message but not responding.

Table 2. Annualized Cost to Respondents Summary Table

Type of Respondent	Type of Instrument	Total Annual Burden	Hourly Wage	Respondent Cost
SA	Questionnaire and Reminders, Invitation Packet, SFA Contact, and Pretest Activities	42.86	$47.49	$2,035.18
SFA	Questionnaire and Reminders, Invitation Packet, and Pretest Activities	484.69	$29.59	$14,342.09
Total	--	527.55	--	$16,377.27

14. Estimates of Other Cost Burden:

No capital, start-up, or ongoing operation or maintenance costs are associated with this data collection.

15. Annualized Cost to Federal Government:

The cost to the Federal Government will depend on the number of hours for federal employees to oversee the survey task and the associated contractor costs. However, we estimate that federal employees will spend approximately 50 hours per year overseeing the Contractors’ data collection activities. Using the hourly wage rate of $44.57 for a GS-12, step 6 federal employee from the 2017 Washington, DC locality pay table, the annual estimated costs equal $2,228.50. Contractor costs to the Federal Government associated with the information collection will total $210,036.93 for 5 months. When combining the federal employee and Contractor costs, the total annual cost to the Federal Government for this collection is estimated at $212,265.43.

16. Explanation for Program Changes or Adjustments:

This is the first survey collection under the SNP QRS umbrella generic collection of information which will decrease the current burden inventory by a total of 528 burden hours and 4,602 responses over the 1-year approval period. Table 3 below shows the remaining total burden hours and responses for the SNP QRS umbrella clearance package.

Table 3. Remaining Total Burden Hours and Responses after Survey Information Collection (IC)

	Approved	Currently Available	Requested for Use with This IC	Difference (remaining)
Responses	250,745	250,745	4,602	246,143
Hours	34,523	34,523	528	33,995

17. Plans for Tabulation and Publication and Project Time Schedule

The plan for data collection from the SAs and SFAs will be from February through March 2018. The data collected will be used to develop data analyses and a report which will be published on the FNS website and sent to Congress, as Congressionally mandated. The plan is to publish this report in the fall or winter of 2018.

18. Respondent Universe and Sampling Methods:

This generic ICR for the QRS will target State and local administrators of CN programs, and will consist of two short QRS, which will be conducted under the approved generic ICR (OMB #0584-0613, Special Nutrition Programs Quick Response Surveys, Expiration date 9/30/19).

Two frames have been constructed for this study. A census frame of N = 53 SAs was constructed for the Survey of State Child Nutrition Directors. A census of all States and territories were surveyed so there was representation in the State Agency Survey for all States and territories. A sampling frame of N = 17,803 SFAs was constructed for the Survey of School Food Authorities. A survey sample of 1,251 SFAs will be drawn from this sampling frame. This sample size was selected to ensure adequate precision for estimates for the full population and key subgroups, which are discussed in section 19 below.

The frame of SAs includes all 50 states; Washington, DC; and 2 U.S. territories—Guam and American Samoa.³ Each SA in the frame includes contact information for the CN director and the Regional Liaison’s email address. The local entity contact information collected for each SNP will serve as the sample frame for the corresponding QRS in that same year. Once respondents are selected from this frame, the contact information will be used to reach sampled respondents.

Expected response rates.

Overall, for the entire collection, the expected response rate is 81 percent. This is 1,070 respondents divided by the entire sample of 1,321 (see Appendix C).

FNS expects to achieve a 100 percent response rate from the 53 SAs invited to participate in the SA survey, and has achieved 100 percent response (or close to it) on other surveys of SAs (for other FNS projects). Given the nature and length of this survey (20 minutes or less, administered via web), full participation is expected.

FNS expects to achieve a response rate of approximately 80 percent, or better, for the full sample of SFAs invited to participate in the SFA survey, and for each target subgroup. If the response rate is less than 80 percent for the full sample or any subgroup, a nonresponse bias analysis will be conducted to determine if there are any systematic differences between respondents and nonrespondents that would impact key survey estimates. FNS will also adjust for any nonresponse in the sampling weights to use for calculating survey estimates. More discussion of response rates is presented in Question 20.

19. Procedures for the Collection of Information:

Methodology for sample selection

Sample frames for the SAs represent a census of all agencies; therefore, no sampling of these entities will take place. For this ICR, the FNS National Office will liaise with FNS ROs to introduce the study to the SAs and SFAs using contact information collected from the earlier sample frame ICR, whereas the States will provide contact information regarding the local entities within each program to create accurate national sample frames for each school meal program. The study team will develop the sampling frames; selecting the samples using a stratified design with stratification based on SFA size (more details below); and implement the survey. The surveys for SAs and SFAs will be self-administered via the web. Respondents will be provided a link to access the web survey.

National School Lunch Program/School Breakfast Program (NSLP/SBP)

The sampling frame is all SAs in the 50 States; Washington, DC; Guam; and America Samoa. States will provide contact information about SFAs to assemble the local entities sample frame for NSLP/SBP. The sampling frames were constructed using administrative data from FNS and the FNS-742 (Verification Summary Report) file for school year (SY) 2014–2015 provided by FNS (which is approved under OMB# 0584-0594 Food Programs Reporting System [FPRS], expiration date 09/30/19) and the NCES Common Core of Data (CCD) LEA (School District) Universe Survey Data file (http://nces.ed.gov/ccd/pubagency.asp).

Expected levels of precision or accuracy

Any statistics calculated from the sample frame information assembled from States and the local entities sampled from these States should represent the population parameters and achieve precision at ±.05 with 95 percent confidence intervals.

Sampling weights will be constructed for the SFAs to account for the sample design and nonresponse, and appended to the final survey data.⁴ A base weight will be the inverse of the probability of selection of an SFA. The data will also be weighted to adjust for unit nonresponse, using a propensity modeling procedure to predict the probability of responding to the survey based on the available data collected on the sampling frame. Under this approach, the responding cases will be weighted by the inverse of the predicted probability of response using a weighting class methodology that divides the propensity scores into classes and assigns the average score within the class to each case. This approach⁵ helps to eliminate large adjustments to the survey weights to increase the survey precision in the estimates. Weights will also be ratio-adjusted, if necessary, so that the final weights are representative of the overall population of SFAs. Given the complex nature of the sample design and the estimates, the 2M research team proposes to utilize a jackknife variance replication method to simplify the computation of the statistical significance of the descriptive statistics and regression-adjusted estimates.⁶ This method will also account for the non-sampling error associated with the nonresponse adjustment and any subsequent post-stratification or calibration.

Estimation procedure

There are no estimation procedures beyond those described above.

Unusual problems requiring specialized sampling procedures

FNS does not anticipate unusual problems requiring specialized sampling procedures.

Any use of periodic (less frequent than annual) data collection cycles to reduce burden.

FNS intends to assemble the sample frames during the first year of information collection, and refresh (i.e., update) the information in the two subsequent years of data collection.

A census of all 53 SAs from the SA sampling frame will be invited to participate in the SA survey. A nationally representative sample of SFAs will be selected for participation in the SFA survey. The SFA sample will be selected from the frame of N = 17,803 SFAs. We will select a stratified systematic sample of 1,251 SFAs. Assuming an 80 percent response rate, that sample should yield at least 1,000 complete cases. The strata will be defined according to SFA size:

• Small: SFAs with less than 1,000 students

• Medium: SFAs with between 1,000 and 4,999 students

• Large: SFAs with between 5,000 and 24,999 students

• Extra Large: SFAs with 25,000 or more students

Within size strata, we will randomly select SFAs, controlling for (through systematic sampling) urbanicity, free and reduced-priced (F/RP) lunch status, and State. The urbanicity variable will include the following categories:

• City

• Suburban

• Town

• Rural

The F/RP lunch variable is defined as follows:

• Low (less than 30 percent of students)

• Medium (30–59.9 percent of students)

• High (60 percent or more of students)

Using the SFA sample frame of N = 17,803 described above, an initial sample of N = 1,251 SFAs was selected. Assuming an overall response rate of 80 percent or higher, this sample should result in 1,000 or more SFA completes. Characteristics of the sample along with the sample frame (population) are presented in Table 4. Table 4 shows that the sampling over the size categories results in sufficient subsample sizes of SFAs that cover all categories of urbanicity and percent of students approved for F/RP lunches. Moreover, all States and territories, with the exception of Guam, will be represented by two or more SFAs in the SFA sample. Guam is not included because, by chance, no SFAs from Guam were randomly selected into the sample.

Table 4. Characteristics of the Sample Frame and Sample for SAs and SFAs

Characteristic	Population		Sample
	N	%	N	%
Size
Small (<1,000)	10,422	58.54	375	29.98
Medium (1,000–4,999)	5,408	30.38	363	29.02
Large (5,000–24,999)	1,684	9.46	325	25.98
Extra-large (≥25,000)	289	1.62	188	15.03
Urbanicity
City	3,046	17.11	279	22.30
Suburban	4,004	22.49	408	32.61
Town	2,448	13.75	171	13.67
Rural	8,305	46.65	393	31.41
Percentage of students approved for F/RP lunch
Low (<30%)	7,423	41.70	512	40.93
Medium (30–59.9%)	4,747	26.66	361	28.86
High (≥60%)	5,633	31.64	378	30.22
State (by FNS region)
Mid-Atlantic
District of Columbia	64	0.36	4	0.32
Delaware	49	0.28	5	0.40
Maryland	44	0.25	9	0.72
New Jersey	710	3.99	44	3.52
Pennsylvania	776	4.36	51	4.08
Virginia	146	0.82	25	2.00
West Virginia	67	0.38	8	0.64
Mountain Plains
Colorado	197	1.11	18	1.44
Iowa	434	2.44	24	1.92
Kansas	388	2.18	25	2.00
Missouri	710	3.99	34	2.72
Montana	241	1.35	9	0.72
North Dakota	192	1.08	11	0.88
Nebraska	363	2.04	16	1.28
South Dakota	198	1.11	8	0.64
Utah	107	0.60	12	0.96
Wyoming	53	0.30	3	0.24
Midwest
Illinois	1,030	5.79	57	4.56
Indiana	499	2.80	33	2.64
Michigan	805	4.52	49	3.92
Minnesota	645	3.62	41	3.28
Ohio	1,152	6.47	70	5.60
Wisconsin	693	3.89	34	2.72
Northeast
Connecticut	183	1.03	12	0.96
Massachusetts	411	2.31	31	2.48
Maine	206	1.16	8	0.64
New Hampshire	94	0.53	7	0.56
New York	1,006	5.65	59	4.72
Rhode Island	58	0.33	6	0.48
Vermont	71	0.40	4	0.32
Southeast
Alabama	154	0.87	14	1.12
Florida	274	1.54	36	2.88
Georgia	223	1.25	24	1.92
Kentucky	181	1.02	14	1.12
Mississippi	166	0.93	13	1.04
North Carolina	182	1.02	24	1.92
South Carolina	104	0.58	14	1.12
Tennessee	181	1.02	20	1.60
Southwest
Arkansas	269	1.51	12	0.96
Louisiana	152	0.85	14	1.12
New Mexico	203	1.14	9	0.72
Oklahoma	558	3.13	26	2.08
Texas	1,181	6.63	101	8.07
Western
Alaska	44	0.25	2	0.16
Arizona	484	2.72	38	3.04
California	1,114	6.26	113	9.03
Guam	2	0.01	0	0.00
Hawaii	27	0.15	3	0.24
Idaho	141	0.79	9	0.72
Nevada	24	0.13	4	0.32
Oregon	234	1.31	19	1.52
Washington	313	1.76	25	2.00
Total	17,803		1,251

20. Methods to Maximize the Response Rates and to Deal with Nonresponse:

Procedures to be followed to maximize the number of SAs and SFAs that provide the requested information include the following:

• The letters notifying SAs and SFAs are crafted to emphasize the importance of this data collection and how the information will help FNS to better understand and address current policy issues related to CN burden.

• A toll-free number and study email address will be provided so that SAs and SFAs can receive assistance with the survey, if needed.

• Surveys have built in checks and skip patterns to aid the respondents to more smoothly and accurately navigate through the questionnaire.

• Reminder emails and phone calls will be used to follow up with nonresponding SAs and SFAs.

• Surveys are brief (20 minutes each)

• Pretesting was conducted to ensure surveys are clear and concise

FNS believes that following all of these steps will result in a 100 percent response rate for the SA survey and an 80 percent response rate or higher for the SFA survey.

21. Test of Procedures or Methods Undertaken:

The Contractor, 2M, and its Subcontractor, Mathematica Policy Research (Mathematica) conducted cognitive pretests with two SAs and four SFAs on November 16, 17, and 20, 2017, using Word versions of the instruments. FNS provided input for selection of SAs and SAs provided input for selection of SFAs. Pretesting enabled FNS to ask the best possible questions and make the most efficient use of SA and SFA staff time.

The primary objective of the cognitive pretests was to understand (1) whether respondents understood the meaning of the questions; (2) how respondents retrieved information and thought through their response to questions; (3) whether response categories were applicable, clear, and comprehensive; (4) if respondents had trouble answering any questions; (5) how respondents used open-ended questions (e.g., to provide additional detail, to address limitations of the question or response categories); and (6) whether the respondent felt the questionnaire addressed the most significant sources of administrative burden. Mathematica asked pretest participants to complete the survey and debriefing interview by telephone to discuss their impressions.

Pretest SAs reported that the survey questions were generally easy to understand and answer, and the survey addressed their most significant sources of administrative burden. In some instances, respondents suggested clarifying introductory text and adding response options. Pretest SFAs reported that the questions in the SFA Survey were easy to understand and answer, and the survey generally addressed their most significant sources of administrative burden. Respondents suggested clarifying the CN programs covered by the survey, adding response options, and modifying questions to make them easier to answer. Changes to the surveys were made based on this feedback from pretest SAs and SFAs.

22. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

FNS has contracted with 2M, and its Subcontractor, Mathematica, to conduct this study, including data collection and analyses. Research analysts at FNS were consulted on the methodology and design of this research.

Name	Affiliation	Telephone Number	Email
Nicholas Beyler	2M	(855) 328-1611	n[email protected]
Liz Gearan	Mathematica	(202) 484-9220	l[email protected]
Ruth Morgan	USDA – FNS	(703) 457-7759	r[email protected]
Jinee Burdg	USDA – FNS	(703) 305-2744	[email protected]

23. Appendices - Research Tools/Instruments, Pretest Materials, Burden Estimate Calculations:

A.1. Recruitment letter: FNS to RO

A.2. Recruitment letter: RO to SA

A.3. Recruitment letter: SA to SFA

A.4. Invitation email: 2M to SA

A.5. Invitation email: 2M to SFA

A.6. Follow-up email reminder: SA

A.7. Follow-up email reminder: SFA

A.8. SA reminder telephone script

A.9. SFA reminder telephone script

A.10. Table example to include with recruitment letter from RO to SA

A.11. Section 28 of the Richard B Russell National School Lunch Act as amended by the Healthy Hunger Free Kids Act of 2010 (HHFKA)

A.12. House Report 114-531

B.1. Data Collection Instrument: SA

B.2. Data Collection Instrument: SFA

C. Estimated Annual Burden for Information Collection and Nonresponse for CN QRS Surveys

D.1. Pretest questionnaire: SA

D.2. Pretest questionnaire: SFA

D.3. Example FNS RO Pretest email and Example RO SA Pretest Invite attachment

D.4. Example RO SA Pretest Invite

1 Link to the H.R. 244 Consolidated Appropriations Act 2017 Public Law 115-31, May 4, 2017.

https://www.congress.gov/resources/display/content/Appropriations+for+Fiscal+Year+2017

2 States will provide 2M the email addresses for the SFAs.

3 Due to the recent natural disasters along the Eastern United States and in the Atlantic Ocean, Puerto Rico and the U.S. Virgin Islands are excluded from the SA sample frame. Their SFAs are also excluded from the SFA sample frame.

4 Sampling weights are not required for the SAs since we are collecting data from a census of all SAs.

5 Wun, L., Ezzati-Rice, T. M., Baskin, R., Greenblatt, J., Zodet, M., Potter, F., . . . Touzani, M. (2004). Using propensity scores to adjust weights to compensate for dwelling unit level nonresponse in the Medical Expenditure Panel Survey. Agency for Healthcare Research and Quality Working Paper No. 04004. Rockville, MD: Agency for Healthcare Research and Quality.

6 Shao, J., & Wu, C. F. J. (1989). A general theory for jackknife variance estimation. Annals of Statistics, 17(3), 1176–1197.

1