Non-Response OMB Part A_Final Sept 2017

Supporting Statement for OMB Clearance for the Study of Non-Response to the School Meals Application Verification Process

Part A

September 22, 2017

Holly Figueroa

Social Science Research Analyst

Office of Policy Support

Food and Nutrition Service

United States Department of Agriculture

3101 Park Center Drive, Room 1014

Alexandria, Virginia 22302

Phone: 703-305-2105

Email: [email protected]

TABLE OF CONTENTS

LIST OF

APPENDICES 2

PART A. JUSTIFICATION 3

1. Circumstances Making the Collection of Information Necessary 3

2. Purpose and Use of the Information 7

3. Use of Information Technology and Burden Reduction 13

4. Efforts to Identify Duplication and Use of Similar Information 14

5. Impacts Small Businesses or Other Small Entities 15

6. Consequences of Collecting the Information Less Frequently 15

7. Special Circumstances Relating to the Guideline of 1320.5(D)(2) 16

8. Comments in Response to the Federal Register Notice and Efforts to Consult with Persons Outside the Agency 17

9. Explanation of Any Payments or Gifts to Respondents 18

10. Assurances of Confidentiality Provided to Respondents 23

11. Justification for Sensitive Questions 24

12. Estimates of Hour Burden Including Annualized Hourly Costs 26

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers 28

14. Annualized Cost to the Federal Government 28

15. Explanation for Program Changes or Adjustments 29

16. Plans for Tabulations and Publication and Project Time Schedule 29

17. Display of Expiration Date for OMB Approval 33

18. Exception to the Certification Statement Identified in Item 19.0 of Form OMB 83-1 33

References 34

TABLES

A2.1 Overview of Data Collection Activities 7

APPENDICES

1 Code of Federal Regulations, Title 7, Part 210

2 Code of Federal Regulations, Title 7, Part 245

3 Improper Payment Information Act (IPIA) of 2002 (PL 107-300)

4 Improper Payment Elimination and Recovery Act (IPERA) of 2010 (PL 111-204)

5 School Meals: USDA Could Improve Verification Process for Program Access (GAO-15-634T)

6 FNS – National School Lunch and School Breakfast Programs (OIG-27601-0001-41)

7 Framework for Case Study of Verification Outcomes

8 The Healthy, Hunger Free Kids Act (HHFKA, Public Law 111-296; December 13, 2010)

9 The 2004 Child Nutrition and WIC Reauthorization Act

10 Verification data request

11 District interview

12.a Household survey—English

12.b Household survey—Spanish

13 Reapplication data request

14 District recruitment letter

15 District frequently asked questions

16 District recruitment call script

17 Verification data request advance email

18 Verification data request template

19 District interview invitation email

20.a Household survey advance letter—English

20.b Household survey advance letter —Spanish

21.a Household survey brochure—English

21.b Household survey brochure—Spanish

22.a Household survey frequently asked questions—English

22.b Household survey frequently asked questions—Spanish

23 Reapplication data request advance email

24 Reapplication data request template

25 Public Comments to 60 Day Federal Register Notice

26 Response to Public Comments

27.a NASS comments

27.b Response to NASS Comments

28 Pretest Memo

29.a Household survey call script—English

29.b Household survey call script—Spanish

30 Contractor Confidentiality Agreement

31 Contractor Federal Wide Assurance

32 Burden Table

33.a Household survey thank you letter—English

33.b Household survey thank you letter —Spanish

34 Verification data request thank you email

35 Verification data request pre-visit telephone protocol

36 Verification data request confirmation email

37 Respondent payment log

38 Reapplication data request thank you email

39 State recruitment letter

40.a Household survey door hanger—English

40.b Household survey door hanger—Spanish

41.a Police letter—English

41.b Police letter—Spanish

42 IRB approval letter

PART A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

Explain the circumstances that make the collection of information necessary. Identify any legal or administrative requirements that necessitate the collection. Attach a copy of the appropriate section of each statute and regulation mandating or authorizing the collection of information.

The National School Lunch Program (NSLP) and the School Breakfast Program (SBP) provide subsidized lunches and breakfasts to over 30 million students each school day. Students are certified eligible to receive free or reduced-price (F/RP) meals through application or direct certification. Districts are required to use direct certification to determine a child’s eligibility based on State-provided data about participation in the Supplemental Nutrition Assistance Program (SNAP) or other programs that grant students categorical eligibility such as Temporary Assistance for Needy Families (TANF) and the Food Distribution Program on Indian Reservations (FDPIR).

Certification errors (which result in improper payments) can lead to (1) the allocation of F/RP school meals to children in families who are ineligible for the level of benefit received, or (2) the denial of benefits to eligible children in need. The certification errors may occur as the result of school district processing errors or household reporting errors. To detect and deter household reporting errors, districts verify a small sample of applications each year as required and described in Code of Federal Regulations, Title 7, Parts 210 National School Lunch Program (Section 18) and 245 Determining Eligibility for Free and Reduced Price Meals and Free Milk in Schools (Section 6a) (Appendices 1 and 2, respectively). The Improper Payment Information Act (IPIA) of 2002 (PL 107-300) (Appendix 3) and the Improper Payment Elimination and Recovery Act (IPERA) of 2010 (PL 111-204) (Appendix 4) require that FNS identify and reduce erroneous payments in the National School Lunch Program, including both underpayments and overpayments. FNS relies upon the Access, Participation, Eligibility and Certification (APEC) Study Series to provide “reliable, national estimates of erroneous payments made to school districts in which the NSLP and SBP operate.” The current study will go beyond the APEC Study Series to investigate outcomes of the district verification process, including the accuracy of the procedures and reasons why certain households do not respond to district verification requests.

This study will examine the accuracy of district verification procedures using a case study approach similar to the Case Study of National School Lunch Program Verification Outcomes in Large Metropolitan School Districts (2004) (OMB Control Number 0584-0516 Evaluation of the NSLP Application and Verification and Pilot Program, expiration date October 31, 2003). Understanding the approaches districts take to verification, the accuracy of the verification process and the results it produces in the context of potential changes since it was last studied supports FNS’s program integrity mission. Findings from this data collection will also help FNS identify potential improvements to the verification process recommended by the General Accounting Office (GAO) in their 2015 report School Meals: USDA Could Improve Verification Process for Program Access (GAO-15-634T) (Appendix 5) and similar recommendations found in the USDA Office of Inspector General (OIG) audit report FNS – National School Lunch and School Breakfast Programs (27601-0001-41) (Appendix 6).

Consistent with the research conducted in 2004, this study will purposively enroll 20 districts in the case study, describe their verification outcomes, and independently verify eligibility for two samples of households approved by application and selected for verification by the district. These samples include: (1) households that did not respond to the verification request (nonresponding households), and (2) responding households with no change in benefits (responding households with no changes). Appendix 7 provides a framework for the verification outcomes of interest for this case study.

The objectives of this study are to: (1) verify incomes of nonresponding households initially approved on the basis of income or categorical eligibility, (2) verify incomes of responding households initially approved on the basis of income or categorical eligibility with no change in benefits due to verification, (3) examine the process of selecting applications “for cause¹,” (4) consider the ultimate results of the verification process (i.e., how many students continue to receive program benefits, how many students have their program benefits reduced/increased), and (5) provide descriptive statistics for all districts and study districts.

This research will expand on the 2004 study by: (1) including at least one rural district in the case study, (2) interviewing district officials about processes for selecting applications for cause, (3) analyzing verification outcomes for applications selected for cause, (4) analyzing household reasons for not responding to district verification requests, and (5) redesigning 2004 report analyses to account for recent policy changes. For example:

• Section 104 of The Healthy, Hunger Free Kids Act (HHFKA, Public Law 111-296; December 13, 2010) (Appendix 8) established the Community Eligibility Provision (CEP), which allows the highest poverty schools and districts to provide free meals to all students without collecting household applications. This study will exclude students enrolled in CEP from the sample because they are not included in the verification sample.

• Section 104 of The 2004 Child Nutrition and WIC Reauthorization Act (Appendix 9) required school districts to establish systems to directly certify children from households that receive SNAP benefits by school year (SY) 2008-2009. Prior to this legislation, not all districts used direct certification. As such, the 2004 study compared findings for districts that used direct certification and those that did not. This study will exclude those comparisons because they are no longer applicable. Similarly, this study will exclude comparisons of district findings depending on whether they used multichild applications because those applications are now mandatory for all districts.

• The types of programs used for direct certification have expanded since the prior study was conducted. Beginning in school year (SY) 2012–2013, some States have participated in demonstrations to add Medicaid to the list of programs used for direct certification (DCM). The initial DCM demonstration allowed direct certification only for students eligible for free meals. Beginning in SY 2016–2017, a second demonstration also allowed States to certify students using DCM for reduced-price meals. Unlike those in SNAP, TANF, and FDPIR, students on Medicaid are not categorically eligible; they must satisfy the income-eligibility requirements. Because students directly certified for F/RP meals are not subject to verification, we will continue to exclude these students from the study.

• The 2004 study compared district findings by the two district verification sampling methods allowable at the time: random sampling or focused sampling. This study will make similar comparisons for some analyses. However, districts’ verification sample size requirements have been updated in recent years; in particular, rather than two possible sampling methods, districts now have three possible methods: standard sampling, and two alternatives (including random sampling). This study will update these comparisons accordingly.

This is a study of a purposive sample of school districts and is not intended to be generalizable or representative of other school districts nationwide. Any analysis of the data will include appropriate qualifications of the data limitations and not be applied generally to other school districts.

2. Purpose and Use of the Information

Indicate how, by whom, how frequently, and for what purpose the information is to be used. Except for a new collection, indicate the actual use the agency has made of the information received from the current collection.

Table A2.1 summarizes the data collection plan. The table shows, for each data collection, the source, mode, purpose, length, and target number of completed interviews.

Table A2.1. Overview of data collection activities

Instrument	Source	Mode	Purpose	Length	Number of respondents	Frequency
Verification data request (Appendix 10)	School district	On-site collection of electronic administrative data	Collect household characteristics and verification results for all households selected for verification	4 hours2	20	Once
District interview (Appendix 11)	SFA3 Director or designee	Telephone	Collect information about the process and reasons for selecting applications for cause	20 minutes	20	Once
Household survey (Appendix 12. a/b)	Household	In person, computer-assisted personal interviews	Collect income information to independently verify eligibility and collect respondent perceptions on the verification process	2.75 hours4	1,4805	Once
Reapplication data request (Appendix 13)	District	Electronic file submission	Collect eligibility status and basis of decision for reapplicants	1.25 hours6	20	Once

School Food Authority (SFA) directors (or their designees) will be asked to participate in three data collection components: (1) the verification data request, (2) the district interviews, and 3) the reapplication data request. While these study requests are voluntary, the SFAs will be strongly encouraged to cooperate with them, as per Section 305 of HHFKA.⁷ Households will also be asked to participate in a voluntary, in-person household survey. Descriptions of each data collection component are provided below.

Verification Data Request (Appendix 10). Following OMB approval, after districts have completed their annual verification processes, the study team will attempt to enroll 20 school districts in this study. Recruiting materials will be sent to SFA Directors in 25 districts, including the District Recruitment Letter (Appendix 14) and District Frequently Asked Questions (Appendix 15). States in which sample districts are located will also receive notification of the study via the State Recruitment Letter (Appendix 39). We will then place recruiting phone calls to SFA Directors using the District Recruitment Call Script found in Appendix 16. After holding recruiting conversations with 25 districts, we will purposively select the 20 districts best suited to participate in the case study. Enrolling 20 school districts will allow the study team to answer related research questions with sufficient precision. Having 20 districts in the study allows for diversity among the districts along several characteristics, while still facilitating the geographic clustering that the operational design requires. A smaller number of districts would increase the household sample targeted per district and limit the sample to the largest districts given the small proportion of households that are selected for verification and fall into our eligible sample groups. A larger group of districts would increase study costs and respondent burden, and would not help the study team to better address the research questions.

After the 20 districts are enrolled, the Verification Data Request Advance Email (Appendix 17) will be sent to SFA Directors to request information for each household that was part of the district’s verification sample. The email will include two attachments: 1) the Verification Data Request (Appendix 10), and 2) the Verification Data Request Template (Appendix 18). If districts are unable or unwilling to submit the requested data electronically using the template provided, the study team will call the district using the Verification Data Request Pre-Visit Telephone Protocol (Appendix 35) to arrange a time for trained research staff to collect the verification data on-site. A Verification Data Request Confirmation Email (Appendix 36) will be sent a few days prior to the scheduled visit. After the verification data have been collected (whether on-site or via electronic submission), the study team will send a Verification Data Request Thank You Email (Appendix 34).

The information requested in the Verification Data Request (Appendix 10) includes: background information on the households selected for verification, such as household size and the number of students enrolled at the time the verification sample was selected; information on the original application for school meal benefits, including the district’s initial determination of eligibility status and whether the household was certified for school meal benefits based on household income reported on the application; information related to the verification process, including an indicator of whether the application was selected for cause and whether the household responded to verification requests; and contact information for the household.

From these data, two groups of households eligible for the in-person household survey will be identified. Eligible households are those whose applications initially were approved as eligible for F/RP meals based on household income or as eligible for free meals on the basis of categorical eligibility. The eligible households will be separated into two subgroups: (1) those who did not respond to verification (nonresponding households), and (2) those whose eligibility status did not change as a result of verification, excluding those verified through direct verification (responding households with no changes). If the estimated number of households in these two categories exceeds the target number to be interviewed in a district (62 nonresponding households and 41 responding households with no changes per district), a subsample of households will be randomly selected to complete the household survey.

District Interview (Appendix 11). In spring 2018, study staff will conduct 20-minute semi-structured telephone interviews with the SFA director or a representative in each of the 20 districts in the sample. SFA directors (or their designated representatives) will be contacted via the District Interview Invitation Email (Appendix 19) inviting them to schedule an appointment to participate in the interview. The interviews will focus on collecting information about the process and reasons for selecting applications for cause, including the following: when and how often the district selects applications for cause; the frequency, procedures, and methods the district uses to contact households about verification; whether the district has formal and/or informal sets of criteria used to identify questionable applications; and the process the district uses to verify questionable applications. The interview will also ask SFA directors about how the district notifies households that they have been selected for verification and whether or not the district uses the household preferred language to communicate. These interviews will help examine why districts select applications for cause and how the selection process works, and will serve as a complement to the data collected in the verification data request.

Household Survey (Appendix 12. a/b). The household survey will be structured to investigate differences between households that responded to verification requests and those that did not, and to verify incomes of nonresponding households and responding households with no change in benefits.

In early 2018, field staff will administer a 45-minute, in-person computer-assisted personal interviewing (CAPI) survey to up to a total of 1,480 households: approximately 840 total nonresponding households and 640 responding households with no changes. The household survey will be conducted in the respondent’s home to maximize response and facilitate collection of income verification data, including producing physical documentation of income amounts when available. Groves and Kahn (1979) found that survey completion rates are higher and survey break-offs are lower for in-person interviews. They also found that in-person respondents reported feeling less nervous about reporting household income.

An advance package will be sent to sampled households within approximately one month of receiving the district verification data. The advance package will include the Household Survey Advance Letter (Appendix 20. a/b) informing households of the study and encouraging them to participate, and the Household Survey Brochure (Appendix 21. a/b) describing the aims of the study. The advance letter will inform the respondent that the interviewer will ask to see copies of income documentation as part of the survey, and will assure respondents that participation in the study will not affect certification for free or reduced-price meals. The Household Survey Brochure will include a toll-free number that households can call if they have questions, and in the event a household calls this number, study staff will use the Household Survey Frequently Asked Questions document (Appendix 22. a/b) to address any concerns that arise.

Within a few weeks of sending the advance package, the study team will contact sampled households by phone using the Household Survey Call Script (Appendix 29. a/b) to schedule the survey. If households are unable to be reached by phone, trained field staff will attempt up to three in-person household recruitment visits. If sample members are not home at the time of the visit, field staff will leave the Household Survey Door Hanger (Appendix 40.a/b). Data collectors will carry photo identification and copies of study materials to validate their visits to neighborhoods and households included in the study. A Police Letter (Appendix 41. a/b) will be sent to police departments in participating school districts explaining the purpose of the study. Additional details about participant recruitment are available in Supporting Statement B, Question 2.

The household survey will collect detailed information on household structure, sources of income, employment history of adult members of the household, and specific income from October 2017, by source, for each member of the household. The household survey will include other self-reported characteristics such as household size, level of education of the respondent, grade of the student, and race/ethnicity of the respondent and student. The survey will also collect information about how often students in the household eat school meals, parents’ perceptions of the verification process, and parents’ perceptions of the school meal programs. The interviewers will ask nonresponding households about the reasons they did not respond to verification requests. Interviewers will ask responding households with no changes why they chose to complete the request. Interviewers will ask both nonresponding households and responding households with no changes about their perceptions of the barriers to responding to verification requests.

Following completion of the survey, participants will complete the Respondent Payment Log (Appendix 37) to acknowledge receipt of the participation incentive. Each participating household will also receive a Household Survey Thank You Letter (Appendix 33.a/b).

Reapplication Data Request (Appendix 13). In spring 2018, the study team will send each study district a Reapplication Data Request (Appendix 13) to obtain updated information on households’ reapplication and certification status as of March 1, 2018. The data collected from this instrument will be used to consider the ultimate results of the verification process. For example, the data will be used to show how many households reapply after they are denied for nonresponse or because their documentation did not support their claim for eligibility. We will send a Reapplication Data Request Advance Email (Appendix 23) to SFA Directors notifying districts about the upcoming data request. The email will include two attachments: 1) the Reapplication Data Request (Appendix 13), and 2) the Reapplication Data Request Template (Appendix 24). Reapplication data will be collected by May 2018. Following receipt of the reapplication data, the study team will send each participating district a Reapplication Data Request Thank You Email (Appendix 38).

Public Use Dataset. At the conclusion of the data collection, we will prepare a public use dataset that removes all personally identifying information so outside researchers may use the data to conduct their own analyses.

3. Use of Information Technology and Burden Reduction

Describe whether, and to what extent, the collection of information involves the use of automated, electronic, mechanical, or other technological collection techniques or other forms of information technology, e.g., permitting electronic submission of responses, and the basis for the decision for adopting this means of collection. Also, describe any consideration of using information technology to reduce burden.

FNS is committed to complying with the E-government Act of 2002 to promote the use of technology. Wherever possible, improved technology has been incorporated into data collection protocols to reduce respondent burden.

All in-person interviews with households will be conducted electronically with 1,480 respondents through a Computer-Assisted Personal Interviewing (CAPI) survey instrument. Use of CAPI will make possible accurate skip patterns, customized wording for State-specific TANF and SNAP names, response code validity checks, and consistent checking and editing, all of which improve the pace and flow of the interviews and thus reduce respondent burden.

All of the school and district records will be requested electronically, in lieu of collecting hard-copy documents or requiring the completion of specific forms. The Verification Data Request Template (Appendix 18) and the Reapplication Data Request Template (Appendix 24) have been designed as Excel workbooks in order to standardize information across school districts. Each school district will be provided with a secure website to upload all electronic records. FNS estimates that out of the 40 total responses for this collection (20 responses to the Verification Data Request and 20 responses to the Reapplication Data Request), approximately 65 percent of the Verification Data Request (13 total) and all of the Reapplication Data Request (20 total) responses will be submitted electronically. We expect a higher rate of electronic response on the Reapplication Data Request because it will be smaller and less time consuming than the initial Verification Data Request. Because there are only 20 district interview respondents, the district interview will be conducted on paper by telephone. This method is cost effective for the sample size.

FNS estimates that out of the 15,072 total responses for this collection (12,156 respondents + 2,916 nonrespondents), approximately 10 percent (1,513) of responses will be collected electronically. All district interviews (20 total) will be conducted by telephone, while 80 percent (32 of 40) of the Verification and Reapplication Data Requests will be collected electronically and 100 percent (1,480 total) of the Household Survey data will be collected electronically.

4. Efforts to Identify Duplication and Use of Similar Information

Describe efforts to identify duplication. Show specifically why any similar information already available cannot be used or modified for use for the purpose described in item 2 above.

Every effort has been made to avoid duplication of data collection efforts. These efforts include a review of USDA reporting requirements, State administrative agency reporting requirements, and special studies by government and private agencies.

FNS has sole responsibility for administering the USDA school meal programs. It funds State agencies which, in turn, fund local SFAs. Within this structure, local education agencies (LEAs) are responsible for certification and verification activities and SFAs are responsible for food service delivery. SFAs report on their activities to the State agency, which reports to FNS by way of seven FNS Regional Offices. Other than extant, district-level administrative data from the SFA Verification Summary Reports (Form FNS-742, approved under OMB Control Number 0584-0594 Food Programs Reporting System (FPRS), expiration date September 30, 2019), the information required for this study is not currently reported to FNS on a regular basis in a standardized form, nor is the information available from any other previous or contemporary study. The Access, Participation, Eligibility, and Certification (APEC) study series collects household income data to provide estimates of erroneous payments made to school districts for the NSLP and SBP. However, the data collected through this series is not sufficient to examine the processes districts use to verify the accuracy of applications, the barriers individuals face when responding to requests for verification, and the procedures districts use when selecting applications for cause.

5. Impacts Small Businesses or Other Small Entities

If the collection of information impacts small businesses or other small entities, describe any methods used to minimize burden.

Information being requested has been held to the minimum required for the intended use. The sample will be composed of moderate to large-sized school districts (i.e., districts with at least 10,000 students) that verify a sufficiently large sample of applications. As a result, we do not anticipate any impact on small entities.

6. Consequences of Collecting the Information Less Frequently

Describe the consequence to Federal program or policy activities if the collection is not conducted or is conducted less frequently, as well as any technical or legal obstacles to reducing burden.

The NSLP and SBP enable FNS to pursue its mission to reduce hunger by providing children and low-income people access to food. FNS certifies eligibility for these programs through application or direct certification, and errors in these processes may lead to improper payments. To reduce the likelihood of such errors, FNS must verify the accuracy of the verification processes used to determine eligibility for NSLP and SBP as prescribed by 7 CFR Parts 210 National School Lunch Program and 245 Determining Eligibility for Free and Reduced Price Meals and Free Milk in Schools (Appendices 1 and 2, respectively). The data collection in this study is essential for examining the accuracy of these district verification procedures. Findings from this data collection will also help FNS investigate potential improvements to the verification processes recommended by the GAO in their 2015 report School Meals: USDA Could Improve Verification Process for Program Access (GAO-15-634T) (Appendix 5) and similar recommendations found in the USDA OIG audit report FNS – National School Lunch and School Breakfast Programs (27601-0001-41) (Appendix 6) that evaluated how the agency has attempted to lower error rates for NSLP and SBP.

Most data being collected in the study involve a one-time data collection with no planned repetition. SFAs will be contacted a second time in the spring of 2018 to learn of changes in students’ certification and enrollment during the school year.

7. Special Circumstances Relating to the Guideline of 1320.5(D)(2)

Explain any special circumstances that would cause an information collection to be conducted in a manner:

• requiring respondents to report information to the agency more often than quarterly;

• requiring respondents to prepare a written response to a collection of information in fewer than 30 days after receipt of it;

• requiring respondents to submit more than an original and two copies of any document;

• requiring respondents to retain records, other than health, medical, government contract, grant-in-aid, or tax records for more than three years;

• in connection with a statistical survey, that is not designed to produce valid and reliable results that can be generalized to the universe of study;

• requiring the use of a statistical data classification that has not been reviewed and approved by OMB;

• that includes a pledge of confidentiality that is not supported by authority established in statute or regulation, that is not supported by disclosure and data security policies that are consistent with the pledge, or which unnecessarily impedes sharing of data with other agencies for compatible confidential use; or

• requiring respondents to submit proprietary trade secret, or other confidential information unless the agency can demonstrate that it has instituted procedures to protect the information's confidentiality to the extent permitted by law.

There are no special circumstances. This collection of information will be conducted in a manner consistent with the guidelines in the Code of Regulations, 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult with Persons Outside the Agency

If applicable, provide a copy and identify the date and page number of publication in the Federal Register of the agency's notice, soliciting comments on the information collection prior to submission to OMB. Summarize public comments received in response to that notice and describe actions taken by the agency in response to these comments.

Describe efforts to consult with persons outside the agency to obtain their views on the availability of data, frequency of collection, the clarity of instructions and recordkeeping, disclosure, or reporting form, and on the data elements to be recorded, disclosed, or reported.

Notice of this study was published in the Federal Register, Volume 81, Number 248, pages 95101-95102, on December 27, 2016. It specified a 60-day period for comment ending February 27, 2017. One comment was received that related to reducing burden for district staff, making study materials available to households with language and literacy variances, examining the prototype income eligibility benefit application, and sharing outcomes and best practices with Child Nutrition Programs. Public comments and responses are included in Appendices 25 and 26, respectively.

In response to the burden concerns, FNS explained that the study data will be collected using electronic means when possible. When electronic means are not possible, FNS stated that the data request activities and processes will be designed to minimize burden on respondents. Additionally, FNS mentioned that the school district personnel that may have a role in the school meals application verification process will be kept informed throughout the study while keeping the burden on these key individuals minimal. A final report will be disseminated to ensure transparency of study outcomes and best practices.

Consultations about the research design, sample design, data sources and needs, and study reports occurred during the study’s design phase and will continue to take place throughout the study. The purpose of these consultations is to ensure the technical soundness of the study and the relevance of its findings and to verify the importance, relevance, and accessibility of the information sought in the study. FNS consulted with the National Agricultural Statistics Service (NASS) of USDA, who reviewed sampling and statistical methodologies. NASS comments and the response to NASS comments appear in Appendices 27a and 27b, respectively. Additionally, the data collection instruments were pretested externally to determine whether questions were written appropriately and whether they captured data most relevant to the research questions and objectives. This feedback was then used to refine and finalize the data collection instruments, as summarized in the Pretest Memo in Appendix 28. Study consultants included:

Pretest Participants
Stephen Protz	FS Regional Supervisor-Federal Meals Application Manager	520-225-4700
Christina Varela	Manager of Food Services	831-796-7082
NASS Staff
Alison Black	Methods Division	202-690-2388

9. Explanation of Any Payments or Gifts to Respondents

Explain any decision to provide any payment or gift to respondents, other than re-enumeration of contractors or grantees.

The target populations for this study are low-income (1) nonresponding households and (2) responding households with no changes in school meals benefits. Including the nonresponding households in the study especially raises concerns about low response and potential nonresponse bias. By group definition, 100% of the nonresponding households did not respond to verification requests from the school district, thus making it likely that they may not respond to requests to participate in the current study. The responding households with no changes in benefits did respond to the districts’ verification requests; however, these households remain low-income and may still be hard to reach as they may have nonworking telephones or limited cell-phone usage (such as “pay-as-you-go” plans with incur costs at a standard rate of $0.10 per minute).

Providing an extrinsic incentive increases cooperation rates, thereby minimizing non-response bias, especially in populations defined as low-income, and a monetary incentive even more so than other incentive types (Groves et al., 2009; James, 1996; Singer, 2002). In addition, improved cooperation rates reduce the need for call backs which decrease survey costs and disproportionately encourage those less interested in the research to participate, thus reducing non-response bias.

As such, permission is requested to offer a monetary incentive to all household survey participants to compensate them for costs they will incur for participating in the survey. These costs include childcare that may be needed during the two hours required to gather income documentation needed to complete the survey, cell phone and data usage costs associated with calls and texts needed to set up appointments and reminders to complete the survey, and printing or copying costs incurred to prepare income documentation. Additionally, transportation costs may be incurred as those without internet access or printing capabilities may have to travel to a location with wireless or printing services. If permission is granted, interviewers will provide respondents with a $25 Visa gift card upon completion of the survey to offset these costs and reduce nonresponse bias in the study. Proposed incentive amounts are based on the following participation cost estimates:

Cost Category	Calculation	Estimated Cost
Child care	Median hourly wage for childcare workers8: $10.18 x 2 hours to collect income documentation	$20.36
Cell phone and data usage	$0.10 standard rate per minute for pay-as-you-go phones x 15 minutes estimated burden for the Household Survey Call Script	$1.50
Printing/Copying	$0.20 per page (IRS reasonable cost estimate9) x 5 pages of household income documentation	$1.00
Transportation	Average population-weighted distance to nearest public library10 (for access to internet and printers): 4 miles roundtrip x $0.535 (federal mileage reimbursement rate)	$2.14
Total		$25.00

A high response rate increases the likelihood that the survey results are illustrative of the entire target population. Survey estimates may be biased if respondents differ substantially from non-respondents. These differences may impact the validity of the survey results. Research has shown that incentives can minimize non-response bias to surveys without compromising the quality of the data (Singer and Kulka 2002; Singer and Ye 2013). Goldenberg et al. (2009) found that monetary incentives increased response rates and data quality over no incentive. Those receiving the incentive were less likely to say “don’t know” or refuse to answer individual items. There is also evidence that incentives bolster participation among those with lower interest in the survey topic (Jäckle and Lynn 2007; Kay 2001; Schwartz et al. 2006), resulting in data that are more complete. Others have found that incentives significantly increase response rates overall, but particularly with those who had previously refused (Zagorsky and Rhoton 2008). Singer and Kulka (2002) examined a number of studies that showed that incentives reduce differential response rates and hence the potential for nonresponse bias. Mercer et al. (2015) conducted a meta-analysis of the dose-response association between incentives and response and found a positive relationship between higher incentives and response rates for household telephone surveys offering post-pay incentives. Further, Singer et al. (1999), in a previous meta-analysis, found that incentives in face-to-face and telephone surveys were effective at increasing response rates, with a one dollar increase in incentive resulting in approximately a one-third of a percentage point increase in response rate from respondents who may otherwise be underrepresented in surveys, such as those from low income and minority populations.

The incentives proposed for this study are based on the characteristics of the study population and experience with conducting in-person surveys with similar low-income populations and in recent studies on food security:

• National Estimates of NSLP and SBP Erroneous Payments (2011-2016) (OMB Control Number 0584-0530 NSLP/SBP Access, Participation, Eligibility, and Certification Study (APECII), expiration date August 31, 2015), provided a $25 incentive to respondents of the household survey resulting in an 82.4 percent response rate.

• Site-specific baseline survey response rates in the USDA-sponsored 2013 Electronic Benefits Transfer for Children (SEBTC) Demonstration: Evaluation Findings for the Full Implementation Year study (OMB Control Number 0584-0559 Evaluation of the Impact of the Household-Based Summer Demonstrations on Food Insecurity Among Children (SEBTC), discontinued as of March 31, 2017) ranged from 39 percent to 79 percent across 14 sites using a $25 incentive. The average unweighted response rate was 67 percent.

• The USDA Healthy Incentives Pilot (HIP) (OMB Control Number 0584-0584, Expiration Date August 31, 2014), surveyed SNAP participants in Massachusetts. HIP respondents were provided a $20 incentive for the baseline interview, which achieved a 63% response rate; $30 for the Round 2 interview (3-6 months after baseline) which achieved an 83% response rate; and $40 for the Round 3 interview (11-13 months after baseline), which achieved an 81% response rate.

• The Evaluation of the Summer Food Service Program Participant Characteristics (OMB Control Number 0584-0595, Expiration Date August 31, 2016), used $25 prepaid VISA cards to increase participation of parents/caregivers of participants and eligible nonparticipants.

These studies were conducted with populations similar to those in this study and, in sum, indicate that a $25 incentive will be sufficient to target response rates of 68 percent of nonresponding households and 78 percent of households with no changes.

In addition to offering an incentive amount shown to reduce nonresponse bias in previous data collections, the proposed incentive also addresses key OMB considerations identified in its “Guidance on Agency and Statistical Information Collections” memorandum and summarized below:

Improved coverage of specialized respondents or minority populations. The target populations are socially disadvantaged groups, namely low-income and some rural households, all of which are considered hard-to-reach (Bonevski et al. 2014). In addition, households in the nonresponding group may be difficult to recruit into the study and their lack of participation jeopardizes the variability that would be observed in a complete sample. Incentives may encourage greater participation among this group. Respondents who would otherwise not consider participating in the surveys may do so because of the incentive offer (Groves et al. 2000).

Past experience. The studies described above demonstrate the effectiveness of incentives for surveys of similar low-income study populations.

Equity. The incentive amounts will be offered equally to all household survey participants. The incentives will not be targeted to specific subgroups or participants in only some of the districts, nor will they be used to convert refusals.

The planned incentives for the household surveys are designed to mitigate non-response bias, promote high data quality, and compensate participants for the costs associated with participating in the survey. No incentives are planned for participants from State or local governments (State CN Directors, SFA Directors or local police).

10. Assurances of Confidentiality Provided to Respondents

Describe any assurance of confidentiality provided to respondents and the basis for the assurance in statute, regulation, or agency policy.

FNS complies with the Privacy Act of 1974 (5 USC §552a). The information gathered in this study will be kept private to the full extent allowed by law. All data collected from the study will be reported in aggregate form so that it cannot be linked back to any individual responses. Individuals responding to the household survey (Appendix 12. a/b) will be assured that participation is voluntary and will not affect any benefits they may be receiving. The household survey advance letter (Appendix 20. a/b), brochure (Appendix 21. a/b), and call script (Appendix 29. a/b) will contain assurances that collected information will not be published in a way that identifies individual respondents. No names, phone numbers, or any other unique identifiers will be linked to the data or included in any public use data sets or reports.

Access to records is limited to those persons who process the records for the specific uses stated in the Privacy Act of 1974. The information will be kept private and will not be disclosed to anyone but the individuals conducting research in this investigation, except as otherwise required by law. FNS published a system of record notice (SORN) titled FNS-8 USDA/FNS Studies and Reports in the Federal Register, volume 56, pages 19078–19080, on April 25, 1991, that discusses the terms of protections that will be provided to respondents. Identifying information will be accessible only by approved research staff who have direct responsibility for providing and maintaining sample information. Interview respondents will be assigned a unique ID number and analysis will only be conducted on data sets that include these unique ID numbers.

All paper records will be physically secured in locked storage cabinets. Electronic data will be maintained on secured, password-protected servers. Study data will be processed and stored on the Contractor’s password-protected local area network (LAN), which is enabled with several security mechanisms available through the network operating system. Access to private information stored on LAN directories is restricted to authorized project staff by means of identification and passwords. In addition, network servers containing private information are kept in a locked area. Names and phone numbers will be destroyed within 12 months of the end of the contract.

All project staff, data collectors, and subcontractors (if applicable) have signed or will be required to sign a Contractor Confidentiality Agreement (Appendix 30). Additionally, the Contractor holds a federal-wide assurance (FWA) of compliance from the U.S. Department of Health and Human Services’ Office of Human Research Protections (DHHS/OHRP) (FWA number FWA00000981, expiration date January 30, 2022) (Appendix 31), which covers all federally supported or conducted research involving human subjects. All of the study instruments and procedures have been submitted to the Health Media Lab Independent Review Board (IRB) for the purpose of safeguarding research participants’ rights and welfare. The IRB approval letter (Appendix 42) was received on September 22, 2017.

FNS will not handle any data containing identifying information. A unique ID number will be assigned to each respondent and the data will be provided to FNS by this ID number.

11. Justification for Sensitive Questions

Provide additional justification for any questions of a sensitive nature, such as sexual behavior or attitudes, religious beliefs, and other matters that are commonly considered private. This justification should include the reasons why the agency considers the questions necessary, the specific uses to be made of the information, the explanation to be given to persons from whom the information is requested, and any steps to be taken to obtain their consent.

With the exception of questions in the household survey about demographics and household composition, sources and amounts of income, receipt of public assistance, and race and ethnicity, the household surveys (Appendix 12. a/b) and district interviews with SFAs (Appendix 11) do not involve questions of a sensitive nature. All household survey respondents will be reminded that they can decline to answer any question they do not wish to answer and that there are no negative consequences for not participating. Respondents will also be assured of privacy at the outset of the interview. All survey responses will be held secure and respondents’ answers will only be reported in aggregate form so that individuals cannot be identified. This research will fully comply with all Government-wide guidance and regulations as well as USDA Office of the Chief Information Officer (OCIO) directives, guidelines, and requirements.

Questions about income and the receipt of public assistance are necessary to establish the family’s actual eligibility for free and reduced-price meal benefits. Without them, the study will not be able to compare students’ certification status with estimated eligibility status to examine the accuracy of the verification process, which is a key objective of this study. Similar questions have been used with no evidence of harm in many FNS studies, including the Access, Participation, Eligibility, and Certification (APEC) Study Series (OMB Control Number 0584-0530 Third Access, Participation, Eligibility, and Certification Study (APEC III), expiration date August 31, 2015) and Case Study of National School Lunch Program Verification Outcomes in Large Metropolitan School Districts (2004) (OMB Control Number 0584-0516 Evaluation of the NSLP Application and Verification and Pilot Program, expiration date October 31, 2003). The data collected by this study will not only examine the accuracy of the verification process, but will also examine barriers individuals face when responding to requests for verification. These data are not available to FNS from any other source as they require one-on-one discussion and in-person reviews of households’ income information and no other studies are conducted with this target population. Race and ethnicity subgroups are key populations of interest for descriptive analyses of low-income households, including those participating in federal nutrition assistance programs and their experiences with responding to verification requests.

The study team has security policies, procedures, and technical safeguards that are consistent with the Privacy Act, which regulates the collection, maintenance, use, and dissemination of information about individuals that is maintained by federal agencies; the Federal Information Security Management Act (FISMA), which requires federal agencies to develop, document, and implement agency-wide programs to provide security for the information and information systems that support the operations and assets of the agency; National Institute of Standards and Technology security standards and guidance as part of FISMA; and OMB memoranda regarding data security and privacy.

To ensure data are kept private, each school district will be provided with a secure web-based site for the verification and reapplication data transfers. Access to the secure site will be accomplished through usernames and passwords. Household survey data will be collected on a secure web site and each sample member will be assigned a secure URL that allows the data collector to access the web-based survey for that sample member.

Data will be encrypted in transit and at rest using Federal Information Processing Standard 140-2 compliant cryptographic modules and securely destroyed at the earliest opportunity. Sensitive data will be stored in designated encrypted project folders with restricted access through the use of usernames and passwords. All passwords will be masked on entry, may not be recycled, are required to be changed no less than every month, and must adhere to strict composition standards. Access to the project folders is authorized on a need-to-know and least privilege basis. Contractor staff can only connect to the company network using a company-issued encrypted desktop or laptop. All contractor desktops, laptops, and servers have anti-malware installed. Operating systems and applications are kept current to the latest stable releases and updated with newly released security patches, service packs, and hot fixes as they are made available by the vendors.

12. Estimates of Hour Burden Including Annualized Hourly Costs

Provide estimates of the hour burden of the collection of information. The statement should:

• Indicate the number of respondents, frequency of response, annual hour burden, and an explanation of how the burden was estimated. If this request for approval covers more than one form, provide separate hour burden estimates for each form and aggregate the hour burdens in Item 13 of OMB Form 83-I.

• Provide estimates of annualized cost to respondents for the hour burdens for collections of information, identifying and using appropriate wage rate categories.

12.A Estimated Total Burden

With this submission, there are 2,144 respondents¹¹, 15,072 responses, and 5,534.73 burden hours. Estimates of the burden associated with this collection of information are provided by respondent type in Appendix 32. Burden estimates were derived from pretest activities conducted in February and March 2017 and are summarized by respondent type below. Additional details on the pretest can be found in Supporting Statement B, Question 4, and a pretest memo describing the complete findings from the pretest can be found in Appendix 28.

Individuals. Burden estimates for households are based on the amount of time required to review study materials, gather requested income documentation, and complete the household survey. Appendix 32 includes individual/household burden estimates for 1,486 respondents and 595 nonrespondents, including pre-test respondents and nonrespondents, for a total burden of 5,382.77 hours.

State and Local Government. Burden estimates for State and local governments are based on the amount of time required to review and respond to recruitment materials, provide the data requested in the verification and reapplication data requests, and complete the district interview. Appendix 32 includes burden estimates for recruitment efforts with 13 States and 25 SFAs, data collection from the 20 SFAs selected to participate in data collection (a subset of the initial 25 recruited), review of study materials for 20 local police departments in selected districts, and pretest activities for 5 SFAs (3 respondents and 2 nonrespondents), for a total burden of 151.96 hours.

12B. Estimated Cost of Burden

The estimated cost of this data collection for individuals is $39,025.08, based on the Federal minimum wage of $7.25 per hour (http://www.dol.gov/general/topic/wages/minimumwage). The estimates of costs to State and local governments (including State CN Directors, SFA Directors, and local police) are based on the burden estimates and use the U.S. Department of Labor, Bureau of Labor Statistics, May 2016 National Occupational and Wage Statistics for Occupational Groups 999200: State Government (https://www.bls.gov/oes/current/naics4_999200.htm), 611000: Educational Services (http://www.bls.gov/oes/current/naics3_611000.htm), and 999300: Local Government (https://www.bls.gov/oes/current/naics4_999300.htm), respectively. Annualized costs are based on the mean hourly wage for each job category. The estimated cost for State government, which includes State child nutrition directors (Occupation Code 11-9030), is $284.83 ($43.82/hr x 6.50 hours). The estimated cost of this data collection for local government, which includes SFA directors (Occupation Code 11-9039, $39.34/hr x 143.46 hours) and local police (SOC Code 33-3050, $30.15/hr x 2 hours), is $5,704.02. The estimated total cost of burden associated with this data collection is $45,013.93.

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

Provide estimates of the total annual cost burden to respondents or record keepers resulting from the collection of information, (do not include the cost of any hour burden shown in items 12 and 14). The cost estimates should be split into two components: (a) a total capital and start-up cost component annualized over its expected useful life; and (b) a total operation and maintenance and purchase of services component.

There are no capital/startup or ongoing operation/maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

Provide estimates of annualized cost to the Federal government. Also, provide a description of the method used to estimate cost and any other expense that would not have been incurred without this collection of information.

The total estimated cost of the study to the federal government is $2,689,543.52 over a period of three years (September 2016 through September 2019), resulting in an annualized cost of $896,514.51. This represents the Contractor’s costs for labor, other direct costs, and indirect costs and includes the salary of the assigned FNS project officer. Contractor costs are $2,635,524.68. The cost of the FNS employee, Social Science Research Analyst/ Project Officer, involved in project oversight with the study is estimated at GS-12, step 6 at $44.57 per hour for an estimated 404 hours per year, totaling $18,006.28 annually. Federal employee pay rates are based on the General Schedule of the Office of Personnel Management (OPM) for 2017 for the Washington, DC locality.

15. Explanation for Program Changes or Adjustments

Explain the reasons for any program changes or adjustments reported in Items 13 or 14 of the OMB Form 83-1.

This is a new information collection request as a result of program changes and will add 5,534.73 hours of burden and 15,072 responses to OMB’s inventory.

16. Plans for Tabulations and Publication and Project Time Schedule

For collections of information whose results are planned to be published, outline plans for tabulation and publication.

Study Schedule. The planned schedule for the activities in the study is as follows¹²:

Project Activity	Dates (or Months after OMB approval)
Sampling of SFAs and Households13	0-3 months after OMB approval
Train Field Interviewers and Coordinate Data Collection Schedule	1 month after OMB approval
Conduct Data Collection	1-5 months after OMB approval
Prepare Data Files	June 2018-September 2019 (or beginning 6 months after OMB approval)
Analyze Data and Prepare Final Report and Briefing	August 2018-September 2019 (or beginning 8 months after OMB approval)

Analysis Plan. To examine the accuracy of the current (SY 2017–2018) verification process and address associated research objectives, the study team will conduct analysis and produce five key sets of results, described below. To obtain average outcomes across all study districts (for example, the percentage of enrolled students certified for F/RP meals), we will calculate the outcome for each district (percentage of enrolled students certified for F/RP meals in each district) and then calculate the mean of these district-level estimates. The team will refer to this as the “mean outcome across districts” in analytical tables in the report. To obtain estimates of aggregate student-level characteristics (for example, the percentage of students across all study districts in grade 1), the team will multiply the percentage of students with that specific characteristic in each district by the proportion of all study students that reside in the district and sum the products.

Compare Districts Selected for the Case Study to All Districts Nationwide (Objective 5). The districts participating in the study will not be representative of all districts nationwide. In interpreting findings based on the study districts, it will be useful to understand how they differ from school districts nationally. Using descriptive analysis methods, the study will compare the sample of districts participating in the study to all districts nationwide across characteristics and verification outcomes drawn primarily from FNS-742 data and calendar year 2017 information from the Census Small Area Income and Poverty Estimates (SAIPE). The district characteristics will be weighted by number of enrolled students to derive student-level characteristics and verification outcomes, and the outcomes for students in the study sample will be compared to those for all students nationwide. Because differences may arise in the study results for districts using either the standard or an alternate sampling approach, the study will show characteristics and verification results for districts using the standard sampling method and those using an alternate and present them in a table.

Provide a Detailed Account of Verification Outcomes in Study Districts (Objective 4). The verification process can result in four types of certification outcomes for households initially approved on the basis of income: no change in status; a change in eligibility status from free meals to reduced-price meals; a change in eligibility status from reduced-price meals to free meals; or a loss of eligibility for F/RP meals. A loss of eligibility can result from (1) the verification process finding that a household’s income is too high to qualify for F/RP meals, or (2) a household not responding to the verification request. Using descriptive analysis methods and SY 2017–2018 student-level administrative data from the districts, the study will examine the frequency of these verification outcomes. Further, using student-level data from the districts on reapplications and changes in certification status, the study will investigate the extent to which households reapply for school meal benefits or encounter changes in certification status from direct certification after having meal benefits reduced or terminated through verification.

Key outcomes will include the percentage of applications selected for verification by certification status before and after verification, and number and percentage of households that reapply or encounter changes in certification status after (1) being denied for nonresponse or (2) being denied because their documentation did not support their claim for eligibility, or (3) receiving reduced-price meals instead of free meals (overall and by combinations of certification status before and after verification).

Independently Verify Eligibility for Nonresponding Households and Those with No Change in Benefits (Objectives 1 and 2). The study team will use household survey results to examine eligibility for school meal benefits among two groups of households initially approved on the basis of income or categorical eligibility and selected for verification. Specifically, this assessment will examine the percentage of nonrespondents who would have been eligible in October 2017, a date that aligns with the time they were selected for verification, for the pre-verification certification status they were approved for in fall 2017, the percentage that were eligible for a lower level of benefits, and the percentage that would have been considered ineligible for F/RP meals. It will also estimate the percentage of no-change cases that the survey finds eligible for their post-verification certification status, and the percentage that are eligible for a different certification status based on the documentation provided. Finally, it will summarize the percentage of households certified accurately by selected verification outcomes (for example, among students approved for F/RP meals in households that did not respond to the verification request, the percentage eligible for exactly the pre-verification certification status they originally had).

This component of the analysis will also examine characteristics of households that failed to respond to the verification request by whether they were eventually reapproved for F/RP meals. Further, it will examine characteristics of the sample of households that experienced no change in certification status by whether they appeared to qualify for a different certification status based on the independent verification determination. Finally, the analysis will examine the reasons households failed to respond to verification requests based on survey responses, distinguishing between those that were eventually reapproved for F/RP meals by March 1 and those that were not. This examination will also compare perceptions of school meals and the verification process for nonresponding households and responding households with no change in certification status. All of the analyses described above will be descriptive and use data from the household survey, often in conjunction with administrative data on these households from the districts.

Assess Reasons for Differences in District and Study Eligibility Determinations (Objective 4). This descriptive analysis will use household survey results, district interview results, and administrative records on households that were part of the verification sample to provide insights into which factors may contribute to differences in a district’s eligibility determinations versus the study team’s independent determination of eligibility. Specifically, it will focus on households that had no change in certification status due to the district verification process but that the study determined to have a different eligibility status based on survey responses.

Differences in the verification- and survey-based eligibility determinations could be explained by changes in household circumstances, inaccurate reporting of circumstances in the verification process, inaccurate reporting of circumstances in the household interview, or district staff errors in processing the data. This study will assess the prevalence of these potential explanations by examining differences in (1) eligibility determination factors such as income and household size, (2) household characteristics, (3) district characteristics, and (4) district verification procedures.

Examine the Process of Selecting Applications for cause (Objective 3).

Based on a qualitative analysis of interview responses from districts, this analysis will examine reasons why districts select applications for cause, and how the selection process works. It will also use administrative data from the districts to examine verification results for applications selected for cause, including nonresponse rates and whether the households reapplied, and characteristics of households selected for verification for cause.

Publication. The Contractor will formally present study findings to FNS at an internal briefing. Following this briefing, the Contractor will submit a Final Report to FNS, and this Report will be appropriate for publication and public dissemination.

17. Display of Expiration Date for OMB Approval

If seeking approval to not display the expiration date for OMB approval of the information collection, explain the reasons that display would be inappropriate.

The agency will display the expiration date for OMB approval of the information collection on all instruments and recruitment materials.

18. Exception to the Certification Statement Identified in Item 19.0 of Form OMB 83-1

Explain each exception to the certification statement identified in Item 19 "Certification for Paperwork Reduction Act."

There are no exceptions to the certification statement. The agency is able to certify compliance with all provisions under Item 19 of OMB Form 83-I.

References

Bonevski, B., Randell, M., Paul, C., Chapman, K., Twyman, L., Bryant, J., et al. (2014). Reaching the hard-to-reach: A systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Medical Research Methodology, 14(1), 42.

Burghardt, J., Silva, T., & Hulsey, L. (2004). Case study of National School Lunch Program verification outcomes in large metropolitan school districts. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of Analysis, Nutrition, and Evaluation.

Collins, A., Briefel, R., Klerman, J., Rowe, G., Wolf, W., Logan, C., et al. (2013). Summer Electronic Benefits Transfer for Children (SEBTC) Demonstration: Evaluation findings for the full implementation year. Alexandria, VA: U.S. Department of Agriculture, Food and Nutrition Service, Office of Research and Analysis.

Determining Eligibility for Free and Reduced Price Meals and Free Milk in Schools, 7 CFR § 245 (2011)

Donnelly, F.P. (2015). Regional variations in average distance to public libraries in the United States. Library & Information Science Research, 37(4), 280-289.

E-Government Act of 2002, Pub. L. 107–347, 116 Stat. 2899, codified as amended at 44 U.S.C. § 101, H.R. 2458/S. 803. Retrieved from https://www.gpo.gov/fdsys/pkg/PLAW-107publ347/pdf/PLAW-107publ347.pdf

Goldenberg, K. L., McGrath, D., & Tan, L. (2009, May). The effects of incentives on the Consumer Expenditure Interview Survey. In Annual Conference, American Association for Public Opinion Research, Ft. Lauderdale, FL. https://stats.bls.gov/osmr/pdf/st090100.pdf

Groves, R., Fowler, F., Couper, M., Lepkowski, J., Singer, E., & Tourangeau, R. (2009). Survey methodology (2nd Ed., pp 205–206). New York: Wiley.

Groves, R. M., & Kahn, R. L. (1979). Surveys by telephone: a national comparison with personal interviews. New York, NY: Academic Press.

Groves, R. M., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. The Public Opinion Quarterly, 64(3), 299–308.

Jäckle, A., & Lynn, P. (2007). Respondent incentives in a multi-mode panel survey: Cumulative effects on nonresponse and bias (No. 2007-01). ISER Working Paper Series. https://www.econstor.eu/bitstream/10419/91911/1/2007-01.pdf

James, T. (1996). Results of wave 1 incentive experiment in the 1996 survey of income and program participation. American Statistical Association, Proceedings of the Survey Research Methods Section (pp 834–839), Alexandria, VA.

Kay, W. R., Boggess, S., Selvavel, K., & McMahon, M. F. (2001, August). The use of targeted incentives to reluctant respondents on response rate and data quality. In Proceedings of the Survey Research Methods Section, American Statistical Association. http://ww2.amstat.org/sections/SRMS/Proceedings/y2001/Proceed/00609.pdf

Mercer, A., Caporaso, A., Cantor, D., & Townsend, R. (2015). How much gets you how much? Monetary incentives and response rates in household surveys. Public Opinion Quarterly, 79(1), 105-129. https://academic.oup.com/poq/article-abstract/79/1/105/2330052

National School Lunch Program, 7 CFR, § 210 (2012).

Privacy Act, Systems of Records, FNS-8 USDA/FNS Studies and Reports, 56 Fed. Reg. 19078 (April 25, 1991). Retrieved from https://www.gpo.gov/fdsys/search/pagedetails.action?sr=543&originalSearch=&st=31+cfr+904&ps=10&na=&se=&sb=re&timeFrame=&dateBrowse=&govAuthBrowse=&collection=&historical=false&packageId=FR-1991-04-25&fromState=.

School Meals: USDA Could Improve Verification Process for Program Access, Before the Subcommittee on Early Childhood, Elementary, and Secondary Education, Committee on Education and the Workforce, House of Representatives, 1 (2015) (testimony of J. Lucas-Judy). Retrieved from http://www.gao.gov/assets/680/670272.pdf.

Singer, E. (2002). The use of incentives to reduce non-response in household surveys. In R. Groves, D. Dillman, J. Eltinge, & R. Little (Eds.), Survey non-response (pp 163–177). New York: Wiley.

Singer, E., Groves, R. M., & Corning, A. D. (1999). Differential incentives: Beliefs about practices, perceptions of equity, and effects on survey participation. Public Opinion Quarterly, 63, 251–260.

Singer, E., & Kulka, R. A. (2002). Paying respondents for survey participation. In M. Ver Ploeg, R. A. Moffitt, & C. F. Citro (Eds.), Studies of welfare populations: Data collection and research issues (pp. 105–128). Washington, DC: National Academy Press.

Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science, 645(112-141).

Schwartz, L. K., Goble, L., & English, E. H. “Counterbalancing Topic Interest with Cell, Quotas and Incentives: Examining Leverage-Salience Theory in the Context of the Poetry In America Survey.” Paper presented at the meetings of the American Association of Public Opinion Research. May 2006.

The Healthy, Hunger Free Kids Act of 2010, Pub. L. 111–296, 124 Stat. 3183. Retrieved from http://childnutrition.ncpublicschools.gov/regulations-policies/public-law/pl-111-296.pdf/view.

U.S. Department of Agriculture, Office of Inspector General. FNS—National School Lunch and School Breakfast Programs. Washington, DC: Author. Retrieved from https://www.usda.gov/oig/webdocs/27601-0001-41.pdf .

U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support. (2015, May). Program error in the National School Lunch Program and School Breakfast Program: Findings from the Second Access, Participation, Eligibility, and Certification Study (APEC II), Volume 1. Alexandria, VA: Author.

Zagorsky, Jay L., and Patricia Rhoton. "The effects of promised monetary incentives on attrition in a long-term panel survey." Public Opinion Quarterly 72.3 (2008): 502-513. https://academic.oup.com/poq/article-abstract/72/3/502/1835648

1 Applications selected for verification for cause are those selected for verification if something on the application is unclear or questionable. Verification for cause is used as a method for local educational agencies (LEAs) to address integrity concerns.

2 The total burden estimate listed for the Verification Data Request includes 30 minutes for the Request (Appendix 10) and 3.5 hours for the accompanying Verification Data Request Template (Appendix 18).

3 School Food Authority (SFA)

4 The total burden estimate for households includes 45 minutes for the household survey, and two hours to gather income documentation.

5 The study team will aim to complete surveys with 840 nonresponding households and 640 responding households with no changes.

6 The total burden estimate listed for the Reapplication Data Request includes 15 minutes for the Request (Appendix 13) and one hour for the accompanying Reapplication Data Request Template (Appendix 24).

7 The HHFKA of 2010 states: “States, State educational agencies, local educational agencies, schools, institutions, facilities, and contractors participating in programs authorized under this Act and the Child Nutrition Act of 1966 (42 U.S.C 1771 et seq.) shall cooperate with officials and contractors acting on behalf of the Secretary, in the conduct of evaluations and studies under those Acts.”

8 U.S. Department of Labor, Bureau of Labor Statistics, May 2016 National Occupational and Wage Statistics for Occupation Code 39-9011: Childcare Workers, https://www.bls.gov/oes/current/oes399011.htm.

9 https://www.irs.gov/charities-non-profits/public-disclosure-and-availability-of-exempt-organizations-returns-and-applications-costs-for-providing-copies-of-documents

10 Donnelly, F.P. (2015). Regional variations in average distance to public libraries in the United States. Library & Information Science Research, 37(4), 280-289.

11 This includes 1,542 respondents and 602 nonrespondents.

12 The data collection and analysis schedule will be adjusted as needed based on OMB approval.

13 As discussed in Supporting Statement B, Question 1, the SFA sample frame will be drawn from administrative data available via the FNS-742 form. These data will be reviewed and an initial sample drawn prior to OMB approval. Recruitment and selection of the final samples, however, will not begin until OMB approval is received.