Memo

Generic Information Collection Request

Overview: The Food and Nutrition Service (FNS) of the U.S. Department of Agriculture (USDA) administers the federal child nutrition programs, including the National School Lunch Program (NSLP) and School Breakfast Program (SBP). The USDA is required under the Improper Payments Act of 2002 to annually report on erroneous payments made by these programs. To comply with this mandate, FNS launched a multi-wave, cross sectional study to determine national estimates of payment error, titled the Access, Participation, Eligibility, and Certification (APEC). The APEC series uses a multi-stage sample design to select representative samples of school districts, schools, and household applicants in the U.S. The initial APEC data collection occurred during the 2005-2006 school year. A second round of data collection took place during the 2012-2013 school year. Planning for the next round is currently underway. APEC collects data from multiple sources using multiple methods, including personal interviews with household applicants, administrative records from sampled schools, surveys of school and School Food Authority administration, and direct observation.

The first two waves of APEC found that there are three key sources of errors that produce erroneous payments. These include (1) incorrect determination a child’s eligibility, such as when a child is found to be eligible for reduced-price meals, but was actually eligible for free meals; (2) incorrect reporting of actual food sales as reimbursable or not, such as when a sold meal is reported as reimbursable by the school, but does not include all required elements of a reimbursable meal (e.g. milk or fruit); and (3) incorrect calculation of reimbursable meals provided by the school.

Since learning of the findings of the initial APEC study, FNS has been making changes to the program to reduce the rates of these errors. One effort is aimed at reducing the first type of error: incorrect classification of eligibility. APEC data has shown that there are multiple factors that produce this error and each must be addressed appropriately. FNS has engaged the U.S. Census Bureau to look at one-specific cause of this error only: household reporting error.

APEC findings suggest that households misreport two key categories of information when completing the school meals application: household roster and size, and household member income. To address these issues, FNS re-designed the application form for use during the 2015-2016 school year. The Census Bureau was asked to test the revised form through cognitive interviews with actual household applicants. An online version of the application is also under development by FNS for intended use during the 2016-2017 school year. The Census Bureau will conduct additional usability testing of that form in calendar year 2017, under a separately submitted request for OMB clearance.

Current Request: The Census Bureau plans to conduct research under the generic clearance for questionnaire pretesting research (OMB number 0607-0978). We propose conducting structured and cognitive research for the Food and Nutrition Service (FNS) of the U.S. Department of Agriculture. The purpose of this qualitative research is to test concepts included in and evaluate usability of the revised Household Application for Free and Reduced Price School Meals.

Purpose: Revisions to the application were made after large-scale evaluations of the National School Lunch Program (NSLP) and the School Breakfast Program (SBP) found inaccurate or incomplete household reporting on measures, including individual and total income and the number of persons living the in household. Qualitative data will be collected from program participants and administrators. This research is intended to improve the application and associated materials; increasing user comprehension of requirements and instructions, and improving the overall usability of the application materials.

Timeline: Beginning September 2016, staff from the Center for Survey Measurement (CSM) will contact administrators from School Food Authorities (SFA) who are responsible for implementing school meal programs for the FNS. All interviews are to be completed by May 2017.

Protocol. CSM will conduct cognitive interviews referencing a paper application designed by FNS, or one modified for local use by states or SFAs. SFA and state-designed forms will be requested during the initial recruitment contacts with the SFAs. Tailored forms must meet specific minimum criteria defined FNS and represented on the FNS-developed application.

At least eleven SFAs, specific schools supported by each SFA, and participating families will be recruited for structured and cognitive interviews.

CSM will conduct approximately 15 structured interviews with school and SFA staff and 165 cognitive interviews with applicants in at least eleven locations nationwide. Cognitive testing will utilize both concurrent and retrospective think-aloud methods.

Interviews will be audio-recorded to facilitate analysis and summary of the results. All participants will be informed that their responses are voluntary and that the information they provide is confidential. Participants will be asked to sign consent forms and give permission to be recorded.

Materials to be used for the study are listed below and provided as attachments.

• Consent for respondent participation

• Protocol for SFA/School Officials

• Protocol for Household Applicants

• FNS prototype application form (developed by USDA)

• FNS applicant instruction guide (developed by USDA)

• FNS Frequently Asked Questions (developed by USDA)

• Additional screening questions for respondents (household applicant type)

Language: Interviews with SFA and school officials will be conducted in English. Respondents who participate in the program may be interviewed in both English and Spanish. The FNS will provide Spanish translations of participant application forms and instructions.

Sample Design: Our sample design is multi-level to ensure representation of key SFA characteristics (such as region, number of students served, public vs. private) and school characteristics (such as urbanity and school type, i.e., elementary, middle, high school). Recruitment of applicant respondents will target

• respondents age 18 or older;

• respondents with a household members in attendance at a sampled school; and

• respondents who completed an application for free or reduced price school meals in the 2016-2017 school year.

In addition to these characteristics, respondents will ideally represent a range of demographic characteristics (e.g., gender, race/ethnicity, socioeconomic status, educational background).

Recruitment: Respondents will be recruited through contacts at the selected SFAs and schools. Interviews will be conducted at the sampled schools or other locations convenient for respondents. Respondents from applicant households will be administered select screening questions from CSMs “universal screener” previously approved by OMB. Additional study-specific screening questions are included in this application for review and clearance.

Use of Incentives: Due to the length and complexity of the task as well as the necessity to travel to interview locations, we plan to offer an incentive to offset the cost of their participation in this research, such as travel and parking. All household application respondents will receive $40. Due to statutory restrictions, staff associated with the administration of school meals program are not allowed to receive such incentives.

Respondent Burden: Burden associated with participating in this research includes time spent during the recruitment and screening process and the administration of the in-person interview. We estimate that each SFA/school interview will take approximately 30 min (7.5 hours total) and each applicant interview will take approximately 60 minutes (165 hours total). Respondent recruitment will take approximately ten minutes per person. We estimate that we will screen four people for each successful recruit (120 hours). Thus, the total estimated burden for this research is 292.5 hours.

Respondent Type	Number of Interviews	Average Administration Time	Burden Hours
Recruitment	720	10 min	120
SFA/School Official	15	30 min	7.5
Household Applicant	165	60 min	165
Total			292.5

In order to ensure that this information collection captures the range of difficulties respondents may have with the FNS application, no fewer than 11 sites will be selected for cognitive interviewing. We are contractually obligated to conduct interviews in this number of locations. This number of sites is necessary to have a range of regions, school types, and levels of urbanicity represented in our sample. We will interview a maximum of 15 household applicants at each location, allowing for a diversity of respondents with regards to language, age, education level, household size, income, and race and ethnicity.

Informed Consent: Respondents will be provided with the Census Bureau’s informed consent statement to read and review (provided as an attachment to this submission). If a respondent chooses to participate, he or she will be asked to complete and sign the document. The consent document also includes a request for permission to audio tape the cognitive interview. Any respondent who opts not to sign the consent form will be thanked for his or her time and will not participate further.

Assurances of Confidentiality: The Census Bureau will conduct this research under the authority of Title 13 of the United States Code and all collected information that identifies individuals will be held in strict confidence according to the provisions of Title 13 United States Code, Section 9. All respondents will be informed that their participation is voluntary. Any personally identifiable information will be stored on secured servers at the Census Bureau or if on paper, in locked file cabinets. Data collected during interviews will used for research purposes only and no identifiable information will be released from which an individual may be identified.

Use of Information Technology to Reduce Burden: This research will not use IT systems to collect data. However, email correspondence with participants may occur during the study recruitment effort to arrange and confirm logistics. Use of email by respondents is not mandatory and will be used when selected by respondents as the preferred communication method.

Data Analysis: Each interview will be summarized using a combination of audio recordings, notes taken and the completed FNS application forms. The summaries will highlight problems respondents faced when completing the application, both anticipated and unanticipated, related to comprehension. Given the nature of the task and the types of information we are collecting, we will highlight barriers respondents face to access and provide this information. Data collected from SFA or school officials, as well as interviewer impressions, will provide important context to understand respondents’ answers. A final, summary report of findings will be prepared for the USDA FNS once all interviews are complete. These results will inform revisions to the application form used during the 2017-2018 school year.

The contact person for questions regarding data collection and the design of this research is listed below:

Jessica Graber

Center for Survey Measurement

U.S. Census Bureau

Room 5K024B

Washington, D.C. 20233

(301) 763-6550

[email protected]

Page 3 of 3