State, Local and Tribal Agencies (Directors, SFA & Schools)

APPENDIX C.10 STATE LEVEL KEY STAFF GUIDE- FNS 10

OMB Number: 0584-NEW

Expiration Date: XX/XX/XXXX

Evaluation of the School Meal Data Collection Process

STATE LEVEL Key Staff-FNS 10 Interview Guide

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER’S NAME] and I work for Westat, a private research company in Rockville, Maryland. With me today is [NOTE TAKER’S NAME] who will be taking notes.

Consent Form

PURPOSE: The federal Food and Nutrition Service (FNS) is interested in understanding and improving the processes used to collect and report program data. FNS hired Westat to conduct a study to describe how schools, School Food Authorities (SFAs), and state agencies collect and report school meal program data via three forms:

• FNS‑10, Report of School Program Operations,

• FNS‑742, School Food Authority Verification Collection Report, and

• FNS‑834, State Agency (NSLP/SNAP) Direct Certification Rate Data Element Report.

We will also identify where errors tend to occur during data collection and transmission. The final product will be a description of the data collection processes for these forms, and recommendations to improve the process and increase accuracy.

HOW YOU WERE SELECTED: We are conducting four state-based case studies that include qualitative interviews with 4 State Child Nutrition and SNAP agencies, 10 SFAs in each State, and 3 schools per SFA in order to describe the data collection processes and gather information to better understand areas for improvement. Westat worked with FNS to select states based on factors such as state size, FNS region, use and type of management information system (MIS) and direct certification system processes.

INFORMATION TO BE COLLECTED: As a key staff person for this study, we will ask about your involvement with data aggregation, processing, and transmission for the FNS‑10 and/or the FNS-742. We will also ask about your collaboration with different departments and any suggestions you may have on how to improve these processes.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-NEW. The time required for the State Child Nutrition Key staff to provide this information collection is estimated to average 90 minutes per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

RISKS AND PRIVACY: There is little risk to being part of this study. We use all data we collect only for the purposes we describe. Your name will not be linked to any of your responses. In our reports we may include quotes from our respondents, but these will be presented without the speaker’s name and in such a way that you could not be identified. Participating in the survey may not help you individually, but it may help us better understand how to improve the completion of FNS‑10, FNS‑742 and FNS‑834.

STUDY COSTS AND COMPENSATION: There is no cost to you to join this study. The interview takes about 90 minutes.

VOLUNTARY PARTICIPATION: Your participation is entirely voluntary. Refusal to participate will not have any impact on your position or child nutrition programs. You can take a break, skip questions or stop participating at any time.

[NOTE TAKER’S NAME] is here with me to take notes, but know that the information you share in this discussion will be kept private.

QUESTIONS: If you have questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protections office at 1‑888‑920‑7631. Please leave a message with your full name, the name of the research study that you are calling about, which is the Evaluation of School Meal Data Collection, and a phone number beginning with the area code. Someone will return your call as soon as possible.

We have planned for this discussion to last 90 minutes. Is that still okay?

With your permission I would like to record this discussion to help us fill any gaps in our written notes. The recordings, transcripts, and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

Do you have any questions? [ANSWER ALL QUESTIONS]

If you agree to participate in this interview, please read the following statement and sign your name below.

I have read the above information about this project and my rights as a participant. I consent to participate in this research and to have this discussion audio-taped.

Signature Date

Printed Name

If you do not agree to be audio recorded, please cross out “and to have this discussion audio-taped” in the sentence above.

May I turn on the audio recorder now?

Warm Up

1. To start, please tell me about your current role and how long you have worked here.

INTERVIEWER: PROVIDE RESPONDENTS WITH A COPY OF FNS-10 AS A REFERENCE.

2. Walk me through your involvement and specific responsibilities with the FNS‑10 form, which collects school meal claims data.

Data System

Before we discuss the details of the school meal program data, I have a few basic questions about the data system or Management Information System (MIS) that contains school meal claims data.

3. I believe the data system that contains school meal claims data is known as [NAME OF MIS SYSTEM]. Is that correct?

IF NO, OR IF INTERVIEWER DOES NOT KNOW NAME OF MIS: What is the name of the data system that contains school meal claims data?

Probes:

1. Does the system(s) include site/school level information? (i.e., site-based claiming)

2. When policy or reporting requirements change, how difficult is it to update the system(s) to comply with the changes?

4. Does your MIS system(s) interface with other state systems? With the Food Programs Reporting System (FPRS)? Explain.

4. Tell me about any challenges with using the system(s).

Processes for the FNS‑10

Now I want to discusshow you help to collect, aggregate, validate, and submit data for the FNS-10.

4. How do the SFAs submit the monthly meal claims data to the State?

Probes:

1. In what format do you receive the data from the SFAs?

If needed: online web portal submission, Excel document, other?

2. Does every SFA submit data the same way? If not, describe variation.

4. Once the data is collected from the SFAs what happens at the State level to review, aggregate, and validate the monthly meal claims data?

Probes:

1. Who is responsible for each part of this process?

2. Which data checks in the system are programmed to happen automatically and which data checks are done manually by staff?

3. Can you tell me how you calculate the data for the 30-day reports?

IF NEEDED: how do you arrive at the “estimated” numbers needed for the 30-day report?

4. What kind of communication or collaboration is there between program staff and Financial Management staff about the form?

4. You just explained how the 30-day reports are completed. What, if anything, is different about the process when you’re preparing the 90-day reports?

4. Is your process any different when preparing the October report? The July report?

4. The FNS‑10 contains data on student enrollment (see 12b, 13b, 14b, 15b on the FNS‑10). Where does the data on enrollment come from?

4. Does the State provide guidance or assistance to SFAs with collecting or submitting the meal claims data? If so, describe.

If needed: Webinars, mailings, one-on-one TA calls

4. Now I’d like you to walk me through how the FNS-10 meal claims data is entered into the Food Programs Reporting System (FPRS).

Probe:

1. Describe any quality control measures programmed in FPRS.

4. How would you describe the user friendliness of FPRS when submitting the data to FNS?

Probe:

1. What improvements would you make to FPRS?

4. Are there any steps of the process to collect, validate, or submit the FNS-10 form that we have not covered? Explain.

Potential Sources of Error

4. When do you believe mistakes are most likely to occur when schools are collecting data on meal claims? When they are submitting that data to the SFA?

4. When do you believe mistakes are most likely to occur when the SFAs are collecting that data from the schools? When they are aggregating that data? When they are submitting that data to the State?

Probes:

1. What are the most common reasons you have to ask SFAs to revise the data?

2. Are the SFAs usually able to make those revisions?

3. [IF NO] Could you tell me more about why it is hard for the SFAs to make revisions?

4. In your opinion, when do you think mistakes are most likely to occur when the data is being validated at the state level? When it is being submitted on FPRS to FNS?

Probe:

1. Are there any data elements that are particularly problematic with Part A? With Part B?

4. Are there any other internal quality control/data checks that take place that we haven’t discussed?

4. When you have questions about the process for completing the FNS-10, who do you turn to for help?

Suggestions

4. If you could, what changes would you make to the FNS‑10 form to improve the accuracy of the data?

Probes:

1. How else do you think the form could be improved?

2. Has the guidance you’ve received thus far on this form been helpful and clear?

3. IF NO: what guidance would be helpful? Be specific.

4. Would additional guidance help or does the form itself need to change?

4. How could the data upload or submission process of the FNS‑10 be improved?

4. Are there any difficulties with the timing of when the reporting forms are due?

Probe: If yes, do you have any suggestions regarding the timing?

4. Where do you think your State could make improvements to complete the FNS-10?

Communication

Now let’s talk about communication with the Regional Office.

4. What kind of communication is there with the Regional Office regarding meal claims data?

Probes:

1. How often do you communicate with the Regional Office about school meal data?

IF NEEDED: Is there a set call or meeting about school meal data?

2. Does the Regional Office provide any technical assistance or other guidance regarding collecting or processing claims data?

[IF NO COMMUNICATION]

3. Is there any guidance or assistance you wish the Regional Office would provide? Describe.

4. Are there data elements on the FNS-10 that are more likely to be flagged by the Regional Office with follow-up questions?

Wrap Up

4. Reflecting on everything we have talked about, what do you think your State does really well with regard to processing meal claim data?

4. Thank you so much for taking the time to talk with us today. Do you have any final questions or thoughts?