State, Local and Tribal Agencies (Directors, SFA & Schools)

APPENDIX-C11. STATE LEVEL KEY STAFF GUIDE FNS-742 & FNS-834

OMB Number: 0584-NEW

Expiration Date: XX/XX/XXXX

Evaluation of the School Meal Data Collection Process

STATE LEVEL Key Staff-FNS-742 & FNS-834 Interview Guide

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER’S NAME] and I work for Westat, a private research company in Rockville, Maryland. With me today is [NOTE TAKER’S NAME] who will be taking notes.

Consent Form

PURPOSE: The federal Food and Nutrition Service (FNS) is interested in understanding and improving the processes used to collect and report program data. FNS hired Westat to conduct a study to describe how schools, School Food Authorities (SFAs), and state agencies collect and report school meal program data via three forms:

• FNS‑10, Report of School Program Operations,

• FNS‑742, School Food Authority Verification Collection Report, and

• FNS‑834, State Agency (NSLP/SNAP) Direct Certification Rate Data Element Report.

We will also identify where errors tend to occur during data collection and transmission. The final product will be a description of the data collection processes for these forms, and recommendations to improve the process and increase accuracy.

HOW YOU WERE SELECTED: We are conducting four state-based case studies that include qualitative interviews with 4 State Child Nutrition and SNAP agencies, 10 SFAs in each State, and 3 schools per SFA in order to describe the data collection processes and gather information to better understand areas for improvement. Westat worked with FNS to select states based on factors such as state size, FNS region, use and type of management information system (MIS) and direct certification system processes.

INFORMATION TO BE COLLECTED: As a key staff in the Child Nutrition office, we will ask about your involvement with data aggregation, processing, and transmission for the FNS‑742 and/or FNS-834. We will also ask about your collaboration with different departments and any suggestions you may have on how to improve these processes.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-NEW. The time required for the State Child Nutrition Key Staff to provide this information collection is estimated to average 90 minutes per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

RISKS AND PRIVACY: There is little risk to being part of this study. We use all data we collect only for the purposes we describe. Your name will not be linked to any of your responses. In our reports we may include quotes from our respondents, but these will be presented without the speaker’s name and in such a way that you could not be identified. Participating in the survey may not help you individually, but it may help us better understand how to improve the completion of FNS‑10, FNS‑742 and FNS‑834.

STUDY COSTS AND COMPENSATION: There is no cost to you to join this study. The interview takes about 90 minutes.

VOLUNTARY PARTICIPATION: Your participation is entirely voluntary. Refusal to participate will not have any impact on your position or child nutrition programs. You can take a break, skip questions or stop participating at any time.

[NOTE TAKER’S NAME] is here with me to take notes, but know that the information you share in this discussion will be kept private.

QUESTIONS: If you have questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protections office at 1‑888‑920‑7631. Please leave a message with your full name, the name of the research study that you are calling about, which is the Evaluation of School Meal Data Collection, and a phone number beginning with the area code. Someone will return your call as soon as possible.

We have planned for this discussion to last 90 minutes. Is that still okay?

With your permission I would like to record this discussion to help us fill any gaps in our written notes. The recordings, transcripts, and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

Do you have any questions? [ANSWER ALL QUESTIONS]

If you agree to participate in this interview, please read the following statement and sign your name below.

I have read the above information about this project and my rights as a participant. I consent to participate in this research and to have this discussion audio-taped.

Signature Date

Printed Name

If you do not agree to be audio recorded, please cross out “and to have this discussion audio-taped” in the sentence above.

May I turn on the audio recorder now?

Warm Up

1. To start, please tell me about your role and how long you have worked here.

INTERVIEWER: PROVIDE RESPONDENTS WITH A COPY OF EACH FORM AS A REFERENCE.

2. Walk me through your involvement and specific responsibilities with the FNS‑742 form, which collects certification and verification data for each SFA.

Now with the FNS-834 form, which reports the number of SNAP children in special provision schools operating in a non-base year.

Systems

Before we discuss the details of the school meal program data, I have a few basic questions about the data system or Management Information System (MIS) that collects and stores school meal data.

3. I believe the data system that contains school meal direct certification and verification data is known as [NAME OF MIS SYSTEM]. Is that correct?

IF NO, OR IF INTERVIEWER DOES NOT KNOW NAME OF MIS: What is the name of the data system that contains school meal certification and verification data?

4. Does the same system contain data on the number of SNAP children in each school?

Probes (if multiple systems, repeat for each):

1. Does the system(s) include site/school level information? (i.e., site-based claiming)

2. When policy or reporting requirements change, how difficult is it to update the system(s) to comply with the changes?

5. Does your MIS system(s) interface with other state systems? With the Food Programs Reporting System (FPRS)? Explain.

6. What are some of the challenges with using the system(s)

Processes for the FNS‑742

Now I want to learn about how you help to collect, validate, and submit data for the FNS-742.

7. How do the SFAs submit the certification and verification data to the State?

Probes:

1. In what format do you receive the data from the SFAs?

If needed: online web portal submission, Excel document, other?

2. Does every SFA submit data the same way? If not, describe variation.

8. Once the data are collected from the SFAs what happens at the State level to review, aggregate, and validate the data?

Probes:

1. Who is responsible for each part of this process?

2. Which data checks in the system are programmed to happen automatically and which data checks are done manually by staff?

9. Does the State provide guidance or assistance to SFAs with collecting or submitting the data? If so, describe.

If needed: Webinars, mailings, one-on-one TA calls

10. The FNS‑742 contains data on student enrollment in various categories (see Column B, sections 1-4 on the FNS-742). Where does the data on enrollment come from?

Probe:

a. Do you know if the enrollment data collected in these sections are consistent with enrollment data on any other forms submitted to FNS, such as the FNS-10 (see 12b, 13b, 14b, 15b on the FNS‑10)?

11. Now I’d like you to walk me through how the FNS-742 certification and verification data is entered into the Food Programs Reporting System (FPRS).

Probe:

1. Describe any quality control measures programmed in FPRS.

12. How would you describe the user friendliness of FPRS when submitting the data to FNS?

Probe:

1. What improvements would you make to FPRS?

13. Are there any steps of the process to collect, aggregate, validate, or submit the FNS-742 form that we have not covered? Explain.

FNS-742: Sources of Error

Now let’s talk about potential issues when collecting and submitting data for the FNS‑742.

14. What are the most common reasons you have to ask SFAs to revise the verification data for the FNS-742 form?

Probe:

a. Are the SFAs usually able to make those revisions? IF NO: Could you tell me more about why it is hard for the SFAs to make revisions?

15a. When do you think mistakes are most likely to occur when the State is validating verification data before submitting it to FNS?

15b. When submitting verification data to FNS?

16. If you could, what changes would you make to the FNS‑742 to improve the accuracy of the data?

1. Would it help to have separate data collection forms, one for certification and one for verification data?

17. Are there any timing difficulties with when the reporting forms are due? For example, the time gap between when the verification process for free and reduced price applications is over in November and when the report is due in March?

Probe: If yes, do you have any suggestions regarding the timing?

18. Where do you think your State could make improvements to complete the FNS-742?

19. When you have questions about either the verification data or the process of completing the FNS-742, who do you turn to for help?

a. Has the guidance you’ve received on this form from FNS been helpful and clear?

b. Would additional guidance help or does the form itself need to change?

Processes for the FNS‑834

Now let’s turn to Data Element #3, the annual report on the number of SNAP children in special provision schools operating in a non-base year.

19. What is the process you use to obtain data for Provision 2, Provision 3 and CEP schools so that you can report Data Element #3?

a. What is challenging about arriving at the number for Data Element #3?

20. In your opinion, what would the special circumstances box at the bottom of the form be used for?

Probes:

a. Have you ever reported a special circumstance on this form?

b. [If yes] What happened as a result?

FNS-834: Sources of Error

21. In your experience, have there been any issues with duplication on the FNS‑742 for the special provision schools?

[Note to interviewer: the FNS‑742 data element is the count of SNAP children who are directly certified for purposes of receiving a meal; the FNS‑834 data is the count of children who would be matched if they were not enrolled in provision schools.]

22. When you have questions about any of the data elements on this form, who do you turn to for help?

a. Has the guidance you’ve received from FNS on this form been helpful and clear?

b. Would additional guidance or clarity in the instructions help or does the form itself need to change?

23. If you could, what changes would you make to the FNS‑834 to improve the accuracy of the data?

24. Are there any other ways FNS‑834 form could be improved?

Communication

Now let’s talk about communication between your office and the Regional Office.

25. What kind of communication is there with the Regional Office regarding certification or verification data?

Probes:

1. How often do you communicate with the Regional Office about school meal certification or verification data?

IF NEEDED: Is there a set call or meeting about school meal data?

2. Does the Regional Office provide any technical assistance or other guidance regarding collecting or processing claims data?

[IF NO COMMUNICATION]

3. Is there any guidance or assistance you wish the Regional Office would provide? Describe.

Wrap Up

26. Reflecting on everything we have talked about, what do you think your State does really well with regard to processing verification data? Direct certification data?

27. Thank you so much for taking the time to talk with us today. Do you have any final questions or thoughts?