State, Local and Tribal Agencies (Directors, SFA & Schools)

APPENDIX C-6. SCHOOL PRE-TEST PROTOCOL

OMB Number: 0584-NEW

Expiration Date: XX/XX/XXXX

School Meals Forms Study

School Guide

Cognitive Interview Protocol

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER NAME] and as you know, I work for Westat, a private research company in Rockville, Maryland we are contacting you on behalf of United States Department of Agriculture (USDA), Food and Nutrition Service (FNS).

The US Department of Agriculture’s, Food and Nutrition Service (FNS) is trying to better understand how school meal program data is collected and reported. FNS has hired Westat to help conduct the School Meals Forms Study to look at how schools, School Food Authorities (SFAs), and State agencies collect and report school meal program data through USDA FNS forms. Part of this study includes talking directly with schools to better understand daily meal counts and other participation information you collect and send to your SFA. Before we conduct these interviews on a broader scale, we are interested in identifying areas for improving the questions – the goal is to ensure that the questions are easy to understand and to answer.

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-NEW. The time required for the State Child Nutrition Director to provide this information collection is estimated to average 45 minutes per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

Did you receive a copy of interview questionnaire and FNS forms?

IF YES: Do you have that in front of you at the moment? If not, could you please get the copy because we’ll need to review it while we’re talking today.

IF NEVER RECEIVED OR LOST: Are you able to access your email?

HAS ACCESS BUT NO EMAIL ADDRESS ON FILE: I can resend right now via email. Can you please give me your email address?

HAS ACCESS TO EMAIL: I’ve resent the documents.

IF NO ACCESS TO EMAIL EVER, OFFER TO RESEND AND RESCHEDULE INTERVIEW. IF CANNOT RESCHEDULE, PROCEED WITH INTERVIEW: I will mail you another copy of the document to you. Let me confirm your address.

Informed Consent

Before we get started there are a few things I should mention.

• This is a research study which means your participation in this discussion is voluntary.

• Refusal to participate will not have any impact on your district or position as school staff.

• The thoughts and opinions that you share during our discussion will be kept private and your name will not appear in the main report or any documentation shared with FNS, the school district, the School Food Authority, or your manager.

• The thoughts and opinions that you share during our discussion will be used for research purposes only. We will combine your responses with those from other school staff. They will not be used as an evaluation of your work, your staff’s work or the district.

• There are no direct benefits to you or your school district for participating in this study.

• Your participation and feedback is important and needed because input from school staff like yourself help us better understand how reporting can be improved.

• Once we start the interview, you can take a break, skip questions or stop participating at any time. Again, there will be no penalty or loss of benefits to your school district.

• Finally, with your permission, I would like to record this discussion. The recording helps us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

We have planned for this discussion to last no more than an hour.

Do you have any questions? [ANSWER ALL QUESTIONS]

Do I have your consent to participate?

May I turn on the audio recorder now?

IF YES: [ONCE RECORDER IS ON, “Now that the recorder is on, today is DATE at TIME. Do I still have your permission to audio record this interview?” MAKE SURE YOU GET AN AUDIBLE “Yes” FROM RESPONDENT.]

Procedures

First I’ll ask you questions about meal counting and reporting at your school. After that part of the interview is over, I’ll ask you some debriefing questions at the end to get your reactions to the questions about counting and reporting meals.

ADMINISTER IN-DEPTH INTERVIEW. NOTE BEGIN TIME: __________________

NOTE DELIVERY, COMPREHENSION AND RESPONSE ISSUES ON TRACKING DIRECTLY ON INTERVIEW GUIDE.

NOTE ANY VERBAL REACTION R SHARES FOR FOLLOW-UP DURING THE DEBRIEF.

FOR ANY NON-VERBAL REACTIONS (E.G., LONG PAUSES), ASK ONLY, Tell me what you’re thinking here. DO NOT PROBE BEYOND THAT UNTIL THE DEBRIEF.

RESEARCH QUESTIONS

• Do respondents understand and interpret the questions as intended?

• Are school staff aware of how data is collected at the school and site level?

• Are school staff able to answer in-depth questions about data collection at the school and site level?

• Are school staff able to answer in-depth questions about how values are calculated and derived?

• Are school staff able to answer in-depth questions about quality assurance procedures for checking data?

• Are school staff able to answer in-depth questions about transmission of data?

• Are school staff able to answer in-depth questions about errors in reporting ?

• Is there any information about the data collection and submission process that is missing from the in-depth interview guide?

Debriefing Questions

NOTE END TIME: _________________________

Thank you. Now I’m going to ask you some questions about it. First, just tell me your overall impressions of the questions I just asked you about the process of counting and reporting meals at your school.

USE RELEVANT PROBES BELOW TO DISCUSS OBSERVED ISSUES FROM INTERVIEW NOTES.

ASK ABOUT THE ISSUES IN THE ORDER THEY OCCURRED.

REPEAT QUESTION TEXT TO REORIENT R.

Clarification Requests

You asked [FILL] for question XX. Can you say more about what you were thinking there?

Hesitation

Can you say more about your hesitation at question XX?

Confusion

What was it about question XX that seemed confusing to you?

IF NEEDED, In your own words, what information would you say [CONFUSING TEXT] is asking for?

Answer Changes

What made you decide to change your answer at question XX?

If R said question did not apply

For question XX you said it did not apply. Can you say more about that?

If R could not answer

For question XX, you said you couldn’t answer. Can you say more about that?

IF NEEDED:

• Is the information collected or tracked in your district? IF YES, Who has access to it?

Wrap-up

Is there anything else from your earlier review of the questions that we have not talked about today?

Which questions did you think were hardest to answer? What makes you say that?

Which questions did you think were easiest to answer? What makes you say that?

Closing and Incentive

IF OBSERVERS ARE PRESENT, CHECK TO SEE IF THEY HAVE FURTHER QUESTIONS.

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

DISCUSS ANY RESPONDENT COMMENTS.

Thank you for your time.

STOP RECORDING.

5