State Child Nutrition Agency (State Director and Key Staff)

OMB Number: 0584-NEW Expiration Date: XX/XX/XXXX

APPENDIX C-3. PRETEST: PREVISIT TELEPHONE INTERVIEW GUIDE

Assessing the Child Nutrition State Administrative Expense Allocation Formula (Child Nutrition SAE)

Pretest of the Pre-visit Telephone Interview Guide

According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-xxxx. The time required for the State Child Nutrition Director to provide this information collection is estimated to average 45 minutes per response, including the time to review instructions, search existing data resources, gather and maintain the data needed, and complete and review the collection of information.

Introduction

Good morning/afternoon. Thank you again for taking the time to talk with me today. My name is [INTERVIEWER NAME] and as you know, I work for Westat, a private research company in Rockville, Maryland.

The U.S. Department of Agriculture’s Food and Nutrition Service (FNS) is trying to better understand spending patterns and factors that affect State use of State Administrative Expense (SAE) funds. FNS hired Westat to conduct the Child Nutrition SAE study to assess the effectiveness of State’s current formulas, and understand what influences States’ spending allocation decisions. The study will include interviews with State agency Directors in 12 States to understand States’ processes for budgeting and spending SAE funds. If there are multiple state agencies that receive SAE funds in a State we will conduct interviews with staff from all of them. Before we conduct these interviews on a broader scale, we are interested in identifying areas for improving the questions – the goal is to ensure that the questions are easy to understand and to answer.

Did you receive a copy of interview questionnaire?

IF YES: Do you have that in front of you at the moment? If not, could you please get the copy because we’ll need to review it while we’re talking today.

IF NEVER RECEIVED OR LOST: Are you able to access your email?

HAS ACCESS BUT NO EMAIL ADDRESS ON FILE: I can resend right now via email. Can you please give me your email address?

HAS ACCESS TO EMAIL: I’ve resent the documents.

IF NO ACCESS TO EMAIL EVER, OFFER TO RESEND AND RESCHEDULE INTERVIEW. IF CANNOT RESCHEDULE, PROCEED WITH INTERVIEW: I will mail you another copy of the document to you. Let me confirm your address.

Informed Consent

Before we get started there are a few things I want to mention.

• This is a research study which means your participation in this discussion is voluntary.

• Refusal to participate will not have any impact on your position or child nutrition programs.

• The thoughts and opinions that you share during our discussion will be used for research purposes only. They will not be used as an evaluation of your work, your staff’s work, or the Child Nutrition program.

• There are no direct benefits to you or your department for participating in this study.

• Your participation and feedback is important and needed because input from staff like yourself help us refine our instruments to best explore the SAE in different states.

• The names of those who participate in these cognitive tests of the interview guides may be shared, and we may quote you if we feel a comment is particularly poignant. However, once we start the interview, you can take a break, skip questions, choose to make comments off the record, or stop participating at any time. Again, there will be no penalty or loss of benefits to your program.

• Finally, with your permission, I would like to record this discussion. The recording helps us recall exactly what was said when we go to summarize our findings. The recordings and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

• If you have any questions about your rights as a participant, please call the Westat Human Subjects Protections office at 1-888-920-7631. Please leave a message with your full name, the name of the research study that you are calling about (Child Nutrition SAE Study), and a phone number beginning with the area code. Someone will return your call as soon as possible.

We have planned for this discussion to last about 45 minutes.

Do you have any questions? [ANSWER ALL QUESTIONS]

Do I have your consent to participate?

May I turn on the audio recorder now?

IF YES: [ONCE RECORDER IS ON, “Now that the recorder is on, today is DATE at TIME. Do you still consent to participate in the testing of this interview guide, and do I have your permission to audio record this interview?” MAKE SURE YOU GET AN AUDIBLE “Yes” FROM RESPONDENT.]

Procedures

First I’ll ask you questions about your State’s SAE budgeting and spending patterns, and then dive into questions about factors that influence SAE allocations and how specific Child Nutrition programs impact SAE. After the interview is over, I’ll ask you some debriefing questions at the end to get your reactions to the overall content, wording, and flow of the interview questions.

ADMINISTER IN-DEPTH INTERVIEW. NOTE BEGIN TIME: __________________

NOTE DELIVERY, COMPREHENSION AND RESPONSE ISSUES ON TRACKING DIRECTLY ON INTERVIEW GUIDE.

NOTE ANY VERBAL REACTION R SHARES FOR FOLLOW-UP DURING THE DEBRIEF.

FOR ANY NON-VERBAL REACTIONS (E.G., LONG PAUSES), ASK ONLY, Tell me what you’re thinking here. DO NOT PROBE BEYOND THAT UNTIL THE DEBRIEF.

COGNITIVE TESTING RESEARCH QUESTIONS TO REMEMBER:

• Do respondents understand and interpret the questions as intended?

• Are State Directors aware of who has a role in budgeting and spending SAE?

• Are State Directors able to answer in-depth questions about SAE Spending Patterns, including budgeting and spending SAE funds, the services or activities that the SAE funds, and the process and reasons to reallocate or return SAE funds?

• Are State Directors able to answer in-depth questions about the factors that influence the State’s use of SAE?

• Are State Directors able to answer questions about SAE funds that pertain to specific Child Nutrition programs?

• Is there any information about the data collection and submission process that is missing from the in-depth interview guide?

Pre-visit Telephone Interview Guide

Warm Up

1. To begin, please tell me about your current role and how long you have worked at the State agency.

Organizational Structure

2. Please explain the organizational structure of your State and how and where Child Nutrition (CN) programs fit into that structure.

• Probes:

  • Have there been any changes to this structure over the last decade? Please explain.

  • [IF APPLICABLE: How does your agency collaborate with the other agencies that administer CN Programs?]

3. Tell me about how the CN program staff in your office are organized.

• Probes:

  • How are work units organized?

  • What functions do the work units perform-- monitoring, technical assistance, reporting?

  • How do the CN program staff interact with the financial staff? Are they part of the same organization?

  • Besides the other State agencies that administer CN Programs (if applicable), are there other departments that have a role in the use of SAE? [IF YES] In what capacity do you work with these other departments?

4. Which work units have responsibility for SAE in your office?

   • Probes:

     • What is their specific role in SAE? (i.e., planning/budgeting, tracking funds usage, allowable cost questions, etc.)

     • Which staff should we interview while we are on-site?

5. What is the role of the regional office in SAE?

   • Probes:

     • How does the regional office interact with the State about SAE? IF NEEDED: (how does the regional office communicate about SAE funds usage, monitoring?)

     • How frequently do you interact with the regional office about issues related to SAE funds?

State Environment

6. Have there been any challenges, issues, or concerns with SAE in your State?

7. What other information about your State’s environment would be helpful for us to know?

• Probes:

  • Are there State priorities or policies that impact SAE spending?

  • How is State leadership (e.g., Superintendent, Governor, etc.) involved in Child Nutrition programs?

  • What about the State legislature?

  • How do advocacy organizations support Child Nutrition programs in the State?

State Documents

8. Are there any State documents related to SAE that you can send us to help prepare for our on-site visit?

• Probes:

  • May we have a copy of the agency’s SAE budget and expenditures?

  • Are there any procedural documents related to SAE?

• May we obtain a copy of your State’s current SAE Plan (on file with the FNS regional office)? Is there anything you want to share about these documents that would be helpful in our preparation for the on-site interviews?

Wrap Up

9. Thank you so much for taking the time to talk with us today. Do you have any final questions or thoughts?

We look forward to meeting you during the site visit. You will receive follow-up information via email from Westat.

Debriefing Questions

NOTE END TIME: _________________________

Thank you. Now I’m going to ask you some questions about the interview overall. First, tell me your overall impressions of the questions I just asked you.

USE RELEVANT PROBES BELOW TO DISCUSS OBSERVED ISSUES FROM INTERVIEW NOTES.

Clarification Requests

You asked [FILL] for question XX. Can you say more about what you were thinking there?

Hesitation

Can you say more about your hesitation at question XX?

Confusion

What was it about question XX that seemed confusing to you?

IF NEEDED, In your own words, what information would you say [CONFUSING TEXT] is asking for?

Answer Changes

What made you decide to change your answer at question XX?

If R said question did not apply

For question XX you said it did not apply. Can you say more about that?

If R could not answer

For question XX, you said you couldn’t answer. Can you say more about that?

IF NEEDED:

• Is the information collected or tracked in your state? IF YES, Who has access to it?

Wrap-up

Is there anything else from your earlier review of the questions that we have not talked about today?

Which questions did you think were hardest to answer? What makes you say that?

Which questions did you think were easiest to answer? What makes you say that?

Those are all the questions I have for you. Is there anything we haven't discussed that you would like to mention?

Thank you for your time.

STOP RECORDING.

Draft Instruments	0