State Agencies, Grantees and SFA

S

OMB No.: 0584-0559

Expiration Date: 3/31/2014

EBTC EVALUATION

July 2011 Site Visits

Interview of Other Organizations Involved in

Parent Training and Nutrition Education

INTRODUCTION

My name is [X], from [Abt Associates/Mathematica Policy Research].

As you may know, Abt and Mathematica are working together to evaluate the Summer EBT for Children demonstrations for the Food and Nutrition Service of USDA. My colleagues and I are visiting each of the five sites and collecting information from a wide range of stakeholders involved in the demonstration in order to learn about the process of implementation, the challenges you encountered, and lessons learned. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues are very helpful.

For stakeholders not receiving grant funds: Your participation is voluntary, and your responses will be kept confidential to the extent provided by law. You may refuse to answer any question and may stop the interview at any time.

Our reports to FNS will describe the range of responses expressed by staff, and may list the names of agencies and partners who contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you.

I expect our conversation will take approximately [XX] minutes.

[OPTIONAL IF INTERVIEWER CHOOSES TO RECORD:] I want to be sure I am keeping track of everything you are saying. May I record our discussion so that I can listen to it later when I write up my notes? No one outside of our research team will have access to the recording. [IF YES:] Thank you. It will be helpful if you speak up, speak clearly, and speak one at a time. [IF NO:] That’s no problem. I’ll take notes as you talk, but I may sometimes need to ask you to slow down or repeat so that I can get all the information.

First, do you have any questions for me about the project in general or what we will be discussing today?

A. BACKGROUND ON RESPONDENT

A.1 What organization do you work for?

A.2 What services does [ORGANIZATION] provide?

A.3 What is your position? What are your day-to-day responsibilities?

A.4 How long have you worked for [ORGANIZATION]?

B. Local Context

I’d like to start by asking you some questions about the food environment in your local area and the availability of food services during the summer months.

Food environment.

B.1 Since the last site visit, has the food environment in the demonstration area changed (such as the number of [SNAP/WIC] retailers, the types of retailers, or local access to those retailers)?

[If so, probe:]

• In what ways?

• For what reasons?

• Are more or fewer resources available to local households?

• What types?

Summer Food Service Program (SFSP).

B.2 Has there been any change this summer in the number or types of Summer Food Service Program sites available in the demonstration area compared to summer 2010?

[If so, probe:]

• What changed?

• For what reasons?

• Are more or fewer summer feeding sites operating?

• Are more or fewer children being served?

• Are the hours, days, or weeks of operation different?

Other existing summer feeding options.

B.3 Since summer 2010, has there been any change in other summer feeding options (such as Seamless Summer Feeding Program, NSLP, kids' cafes, Child and Adult Care Food Programs) were available in the target communities?

[If so, probe:]

• What changed?

• For what reasons?

• Are more or fewer summer feeding options for children?

• Are more or fewer children being served?

• Are the hours, days, or weeks of operation different?

Gaps in summer food services.

B.4 Do gaps exist this summer in the availability of summer food services for children? How large is the unmet need?

B.5 What are reasons for the gaps in services?

[Probe:]

• Limited funding

• Not enough sites

• Poor transportation to sites

• Not open the full summer

• Limited outreach

• Lack of awareness

• Stigma

• Limited hours or days of operation

• Parents concerns’ about neighborhood safety

• Lack of program activities

• Other reasons?

C. INVOLVING LOCAL ORGANIZATIONS OR COMMUNITY ORGANIZATIONS

I’d now like to ask you about your work on [PROGRAM].

Organizational structure for the work on the demonstration.

C.1 What are your and/or your organizations responsibilities as they relate to [SEBTC PROGRAM NAME]?

C.2 Have your responsibilities changed since the last site visit?

[If so, probe:]

• How have your responsibilities changed?

• Why did they change?

Agreements between organizations.

C.3 Since the last site visit, has the agreement or contract between your organization and [GRANTEE] changed?

[If so, probe:]

• What changed?

• Why?

• Can we get a copy of the agreement or contract?

Staffing structure.

C.4 How many staff members from your organization are involved in the demonstration activities?

C.5 What are their roles on the project?

[Probe:]

• What type of backgrounds and qualifications do each have?

C.6 Has the staffing structure changed since the last site visit?

[If so, probe:]

• How has it changed?

• Why did it change?

C.7 Has there been staff turnover since the last site visit?

[If so, probe:]

• In what positions and for what reasons?

• What were the effects of this turnover?

C.8 Were any new staff members hired since the last site visit as a result of the demonstration?

Communication between grantees and key partners.

C.9 How often do you communicate with [GRANTEE] and for what reasons?

C.10 Have there been challenges to maintaining communication?

[If so, probe:]

• In what ways and for what areas could communications be improved?

C.11 What forms of communication and collaboration have been most helpful? In what ways?

Successes and challenges of the collaborations

C.12 What are the most effective elements of the collaboration with [GRANTEE], and why? What has worked best in this partnership?

C.13 What aspects of this collaboration could be improved?

C.14 What have you learned about establishing and maintaining this collaboration?

D. INFORMATION AND TECHNICAL ASSISTANCE

FOR COMMUNITY ORGANIZATIONS

Experiences of and local organizations with training

D.1 Since the last site visit, have you received any training or technical assistance for the [SEBTC PROGRAM NAME]?

[If not, ask:]

• What types of training or TA would have been helpful?

[If so, ask the remainder of this section:]

D.2 What type of training or TA did you receive (e.g., information materials distribution, one-on-one, group, computer)?

[If you received written information, probe:]

• What was the focus of the materials (e.g., developing lists of eligible children, the use of benefits and EBT cards, outreach and education to eligible households)?

• When did you receive them?

• Was the level of information sufficient to understand the program and complete your role?

[If you received training, ask the remaining questions in this section:]

D.3 What was the focus of the training (e.g., developing lists of eligible children, the use of benefits and EBT cards, outreach and education to eligible households)?

D.4 Who conducted the training?

D.5 How long was the training?

D.6 Was there any follow-up training provided?

D.7 Was the level of training sufficient to understand the program and complete your role in the demonstration?

D.8 Was the type of training appropriate?

D.9 Would you suggest conducting more/less of certain types?

D.10 What changes would you suggest the state makes to the training in the future (i.e., content, timing, length, location)?

E. INFORMATION AND TRAINING FOR PARENTS ON SEBTC BENEFITS

[NOTE TO VISITOR: ASK THIS SET OF QUESTIONS ONLY ORGANIZATION OFFERS TRAINING FOR PARENTS.]

Training for parents

E.1 Beyond the information letter that was sent to parents about participating in the [SEBTC PROGRAM NAME] program, have or will parents in the demonstration sites receive training or additional information packets?

E.2 Do materials or training differ for certain populations (e.g., [SNAP/WIC] recipients versus non-recipients)?

Content of materials and training

[Note to visitors: Please get copies of all materials.]

E.3 What is the focus of the materials or training?

[Probe:]

• Information about the [SEBTC PROGRAM NAME]

• What the EBT card is and how to use it (including activation)

• What types of foods are allowed, locations for accessing benefits

• Who should receive the benefit (children)?

E.4 Do the materials or training provide broader information about eligibility for other programs (e.g., SNAP, WIC, summer food programs in the community)?

E.5 Do the materials or training address potential concerns about immigration status and benefit receipt?

E.6 Does the training include information on nutrition education (if so, we will talk in more detail about this later)?

Format of training

E.7 How are materials or training being provided (e.g., information packets, one-on-one, classes, computer, web-based)?

E.8 Who prepared and distributed materials?

[If using group training:]

E.9 Where are trainings conducted?

E.10 How large will the classes be?

E.11 Who is conducting training?

E.12 How long were the training classes?

E.13 How many training sessions were held? And how often?

E.14 Are materials and training being distributed in multiple languages? Which languages are included?

E.15 Is follow-up training planned?

Partner agencies’ experiences with training.

E.16 Was the type of training appropriate?

E.17 Was the level of training appropriate? Would you suggest conducting more/less of certain types?

E.18 How much follow-up training/support was needed?

E.19 Overall, was more training needed/offered?

E.20 For classes, was the size and turnout about what you thought it would be?

E.21 Were the training locations convenient for parents/guardians?

E.22 Were the times of trainings convenient for parents/guardians?

E.23 How did the parents respond to the training?

E.24 Would you suggest making changes to the training (i.e., content, timing, length, location) for the next summer?

Additional support for families [Ask only if community-based organization provides training and support to parents]

E.25 Is there a contact number for households to call if they have questions about the program, benefits, or card?

[Probe:]

• Is this contact number the same as [SNAP/WIC]?

• Who are they contacting (e.g., eligibility worker, CBO, vendor, automated)?

• What are the hours of operation for the customer service number?

• Have customer service representatives been trained on how to respond to questions?

E.26 What happens if a customer service representative cannot answer a question?

[Probe:]

• Is there a local contact number? Who answers that line?

• Is there any face-to-face support offered? Who provides the assistance?

E.27 Is there a website or webpage dedicated to [SEBTC PROGRAM NAME] information? [If so:] How is access controlled?

Household reaction to the EBT benefits

E.28 Have you heard from parents about the Summer EBT for Children program?

[If yes]

• In general, how have they responded?

[Probe:]

• About how many inquires/calls have you received?

• What kinds of questions did parents ask?

• What types of concerns were most often discussed?

• Did parents tend to be more positive or negative about the program?

E.29 What has been the reaction of households to the $60 per child monthly benefit?

[Probe:]

• Do they believe the level is too high, too low, or appropriate?

• Has this affected the consent rates for the demonstration in any way? If so, how?

F. NUTRITION EDUCATION

[NOTE TO VISITOR: ASK THE FOLLOWING QUESTIONS ONLY IF ORGANIZATION IS CONDUCTING OR HAS CONDUCTED NUTRITION EDUCATION SINCE THE LAST VISIT]

F.1 Can you describe the nutrition education that you are have done for families in the demonstration area since the last site visit? Is your nutrition education program directly tied to the Summer EBT for Children?

F.2 What is the structure of the services (e.g. written materials versus training)?

[If training, then probe:]

• Group or one-on-one?

• How long are the classes?

• How often are they offered?

F.3 What is the focus of the nutrition education?

[Probe: What are some examples of handouts or lessons covered?]

• Stretching your shopping dollar

• (for WIC sites only) List of SEBTC foods available, portion sizes allowed

• Fruits and vegetables

• Whole grains

• Eating seasonal foods

• Farmer’s markets (if able to use card there)

• Using low-fat or fat-free milk

• Healthy snacks for kids

• Introducing new foods

F.4 Do you provide these services throughout the demonstration area? If not, which communities are targeted?

F.5 Who are the key developers of the nutrition education approach and materials?

F.6 Who are the staff members that will deliver the nutrition education?

[Probe:]

• How many staff members will provide education?

• What percentage of their time will be allocated to educating participants?

F.7 Are there referrals to sources of more in-depth nutrition counseling in the community?

[If nutrition education is tied to the Summer EBT, ask the remaining questions in the section.]

F.8 What individuals are being targeted for nutrition education?

[Probe:]

• Are only households that receive the [SEBTC PROGRAM NAME] benefit participating or are households who do not receive the benefit also participating in nutrition education?

• Are subgroups of individuals targeted for particular education components (for example children only, parents/guardians but not children, etc.)?

• Why did you target these subgroups?

• How often will parents/guardians and children receive education and through what modes?

F.9 Do households have to participate in nutrition education to receive an EBT card? Do you know why was this approach taken?

F.10 Since the last site visit, what activities have been used to promote the awareness and use of nutrition education among targeted households and children (for example, have you distributed information, provided support services, or made referrals)?

[Probe:]

• Why did you to select these promotion activities?

• Do you perceive these activities as helpful in encouraging targeted households to understand or take advantage of the education services?

F.11 Does the nutrition education you are offering for Summer EBT for Children differ from what you typically offer?

[Probe:]

• What aspects are similar, and what aspects are different in terms of the delivery of education and materials used?

• Why are there differences between typical nutrition education and [SEBTC PROGRAM NAME] nutrition education?

• How will staff be advised of the change?

• Will staff be trained to use the adapted materials?

Attitudes towards [SEBTC PROGRAM NAME] nutrition education.

F.13 What has been the reaction of the your staff towards the nutrition education component of the demonstration (e.g., receptive, enthusiastic, indifferent, etc.)?

F.14 What have participants’ reactions been?

[Probe:]

• Are there particular components or modes of education that they prefer?

• If so, which ones and for what reasons?

Successes and challenges of [SEBTC PROGRAM NAME] nutrition education.

F.15 How successful has this nutrition education component been in meeting your goals?

F.16 What has been successful about the planning process and delivery of education? What factors have led to these successes?

F.17 What challenges have you faced? What are you and local agencies doing to overcome these challenges?

F.18 What kinds of improvements could be made in the future?

G. SUCCESSES, CHALLENGES, AND SOLUTIONS

Perceived impacts of the demonstration.

G.1 To what extent do you believe the demonstration has affected children’s food security during the summer?

G.2 To what extent has it affected child nutrition during the summer?

G.3 To what extent has it affected household food security?

G.4 To what extent has it affected new applications for [SNAP or WIC]?

G.5 To what extent has it affected other summer feeding programs such as the SFSP, or emergency food services such as food banks and kitchens? Did they experience an increase or decrease in demand for food?

G.6 Are effects more apparent for certain groups of populations? [If so,] Which groups?

G.7 If [STATE] implemented services beyond the EBT benefit, to what extent does it appear that these services affected children’s outcomes?

G.8 What events during the summer outside of the demonstration may have affected its impacts?

Successes.

G.9 Thus far, what have been the greatest successes of the demonstration?

G.10 What factors contributed significantly to this success?

G.11 What, if anything, could have been done differently to achieve greater improvements in children’s food security?

Challenges.

G.12 Thus far, what have been the biggest challenges to meeting the goals of the demonstration?

G.13 Have these challenges been resolved?

• If so, how?

• If not, why?

G.14 What could the [GRANTEE] or other agencies have done differently to meet demonstration goals more effectively?

G.15 How have these challenges affected the effectiveness of the demonstration?

H. Feasibility of Continuing and Replicating DemonstrationS

Conditions needed to continue the demonstration.

H.1 What conditions would need to be maintained for the demonstration to continue successfully next year?

H.2 What program components or implementation practices would need to be maintained?

H.3 What changes, if any, would need to be made in order to operate successfully next year?

Conditions needed for replication in other sites.

H.4 What state and local conditions are critical to replicate the demonstration in other areas of the country?

H.5 What program components or implementation practices are essential for replication?

H.6 Are there practices you would advise other states and local areas to avoid?

[If so, probe:]

• What are they?

• Why would you suggest avoiding them?

H.7 What advice would you give to other states and local areas interested in replication of the demonstration?

I. Implementation and Operational Costs

Matched, volunteer, and donated resources [Ask if organization is receiving grant funds].

I.1 Since the last site visit, have any resources been used beyond the grant to support implementation of the grant?

[If so, probe:]

• What was the source?

• How much was used?

• How were these resources used?

I.2 Since the last site visit, have volunteers been involved in providing services?

[If so, probe:]

• Can you estimate how much volunteer time was spent?

• What was the role of these volunteers?

I.3 Since the last site visit, were any other resources donated for the demonstration (e.g. office space, materials)?

[If so, probe:]

• What were they?

• Who donated them

• How much was used?

Matched, volunteer, and donated resources [Ask if organization is not receiving grant funds].

I.4 Since the last site visit, what resources have been used to support implementation of the grant?

[Probe:]

• How much was used?

• How were these resources used?

I.5 Since the last site visit, have volunteers been involved in providing services?

[If so, probe:]

• Can you estimate how much volunteer time was spent?

• What was the role of these volunteers?

I.6 Since the last site visit, were any other resources donated for the demonstration (e.g. office space, materials)?

[If so, probe:]

• What were they?

• Who donated them

• How much was used?

Staff time spent on [SEBTC PROGRAM NAME].

I.7 What percentage of your time do you typically spend on the program?

[Probe:]

• Has this changed over time?

• How did the percentage compare before the school year ended compared to after the school year ended?

• How much of your time is charged to the grant?

• How much is paid by other funding sources?

• Do you work overtime hours on the project? If so, how much is paid versus unpaid?

CLOSING

Is there anything you think is important for the FNS to know about the [STATE]’s [SEBTC PROGRAM NAME] demonstration that we did not ask about?

Thank you for your time and helpful feedback. The information you have shared will be valuable to our team as we look across states and localities for themes and ideas that we can share with FNS.

Are there any questions you have for me before we finish?

END OF INTERVIEW

25