SEBTC Evaluation
April 2011 Site Visits
Interview with Grantee and Major Partners
[Note to site visitors:] Unless otherwise noted, all questions will be asked of a single respondent only. Site visitors will identify the roles of respondents during the site visit planning process and determine which respondent should address each major topic area. The site visit guide also indicates questions within each topic area that should be asked of more than one respondent.
INTRODUCTION |
My name is [X], from [Abt Associates/Mathematica Policy Research].
As you may know, Abt and Mathematica are working together to evaluate the Summer EBT for Children demonstrations for the Food and Nutrition Service of USDA. My colleagues and I are visiting each of the five sites and collecting information from a wide range of stakeholders involved in the demonstration in order to learn about the process of implementation, the challenges you encountered, and lessons learned. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues are very helpful.
For stakeholders not receiving grant funds: Your participation is voluntary, and your responses will be kept confidential to the extent provided by law. You may refuse to answer any question and may stop the interview at any time.
Our reports to FNS will describe the range of responses expressed by staff, and may list the names of agencies and partners who contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you.
I expect our conversation will take approximately 180 minutes.
[OPTIONAL IF INTERVIEWER CHOOSES TO RECORD:] I want to be sure I am keeping track of everything you are saying. May I record our discussion so that I can listen to it later when I write up my notes? No one outside of our research team will have access to the recording. [IF YES:] Thank you. It will be helpful if you speak up, speak clearly, and speak one at a time. [IF NO:] That’s no problem. I’ll take notes as you talk, but I may sometimes need to ask you to slow down or repeat so that I can get all the information.
First, do you have any questions for me about the project in general or what we will be discussing today?
A. BACKGROUND ON RESPONDENT |
[Ask this section of all respondents]
A.1 What organization do you work for?
A.2 What services does [ORGANIZATION] provide?
A.3 What is your position? What are your day-to-day responsibilities?
A.4 How long have you worked for [ORGANIZATION]?
A.5 When did you begin working on [SEBTC PROGRAM NAME]?
A.6 What are your specific responsibilities?
B. Demonstration Planning and the Application Process |
Impetus for applying.
B.1 Why did [STATE] choose to apply for the demonstration grant?
B.2 What were the perceived opportunities of this grant?
B.3 What were the grant’s main challenges?
Goals of the demonstration.
B.4 What goals do you hope to achieve?
B.5 To what extent are the goals quantifiable?
B.6 How will they be measured?
Organizations involved in application process.
B.7 Why was it decided that [AGENCY NAME] was the lead agency for the grant?
B.8 Were other state agencies involved in the application process?
[Probe for each organization type and their roles:]
Other state agencies
Local School Food Authorities (SFAs)
EBT vendor
Community based organizations
Selection of local area.
B.9 Why was the local area selected?
B.10 Were other areas considered?
The planning process and issues addressed
B.11 Which of the organizations we talked about were involved in the planning process?
B.12 What were the roles of the key players?
B.13 What mechanisms were used during this planning process (e.g., workgroups, steering committees, focus groups)?
B.14 Over what period of time did the planning process occur?
B.15 What were the main issues of focus and/or concern during the planning process? How were they addressed?
B.16 Which organization took the lead on writing the grant application?
B.17 What challenges did you encounter during the planning and application process? How were they addressed?
C. local Context |
Food environment [for WIC sites ONLY]
C.1 How many WIC retailers are there in the demonstration area?1
C.2 Are there WIC-only stores in the demonstration area? Do They offer fresh food?
C.3 How many are large retailers such as national or regional franchises?
C.4 How many are small local stores?
C.5 Are there farmer’s markets or community gardens in the area?
[If yes]
Do they accept WIC paper vouchers? If so, what proportion?
Do they accept WIC EBT cards? If so, what proportion?
Do they accept Cash Value Vouchers (CVV) for fruits and vegetables? If so, what proportion?
Food environment [for SNAP sites ONLY]
C.6 How many SNAP retailers are there in the demonstration area?2
C.7 How many are large retailers such as national or regional franchises?
C.8 How many are small local stores?
C.9 Are there farmer’s markets or community gardens in the area?
[If yes]
Do they accept SNAP paper vouchers? If so, what proportion?
Do they accept SNAP EBT cards? If so, what proportion?
Barriers to accessing SNAP/WIC retailers
C.10 Are there any barriers for some local residents to accessing [SNAP/WIC] retailers?
[If so]
Do any agencies offer services to help with access (such as help with transportation, ability to order groceries for delivery to neighborhood centers)?
Summer Food Service Program (SFSP).
[Confirm information received on SFSP]
C.11 How many SFSP sponsors are there in the target areas?
C.12 How many and what types of organizations (SFA, government, non-profit) sponsor the SFSP and how many feeding sites did they run?
C.13 Where are sites typically located? (for example, community centers, parks, playgrounds)
C.14 What percentage are open sites?
C.15 How many children were served in 2010?
C.16 How many weeks were the programs open?
C.17 What kinds of issues arose during the summer of 2010?
Other summer feeding options.
C.18 Beyond the SFSP, what other summer feeding options (such as Seamless Summer Feeding Program, NSLP, kids' cafes, Child and Adult Care Food Programs) were available in the summer of 2010 in the target communities?
[Probe:]
Can you please describe them?
What types and how many organizations administered these programs?
How many children were typically served by each?
C.19 What food bank(s) or food rescue organization(s) provide emergency food in the target areas?
[Probe:]
Are they part of a national or regional network (Feeding America, etc.)?
How many food pantries, soup kitchen(s), and emergency shelters are available?
Do they serve families with children?
Gaps in summer food services.
C.20 Do gaps exist in the availability of summer food services for children?
[If so, probe:]
How large is the unmet need?
What are reasons for the gaps in services (e.g., limited funding, not enough sites, poor transportation to sites, not open the full summer, limited outreach, lack of awareness, stigma, limited hours or days of operation, parents concerns’ about neighborhood safety, lack of program activities)?
D. Basic parameters of the summer EBT MODEL |
Branding.
[Confirm the “brand” or name used for outreach purposes]
D.1 Why was that name selected?
D.2 Did you develop a logo? What does the logo represent?
EBT cards.
[For SNAP sites]
D.3 Why did the agency choose to issue separate card/add benefits to existing card (e.g. easier administratively, cost savings, easier for households)?
Level of EBT benefits. [Ask of grantee and all major partners]
D.4 What do you think of the $60 per child monthly benefit level specified by FNS?
[Probe:]
Do they believe the level is too high, too low, or appropriate?
D.5 What do you think of the fact that children of all ages are receiving the same amount of benefits and that multiple children in a household each receive the full benefit (no accounting for economies of scale)?
Mix of benefits in WIC sites. [Ask of grantee and all major partners]
D.6 What has been the reaction of participating organizations to the WIC foods selected for the demonstration?
[Probe:]
-Overall mix of foods
-Split between food and cash voucher for fruits and vegetables
-Same types and amounts of food for children of all ages
Timeframe of EBT benefits.
[Confirm state plans about pro-rating benefits for the first and last months of the summer.]
D.7 What implications are there for the pro-rated benefits for the first and last months of the summer for implementation?
D.8 What implications might it have for households accessing benefits?
Services beyond the EBT benefit.
D.9 Are families offered any services, such as nutrition education, beyond the EBT benefit as part of the demonstration?
[If so, probe:]
- Can you give me a brief description?
- When were or will these services be offered?
- To whom?
E. Project Organization and Management |
Organizational structure to administer the demonstration. [Ask of grantee and all major partners]
E.1 Have the responsibilities of each organization changed from what was proposed in the grant application?
[If so]
How and why?
E.2 What are the main advantages to the current organizational structure?
E.3 Are there any disadvantages to your current organizational structure?
Agreements between organizations.
E.4 Are there agreements or contracts in place between [GRANTEE] and [EACH PARTICIPATING ORGANIZATION]?
[If so, probe:]
- Do these agreements or contracts identify the goals, objectives, roles, and responsibilities as they pertain to the demonstration?
- Have these agreements been formalized? If yes, how so? [Can we get copies?]
Staffing structure [Ask of grantee and all major partners]
E.5 What is your current staffing structure for administering demonstration services?
[Probe:]
- What roles do key staff members play? What type of backgrounds and qualifications do they have?
- What roles do support staff play? What type of backgrounds and qualifications do they have?
- Are there shared responsibilities across organizations? Who leads or coordinates them?
E.6 What percentage of time do staff members dedicate to the demonstration?
E.7 Has the staffing structure changed since the inception of the grant? How and why?
E.8 Has there been staff turnover? In what positions and for what reasons? What were the effects of this turnover?
E.9 Were any new staff members hired as a result of the demonstration? Which ones?
E.10 What new skills are required of existing staff to carry out demonstration services? Did these staff receive training to carry out their new responsibilities?
Communication between grantees and key partners. [Ask of grantee and all major partners]
E.11 What levels of staff communicate across the lead agencies?
E.12 How often? For what reasons? What topics are discussed?
E.13 In what format are meetings held (large group meetings, conference calls, one-on-one with each partner)?
E.14 Have there been challenges to maintaining communication? In what ways and for what areas could communications be improved?
E.15 What forms of communication and collaboration have been most helpful? In what ways?
F. Involving Local Agencies and Community Organizations |
Identification of collaborating agencies.
F.1 What local agencies and community organizations that do not receive grant funds are have a role in the demonstration (e.g., outreach, training, nutrition education)?
[Probe:]
For example, SFSP sites and other summer feeding programs, emergency food providers, hunger advocacy groups, other faith or community-based organizations?
F.2 How did you identify these organizations?
F.3 Did you consider or approach others? Why are they not involved?
F.4 Did any planned partnerships fail to materialize? Which ones? Why weren’t these partnerships realized?
Roles and responsibilities of collaborating agencies.
F.5 What are the roles and responsibilities of these collaborating agencies?
[Probe:]
For example, an advisory role in planning and decision making, volunteer resources and staff time for training and activities, provide complementary services to beneficiaries, provide outreach and referral, or something else?
F.6 What are the sources of funding for partners’ activities?
Initiation of the collaborations.
F.7 Did you have working partnerships with these local organizations before this demonstration grant?
[If so, probe:]
Describe your prior relationship.
In what ways were these prior relationships useful for the demonstration?
Collaborative agreements.
F.8 Did you establish a formal or written agreement about the role of the collaborator in the demonstration?
[If so, probe:]
How? [Can we get a copy of the contract or MOU?]
[If not, probe:]
Why?
Successes and challenges of the collaborations.
F.9 What are the most effective elements of the collaborations, and why?
F.10 What have you learned about establishing and maintaining these collaborations?
F.11 What lessons would you share with other grantees?
F.12 What has worked best in these partnerships?
F.13 What partnership aspects could be improved?
F.14 From which partners would you have liked more involvement?
G. Identifying Eligible Children and Households |
Identifying children eligible for FRP meals.
G.1 Who at the SFA was responsible for identifying children eligible for FRP meals (e.g. the food service department, the central administration or both)?
G.2 How did you coordinate with the SFAs during the process of identifying children?
With whom at the SFAs did you work (e.g. the food service department or the central administration or both)?
How did you determine which data elements you needed and could get from the schools?
Did the SFAs share the information on children with the grantee or its partner? Or was data held confidential at the SFAs until the consent process was finalized?
G.3 What was the process at the local SFAs for identifying eligible children?
[Probe:]
What data sources were used?
Did you need to combine data from multiple databases?
Are data on children who are directly certified maintained in the same database as data on children who submit a school lunch application?
If not, what were the implications?
Were you able to pull the most recent contact information from the student records database?
Were you able to provide all the data elements requested by [GRANTEE]? If not, which were unavailable?
How many staff hours did the process take?
G.4 What criteria did you use for including a child in the file of eligible children?
[Probe:]
What grades were included? Was pre-k included? Were seniors included?
Were any students excluded from the file even if they qualified for FRP meals?
What cutoff date did you use for school lunch applications to be included?
What percentage of children are directly certified in your SFA?
How often is direct certification conducted? Did you include students from the most recent certification?
G.5 What is your impression of the quality of the data from each source that you used?
[Probe:]
How often is data from school lunch applications entered into your database?
How often is student contact information updated?
How and for what purposes is contact information updated?
G.6 In what month did the process of identifying eligible children begin?
G.7 Have there been any challenges in compiling the data?
[Probe:]
Files from various school districts in different formats?
Some data items not available from some districts?
Not all eligible children were identified?
Data came from different sources, such as the NLSP applications and school records
G.8 How many children were identified in the target area?
Aggregating data into household level files.
G.9 Who was responsible for aggregating the data on children into household level files (e.g. the SFAs, the grantee, or another partner)?
G.10 Was there a household or family identifier in the data already?
[If so, probe:]
What was the identifier (e.g. SNAP ID, family ID used by the district)?
Did vary by SFA? How?
[If not, probe:]
How did you match children to households?
Did you match by certain variables such as parent/guardian name, address, and phone?
Was the matching done manually or electronically?
G.11 How did you handle it when more than one family lived in the same household? How many of instances did you identify in your SFA?
G.12 How difficult was it to create the household file for the [GRANTEE]?
[Probe:]
How many staff hours did it take?
Over what calendar period?
G.13 Were there any challenges in compiling the data (e.g. matching children without [SNAP/WIC] identifier, difficulty matching parent/guardian names, addresses, and telephone numbers across databases)?
[If so, probe:]
Were they resolved? How?
Do you have any concerns about the quality of the resulting file?
G.14 How many households did you identify in the target area?
H.
OUTREACH AND Consent for the Demonstration |
Outreach and recruitment efforts.
H.1 What outreach efforts have been conducted or are planned to inform parents/caregivers about the demonstration?
H.2 What methods are being used (e.g., mail, flyers, radio, television)?
H.3 What are the main messages in outreach materials? How widely are materials and information being distributed?
H.4 In what ways are outreach efforts different for households already enrolled in SNAP/WIC versus those that are not?
H.5 Did materials address potential concerns about immigration status and benefit receipt?
H.6 What organizations are responsible for outreach?
H.7 What languages were used for the materials?
H.8 When did outreach efforts begin?
H.9 Of these activities, what has been done so far? What is the planned schedule for remaining outreach efforts?
H.10 Did you hear about any reactions to the efforts from parents or others in the community? What were they?
Consent collection process.
H.11 Were there adaptations made to the planned approach to distributing and collecting consent forms from households?
[If so, probe:]
Why what were they?
Why were they made?
Tracking and managing consent distribution and receipt.
H.12 How did you track how and when letters went out and were sent back?
[Probe:]
How are returned letters handled, if addresses are incorrect?
How are with incomplete or missing information handled?
How are the data to track the return of consent forms stored and cleaned?
How are the data completed by parents/guardians on consent forms stored and cleaned?
Who has access to the data?
Staff responsibilities related to consent.
H.13 Who has primary responsibility for managing the production, distribution, and collection of consents?
H.14 Who is responsible for tracking information about consent mailings and returns (such as the number of letters that have been mailed out, returned as undeliverable, positive responses [active consent only], refusal?
Consent return rates [IF ACTIVE CONSENT REQUIRED] for active sites.
H.15 How many consent packages were sent in the initial mailing?
H.16 What percentage of households that were sent mailings have returned forms with complete information? What were common problems with how the forms were filled out?
H.17 What percentage returned forms with incomplete or missing information?
H.18 What percentage of letters were returned undeliverable?
H.19 [Ask if there was a second mailing for the consent process] For the second mailing, what percentage of households returned forms with complete information? What were common problems with how the forms were filled out?
H.20 Approximately how many parents called with questions or concerns?
H.21 What percentage of households received multiple contacts?
Consent return rates [IF PASSIVE CONSENT REQUIRED.]
H.22 How many consent packages were sent in the initial mailing?
H.23 What percentage of households that were sent mailings opted out or responded that they did not want to be part of the demonstration?
H.24 Was there any effort to encourage these households to reconsider?
H.25 What percentage of letters were returned undeliverable? Was an attempt made to determine the correct address and then mail the consent forms again?
H.26 [Ask if there was a second mailing for the consent process] For the second mailing, what percentage of households returned forms with complete information? What were common problems with how the forms were filled out?
H.27 Approximately how many parents called with questions or concerns?
Maximizing consent response, [IF ACTIVE CONSENT REQUIRED].
H.28 Beyond the outreach efforts described above, did you do other things to maximize response rates from households?
[Probe:]
Did anyone try to contact nonresponsive households?
Who tried to contact them and how (second letters, phone calls)?
Were specific households targeted?
Did you try to contact households more than once?
What was the reaction of households that were reached?
H.29 For how many weeks did you attempt to contact nonresponsive households before finalizing the list of consenting households?
H.30 What challenges have you faced in encouraging parents to apply for the benefit? Did you overcome them and how?
H.31 What kind of technical assistance or support would have been most useful?
H.33 How successful were efforts to contact nonresponsive households?
Household reaction to the Summer EBT benefits.
H.34 In general, how have households responded to the new program?
[Probe:]
About how many inquires/calls have you received?
What kinds of questions did parents ask?
What types of concerns were most often discussed?
Did this vary by school, age of the child, or some other factor?
Did parents tend to be more positive or negative about the program?
H.35 What has been the reaction of households to the $60 per child monthly benefit level?
[Probe:]
Do they believe the level is too high, too low, or appropriate?
Has this affected the consent rates for the demonstration in any way? If so,how?
Lessons learned from consent process [ASK IF PROCESS COMPLETE].
H.36 What aspects of the consent process could have been improved? Why?
H.37 What advice would you give others about tracking and managing the process?
H.38 What kind of technical assistance or support would have been most useful?
I. Informing households about Random Assignment Results AND ENROLLING THEM IN [SEBTC PROGRAM NAME] |
Informing households that were not selected for [SEBTC PROGRAM NAME].
I.1 Are you planning to notify both families who were and were not selected to get the Summer EBT for Children benefit?
[If yes:]
What information was provided?
Were they given materials about other summer feeding options, [SNAP/WIC], nutrition education, or other topics? If so, what was provided? [GET COPIES OF MATERIALS]
I.2 What was parents’ response to these materials?
[Probe:]
Did you receive any complaints about households not being selected to receive benefits?
If so, how many?
Who responded to these households and how?
Informing households that were selected for [SEBTC PROGRAM NAME].
I.3 We will ask more detailed questions about the materials later, but can you provide a brief description of the materials sent to households that were selected?
I.4 What was parents’ response to these materials?
[Probe:]
Did you receive comments that households were pleased to be accepted? If so how many?
Timeline for informing households of random assignment results.
I.5 Have you begun to inform households of the results?
[If so, probe:]
In what month did you begin to inform households?
[If no, probe:]
When do you anticipate informing households?
I.6 Do you feel the timeline for this process was/will be reasonable and early enough for parents to plan for the summer?
[If not, probe:]
How much earlier would you like to complete the process in future summers?
Enrolling households in [SEBTC PROGRAM NAME].
I.7 How are households assigned to the demonstration group officially enrolled in the program?
[Probe:]
What database are you using?
Why did you choose this database?
Did you have to make any modifications to the database for this demonstration?
What child or household information had to be entered?
Who was responsible for entering this data?
How long did it take?
I.8 Did you encounter any challenges while enrolling households?
[If so, probe:]
Can you describe them?
Were you able to resolve these challenges? How?
What would you do differently in the future if the program were continued?
J. EBT SYSTEMS PREPARATION AND distributiOn OF ebt cards |
EBT systems preparations
J.1 Are changes to the EBT system or third-party processor (TPP) systems needed to implement [SEBTC PROGRAM NAME]?
[If so, probe]
What changes are needed?
What is the status of these changes?
What tasks remain to be done?
J.2 Are changes to the point of sale (POS) system programming or data loads needed to implement [SEBTC PROGRAM NAME]?
[If so, probe]
What changes are needed?
What is the status of these changes?
What tasks remain to be done?
J.3 Are changes to the interactive voice response (IVR) system, customer service scripts, or user website needed to implement [SEBTC PROGRAM NAME]?
[If so, probe and request documentation of new scripts and screens]
What changes are needed?
What is the status of these changes?
What tasks remain to be done?
J.4 Are changes to the settlement and reconciliation processes needed to implement [SEBTC PROGRAM NAME]?
[If so, probe]
What changes are needed?
What is the status of these changes?
What tasks remain to be done?
J.5 Please describe the work that has been done on the changes to the EBT system, the POS system, or the IVR system so far.
Who is involved with this process?
What design, development, implementation, and testing steps have been completed? How long did this take?
What activities took the most effort?
What issues were encountered?
J.6 What is the status of system preparations?
What is the schedule for completing preparations and going live?
What are the critical milestones and when are they scheduled to happen?
What do you see as the main challenges remaining?
EBT card distribution.
J.7 When will EBT cards be distributed to households? Have you experienced any delays that have affected that timeline?
Additional materials sent to households with EBT cards.
J.8 Will letters or information materials accompany the EBT card?
What type of information will be included (how to activate/access the card, where to use, contact information, etc.)? [GET COPIES]
J.9 Will training be offered to households unfamiliar with the EBT process? [More questions about specifics of training below].
EBT card activation.
[Note to visitors: Review grant application and confirm your understanding of this process, including how households will activate their cards, who they will interact with to do so, whether they get a PIN and can change it, and whether this process is the same as SNAP/WIC.]
J.10 Has the process that you proposed in your grant application for households to activate the card changed (e.g., activated upon receipt or upon use, or call a number to activate.)? Is so, how and why?
K. information to and technical assistance for schools AND OTHER COmMUNITY PARTNERS |
Types and extent of information and technical assistance.
K.1 Have or will key staff at the participating schools and community partners receive information or training about the demonstration?
[If not, ask:] Why?
[If so, ask the remainder of this section:]
K.2 What type of information or training have been or will be provided (e.g., information materials distribution, one-on-one, group, computer)?
[If written information, probe:]
What was the focus of the materials (e.g., developing lists of eligible children, the use of benefits and EBT cards, outreach and education to eligible households)?
When was the information provided?
Was the level of information sufficient for local staff to understand the program and complete their roles?
[If training or technical assistance, ask the remaining questions in this section:]
K.3 What was the focus of the training or technical assistane (e.g., developing lists of eligible children, the use of benefits and EBT cards, outreach and education to eligible households)?
K.4 Who provided the training and technical assistance?
K.5 How long did it take?
Experiences providing training.
K.6 Was there any follow-up training or TA provided? How much?
K.7 Do you think the level of training and TA was sufficient for local staff to understand the program and complete their roles in the demonstration?
K.8 Was the type of training and TA appropriate?
K.9 Would you suggest conducting more/less of certain types?
K.10 How did local staff respond?
K.11 What changes would you suggest making to the training in the future (i.e., content, timing, length, location)?
L. information and training for parents on [SEBTC PROGRAM NAME] benefits |
[Note to visitor: Ask this set of questions only if training for parents has already started at the time of the visit. Otherwise, questions will be asked during the summer site visit]
Training for parents.
L.1 Beyond the information letter that will be sent to parents about participating in the [SEBTC PROGRAM NAME] program, have parents in the demonstration sites received training or additional information packets?
L.2 Do the materials or training differ for different groups of families (e.g., [SNAP/WIC] recipients versus non-recipients, households that do not speak English)?
Content of materials and training.
[Note to visitors: Please get copies of all materials.]
L.3 What is the focus of the materials or training?
[Probe:]
Information about the [SEBTC PROGRAM NAME]
What the EBT card is and how to use it (including activation)
What types of foods are allowed, locations for accessing benefits
Who should receive the benefit (children)
L.4 Do the materials or training provide broader information about eligibility for other programs (e.g., SNAP, WIC, summer food programs in the community)?
L.5 Do the materials or training address potential concerns about immigration status and benefit receipt?
L.6 Does the training include information on nutrition education (if so, we will talk in more detail about this later)?
Format of training.
L.7 How are materials or training being provided (e.g., information packets, one-on-one, classes, computer, web-based)?
L.8 Who prepared and distributed materials?
[Ask the following questions if there are group training sessions being conducted]
L.9 Who is conducting training?
L.10 Where are trainings being conducted?
L.11 How large are the classes?
L.12 How long do the training classes last?
L.13 How many training sessions will be held? And how often?
L.14 Are materials and training distributed in multiple languages? Which languages will be included?
L.15 Is follow-up training planned?
Additional support for parents.
L.16 Will households have access to an 800 telephone number or a local number to ask questions about the demonstration? Who will answer those calls?
L.17 Is there a location where household members could go to get help (face-to-face)?
[If so, probe:]
Who will provide that help (e.g., local benefit office, CBO, school staff)?
How convenient are these locations for parents/guardians?
L.18 Will there be a website or webpage dedicated to [SEBTC PROGRAM NAME] information? How will access be controlled?
L.19 What other types of media will be used to train/inform participants (e.g., training videos, media campaigns, radio ads)?
M. training and support for retailers |
[Note to visitor: Ask this set of questions only if retailer training has already started at the time of the visit. Otherwise, these questions will be asked during the summer site visit.]
Retailer training.
M.1 From a retailer’s perspective, will the [SEBTC PROGRAM NAME] be very similar to [SNAP/WIC] or will there be a distinction?
M.2 To what extent do retailers need informational materials or training?
M.3 Will all [SNAP/WIC] retailers in the demonstration areas receive information/training?
M.4 [For WIC sites] How will information on the WIC food package for [SEBTC PROGRAM NAME] households be communicated to retailers?
Type of materials or training.
M.5 What will be the focus of the materials or training (e.g., information about the [SEBTC PROGRAM NAME], the look of the EBT card, use of the EBT card, the new WIC package)? What type of materials or training will be provided (e.g., information packets, one-on-one, group, computer, web-based)?
M.6 Who prepared and distributed materials?
M.7 Are materials or training offered in multiple languages? Which languages are included?
M.8 Are face-to-face trainings being conducted at the retailer’s location or with multiple retailers?
M.9 Who is conducting the training?
M.10 How long do training classes last?
M.11 How many trainings will there be?
M.12 What retailer personnel will be invited to the training?
M.13 What type of information will be discussed?
M.14 How much training has been conducted so far? Please describe what has been done so far and what is planned for the future.
M.15 Is follow-up training planned after retailers receive the initial training or as new staff members join the retailers?
Additional retailer support.
M.16 Will retailers be expected to use the existing EBT retailer service number?
[Probe:]
Who answers this number (vendor personnel, automated, State, etc.)?
What are the hours of operation?
M.17 Will there be a special customer service/contact number for retailer questions about SEBTC?
[Probe:]
Who will answer this number (vendor personnel, automated, State, etc.)?
What will be the hours of operation for the special SEBTC retailer service number?
M.18 Are any other supports planned for retailers? If so, please describe them.
N. NUTRITION EDUCATION |
[Note to visitor: Ask the following questions only if nutrition education has already started or was conducted as part of outreach. If not, these questions will be asked during the summer site visit]
[SEBTC PROGRAM NAME] nutrition education.
N.1 Earlier, you provided a brief description of the nutrition education services that are being or will be offered to household members as part of the demonstration.
N.2 Were these nutrition education services designed specifically for the demonstration? Or are they part of on-going education offered in the local area? Or both?
N.3 What organization(s) is/are responsible for developing the nutrition education approach and materials?
N.4 Are comparable activities being provided across the entire demonstration area? If there is variation across local areas or agencies, please describe that variation.
N.5 Who is being targeted for nutrition education?
[Probe:]
Are only households that receive the Summer EBT benefit participating or are households who do not receive the benefit also participating in nutrition education?
Are subgroups of individuals targeted for particular education components (for example children only, parents/guardians but not children, etc.)?
N.6 What is the structure of the services (e.g. written materials versus training)?
[If training, then probe:]
Group or one-on-one?
How long are the classes?
How often are they offered?
N.7 What is the focus of the nutrition education?
[Probe: What are some examples of handouts or lessons covered?]
Stretching your shopping dollar
(for WIC sites only) List of SEBTC foods available, portion sizes allowed
Fruits and vegetables
Whole grains
Eating seasonal foods
Farmer’s markets (if able to use card there)
Using low-fat or fat-free milk
Healthy snacks for kids
Introducing new foods
Promoting nutrition education services.
N.8 Will households have to participate in nutrition education to receive an EBT card?
[If so]
Why did you take this approach?
N.9 What activities will be used to promote the awareness and use of nutrition education among targeted households and children (for example, have you distributed information, provided support services, or made referrals)? Why did you to select these promotion activities?
N.10 Do you perceive these activities as helpful in encouraging targeted households to understand or take advantage of the education services?
Other nutrition education services offered in local areas.
N.11 Beyond what is being offered as part of the demonstration, what other nutrition education services are available to local residents in the target areas?
N.12 What organization(s) offer those services (such as (regular) WIC, cooperative extension, SNAP program, hospitals, Head Start, community groups, schools)?
N.13 Do available services vary across local areas? If so, how?
N.14 How many and what types of residents tend to receive these services?
N.15 What is the focus of the nutrition education?
[Probe:]
Stretching your shopping dollar
Fruits and vegetables
Whole grains
Eating seasonal foods
Farmer’s markets (if able to use card there)
Using low-fat or fat-free milk
Healthy snacks for kids
Introducing new foods
O. Successes, Challenges, and Solutions [Ask of grantee and all major partners] |
Successes. [Ask of grantee and all major partners]
O.1 Thus far, what have been the greatest successes of the demonstration?
O.2 What factors contributed significantly to this success?
O.3 What, if anything, could be done differently to improve the planning process or the initiation of the program?
Challenges. [Ask of grantee and all major partners]
O.4 Thus far, what have been the biggest challenges to meeting the goals of the demonstration?
O.5 Have these challenges been resolved?
[If so]
How?
[If not]
Why?
O.6 What could [STATE] have done differently in the early phases of implementation to meet its goals more effectively?
P. Implementation and Operational Costs |
Matched, volunteer and donated resources. [Ask of grantee and major partners receiving grant funds]
P.1 Have any resources been used beyond the grant to support implementation of the grant?
[If so, probe:]
What was the source and how much was used?
How were these resources used?
P.2 Are volunteers involved in providing services?
[If so, probe:]
Can you estimate how much volunteer time was spent?
What was the role of these volunteers?
P.3 Were any other resources donated for the demonstration (e.g. office space, materials)?
[If so, probe:]
What were they?
Who donated them?
How much was used?
Matched, volunteer and donated resources [Ask of major partners not receiving grant funds].
P.4 What resources have been used to support implementation of the grant?
[Probe:]
How much was used?
How were these resources used?
P.5 Are volunteers involved in providing services?
[If so, probe:]
Can you estimate how much volunteer time was spent?
What was the role of these volunteers?
P.6 Were any other resources donated for the demonstration (e.g. office space, materials)?
[If so, probe:]
What were they?
Who donated them?
How much was used?
Staff time spent on [SEBTC PROGRAM NAME]. [ASK OF EACH RESPONDENT AT GRANTEE AND SUBCONTRACTORS SEPARATELY]
P.7 What percentage of your time do you typically spend on the program?
[Probe:]
Has this changed over time?
How much of your time is charged to the grant?
How much is paid by other funding sources?
P.8 Do you work overtime hours on the project?
[If so,]
How much is paid versus unpaid?
CLOSING |
Is there anything you think is important for the FNS to know about the [STATE]’s [SEBTC PROGRAM NAME] demonstration that we did not ask about?
Thank you for your time and helpful feedback. The information you have shared will be valuable to our team as we look across states and localities for themes and ideas that we can share with FNS.
Are there any questions you have for me before we finish?
1 Questions about WIC retailers will be addressed using State WIC Agency reports to the extent possible.
2 Questions about SNAP retailers will be addressed using State WIC Agency reports to the extent possible.
| File Type | application/msword |
| File Title | EVALUATION OF THE SUMMER EBT FOR CHILDREN |
| Author | Jeanne Bellotti |
| Last Modified By | hwilson |
| File Modified | 2011-03-18 |
| File Created | 2011-03-17 |