Business- Profit & Not -for- Profit

SEBTC Evaluation

September 2011 Site Visits

Interview with EBT Processors

[Note to site visitors:] Unless otherwise noted, all questions will be asked of a single respondent only. Site visitors will identify the roles of respondents during the site visit planning process and determine which respondent should address each major topic area. The site visit guide also indicates questions within each topic area that should be asked of more than one respondent.

INTRODUCTION

My name is [X], from [Abt Associates/Mathematica Policy Research].

As you may know, Abt and Mathematica are working together to evaluate the Summer EBT for Children demonstrations for the Food and Nutrition Service of USDA. My colleagues and I are visiting each of the five sites and collecting information from a wide range of stakeholders involved in the demonstration in order to learn about the process of implementation, the challenges you encountered, and lessons learned. I want to start by thanking you for taking time to speak with us today. Your perspective and insights on these issues are very helpful.

For stakeholders not receiving grant funds: Your participation is voluntary, and your responses will be kept confidential to the extent provided by law. You may refuse to answer any question and may stop the interview at any time.

Our reports to FNS will describe the range of responses expressed by staff, and may list the names of agencies and partners who contributed information, but we will not quote you or anyone by name or title. However, because of the relatively small number of organizations participating in the study, there is a possibility that a response could be correctly attributed to you.

I expect our conversation will take approximately 120 minutes.

[OPTIONAL IF INTERVIEWER CHOOSES TO RECORD:] I want to be sure I am keeping track of everything you are saying. May I record our discussion so that I can listen to it later when I write up my notes? No one outside of our research team will have access to the recording. [IF YES:] Thank you. It will be helpful if you speak up, speak clearly, and speak one at a time. [IF NO:] That’s no problem. I’ll take notes as you talk, but I may sometimes need to ask you to slow down or repeat so that I can get all the information.

First, do you have any questions for me about the project in general or what we will be discussing today?

A. BACKGROUND ON RESPONDENT

[Ask this section of all new respondents]

A.1 What organization do you work for?

A.2 What services does [ORGANIZATION] provide?

A.3 What is your position? What are your day-to-day responsibilities?

A.4 How long have you worked for [ORGANIZATION]?

A.5 When did you begin working on [SEBTC PROGRAM NAME]?

A.6 What are your specific responsibilities?

B. Demonstration Planning

Goals of the demonstration.

B.1 Since the last site visit in [MONTH], have the goals of the demonstration changed?

[If so, probe:]

- How?

- For what reasons?

- What new goals do you hope to achieve?

- To what extent are the goals quantifiable?

- How will they be measured?

C. Local Context

Food environment.

C.1 Since the last site visit, has the food environment in the demonstration area changed (such as the number of [SNAP/WIC] retailers, the types of retailers, or local access to those retailers)?

[If so, probe:]

- In what ways?

- For what reasons?

- Are more or fewer resources available to local households?

- What types?

D. Basic parameters of the summer EBT MODEL

EBT cards.

D.1 Since the last site visit, have there been changes to the approach for the distribution of EBT cards? If so, please explain.

Timeframe of EBT benefits.

D.2 Since the last site visit, have there been any changes to the schedule for issuing EBT benefits?

[If so, probe:]

- Can you explain the changes?

- Why were the changes made?

- What has been the reaction of households?

E. Project Organization and Management

Organizational structure to administer the demonstration.

E.1 Has your organization’s structure for administering the demonstration changed since the last site visit?

[If so, probe:]

- Can you please describe how?

E.2 Are any new units or subcontractors participating?

[If so, probe:]

- What are their responsibilities?

- How were they selected?

E.3 Did any units or subcontractors stop working on the demonstration?

[If so, probe:]

- Why?

- Which organizations have taken over their responsibilities?

E.4 What are the main advantages to the current organizational structure?

[Probe:]

- What aspect(s) would you recommend that other demonstration programs adopt?

- Why?

Agreements between organizations.

E.5 Has your firm’s contract with [GRANTEE] changed since the last visit?

[If so, probe:]

- How

- Why?

- Can we have copies of the new agreements?

Changes in staffing and staff turnover.

E.6 Has the staffing for [SEBTC PROGRAM NAME] changed since the last interview? How and why?

E.7 If there were temporary staff hired for the demonstration, when were/will they be terminated?

E.8 Has there been other staff turnover?

[If so, probe:]

- In what positions?

- For what reasons?

- What were the effects of this turnover?

E.9 Overall, where were the strengths and challenges of staffing for the demonstration?

Communication with grantee and key partners.

E.10 Thinking back over the course of the spring and summer, have there been any challenges to maintaining communication with key partner agencies or organizations?

E.11 In what ways and for what areas could communications be improved?

E.12 What forms of communication and collaboration have been most helpful? In what ways?

F. EBT SYSTEMS MODIFICATIONS AND OPERATIONS

EBT system modifications.

F.1 Since the last interview, what modifications were made to the EBT system or third-party processor (TPP) systems for [SEBTC PROGRAM NAME]?

[Probe:]

- Who was involved with this process?

- How long did this take?

- What activities took the most effort?

- What issues were encountered in completing implementation?

EBT system interfaces.

F.2 Since the last interview, what modifications were made to system interfaces with the [SEBTC PROGRAM NAME] eligibility or certification system?

[Probe:]

- Who was involved in this process?

- How long did this take?

- What activities took the most effort?

- What issues were encountered?

EBT system settlement and reconciliation.

F.3 Since the last interview, what changes were made to the settlement and reconciliation of the [SEBTC PROGRAM NAME] EBT system?

[Probe:]

- Who was involved in this process?

- How long did it take?

- What activities took the most effort?

- What issues were encountered?

EBT retailer coordination.

F.4 Since the last interview, what changes were made the way that EBT retailers and third party processors participate in [SEBTC PROGRAM NAME]?

[Probe:]

- Why were the changes needed?

- Who was involved in this process?

- How long did it take?

- What activities took the most effort?

- What issues were encountered?

Point of sale (POS) system.

F.5 Since the last interview, what changes were made to the POS system programming or data loads for [SEBTC PROGRAM NAME]?

[Probe:]

- If programming or data loads were required, were you able to do this remotely?

- Who was involved with this process?

- How long did this take?

- What activities took the most effort?

- What issues were encountered in completing implementation/changes?

Automated customer service system.

F.6 Since the last interview, what changes were made to the interactive voice response (IVR) system, customer service scripts, or user website for [SEBTC PROGRAM NAME]?

[Probe:]

- Who was involved with this process?

- How long did this take?

- What activities took the most effort?

- What issues were encountered in completing implementation?

- How well do the automated customer service systems (interactive voice response (IVR) system, customer service scripts, user website) meet customer needs?

Operational issues.

F.7 Since the start of operations for [SEBTC PROGRAM NAME], what operational issues have been encountered?

[Probe:]

- What parts of system operations were affected? [host, TPP, reconciliation/settlement, POS, IVR, customer service]

- How were service levels affected?

- How were the problems addressed?

- Are these problems resolved or ongoing?

- What changes are you considering for next year?

Recommendations for next year.

F.8 What changes do you recommend for the design and operations of systems for [SEBTC PROGRAM NAME] for next year, both in [SITE] and in other sites?

[Probe:]

- What parts of system design and operations should be different? [host, TPP, reconciliation/settlement, POS, IVR, customer service]

- Why are changes recommended?

- How do your recommendations vary across different sites?

G. replacing ebt cards and other participant support

I’d like to turn now to some operational questions about customer service for [SEBTC PROGRAM NAME] households, focusing on the time since our last interview.

Extent of card replacements.

G.1 During the late summer months, how often did you have to replace EBT cards that were lost, stolen, or damaged?

G.2 What were the most common reasons?

G.3 Is the rate of card replacements for households that just get [SEBTC PROGRAM NAME] benefits different from households that get [SNAP/WIC]?

[Probe:]

- Is the rate higher or lower for households that just get [SEBTC PROGRAM NAME] benefits?

- What are the reasons for this difference?

- [If there is a problem] What actions have been taken to reduce the rate of replacements?

- How has the strategy for issuing cards [one card/multiple cards per household] affected the rate of replacements?

G.4 Were there any issues with the process for replacing the cards?

G.5 Did households seem to understand the process?

G.6 How long did it take for households to receive replacement cards?

G.7 Would you make any changes to the process for the next summer?

Other participant support provided.

G.8 Were there any changes to participant supports (such as 800 numbers, local information lines, a website)? If so, how and why?

H. Expunging funds/Expiring BENEFITS

Timing and process for expunging funds – SNAP approach.

H.1 Have unused funds been expunged from accounts for [SEBTC PROGRAM NAME]? If not, when?

H.2 What process was/will be used to expunge funds from the cards?

[Probe:]

- Is this similar to the process used for SNAP when the cards are inactive?

- What happens to unused funds?

- What notification or reminders will households receive before their benefits are expunged?

Challenges with expunging funds – SNAP approach.

H.3 Has the process for expunging funds worked as anticipated?

H.4 Have there been any problems with expunging funds for the [SEBTC PROGRAM NAME] benefits?

H.5 Did you have to change the process after implementation?

H.6 Is there anything you would do differently for the next summer?

Deactivating SEBTC cards.

H.7 Will [SEBTC PROGRAM NAME] cards be deactivated? If so, when and how?

Challenges with closing out SEBT for Children accounts.

H.8 Did the process for closing out accounts work as anticipated?

[If not, probe:]

- What did not go as planned?

- What were the problems with the process?

- Were they resolved? How?

H.9 What were the responses of households?

[Probe:]

- Did you encounter any households that were unaware that their benefits were ending?

- How many?

- Did households raise any other issues?

H.10 Is there anything you would do differently for the next summer?

I. training and support for retailers

[NOTE TO VISITOR: ASK FOLLOWING QUESTIONS IF RETAILER TRAINING OR OTHER SUPPORT PROVIDED]

Retailer training.

I.1 Since the last interview, has your organization provided information/training about [SEBTC PROGRAM NAME] to retailers?

I.2 Have all [SNAP/WIC] retailers in the demonstration areas received information/training?

Type of material or training.

I.3 What was the focus of the materials or training (e.g., information about the [SEBTC PROGRAM NAME], the look of the EBT card, use of the EBT card, the new WIC package)? What type of materials or training will be provided (e.g., information packets, one-on-one, group, computer, web-based)?

I.4 Who prepared and distributed materials?

I.5 Were materials or training offered in multiple languages? Which languages are included?

I.6 Were face-to-face trainings conducted at the retailer’s location or with multiple retailers?

I.7 Who conducted the training?

I.8 How long did training classes last?

I.9 How many trainings were there?

I.10 What retailer personnel were invited to the training?

I.11 What type of information was discussed?

I.12 What notice did you provide to retailers about the end of the [SEBTC PROGRAM NAME] demonstration?

Assessment of retailer training.

I.13 What were the strengths and weaknesses of the retailer training?

I.14 How much follow-up training/support was needed?

I.15 How did the retailers respond to the training?

I.16 Did any issues arise that were not anticipated or addressed through training?

[Probe:]

- Stocking issues due to high demand

- Concerns/reactions from retailers about the new program

- Problems with cashiers recognizing and accepting cards

- Other issues?

I.17 Would you suggest making changes to the training (i.e., content, timing, length, location) for the next summer?

Additional retailer support [ASK IF PROCESSOR PROVIDES CUSTOMER SERVICE NUMBER OR OTHER SUPPORTS FOR RETAILERS].

I.18 Since the last interview, have there been any changes in the customer service hotline or other supports for retailers for SEBTC?

[Probe:]

- What changes were made?

- Why were they made?

- How did the changes affect retailer service and retailers’ ability to serve SEBTC customers?

Extent of retailer support provided.

I.91 How often do retailers call the customer service number about SEBTC? Were you able to track this?

I.20 What are the common issues they raised?

I.21 Did questions arise from these calls that you would consider including in future retailer training?

I.22 Were there any issues with the customer service process?

I.23 Would you make any changes to the process for the next summer?

J. Successes, Challenges, and Solutions

Successes.

J.1 Thinking back over the summer, what were the greatest successes of the demonstration?

J.2 What factors contributed significantly to this success?

J.3 What, if anything, could have been done differently to make the demonstration work better?

Challenges.

J.4 What were the biggest challenges to meeting the goals of the demonstration?

J.5 Were these challenges resolved?

- If so, how?

- If not, why?

J.6 What could your organization have done differently to meet demonstration goals more effectively?

J.7 How did these challenges affected the effectiveness of the demonstration?

K. Feasibility of Continuing and Replicating DemonstrationS

Conditions needed to continue the demonstration.

K.1 What conditions would need to be maintained for the demonstration to continue successfully next year?

K.2 What program components or implementation practices would need to be maintained?

K.3 What changes, if any, would need to be made in order to operate successfully next year?

Conditions needed for replication in other sites.

K.4 What state and local conditions are critical to replicate the demonstration in other areas of the country?

K.5 What program components or implementation practices are essential for replication?

K.6 Are there practices you would advise other states and local areas to avoid?

[If so, probe:]

- What are they?

- Why would you suggest avoiding them?

K.7 What advice would you give to other states and local areas interested in replication of the demonstration?

L. Operational Costs

Now I would like to turn to some questions related to the cost of operating the demonstration. I brought copies of the grant expenditure data that you provided before the site visit and would like to review them with you in detail.

Review of grant expenditures. [Review cost forms]

L.1 Can you please briefly describe the responsibilities of each staff member listed?

L.2 What staff title applies to each person (for example, senior executive, project manager, IT/MIS manager, IT/MIS developer, customer service, administrative support)?

L.3 Can you please describe what is included in the other direct costs by category?

Sufficiency of funds.

L.4 Do you believe that your organization had sufficient funds through your grant to operate the [SEBTC PROGRAM NAME] benefits successfully?

[If not, probe:]

- Why was there a gap between funds and needed resources?

- How large was this gap?

- How did you deal with this gap?

- What would you do differently if you didn’t have this gap?

Accounting systems.

L.5 Did make any changes to your accounting systems for the SEBTC grant?

[If so, probe:]

- Please describe how.

- Did these changes influence other aspects of your organization’s work?

- Will these revised procedures be used within the organization beyond the grant?

Tracking expenditures.

L.6 Did you experience any challenges in tracking grant expenditures over the course of the summer? If so, please describe.

CLOSING

Is there anything you think is important for the FNS to know about the [STATE]’s [SEBTC PROGRAM NAME] demonstration that we did not ask about?

Are your any questions you have for me before we finish?

Thank you for your time and helpful feedback. The information you have shared will be valuable to our team as we look across states and localities for themes and ideas that we can share with FNS.

END OF INTERVIEW

11