Businesses

L. In-Depth Interview Protocol

OMB Number: 0584-NEW Expiration Date: MM/DD/20YY

ASSESSMENT OF THE ADMINISTRATIVE COSTS OF ELECTRONIC HEALTHY INCENTIVES PROJECTS

Interview Guide for Video Interviews

Introduction

Good morning/afternoon. Thank you for taking the time to talk with me today. My name is [INTERVIEWER’S NAME] and I work for Westat, a private research company based in Rockville, Maryland.

Purpose

The Food and Nutrition Service (FNS) is interested in understanding the costs of integrating SNAP nutrition incentive programs into State EBT systems. FNS is also interested in understanding how these costs are different from the costs of setting up and running incentive programs that aren’t integrated in EBT systems. FNS hired Westat to conduct a study that will collect cost data on the eHIP projects and break down these costs by what the cost is for (for example, changing EBT systems), who is incurring this cost (the state, EBT processors, retailers, or someone else), and whether this is a one-time cost of setting up the project or an ongoing cost of administering eHIP. Westat will be producing a report that discusses these costs across all three states in the project as well as estimates of the costs of nationwide implementation of eHIP and estimates of how eHIP costs compare with the costs of non-EBT integrated nutrition incentive projects.

Information to be collected

During our conversation today, we’d like to ask you more about the data that you or your organization submitted, any additional sources of funding outside the grant that you may have used for your work, any unexpected costs that may have arisen, and any lessons you’ve learned related to costs so far.

Risks and privacy

There is little risk to being part of this study. We use all data we collect only for the purposes we describe. FNS has directed all the eHIP States to participate in the cost study. While we will be comparing the costs of the eHIP projects across States, the results of this study will have no impact on the eHIP project funding or on any future funding each State receives. FNS knows that we will be interviewing individuals from all the eHIP States. However, we will not be revealing to FNS the names or specific positions of the people interviewed in each State. We will be presenting the results of these interviews in aggregate. Your name will not be linked to your responses. In our reports, we may include direct quotes, but these will be presented without the speaker’s name, position, organization, or State so that report readers should not be able to identify speakers.

Study costs and compensation

These is no cost to you to participate apart from the time you spend with us for this interview. There is no compensation. The interview should take about 60 minutes.

Voluntary participation

Your participation is entirely voluntary. Refusal to participate will not have any impact on your position, your organization, or your state. You can take a break, skip questions, say something off the record, or stop participating at any time.

Questions

If you have any questions about your rights and welfare as a research participant, please call the Westat Human Subjects Protections office at 1-888-920-7631. Please leave a message with your full name, the name of the research study you are calling about, which is eHIP, and a phone number beginning with the area code. Someone will return your call as soon as possible.

We have planned for this discussion to last about 60 minutes. Is that still okay?

With your permission, I would like to record this discussion to help us fill any gaps in our written notes. The recordings, transcripts, and any notes we have will be stored on Westat’s secure server and will be destroyed after the project is complete.

Do you have any questions? [ANSWER ALL QUESTIONS]

May I start the recording now?

[START RECORDING IF GIVES CONSENT]

Now that the recording is started, do you agree to participate? [PAUSE FOR RESPONSE]

And do you consent to be recorded? [PAUSE FOR RESPONSE]

1. Warm up and context

[IF FIRST INTERVIEW WITH RESPONDENT]

• To start, please tell me about your role at [EMPLOYER] and how long you have worked here.

• What is your role on eHIP?

  • How long have you worked on eHIP?

  • How, if at all, has your role on eHIP changed over time?

  • Do you work with any other organizations on eHIP? [IF STATE EMPLOYEE] Do you work with other State offices?

• What types of day-to-day work are you doing on eHIP right now?

[IF SECOND INTERVIEW WITH RESPONDENT]

• In our first interview, your role was [ENTER ROLE FROM FIRST INTERVIEW]. Has your role changed since that time? If so, how?

• In the last interview, you said you worked with [LIST ORGANIZATIONS/OTHER STATE OFFICES]. Do you still work with all of these? Are there other organizations or State offices that you now work with?

• What are you currently working on for eHIP?

[FIRST INTERVIEW ONLY]

[GO TO SECTION 2 FOR ALL INTERVIEWS IF THERE ARE COST DATA QUESTIONS—REFER TO QUALITATIVE INTERVIEW WORKSHEET. IF THERE ARE NO QUESTIONS, MOVE TO SECTION 3 FOR STATE EMPLOYEES AND SECTION 4 FOR EBT/TPP/RETAILERS]

2. Cost data questions

• Thank you again for the cost data [you/your organization] submitted. I wanted to walk through some of these data with you now.

[SHARE SCREEN WITH DATA; IF CANNOT SHARE, EMAIL DOCUMENTS TO RESPONDENT]

[FOR ITEMS WE HAVE QUESTIONS ON]

• We had a question on a few items…

• Could you explain, in your own words, what costs enter into this item?

  • Probe: What type of work is included in this item?

  • Probe: Does this cost include [LIST ITEM]? If not, is it included elsewhere?

  • Probe: What staff are involved in this work?

  • Probe: Is this work charged against the grant? Why or why not?

[FOR ITEMS WHERE NOTHING WAS ENTERED]

• We noticed you left this item blank. Let’s talk through an estimate now.

[BRING UP QUALITATIVE INTERVIEW WORKSHEET]

[REPORT REVIEW TAB]

• As you can see, you have fully covered estimates for [LIST ALL AREAS THAT HAVE BEEN IDENTIFIED AS FULLY COVERED]. We’d like to work through some estimates for [LIST ITEMS THAT ARE PARTIALLY COVERED AND NOT COVERED].

[FOR EACH CELL MARKED P OR U: IF SALARIES AND WAGES, GO TO TASK TIME WORKSHEET AND THEN SALARY INFORMATION. IF CONTRACTUAL EXPENSES, SUPPLIES, PRINTING, OTHER, OR INDIRECT, GO TO ODC WORKSHEET. IF OTHER FUNDING, GO TO OTHER FUNDING WORKSHEET].

Task time worksheet

• I want to talk about [ACTIVITY]. What are some of the steps that go into this activity? That is, how is this activity accomplished? [PROBE ON STEPS]

• [FOR EACH STEP] What type of worker does this step? What is their title?

• How often does this step have to be done?

• How long does it take to do this step?

• Is the time used to complete this task charged to the grant? [IF NOT] What funding is used for this task?

Salary information

• Okay, now I want to see for each of the staff we just talked about, what is their salary?

[LIST JOB TITLES FROM TASK TIME WORKSHEET]

• What is their pay rate?

• How often are they paid, and on what basis?

• What is their fringe benefit? An estimate is fine.

• What is the basis on which their fringe benefit is paid?

ODC worksheet

• Okay, finally let’s talk about the other direct costs. Here are some common types of other direct costs. Can you estimate how much each of these costs is? Maybe you have a general idea of how much these costs are over a week, a month, a quarter, or a year?

• What types of activities are these costs going to? [REVIEW LIST OF ACTIVITIES; TRY TO ALLOCATE COSTS TO ACTIVITIES]

3. General cost questions

• Were there any costs that you found unexpectedly large? Why was that?

• Were there any type of costs that you didn’t expect at all? Which ones? Why do you think that was?

• Are there any types of costs that you noticed weren’t included in the data request? [SHOW DATA THEY SUBMITTED]

  • Probe: What activities would be included in this cost?

  • Probe: Who worked on this item? We’re looking for the worker type—meaning their job title or pay classification.

    • What are their wage rates? What are their fringe benefit rates?

    • How much time did this item take?

  • Probe: Were there non-personnel costs that were not included? Please describe and provide an estimate. [USE ODC WORKSHEET]

  • Probe: Was this cost charged against the grant? Why or why not?

  • Probe: What would you estimate the total cost of this item to be?

4. Partner costs

[ONLY FOR EBT, TPP AND RETAILER RESPONDENTS]

• I’d like to go over some of the costs that you may have incurred in implementing eHIP.

  • How many hours were spent on implementing eHIP in total?

  • What would you say was an average loaded hourly rate for staff who worked on eHIP?

• Let’s talk about these costs by activity.

  • What types of systems changes have you had to implement or are you in the process of implementing for eHIP?

    • What types of staff worked or have worked on these changes? What is their approximate wage rate (loaded and unloaded)?

    • About how much time did these changes take from planning to completion? Or if the work is ongoing, how much time has it taken from planning to now?

  • What type of project management have you needed for eHIP? That is, how much oversight has been needed for eHIP as a whole?

    • What types of staff worked on this? What is their approximate wage rate (loaded and unloaded)?

    • About how much time did this take?

  • Have you had additional accounting costs for eHIP?

    • What types of staff worked on this? What is their approximate wage rate (loaded and unloaded)?

    • About how much time did this take?

  • Have you had additional customer service costs for eHIP? For example, have you had to provide additional support for [IF EBT OR TPP—STORES; IF RETAILERS—CUSTOMERS] who are participating in eHIP, compared with the support provided to those not participating in eHIP?

    • What types of staff worked on this? What is their approximate wage rate (loaded and unloaded)?

    • About how much time did this take?

[ALL SECTIONS FROM HERE ON ARE FOR ALL RESPONDENTS]

5. Other funding

• Outside of the grant, has your organization used any other source of funding for the eHIP work?

  • Probe: SNAP funding?

  • Probe: Other funding? What sources?

• [PULL UP OTHER FUNDING TAB] [FOR EACH SOURCE OF FUNDING]

  • How much other funding was used for eHIP?

  • What was this funding used for?

[FIRST AND SECOND INTERVIEW]

6. Estimating costs going forward

• Let’s think about the ongoing costs of eHIP, including customer service, settlement and accounting, project management, fraud prevention and detection, and retailer support.

  • Which of these costs do you expect to increase over time?

    • Is this due to an expansion of the project? If so, how is it expanding?

  • Probe:

  • Which of these costs do you expect to decrease over time?

  • Which costs do you expect will stay about the same?

  • What factors do you think will determine whether costs go up or down as the project continues?

• Let’s look at these costs one by one.

  • [ALL RESPONDENTS] Project management: How do you expect project management costs to change over the next six months? How do you expect project management costs to change through the end of the grant?

  • [STATE RESPONDENTS ONLY] Retailer recruiting and startup support: How do you expect retailer recruiting and startup support costs to change over the next six months? How do you expect retailer recruiting and startup support costs to change over through the end of the grant?

  • [STATE RESPONDENTS ONLY] Retailer training/TA and support: How do you expect retailer training/TA and support costs to change over the next six months? How do you expect retailer training/TA and support costs to change over through the end of the grant?

  • [STATE RESPONDENTS AND RETAILERS] eHIP promotion (this includes all communication with SNAP participants to raise awareness, understanding, readiness, and intent to use benefits): How do you expect eHIP promotion costs to change over the next six months? How do you expect eHIP promotion costs to change through the end of the grant?

  • [ALL RESPONDENTS] Customer service: How do you expect customer service costs to change over the next six months? How do you expect customer service costs to change through the end of the grant?

  • [STATE RESPONDENTS AND EBT] Settlement and accounting (this includes accounting for incentives and resolving disputes around settlement, as well as administrative costs): How do you expect project settlement and accounting costs to change over the next six months? How do you expect settlement and accounting costs to change through the end of the grant?

  • [ALL RESPONDENTS] Systems changes (this includes maintenance and trouble-shooting): How do you expect systems costs to change over the next six months? How do you expect systems costs to change through the end of the grant?

[SECOND INTERVIEW ONLY]

7. National estimates

• Thinking just about the planning for eHIP, before the project launched:

  • What costs have been higher than you expected? What made these costs higher?

  • What costs have been lower than you expected? What made these costs lower?

  • What factors helped keep implementation costs low?

    • Probe: Existing systems; prior integration efforts; experience in other projects; relationships between organizations; other

    • Probe: Sometimes things going not so well can reduce costs, like less retailer or consumer take-up than expected. Has this impacted implementation costs for you?

  • What factors may have driven implementation costs up?

    • Probe: Systems changes; administrative hurdles; project buy-in; other

• Thinking about the ongoing costs of eHIP:

  • Have costs been higher or lower than you expected?

    • What costs have been higher than you expected? What made these costs higher?

    • What costs have been lower than you expected? What made these costs lower?

  • What factors have helped keep ongoing costs low?

  • What factors may have driven ongoing costs up?

  • Have there been any actions taken to contain costs? If so, what are these?

  • Have incentive take-up/redemptions been higher or lower than you expected? What do you think has led to this?

    • Have there been any actions taken to increase incentive redemptions? If so, what are these?

• Imagine that eHIP were being expanded across [STATE].

  • What would need to be changed to expand across [STATE]?

    • Probe: Any changes to systems? To staffing?

      • How are these different than the changes that have already been made?

  • Probe for non-retailers:

    • What changes would retailers need to make?

      • How would these be different than the changes participating retailers have made?

    • What changes would be needed to expand to retailers that don’t currently participate?

      • Are these changes different for different types of retailers? Grocery stores, farmers’ markets, etc.

  • Which of these changes would be most costly, and why? Which would be least costly, and why?

    • Would these changes be more or less costly than the initial implementation costs?

      • Probe: Would the costs of expansion be proportional to the increase in scale? Would there be any savings in expansion? If so, where?

  • How would ongoing costs of eHIP differ if the project were statewide?

• Imagine that eHIP were being expanded to cover other types of eligible foods, like fluid milk.

  • Would it be possible to expand eHIP to other types of eligible foods?

    • Probe: Would it be feasible for all types of retailers? What about retailers who don’t have an integrated POS?

  • What would need to be changed to expand to other eligible foods?

    • Probe: Any changes to systems? To staffing? To training?

      • How are these different than the changes that have already been made?

    • Probe for non-retailers: What changes would retailers need to make?

      • How would these be different than the changes participating retailers have made?

    • Which of these changes would be most costly, and why? Which would be least costly, and why?

  • How do you think ongoing costs would compare if eHIP were to expand to other eligible foods?

• [FOR EBT PROCESSOR AND TPP] What changes, if any, would you need to implement eHIP in another State in which you already operate?

• [FOR NATIONAL RETAILER] What changes, if any, would you need to implement eHIP in another State in which you already operate?

[SECOND INTERVIEW ONLY]

8. Lessons learned

• [FOR STATE/EBT PROCESSOR/TPP] Imagine that another [STATE/EBT PROCESSOR/TPP] came to you for advice on implementing eHIP in a different state.

  • What would you tell them?

    • Probe: What advantages of eHIP would you mention? What disadvantages?

  • What factors would make it hard for them to implement eHIP?

  • What factors would make it easier for them?

  • What lessons have you learned in implementing eHIP so far?

• [FOR RETAILER] Imagine that another retailer similar to you came to you for advice on implementing eHIP in this state.

  • What would you tell them?

    • Probe: What advantages of eHIP would you mention? What disadvantages?

  • What factors would make it hard for them to implement eHIP?

  • What factors would make it easier for them?

  • What lessons have you learned in implementing eHIP so far?

9. [FIRST AND SECOND INTERVIEW]

Wrap-up

• Is there anything I haven’t touched on that you’d like to discuss?

Thank you so much for taking the time to speak with me today. Would it be all right if I follow up via email if I have any follow-up questions?

This information is being collected to provide the Food and Nutrition Service (FNS) with key cost information on Electronic Healthy Incentives Projects conducted by SNAP State agencies. This is a voluntary collection, and FNS will use the information to examine costs of Electronic Healthy Incentives Projects for SNAP. This collection requests personally identifiable information under the Privacy Act of 1974. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-NEW. The time required to complete this information collection is estimated to average 1.5 hours (90 minutes) per response. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to U.S. Department of Agriculture, Food and Nutrition Service, Office of Policy Support, 1320 Braddock Place, 5th Floor, Alexandria, VA 22314 ATTN: PRA (0584-NEW). Do not return the completed form to this address. If you have any questions, please contact the FNS Project Officer for this project, Kathleen Patton, at [email protected] or 703-305-2813.