Business Responsive

Appendix F.2

WIC and FMNP vendor/outlet case study interview protocol

OMB #0584-XXXX

Public Burden Statement

This information is being collected to assist the Food and Nutrition Service to better understand the implementation and impact of the Special Supplemental Nutrition program for Women, Infants and Children (WIC) modernization efforts on WIC State and local agencies, WIC vendors and WIC participants. This is a voluntary collection and FNS will use the information to monitor and strengthen WIC program modernization efforts. This collection does not request any personally identifiable information under the Privacy Act of 1974. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-[xxxx]. The time required to complete this information collection is estimated to average 60 minutes (1.0 hours) per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Service, Evidence, Analysis, and Regulatory Affairs Office, 1320 Braddock Place, 5th Floor, Alexandria, VA 22306, ATTN: PRA (0584-xxxx). Do not return the completed form to this address.

Expiration Date: XX/XX/20XX

WIC & FMNP MODERNIZATION EVALUATION
WIC & FMNP Vendor/Outlet Staff Case Study Interview Protocol

NOTE: This protocol is a guide, not a script. It has been developed to use with vendor/farmer/market staff participating in the case studies. Interviewers will need to ask the questions that are relevant to the specific vendor and tailor or add probes to further explore the responses provided.

A. Introduction [5 minutes]

Hello, my name is [NAME] and I work for [Mathematica/MEF/NPI], a firm that is conducting research in partnership with the U.S. Department of Agriculture’s Food and Nutrition Service, or FNS. FNS is working with key partners on a major national effort to modernize the Special Supplemental Nutrition Program for Women, Infants, and Children, also called WIC, and the WIC Farmers’ Market Nutrition Program (FMNP).

The WIC program asked us to help them better understand what it’s like for [WIC/FMNP] [vendor/farmer/farmers market] staff taking part in these shopping improvement activities. We are hoping to get your thoughts today on how the program’s efforts are affecting [vendor/farmer/farmers’ market] staff like you.

This interview should take about one hour, and we will give you a $50 gift card to thank you for your time. Before we start, I want to let you know that participating in this interview is voluntary. Your identity and that of your company or organization will be kept private, except as required by law. We’d like you to answer as many questions as you can, but you may refuse to answer any question and you can stop the discussion at any time. There are no risks or benefits to participating in this study and interview. We can leave a copy of this consent statement with you for your records.

We will take notes during the interview and, with your permission, record it to help inform our notes. [IF INTERVIEW IS ON A VIDEOCONFERENCE PLATFORM: we would like to audio and video record this interview]. No one outside our research team will listen to the recording or have access to the notes. We will destroy the recordings and the notes at the end of the project. We will combine the information gathered from this interview with data from other interviews and focus groups for our evaluation reports. You can contact the survey director, Caroline Lauver, at [EMAIL] or [PHONE] if you have any questions.

Do you give your permission for us to record this interview? (Yes/No) IF YES – PRESS RECORD

Do you agree to participate in this study and this interview? (Yes/No)

Do you have any questions before we start?

B. Respondent and vendor background

Interviewer instructions: Ask this set of questions for all respondents – use these questions to confirm information you may already have.

I’m going to start with some questions to learn more about your [retail outlet/farmers’ market/farmers market stand/roadside produce stand], and your job position.

1. How long has [VENDOR ORGANIZATION] been a [WIC/Farmers’ Market Nutrition Program (FMNP)] vendor/authorized outlet? Your best estimate is fine.

2. Can you confirm your official job title or position at [VENDOR ORGANIZATION]?

3. [FARMERS/MARKET STAFF ONLY] Please describe what the staffing looks like at [FARMERS’ MARKET/ROADSIDE PRODUCE STAND].

Interviewer instructions: For farmers/farmers’ market/roadside produce stand staff, please skip down to Section E. Farmers’ market shopping experience and continue the interview from there. For all other vendors, please go to Section C. Online shopping activities, or Section D. In-person shopping activities, depending on the type(s) of activities the vendor is involved in.

C. Online shopping activities

Interviewer instructions: Customize this set of questions based on the online shopping activities the vendor participates in (this will be tied to the modernization activities the associated State/local agency is implementing). Use information learned from the document review you completed before the interview. You may use some of these questions to confirm information learned through the document review.

I’d like to ask you some questions about what it was like carrying out activities that made online shopping possible and improved the experience for WIC participants in (LA SERVICE AREA).

1. Tell me about the activities you are doing to make online shopping possible for WIC participants. How are you and your staff putting these activities in place? (INTERVIEWER INSTRUCTIONS: Be sure to specifically ask about activities mentioned in the document review or other sources. These activities could include making online shopping available, increasing access to online shopping, making online shopping better for WIC shoppers, or offering online shopping with curbside pickup and payment for WIC participants.)

1. Did you carry out the activities all at once or in stages? Please describe what this looked like for each activity.

2. Have you carried out all the planned activities, or are there some activities you didn’t carry out? Please describe.

3. Looking back, what are your thoughts around your organization’s capacity to implement WIC online shopping?

2. When did you start offering online shopping and other related activities to WIC participants? (INTERVIEWER INSTRUCTIONS: please get the month and year for all activities, or a close estimate if they can’t provide the exact month.)

1. Were there changes in the timeline for carrying out these activities? What led to these changes? Please describe.

3. What is the current process for shopping for WIC products online with [VENDOR ORGANIZATION]?

1. How are products marked as WIC eligible online?

2. For online shopping, does [VENDOR NAME] do online payment and/or in-store payment?

3. Can WIC participants get delivery of the groceries they order online or do curbside pickup? How do they pick up groceries?

4. How are mixed basket transactions (which are transactions that include both WIC and non-WIC eligible items) handled in online orders and online payments?

5. How do you handle situations when a substitution of a product is needed when WIC participants are shopping for WIC products online?

6. What services or resources are available for WIC participants with disabilities using online shopping?

7. Is your online shopping service available in multiple languages? If so, which languages?

4. What has gone well with offering online shopping and other related activities to WIC participants (for example, allowing participants to redeem WIC benefits through online shopping, curbside pickup, delivery)?

1. What activities were easy to carry out and why?

2. What factors led to these successes?

3. What support did the WIC program provide to help make the activities a success?

5. What has been challenging with offering online shopping and other related activities to WIC participants?

1. How did you and your staff address these challenges?

2. How did the WIC program help you overcome these challenges (such as providing software, hardware, staff training for online shopping activities)?

6. What is needed to ensure the current WIC online shopping activities can continue into the future?

1. What additional resources or support are needed to continue to offer online shopping and other related activities?

2. What factors would be important in the decision to make WIC online shopping available in additional store locations?

7. What advice would you give to other WIC vendors interested in pursuing WIC online shopping?

D. In-person shopping activities

Interviewer instructions: Customize this set of questions based on the in-person shopping activities the vendor participated in. Use the information learned from the document review you completed before the interview. You may use some of these questions to confirm information learned through the document review.

I’d like to ask you some questions about what it was like carrying out activities meant to improve the in-person shopping experience for WIC participants within (LA SERVICE AREA) – we know that many of the WIC participants from this area shop at [FILL ADDRESS OF VENDOR LOCATION]. By in-person shopping, we mean shopping that WIC participants themselves are doing in your store (i.e., this doesn’t include shopping that may be done by store staff or third parties to fulfill online orders).

1. Tell me about the activities you are doing or participating in to improve in-person shopping for WIC participants. How are you and your staff putting these activities in place? (INTERVIEWER INSTRUCTIONS: Be sure to specifically ask about activities mentioned in the document review or other sources. For example, these activities could include allowing WIC participants to redeem their benefits using self-checkout, placing WIC items together (sometimes called a WIC aisle or a WIC store within the store), improving signage and labels for WIC-eligible items, communicating with the State or local WIC office to discuss issues that are occurring in store, and training employees about WIC and how to handle WIC related issues and provide good customer service to WIC participants at checkout.)

1. Did you carry out the activities all at once or in stages? Please describe what this looked like for each activity.

2. Have you carried out all the planned activities, or are there some activities you’d still like to carry out? Please describe.

3. What are some of the most common challenges WIC participants face when shopping in the store? How are these activities you are implementing meant to address those challenges?

2. When did you start making these changes to the in-person shopping experience? (INTERVIEWER INSTRUCTIONS: please get the month and year for all activities, or a close estimate if they cannot provide the exact month.)

1. Were there changes in your timeline for carrying out these activities? What led to these changes? Please describe.

3. How have the modernization activities you’ve been working on changed the in-person WIC shopping experience?

4. What has gone well with these in-person shopping activities?

1. What activities were easy to carry out and why?

2. What factors led to these successes?

3. What support did the WIC program provide to help make the activities a success?

5. What has been challenging with making these changes to the in-person shopping experience?

1. How did you and your staff address these challenges?

2. How did the WIC program help you overcome these challenges (such as providing funding, equipment, or policy changes such as allowing self-checkout in WIC)?

6. Are these in-person shopping activities still in place? Do these in-person shopping activities feel like they are sustainable for the long-term?

1. What can the WIC program do or what resources could it provide to help you continue to offer these activities?

2. What kinds of changes would you make to the in-person shopping activities to make them sustainable in the long term? Please describe.

7. I’d like to ask you a few questions about how [VENDOR NAME] marks items as being WIC-approved so WIC participants can easily locate them when shopping. Please describe the process for marking items as WIC-approved.

1. What is the process for keeping the WIC-approval markings up-to-date? Is this easy or difficult to carry out?

2. In your opinion, how easy or difficult is it for WIC participants to find WIC-eligible products in your store? Please describe.

3. Does [VENDOR NAME] need any assistance from the WIC program to maintain or improve the process for labeling products as WIC-approved? Please describe.

8. In your opinion, what else would improve the in-person WIC shopping experience?

   1. What resources or support from the WIC program would [VENDOR NAME] need to implement it?

E. Farmers’ market shopping experience

Interviewer instructions: Farmers’ markets/roadside produce stands could accept both WIC CVB and FMNP and could be using a different eSolution for each. If both, ensure data is documented separately for each. Use the information learned from the document review you completed before the interview to customize the interview questions. You may use some of these questions to confirm information learned through the document review.

I’d like to ask you some questions about what it was like [moving from paper coupons to an eSolution for FMNP, and/or enabling WIC CVB use] at farmers’ markets/roadside produce stands]. eSolutions are technology modernizations that enable FMNP participants to electronically redeem their FMNP benefits, and in some cases the WIC CVB, at a local farmers’ market or roadside produce stand. Enabling WIC CVB use at farmers’ markets/roadside produce stands may involve accepting a different electronic payment method and using a different point of sale device than an eSolution.

1. Do you accept WIC CVB or electronic FMNP benefits (through what we call “eSolutions” like an EBT card, a mobile app, or a QR code)? If so, when did you start accepting these eSolutions? (Interviewer – please get the month and year for all activities, or a close estimate if they cannot provide the exact month.)

2. Tell me about the activities you have completed and those you are carrying out to accept [WIC CVB /electronic FMNP benefits]. How are you [and your staff] preparing to accept or already accepting these electronic benefits? (INTERVIEWER INSTRUCTIONS: Be sure to specifically ask about activities mentioned in the document review or other sources.

1. How has [the FMNP eSolution/accepting WIC CVB] changed your experience as a WIC/FMNP authorized market or stand (for example, tracking transactions, obtaining reimbursements)?

3. What is the process for electronically transacting [WIC CVB or electronic FMNP] benefits at the farmers’ market or roadside produce stand?

1. Have you noticed an increase in WIC/FMNP participants transacting their benefits at the farmers’ market and/or roadside produce stand?

4. What has gone well with making the [FMNP eSolution/accepting WIC CVB] available at the farmers’ market and/or roadside produce stand?

1. What led to [making the eSolution/accepting WIC CVB] a success?

2. What support did the [FMNP/WIC program] provide to help make the [FMNP eSolution/accepting WIC CVB] a success?

5. What has been challenging with [using the FMNP eSolution/accepting WIC CVB] at the farmers’ market and/or roadside produce stand?

1. What types of challenges have [WIC/FMNP] participants had with electronically transacting their benefits?

2. How did you and your staff address these [WIC/FMNP] challenges?

3. How did the [FMNP/WIC program] help you overcome these challenges (for example, providing software or hardware such as iPads, iPhones, or hotspots; or providing other forms of support)?

4. What could the FMNP and/or WIC program have done to better help you address these challenges?

6. How has [the FMNP eSolution/accepting WIC CVB] changed your WIC/FMNP reimbursement process and timeline?

7. Does it feel sustainable to continue to make the [eSolution/WIC CVB] available for the long-term? What kinds of changes would you make to the [eSolution/WIC CVB] to make its use sustainable in the long term? Please describe.

F. Vendor resources, outreach, and training

Instructions for interviewers: Ask all vendor/farmer/market staff questions from this section. Customize this set of questions based on the resources, outreach, and training activities the vendor or farmer/market staff may have been exposed to. Use the information learned from the document review you completed before the interview. You may use some of these questions to confirm information learned through the document review.

I’d like to ask you some questions about your experience with the resources, outreach, and training the FMNP and/or WIC program has provided to you as staff at a WIC vendor and/or an authorized farmer/market.

1. [IF STATE HAS A WEBSITE/ONLINE SYSTEM] Do you use the [FMNP/WIC program’s] website or online system to obtain information about the [WIC/FMNP] program?

1. Is the website or system easy to use? Why or why not?

2. What type of information are you trying to find, or tasks are you trying to accomplish through the website? Are you able to find what you need? Please describe.

3. Is there information about the [FMNP/WIC program] that you need and are unable to find? Please describe.

2. [IF STATE HAS A WEBSITE/ONLINE SYSTEM] Have you used the [FMNP/WIC program’s] website or online system for [VENDOR / farmer or market NAME] to apply to become a [WIC vendor or WIC/FMNP farmer/market/roadside produce stand], or to renew that authorization? IF YES: Please describe your experience using the online system for these tasks.

1. How did you learn about the online application and renewal system?

2. Did the [FMNP/WIC program] offer any assistance on how to use the system? Please describe.

3. [IF THEY HAD PREVIOUSLY USED A DIFFERENT SYSTEM]: Is the online system easier to use, more efficient, or preferable to the paper application and renewal system? Why or why not?

3. [IF STATE HAS A WEBSITE/ONLINE SYSTEM] How easy or difficult was it to use the [FMNP/WIC program’s] website or online system to apply to become a [WIC vendor or WIC/FMNP farmer/market/roadside produce stand] or to renew [VENDOR NAME]’s that authorization? Please describe.

1. How did you overcome any challenges?

2. How can the [FMNP/WIC program] adjust the system to make it easier to use or more useful for you?

4. Please tell me about the communication from the [FMNP/WIC program] about the [online/in-person/farmers’ market/roadside produce stand] shopping activities that you carried out as part of the modernization efforts.

1. What types of information, training, and technical support did the [FMNP/WIC program] share with you about these activities and carrying them out?

2. Is there information you wish the [FMNP/WIC program] would’ve shared with you that might have improved your experience carrying out these activities? Please describe.

5. Do you feel like you had sufficient support from the [FMNP/WIC program] as you carried out these [online/in-person/farmers’ market/roadside produce stand] shopping activities? Please describe.

1. Did you have someone at the [FMNP/WIC program] who you could contact for questions or additional support?

2. How can the [FMNP/WIC program] improve its support for [vendors/farmers/farmers’ markets] staff like you?

6. Have you or your staff received any trainings from the [FMNP/WIC program] in the past [TIMEFRAME] to help you better serve [WIC/FMNP] participants? Have any of these trainings been directly related to the [online/in-person/farmers’ market/roadside produce stand] shopping activities? Please describe.

1. How many trainings were provided? How were new staff trained on [WIC/FMNP] program updates?

2. What topics did these trainings cover?

3. Were these trainings helpful? Why or why not?

4. How can the [FMNP/WIC program] improve these trainings or make them more useful?

5. Are there other topics you’d like to receive training on?

G. Closing questions

Instructions for interviewers: Ask all vendor/farmer/market staff questions from this section. Customize this set of questions based on information learned through the document review and the interview.

I’d like to close out this interview with some final questions about the experience of [VENDOR NAME] being a WIC vendor/FMNP authorized market/stand and serving WIC/FMNP participants.

1. Please describe your experience working with WIC/FMNP while implementing the modernization activities we discussed. Was this experience positive or negative and why?

2. Do you feel the [online shopping/in-person shopping/farmers’ market/roadside produce stand shopping experience] modernization activities helped you better serve WIC/FMNP participants? Why or why not?

1. How would you adjust the activities so you can better serve WIC/FMNP participants?

3. In your opinion, have the modernization activities you carried out improved WIC/FMNP participants’ shopping experience? Why or why not?

1. Were any activities more effective than others?

2. Have you heard any feedback from WIC/FMNP participants on their shopping experience?

4. Finally, is there anything else you wanted to mention about these activities that we haven’t already discussed?

Thank you for participating in this interview!