Individual and Household Responsive

Appendix F.4

WIC participant case study focus group guide

OMB #0584-XXXX

Public Burden Statement

This information is being collected to assist the Food and Nutrition Service to better understand the implementation and impact of the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC) modernization efforts on State and local WIC agencies, WIC vendors, and WIC participants. This is a voluntary collection and FNS will use the information to monitor and strengthen WIC program modernization efforts. This collection does not request any personally identifiable information under the Privacy Act of 1974. According to the Paperwork Reduction Act of 1995, an agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a valid OMB control number. The valid OMB control number for this information collection is 0584-[xxxx]. The time required to complete this information collection is estimated to average 90 minutes (1.5 hours) per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to: U.S. Department of Agriculture, Food and Nutrition Service, Evidence, Analysis, and Regulatory Affairs Office, 1320 Braddock Place, 5th Floor, Alexandria, VA 22306, ATTN: PRA (0584-xxxx). Do not return the completed form to this address.

Expiration Date: XX/XX/20XX

WIC & FMNP MODERNIZATION EVALUATION
WIC Participant Case Study Focus Group Guide

NOTE TO FACILITATOR: This protocol is a guide, not a script. It has been developed to use with WIC/FMNP participants who have been exposed to activities funded by at least one of the modernization grants. Groups will have some flexibility to focus on one modernization activity area while others may cover multiple activity areas. Which area(s) you will cover in the focus group is indicated by an X in the Topic for Focus Group column below.

Focus group facilitators will need to tailor questions and/or add probes based on the activities and services provided under the grant(s). For each focus group covering a single area, the facilitator will ask all relevant questions for that activity area. For focus groups covering multiple areas, the study team will select up to 20 questions from across the focus group guide based on the area(s) of focus.

Before starting the group, please complete the following table with the details related to the activity(ies) selected to help customize the discussion. All groups should start at Section A – Introduction, then go to the section(s) associated with the activity(ies) of interest (Sections B, C, D, or E) and all finish with Sections F and G – Closing.

WIC LOCAL AGENCY NAME: __________________________________

TYPE OF ACTIVITY AREAS	TOPIC FOR FOCUS GROUP	DETAILS ABOUT MODERNIZATION EFFORTS IN TOPIC AREA (mark each that applies to this local agency’s effort)
Outreach to eligible non-participants and communication with participants (Section B)		___ Text messaging ___ Billboards ___ Community flyers or brochures (e.g., at doctor office, health clinic, food pantry, etc.) ___Posters (e.g., in buildings, on buses, etc.) ___Your State‘s WIC website ___A healthcare provider ___Speaking with a WIC staff member ___Social media content or ads ___Digital ads online ___ Radio ads (e.g., local radio or streaming) ___TV ads ___Event in my community ___ Automated reminder messages ___ [STATE WIC APP NAME] ___ WIC branding activities to increase public awareness ___ OTHER, DESCRIBED BELOW: ____________________________________________
The Shopping experience (Section C)		___ The general WIC shopping experience ___ Shopping education ___ Checking out at the register ___ Using the self-checkout ___ Using the [WIC SHOPPING APP NAME] ___ Availability of the [WIC SHOPPING APP NAME] in another language ___ Online shopping ___ Curbside pickup when online shopping ___ Grocery delivery when online shopping ___ WIC foods meeting cultural/personal/religious needs ___ OTHER, DESCRIBED BELOW: _____________________________________
Technology enhancements for participants (Section D)		___ Online applications ___ Virtual appointment platform ___ Text messaging tool ___ AI (e.g., chatbot) ___ Participant portal ___ Electronic document sharing tool/electronic document signatures tool ___ Appointment scheduling tool ___ Loading EBT cards remotely ___ WIC Website (online web info) ___ [STATE WIC APP NAME] ___ Online materials (self-guided) ___ Webinars (live) ___ OTHER, DESCRIBED BELOW: ____________________________________________
Expanding Access to Farmers’ Markets in WIC (Section E)		___ Marketing to WIC participants about using WIC/FMNP at farmers markets’ and/or roadside produce stands ___ Using WIC CVB at farmers’ markets and/or roadside produce stands ___ Using FMNP eSolutions at farmers’ markets and/or roadside produce stands ___ Mobile certification and/or collection of anthropometric data and documents at farmers’ markets. ___ Recipes/Classes cooking foods available at farmers’ markets and/or roadside stands ___ OTHER, DESCRIBED BELOW: ____________________________________________
Supporting the WIC Workforce (Section F)	X (asked of all participants)	___ Engaging with participants ___ Empathy training ___ OTHER, DESCRIBED BELOW: ____________________________________________

A. Introduction [10 minutes]

Hello, my name is [NAME], this is my colleague [NAME], and we are from Mathematica, a research company. The WIC program asked us to help them better understand what it’s like to participate in WIC. They are working to update and modernize the program to make it easier for families to use. We are hoping to get your thoughts today about how those changes are affecting WIC participants like you.

Today’s discussion will focus on changes WIC has made around [INSERT TYPE(S) OF ACTIVITIES FROM FIRST COLUMN]. Our discussion today should take about 90 minutes. We will use the information you share to write a summary of what we have learned, but we will not use any of your names in any reports, so please feel free to talk openly about your opinions and experiences. Although we will be [audio and/or video] recording our discussion and taking notes, we will not use any names. If we use quotes in our reporting, we won’t attribute them to anyone. We value the information you share with us and want to make sure we capture it all. Only the study team will listen to the recordings, and we will destroy the recording and the notes at the end of the project.

What is shared during this discussion today should not be shared outside of this gathering. The researchers cannot guarantee confidentiality, as this confidentiality depends on all participants to keep this discussion private. There are no risks or benefits to participating in this study and focus group. We can leave a copy of this consent statement with you for your records. You can contact the survey director, Caroline Lauver, at [EMAIL] or [PHONE] if you have any questions.

As a thank you for sharing your thoughts, each of you will receive a $50 gift card. Your participation is voluntary, and you don’t have to answer any questions you don’t want to answer. The WIC benefits and services you receive will not be affected in any way by taking part in today’s discussion.

Is it okay for us to start recording the discussion? IF YES – START RECORDING

[FACILITATOR INSTRUCTION: Once recording is started, obtain verbal consent from each participant that they agree to participate in the focus group].

Before I can get started, I am going to go around the table and ask each of you if you agree to be part of today’s discussion, so I have it on the recording.

Do you consent to participate in this study and this focus group?

Housekeeping notes for today’s discussion

Great. Now I want to go over housekeeping notes:

• As I mentioned earlier, the discussion will last about 90 minutes. We won’t have any formal breaks, but please feel free to stretch or go to the bathroom if needed.

• There are no right or wrong answers to the questions we will ask. We are interested in learning each of your honest opinions. You are the experts, and we are here to learn from you.

• We would like to hear from all of you today but will ask that only one person talk at a time so the recording can pick up everyone’s comments.

• Everyone’s circumstances are different. Please be respectful of others’ opinions and experiences, even if you do not agree with them.

Participant introductions

To get things started, I’d like for each of you to tell us your first name, how many children you have and their ages (or if you are in WIC because you are pregnant), and how long you’ve participated in WIC. And lastly, what your favorite WIC food is.

[FACILITATOR INSTRUCTIONS: Go around the table and make a cheat sheet with their answers to help you target questions later in the discussion.

Move on to the appropriate section based on the topic area]

B. Outreach to Participants [45 minutes]

Okay, I’d like to start our discussion with some questions about how you learned about WIC and how they communicate with you and how you communicate with them.

Let’s begin with how you learned about WIC. Let’s start with those of you who are fairly new to WIC.

1. How did you learn about WIC in general? (START WITH THOSE WHO HAVE PARTICIPATED THE SHORTEST TIME). And what about those of you who have participated for a while? How did you first learn about WIC?

2. WIC agencies reach out to the public in different ways. Has anyone seen any advertising or outreach about WIC in the area, and if so, what did you see? (For example, text messaging, billboards, community flyers or brochures, posters, your State’s WIC website, information from a healthcare provider, social media content or ads, digital ads online, radio ads, TV ads, speaking directly with a WIC staff member, or community events).

3. ASK ABOUT SPECIFIC THINGS INCLUDED IN THEIR STATE AGENCY’S GRANT ACTIVITIES. Has anyone seen [INSERT FROM TABLE AT BEGINNING]? What did you think about it? [CONTINUE WITH ALL OUTREACH LISTED]

4. What are your thoughts about these different ways of letting people know about WIC?

5. Did the advertising or outreach influence your decision to enroll in WIC? How? What got your attention? What would you change?

6. What do you think are the best ways to reach people who could use WIC?

7. Did anyone other than WIC staff ever talk to you about WIC before you applied? Who was it? I don’t want to know anyone’s name, but just their relationship to you, like a friend, family member, nurse/doctor, or healthcare provider.

Let’s switch gears now and talk a bit about how your WIC clinic communicates with you.

1. Can you tell me some of the ways your WIC clinic reaches out and communicates with you? (IF NEEDED: Some examples might be emails, calls, texts, app messages, or other ways)

2. Can you share what works for you? And why you prefer that method? Did WIC ask you how you’d like to be contacted and what you’d like to be contacted about?

3. How many of you are getting emails from WIC (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING)? How often do you receive emails from WIC? What type of emails do you get (for example, appointment reminders, recipes, nutrition education)? What do you think about the emails?

4. What information do you wish you were receiving from WIC through email?

5. Is there anything WIC shares with you by email that you wish they would share with you in some other way? In what way would you prefer to get this information and why?

6. And what about texts? How many of you are getting texts from WIC (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING)? Are these informational texts only or are you able to respond – and do you respond? Do you like getting text messages from the WIC clinic? What kinds of information do you like getting from WIC by text and why? About how often do you receive texts from WIC?

7. What information do you wish you were receiving from WIC through text?

8. Is there anything WIC shares with you by text that you wish they would share with you in some other way? In what way would you prefer to get this information and why?

9. And what about phone? How many of you are getting phone calls or voice mail messages from WIC (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING)? Do you like getting calls and voice messages from the WIC clinic?

10. What days and time of day do you like getting calls or voice messages from the WIC clinic? How often do you typically receive phone calls from WIC?

11. What kinds of information do you wish WIC would call about?

12. Is there anything WIC shares with you by phone that you wish they would share with you in some other way? In what way would you prefer to get this information and why?

IF [STATE WIC APP NAME] CHECKED IN TABLE ASK:

1. How many of you have heard about [STATE WIC APP NAME] for your phone or tablet? Have any of you downloaded it to your phone or tablet? (SAY THE NUMBER OR PROPORTION ALOUD FOR THE RECORDING)

1. FOR THOSE WHO DON’T USE IT: Why have you decided not to use [STATE WIC APP NAME]?

2. Okay for those of you who have [STATE WIC APP NAME], I want to start by asking about how WIC communicates with you through the app. Does WIC send you messages through the app? What information do they share though these messages? How often do you get these messages?

3. What do you think of the messages from WIC through [STATE WIC APP NAME]? Can you respond to these messages through the app? Do you like this way of communicating with WIC? Why or why not?

4. Besides communication, what other kinds of information are you getting through the app?

5. Do you like receiving this type of information from WIC via the app? Why or why not?

6. What else would you like the app to tell you about, such as appointments, store locations, classes, recipes, and information on other services you may be interested in (for example, low-cost health clinics, help to stop smoking, etc.)?

7. Last question about how WIC communicates with you. Do you get the same information through multiple forms of communication? How would you describe the amount of communication from WIC – too much, too little, just right? What would be your ideal communication from WIC?

Okay let’s shift gears and talk about how you connect with staff at your local WIC clinic.

1. What do you find is the best way to reach out to your local WIC staff? For example, to schedule an appointment, ask a question about your benefits, or ask questions about a breastfeeding challenge.

2. Does the way you reach out to them change depending on what you want to discuss? How does that work? For example: What types of information do you like to tell them by phone, what by text, what by email? or the app?

3. And finally, can you talk a bit more about your experiences communicating with staff from the local WIC clinic? What they do well, and what could they do to improve your WIC experience?

END OF SECTION B GO TO SECTION F – WORKFORCE AND THEN G – CLOSING

C. Shopping Experiences [45-60 minutes]

Okay, I’d like to start our discussion with some questions about your shopping experiences both in-person and online as they apply to you when getting your WIC benefits. When we mention EBT in today’s discussion, we’re talking about WIC benefits you redeem electronically by using a card at checkout – and we only mean WIC EBT (not EBT from another program like SNAP).

1. What stores do you usually go to for your WIC groceries – for example, Walmart or Super Target, [LIST CHAIN STORES LOCAL TO THE AREA], convenience stores, or somewhere else?

2. Now I just have a few questions about the checkout process [IF THE SITE ALSO HAS SELF-CHECKOUT: Right now I want to focus on checking out at the register – we can discuss the self-checkout process in a few minutes].

3. Can you tell me about your experience checking out at the register when using your WIC benefits?

4. Could you tell me about any challenges you run into when using your WIC benefits at the register?

5. How helpful are store staff (such as the clerk running the register) with helping you address the challenges?

6. What would make it easier to redeem your benefits? Is there information about the products that you would need before the checkout to make the checkout process easier, for example?

IF SELF-CHECKOUT WITH WIC EBT CHECKED IN TABLE ASK:

Now let’s talk about self-checkout when shopping in person.

1. How many of you have ever used the self-checkout when getting WIC foods? (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING)

2. Can you tell me what your experience has been using your WIC benefits using self-checkout.

3. How does your experience using self-checkout for WIC items compare to going through a regular checkout aisle? Which do you prefer?

4. What would make it easier to use your benefits with self-checkout?

5. What do you do when you run into a problem using your benefits at self-checkout?

6. How helpful are the store staff in solving those problems?

7. For those of you who choose not to use self-checkout, can you share with us why you prefer the regular checkout over the self-checkout?

IF [WIC SHOPPING APP] CHECKED IN TABLE ASK:

1. [IF NOT ALREADY COVERED IN OUTREACH SECTION] How many of you have heard about the [WIC SHOPPING APP NAME] – which is the app used to help with shopping? (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING) How many of you used it? (SHOW OF HANDS, SAY THE NUMBER ALOUD FOR THE RECORDING)

2. How did you hear about the [WIC SHOPPING APP NAME]? How did you learn to use it?

3. For those who have heard about the [WIC SHOPPING APP NAME] but are not using it, we would like to hear why you are not using it?

4. Please tell me about your experience using the [WIC SHOPPING APP NAME] when you are shopping. What is the process like for you?

5. How often do you use the [WIC SHOPPING APP NAME] when you shop in-store?

6. Which parts of the [WIC SHOPPING APP NAME] do you use to help you prepare to shop or while you are shopping? (Probe: check benefits balance, find a WIC approved store, use the UPC scanner, check the food list, report a shopping issue to a WIC clinic, etc.) What aspects are the most helpful for preparing for shopping and shopping in-store?

7. And what things about the app could use improvements to help you shop? What changes do you think would make it more useful while you are shopping?

1. Do you ever encounter an issue where the app says an item is WIC approved but the item is denied at checkout? How often? How does this affect you when it happens?

8. Does anyone here speak a language other than English? [SHOW OF HANDS, SAY ALOUD FOR RECORDING]. What language(s) do you speak? Is the [WIC SHOPPING APP NAME] available in those languages – and have you used it in those languages? IF YES: What has been your experience using it?

IF ONLINE SHOPPING CHECKED IN TABLE ASK:

I see online shopping is available for WIC participants in your area…

1. Have you tried purchasing WIC foods using online shopping? [SHOW OF HANDS, SAY ALOUD FOR RECORDING]. What store(s) have you shopped at online for WIC foods? [PROVIDE EXAMPLES IF KNOWN WHICH LOCAL STORES OFFER ONLINE SHOPPING FOR WIC FOODS].

2. And if so, what has been your experience? How comfortable do you feel shopping for WIC foods online?

3. Was it easy to identify WIC approved foods when shopping online? What did that process look like? IF NO: Do you have any suggestions for how to make this easier?

4. Have you ever had to make changes to your online WIC order? IF YES: What did that process look like? Was it easy or hard to do?

5. What does the checkout process look like when shopping for WIC foods online? Was it easy or difficult to use different payment methods, if you needed to?

6. Have you ever run into issues shopping for WIC foods online (for example, difficulty finding WIC foods online, difficulty checking out, technical issues with the app, etc.)? How did you resolve these issues? Did you get the support you needed?

7. How often do you shop for WIC foods online?

8. How do you think shopping for WIC foods online compares to shopping in person? Does it save you time?

9. Do you get the same amount, more, or fewer of your WIC foods when you shop online compared to shopping in-person? Why?

10. What aspect of online shopping for WIC foods do you like best?

11. Have you experienced any difficulties shopping for WIC foods online? What were the difficulties and how often do you experience them when shopping online?

12. If you experienced difficulties shopping for WIC foods online, was it clear how to get support?

13. What could WIC do to make online shopping better?

14. For those who haven’t shopped for WIC foods online, what are the reasons why you haven’t tried it?

15. Are there things that WIC could do to make online shopping for WIC foods possible or more comfortable for you?

Now I would like to ask your opinion about a couple of specific aspects of the online shopping experience… [ASK SECTION BASED ON WHAT IS CHECKED IN THE TABLE]

Let’s start with Curbside Pick-Up …

1. Have you tried using curbside pickup? [SHOW OF HANDS, SAY ALOUD FOR RECORDING] If so, how often?

2. How easy or hard is it to use curbside pickup and why?

3. In the past, have you experienced substitutions with your order? If so, have you had any issues with the substituted item(s)?

4. If you ordered fruits, vegetables, or other produce, please describe whether you were pleased with the quality. Why were you/why were you not pleased with the quality? 

5. What do you like best about curbside pickup?

6. What are the biggest challenges using it, and how have you dealt with the challenges?

7. What would make curbside pickup better?

What about Grocery Delivery…

1. Have you tried using grocery delivery for your WIC foods? [SHOW OF HANDS, SAY ALOUD FOR RECORDING] If so, how often?

2. How easy or hard is it to use grocery delivery and why?

3. In the past, have you experienced substitutions with your order? If so, have you had any issues with the substituted item(s)?

4. If you ordered fruits, vegetables, or other produce, were you pleased with the quality? 

5. What do you like best about grocery delivery?

6. What are the biggest challenges about using it, and how have you dealt with the challenges?

7. What would make using grocery delivery better?

ASK ALL (IN SHOPPING TOPIC AREA)

Let’s talk about learning to shopping for WIC foods and then discuss the foods you get from WIC.

1. What type of shopping education did you receive during your first WIC certification appointment – what types of things did you learn about? Was this shopping education helpful? Why or why not?

2. Do you feel like the WIC program helped you feel ready to shop for WIC-approved foods when you first joined the program? Why or why not? What could they do differently to help you feel better prepared?

3. How many of you have been shopping for WIC-approved pretty regularly for awhile now? FOR THOSE WHO RAISED HANDS: Now that you are familiar with shopping for WIC foods, looking back what would you change about the shopping education experience to make it more useful for new WIC participants?

4. Do you receive shopping or food package education help during your recertification appointments? IF YES: How is this education helpful? IF NO: What other type of information or education would be more helpful during these recertification appointments to help you shop for WIC-approved foods?

5. How do WIC staff help you when you have a problem finding WIC-approved products in the store or encounter any other problems when shopping in the store?

6. Do you usually buy or get all the WIC foods you are eligible for in a month? IF NO: What are some reasons why you don’t?

7. We have heard from some other WIC participants that they usually buy the same foods each month. How does this compare to your experience?

8. If you buy the same foods each month, are you happy with them or would you prefer to buy a greater variety? How could WIC help you meet that goal?

9. Are you ever unable to redeem all the benefits you have? And if so, what are the reasons for that? And how often does it happen?

10. What could the WIC program do to help you use more or all of your monthly benefits?

11. How have WIC staff helped you tackle problems in the checkout line, problems with your EBT card, or other problems you run into when redeeming your benefits? What else could WIC staff do to be helpful to you?

12. How do you find WIC eligible products in the store? What is that process like for you?

13. What are some of the challenges you face when identifying WIC eligible foods? What helps you with that process? How do WIC staff help you?

14. How satisfied are you with the selection of WIC eligible items? What would you change about the foods that are available in WIC?

15. And what is your experience finding WIC approved items to meet your cultural, personal, or religious needs? Does the WIC agency help you understand which WIC foods are available that may meet your needs?

16. Thinking about all aspects of redeeming your WIC benefits when shopping in person, has your experience changed at all over the past [TIMEFRAME]? In what way?

17. Can you tell me some ways it has been a positive change? What factors led to the positive change?

18. And in what ways has it been a negative change? What factors led to the negative change?

19. In general, if you run into issues (with the app, EBT card, at check out, etc.), how do you work with WIC staff to resolve them? What does that process usually look like for you? Who do you think is responsible for solving these problems?

20. Do the stores you like/prefer to shop in accept your WIC benefits? IF NO: What stores do you like to shop at that you would like to also accept your WIC benefits?

END OF SECTION C GO TO SECTION F-WORKFORCE AND THEN SECTION G – CLOSING

D. Technology Enhancements [30-45 minutes]

Okay, I’d like to start our discussion with some questions about some technological changes WIC has made over the past few [or timeframe] months.

ASK IF ONLINE APPLICATIONS CHECKED IN TABLE

1. First, I would like to ask the folks here who have only been on WIC for a relatively short time [NAME TIMEFRAME]. Can you tell me about what your application process was like? Probe: did you apply online, virtually, and/or in person?

2. Now this question is for everyone. How many of you are aware that you can now apply to WIC online? How did you learn about this?

3. Did anyone here use an online tool to check your eligibility before you applied for WIC? If so, what was your experience like? Did the tool tell you that you are likely to be eligible? Did it help you figure out next steps to apply for WIC?

4. Who here has used the online application process to enroll in WIC? What was your experience like? Were you able to upload your documents or did you have to provide documents/finish your application in another way (for example, email, drop off at clinic). Did you have to go in person to finish your application or were you able to be certified without ever going in?

5. What worked well with using the online application?

6. What was difficult when using the online application?

7. Once you submitted your online application, how did you know for sure that WIC received it (for example, did you get a text or an email letting you know)?

8. How long after you submitted your online application did WIC staff contact you to set up your first appointment? Did this feel too soon or too long of a wait? Was your first appointment virtual or in person? Did you get to choose?

9. Has anyone tried using the online application, but given up? Can you tell me a bit about your experience? (Probe: did they apply using a mobile device or computer? Why did they give up?)

10. And for those of you who have tried or used the online application, did any of you need help from WIC staff to complete it? What kind of help did you need and were you able to easily get it?

11. Was there support that you wish you received but didn’t? Can you describe this?

12. What do you think WIC can do to make the online application easier to use?

ASK IF VIRTUAL APPOINTMENTS CHECKED IN TABLE

1. Have any of you heard about virtual WIC appointments? These appointments could be to enroll you for WIC or update your enrollment (called certification) or for nutrition education and could be a video call or phone appointment. What have you heard?

2. Do you know which types of appointments your clinic offers? For example, in-person, over video, over phone?

3. What types of activities/topics do the virtual appointments cover (for example, certification, recertification, nutrition education)? Are you still required to attend an in-person appointment to cover any remaining activities/topics?

4. Have any of you done a virtual appointment? Was it for certification, nutrition education, or something else? Were you offered a virtual appointment, or did you have to ask the clinic for one? Did you use phone, video, or something else?

5. IF HAD VIRTUAL VISIT: Did you have to submit any documents? If so, how do you submit the documents (examples, via email, text, online portal, etc.)?

6. IF HAD VIRTUAL VISIT: What did you like about the virtual appointments?

7. IF HAD VIRTUAL VISIT: What didn’t you like about the virtual appointments? Do you have any suggestions for improvements?

8. Has anyone done both in person and virtual appointments? IF YES: Can you compare your experiences with the two different types of visits? For example, what did you like better about in person appointments and what did you like better about virtual appointments when you compare the two?

9. Which do you prefer – virtual or in person – and why? Are there types of topics or activities you prefer were covered in person or virtually? What worked better in person and what worked better virtually? 

10. ASK ALL: Do you have any suggestions on how to improve virtual appointments or make them more useful for participants like you?

Now I’d like to talk about some other technology features that your local WIC clinic has put in place.

IF APPOINTMENT SCHEDULING TOOL CHECKED OFF IN TABLE:

1. Are you aware that you can now schedule your appointments online? IF YES: How did you learn about this?

2. Have you tried scheduling an appointment using the appointment scheduling tool? IF YES: If so, how often?

3. Are you able to reschedule or cancel appointments using the appointment scheduling tool? What was that experience like?

4. How easy or hard is it to use the appointment scheduling tool and why?

5. What do you like best about the appointment scheduling tool?

6. What are the biggest challenges about using it, and how have you dealt with the challenges?

7. What would make using the appointment scheduling tool better?

IF REMOTE BENEFIT ISSUANCE/ LOADING OF EBT CARDS CHECKED OFF IN TABLE:

1. Are you aware that you can now have your EBT card loaded remotely, so that you no longer have to go into the clinic to have them loaded? IF YES: How did you learn about this?

2. Have you tried getting your EBT card loaded remotely? IF YES: If so, how often?

3. How easy or hard is it to load your EBT card remotely and why?

4. What do you like best about loading EBT cards remotely as compared to going into the clinic?

5. What are the biggest challenges about getting your WIC benefits loaded remotely, and how have you dealt with the challenges?

6. What would make loading your EBT card remotely easier or better?

7. Is there anything WIC can do to make using the remote system for loading EBT cards better?

IF TEXT MESSAGING TOOL CHECKED OFF IN TABLE:

1. Are you aware if you can receive text messages from WIC? These messages could be reminders or sharing information about things like nutrition? IF YES: How did you learn about this?

2. Have you received any text messages from WIC? IF YES: If so, about how often? What types of things do they text you about?

3. Are the texts automatic/one-way messages telling you about something, or can you have a two-way conversation with someone at the WIC office via text? If you can have a conversation, what kinds of things can you talk about? (examples: appointment scheduling, nutrition advice, help shopping, breastfeeding questions)

4. Are you able to receive text messages in your preferred language? IF NO: would it be helpful for you to receive text messages in your preferred language?

5. What do you like best about text messaging from WIC?

6. What are the biggest challenges about WIC text messaging, and how have you dealt with the challenges?

7. What could WIC do to make text messaging more useful or better?

8. Does your mobile app allow for notifications from WIC? IF YES: Do you prefer receiving one-way messaging via text messaging or your mobile app or both?

ASK IF AI (e.g., CHATBOT) CHECKED IN TABLE

1. Have you heard anything about the chatbot you can use to communicate with the WIC program? This chatbot uses artificial intelligence, or AI. IF YES: what have you heard about it?

2. Have you ever used the AI chatbot? IF YES: What was that experience like? How often do you use the AI chatbot?

1. What do you like about the AI chatbot?

2. What are things WIC could do to improve the AI chatbot?

IF WIC WEBSITE (ONLINE WEB INFO) CHECKED OFF IN TABLE:

1. Have you heard anything about a WIC website where you can get information about WIC in general as well as learning information about your local WIC clinic? IF YES: How did you learn about the website?

2. Have you ever visited the WIC website IF YES: If so, about how often? What types of information were you looking for?

3. How easy or hard was it to find what you were looking for?

4. IF CHAT FUNCTION IS AVAILABLE: Did you use the chat feature on the website to talk to someone virtually? IF YES: Can you tell me a bit about your experience chatting through the website?

5. IF YOU DIDN’T USE THE CHAT FUNCTION: For those of you who didn’t use the chat function, did you know the website had a chat feature? If so, why did you not use it?

6. Do you use the WIC website for other features, such as locating a clinic nearby, checking eligibility requirements, applying online, downloading the app, finding the WIC approved food list, or something else?

7. What could WIC do to make their website more useful or better?
   And what about the chat function specifically, what could WIC do to improve the chat part of the WIC website?

ASK IF [STATE WIC APP NAME] CHECKED IN TABLE

1. Have you heard anything about [STATE WIC APP NAME]? This app could be used for things like checking your family’s food balance, tracking your upcoming WIC appointments, and finding stores that accept WIC. IF YES: What have you heard about it?

2. Have you used the [STATE WIC APP NAME]? IF YES: what was that experience like? How often do you use the [STATE WIC APP NAME]? What features do you use the most?

3. What do you like about [STATE WIC APP NAME]? What ways has it been helpful to you as a WIC participant?

4. What are things that WIC could do to improve the [STATE WIC APP NAME]?

5. Would you recommend the [STATE WIC APP NAME] to other WIC participants? Why or why not?

ASK IF PARTICIPANT PORTAL CHECKED IN TABLE

1. Have you heard anything about a WIC participant portal where you can login and manage things like your appointments or upload documents to WIC for eligibility, prescriptions, or food and formula request forms? IF YES: what have you heard about it?

2. Have you ever used the WIC participant portal? IF YES: What was that experience like? How often do you use the WIC participant portal? What features do you use the most?

3. What do you like about the WIC participant portal?

4. What are things WIC could do to improve the participant portal?

5. Have you ever had to troubleshoot technical problems when using the WIC participant portal? IF YES: How did that go? Were WIC staff able to help you?

ASK IF ELECTRONIC DOCUMENT SHARING TOOL/ELECTRONIC DOCUMENT SIGNATURES TOOL CHECKED IN TABLE

1. Have you heard anything about how WIC applicants and participants can now upload documents to WIC (rather than having to provide them in person) and/or sign documents electronically? IF YES: what have you heard about this?

2. Have you ever uploaded documents or signed documents electronically? IF YES: What was that experience like? How often have you done it?

3. What do you like about being able to upload and/or sign documents electronically?

4. What are things WIC could do to improve how WIC applicants and participants upload and/or sign documents electronically?

5. Have you ever had to troubleshoot technical problems when uploading and/or signing documents electronically? IF YES: How did that go? Were WIC staff able to help you?

ASK IF ONLINE MATERIALS (SELF-GUIDED) CHECKED IN TABLE

1. Have you heard anything about the WIC program’s self-guided online nutrition education materials – these can be used to learn more about ways to help you and your family eat healthy? IF YES: what have you heard about them?

2. Have you ever used the WIC program’s self-guided online nutrition education materials? IF YES: What was that experience like? How often do you use the self-guided online materials? Are there topics you are specifically looking for?

3. What do you like about the self-guided online materials? Were they helpful or easy to use?

4. What are things WIC could do to improve the self-guided online materials?

ASK IF WEBINARS CHECKED IN TABLE

1. Have you heard anything about the WIC program’s live webinars? IF YES: what have you heard about them?

2. Have you ever attended a live webinar through the WIC program? IF YES: What was that experience like? How often do you attend WIC live webinars? What topics are you interested in when you are choosing a live WIC webinar?

3. What do you like about the WIC live webinars?

4. What could WIC do to improve the WIC live webinars?

END OF SECTION D GO TO SECTION F-WORKFORCE AND THEN SECTION G – CLOSING

E. Farmers’ Markets [20-30 minutes]

Okay, I’d like to start our discussion with some questions about getting fresh fruits and vegetables.

1. Where do you usually shop to get fresh fruits and vegetables? Why do you shop there?

2. Have you ever shopped at a farmers’ market and/or roadside produce stand to buy fruit and vegetables?

3. IF YES: Besides food available for purchase, were there any other kinds of activities or information available at the farmers’ market and/or roadside produce stand, and what were they? (IF NEEDED: This might be things like demonstrations on how to prepare certain vegetables or recipes being handed out.)

4. About how close to where you live is the closest farmers’ market and/or roadside produce stand?

5. If you were to go, how would you get there? How easy or difficult is it, or would it be, to get there? And why?

6. What do you like best about shopping at a farmers’ market and/or roadside produce stand?

7. And what do you like least about shopping there?

8. Have you seen any marketing or advertising about farmers’ markets and/or roadside produce stands near you? What did you see?

9. Do you know if the farmers’ market and/or roadside produce stands near you accepts WIC fruit and vegetable benefits? Do you know of any farmers’ markets and/or roadside produce stands that do? How did they communicate that they accept WIC?

10. Do you know if the farmers’ market and/or roadside produce stand near you accepts seasonal WIC farmers’ market (FMNP) benefits? How did they communicate that they accept FMNP?

11. Has anyone here today bought food from a farmers’ market and/or roadside produce stand using your WIC fruit and vegetable benefits (CVB) and/or your seasonal WIC farmers’ market (FMNP) benefits? IF YES: Can you tell me about your experience doing so? [ASK ABOUT OTHERS’ EXPERIENCES.]

12. IF YES: Overall, how satisfied are you with redeeming your benefits at a farmers’ market and/or roadside produce stand? Why do you feel that way?

13. And has your experience or satisfaction with redeeming your benefits at farmers’ markets and/or roadside produce stands changed at all over the past [TIME FRAME]? And if so, in what ways? (IF NEEDED: What factors led to this change in your experience or satisfaction?)

14. How do you purchase fruits and vegetables at farmers’ markets/roadside produce stands using your WIC CVB? What was your experience like? (IF NEEDED: You might use your WIC EBT card, mobile pay, or a printed or digital QR code for instance.)

15. How do you purchase fruits and vegetables at farmers’ market and/or roadside produce stands using your seasonal FMNP benefits? What was your experience like? (IF NEEDED: You might use paper coupons, a printed or digital QR code, or an EBT card for instance.)

16. How easy or hard was it to complete the purchase?

17. What kind of challenges did you run into? And how did you deal with them?

18. What could WIC/FMNP do to make shopping at a farmers’ market and/or roadside produce stands easier for you and other participants like you?

19. I have just a few more questions related to WIC at farmers’ markets. Have any of you participated in any of the following at a farmers’ market: mobile certification, had WIC staff collect information about you or your child’s height/weight, or submitted documents to WIC staff?

20. IF YES: what was that experience like? What did you like about it? Are there any changes that could make this experience better or more helpful/convenient for you?

21. Have any of you participated in or received any recipes or cooking classes from WIC at the local farmers’ market? IF YES: what was that experience like? What did you like about it and how could it be improved?

END OF SECTION E. CONTINUE TO SECTION F-WORKFORCE AND THEN SECTION G – CLOSING

F. Workforce Issues [10-15 minutes]

Okay, I’d like to cover one last topic today, and that is about the WIC staff at your local clinic. When answering these questions, please think about both the in person and virtual interactions you’ve had with WIC staff.

1. Before we dive into the specifics, I want each of you to give me one word that describes how staff at WIC make you feel. [GO AROUND THE ROOM]

2. WIC wants participants to feel respected, welcomed, and valued, but we know every clinic and every staff person is different. Tell me about how the staff at WIC make you feel?

3. In what ways do staff at WIC make you feel welcome? What more could they do, or how could they improve?

4. And in what ways do they make you feel valued and respected? What more can they do, or how could they improve?

5. How often do you interact with WIC staff, including both in person and virtual interactions? Does this feel like too often, not often enough, or just right?

6. And what are the different ways you interact – in person, by phone, email, text, etc.? Which ways do you like best and why? Do you have any suggestions for improvement?

7. When you interact with staff, do you feel like you have their full attention? Do you have any suggestions for how these interactions with WIC staff could be improved?

8. Do you feel comfortable interacting with WIC staff and approaching them with questions?

9. Are you able to easily communicate with staff in your preferred language? Do you have any suggestions for ways this communication could be improved?

10. How do WIC staff show that they respect and understand your culture when you interact with them?

11. How do staff try to understand your needs? In what ways do they do this well and in what ways could they do better?

12. How do the nutrition education and breastfeeding recommendations from staff fit into your lifestyle and cultural preferences? If they don’t fit, how should staff change the recommendations so they fit better?

G. Closing [5 minutes]

Facilitator instructions: Conclude the focus group by thanking the participants for their time and letting them know we will be giving them a gift card before they leave the focus group to thank them for their time and participation in the focus group.

Thank you all so much for sharing your thoughts and opinions today. Your insight is invaluable, and we’re so thankful that you took the time to help us improve the WIC Program! I don’t have any further questions. Do you have anything more you’d like to share? If not, have a nice rest of your day. And thanks again.