6100-082 Discussion Guide - Pretesting Outreach Materials for Fis

OMB Control Number: 2030-0051

Exp. Date: 05/31/2024

Pretesting Outreach Materials for Fish Consumption Advisories to Reach Vulnerable Populations

Discussion Guide

This collection of information is approved by OMB under the Paperwork Reduction Act, 44 U.S.C. 3501 et seq. (OMB Control No. 2030-0051. Responses to this collection of information are voluntary. An agency may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. The public reporting and recordkeeping burden for this collection of information is estimated to be 55 minutes per response. Send comments on the Agency’s need for this information, the accuracy of the provided burden estimates and any suggested methods for minimizing respondent burden to Director, Information Engagement Division, U.S. Environmental Protection Agency (2821T), 1200 Pennsylvania Ave., NW, Washington, D.C. 20460. Include the OMB control number in any correspondence. Do not send the completed form to this address.

Discussion Guide

Background and Moderator Introduction

• Hello and welcome to this focus group. Before we begin, I want to get your consent to record this discussion. I am an independent researcher. I don’t work for the sponsors and want to assure you that everything you say will be confidential and will be used for research purposes only.

  • Do I have your consent to record and to conduct the research? (Begin recording after consent is received from all participants)

• Hello and welcome again! Thanks for taking the time to talk with me today and consenting to the recording and research. My name is [placeholder to share name].

• Can you please raise your hand if you are able to see and hear me?

  • Moderator to check sound and video quality of each respondent and confirm both sound and video quality are adequate.

• Thank you for spending the next hour talking with me. I’m conducting these groups to get your reactions about communications on eating fish.

• Please keep in mind, because I am asking about your experiences and opinions, there are no right or wrong opinions. If everyone in the group has a different opinion, that is fine. I need to know every opinion.

• We will use Zoom polls during this session.

  • Moderator to describe the functionality of the poll feature and ensure participants can use the poll.

  • Moderator to share with the participants that their answers will be anonymous and are not able to be identified by other participants.

  

  Instruction to Interviewer/Moderator:

  Interviewer shares all poll questions via the Zoom online platform.

  The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

  Only the moderator can view the responses.

  
  
  

• Do you have any questions before we begin?

Before we get started looking at the images, I have a few questions for you about eating fish. For some of these questions, I will ask you to provide your responses using an online poll. Remember, only I will be able to view your response.

1. Which parts of a fish do you eat? Think about all of your fish consumption, all types of fish, all the ways fish is prepared. Using the poll feature, please check every answer that is true for you. (Poll Question 1)

   1. Fat

   2. Fillet

   3. Fins

   4. Fish steak

   5. Head

   6. Organs

   7. Skin

   8. Tail

   9. Bones

   10. Roe/fish eggs

   11. Other [specify]

   12. Prefer not to answer

2. Thank you for sharing your answers using the poll. Now, I need a show of hands, have you ever heard or seen information about which parts of a fish are safe or unsafe to eat? Please raise your hand if you have heard or seen fish safety information. Have you ever heard or seen information about which parts of a fish are safe or unsafe to eat?

MODERATOR: Looks like [moderator to provide number of respondents who stated “yes”] have seen this information.

3. If you raised your hand that you have heard or seen information about which parts of a fish are safe or unsafe to eat, where have you heard or seen this information?

4. Now, we’re going to ask you to answer another question using our online poll. Remember there are no right or wrong answers, I just need to know your thoughts. Just like the first poll, I need you to check every box that you say “yes” to. To the best of your knowledge, which parts of a fish are safe to eat? Select all that apply. (Poll Question 2)

   1. Fat

   2. Fillet

   3. Fins

   4. Fish steak

   5. Head

   6. Organs

   7. Skin

   8. Tail

   9. Bones

   10. Roe/fish eggs

   11. Other [specify]

   12. Prefer not to answer

Comprehension and Receptivity to Images with and without QR Codes:

Instruction to Interviewer/Moderator:

The two primary images (Picture_fish with QR code and Picture_fish without QR code) will be shared with participants and rotated across interviews to mitigate order effects.

The interviewer will screen share the first image in the rotation (in this case, Picture_fish with QR code) to show the participants this image.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the moderator can view the responses.

Picture_fish with QR code

Next, I’m going to show you some images. We are going to talk about each of them one by one.

5. Now that you’ve looked at this image, I’m going to ask you to take a quick poll. For this poll, I need you to choose only ONE answer. How much do you agree or disagree with this statement? (Poll Question 3)

This image is useful to me.

1. Strongly disagree

2. Disagree

3. Neither disagree nor agree

4. Agree

5. Strongly agree

6. Thanks for completing the poll. Now I’m going to ask each of you to talk to me more about this image. Let’s begin with this image: Picture_fish with QR code. Tell me, what do you think about this image?

MODERATOR PROBE: Is this an image that would catch your attention? If yes, tell me what catches your attention?

If no, why would this image not catch your attention? If it didn’t catch your attention, did you notice it?

What would make it catch your attention?

Who else has thoughts on this? Who feels this would catch their attention? Who feels it would not?

Let’s talk about the information on this image – some of you agreed this information was useful and others did not. Who would like to tell me why this information is useful? What in particular is useful?

MODERATOR PROBE:

If all participants say the information on this image is useful: Let’s talk about the information on this image. All of you agreed this information was useful. Who would like to tell me why this information is useful? What in particular is useful?

If all participants say the information on this image is not useful: Let’s talk about the information on this image. All of you agreed this information was not useful. Who would like to tell me why this information was not useful? What in particular is not useful?

Is this new information? Did you learn something new? Please raise your hand to answer.

Tell me more about this – What was new information? How would you use that information?

Is there any information you feel is missing?

7. What information do you think this image is trying to communicate? Who agrees with the statement that participant “X” gave?
   
   
   

Is there any other information you think the image is providing? Who agrees with the statement that participant “X” gave?

MODERATOR PROBE: Is there anything that someone might find confusing? Tell me more about that.

8. Okay, now let’s talk about some specific areas of this image. First, the fish parts diagram at the top of the image. What parts are being pointed out as safe to eat?

MODERATOR PROBE: Let’s have a show of hands [Moderator names the following fish parts: Fat; Fillet; Fins; Fish steak; Head; Organs; Skin; Tail; Bones; Roe/fish eggs; Other parts specified in participants’ answers to Question 4] is safe. Who agrees that is being pointed out as safe to eat?

Repeat above for various fish parts people say are safe to eat.

What should someone do with the other parts? Show of hands, who agrees that this is what should be done with the other parts?

Are there any other thoughts about the parts being pointed out?

9. Now let’s look at the section under the fish diagram. Do you notice any symbols? (Reminder to moderator: If QR code is mentioned as a symbol, note that we will talk about the QR code in a few minutes and ask if any other symbols stand out.)

MODERATOR PROBE: I’m hearing the knife and fork are a symbol, what do you think those symbols mean? Does anyone else have a different meaning for the knife and fork? Tell me about that.

Some of you mentioned the trashcan, what do you think that symbol means?

Does anyone else have a different meaning for the trashcan? Tell me about that.

10. Now let’s just talk about the colors used on this image. What do those colors mean to you?

MODERATOR PROBE: Orange/red, tell me more. How would you describe its’ meaning? How would you describe this color – orange, red?

Blue/green, tell me more. How would you describe its’ meaning?

11. We’ve talked about many things on this image. Is there anything you like?

MODERATOR PROBE: If yes, tell me why you like that? Does anyone have a thought on [insert language used by participant]? Tell me more.

12. I hear what you like, is there anything you dislike?

MODERATOR PROBE: If yes, tell me why do you dislike that? Does anyone else have a thought on [insert language used by participant]? Tell me more.

13. If you were to change anything about this image to make it more clearly communicate the parts of a fish that are safe to eat, what change would you make?

MODERATOR PROBE: Who else would make that change? Any other changes?

Instruction to Interviewer/Moderator:

The two primary images (Picture_fish with QR code and Picture_fish without QR code will be shared with participants and rotated across interviews to mitigate order effects.

The interviewer will screen share the second image in the rotation (in this case, Picture_fish without QR code) to show the participants this image.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Picture_fish without QR code

Who would not make any changes? Why?

14. Now I’m going to show you another version of this image: Picture fish without QR code. On this image, the QR code has been removed. How do you feel about removing the QR code?

MODERATOR PROBE: Does removing the QR code change the meaning of this image in any way?

If yes, how does it change the meaning?

Instruction to Interviewer/Moderator:

The interviewer will screen share the next image (Pictures_fish with QR code_ VS._without QR code) to show the participants this image.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Pictures_fish with QR code_ VS._without QR code

If no, tell me why removing the QR code doesn’t change the meaning?

15. Now I’m going to show both versions of this image side by side (Picture_fish with QR code and Picture_fish without QR code_side by side). You can see a QR code has been added. How do you feel about adding a QR code?

MODERATOR PROBE: Does adding the QR code change the meaning of this image in any way?

If yes, how does it change the meaning.

If no, tell me why adding the QR code doesn’t change the meaning?

16. Raise your hands if you prefer the image WITH the QR code. Raise your hands if you prefer the image WITHOUT the QR code.

MODERATOR PROBE: if someone did not raise their hand – I may have missed your preference. Do you prefer the image with or without the QR code?

If you preferred the image with the QR code, tell me why? If you preferred the image without the QR code, tell me why the image is better without the QR code?

Instruction to Interviewer/Moderator:

The two primary images (Language_Parts tend to have and Language_Parts generally have) will be shared with participants and rotated across interviews to mitigate order effects.

The interviewer will screen share the first image in the rotation (in this case, Language_Parts tend to have) to show the participants this image.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Language_Parts tend to have

Now I’m going to show you another image (in this case, Language_Tend to have) except this time, I want to focus just one section, that’s why the rest of the image is greyed out. The part that is greyed out is the same as what we’ve just discussed.

Now, we’re going to just focus on the wording in the orange/green section below the image of the fish.

17. What’s your first reaction when you read the wording?

MODERATOR PROBE: Tell me more about that reaction. Did anyone else have that reaction? A different reaction?

Instruction to Interviewer/Moderator:

The two primary images (Language_Parts tend to have and Language_Parts generally have) will be shared with participants and rotated across interviews to mitigate order effects.

The interviewer will screen share the second image in the rotation (in this case, Language_Parts generally have) in rotation to show the participants this image.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Language_Parts generally have

18. What’s your first reaction when you read the wording below the image of the fish?

Instruction to Interviewer/Moderator:

Interviewer shares the image (Language_Parts tend to have_VS._Parts generally have) with participants.

Moderator asks Question 19 and shares they will read the text beneath both images (Language_Parts tend to have_VS._Parts generally have) out loud and invites participants to read along silently. Moderator asks the participants to enter their responses directly into poll on their tablet or computer. The response to the question will show each version of the text and the participant will select the one that they prefer. Moderator follows up by asking the group why they prefer the text they chose and captures responses by taking qualitative notes.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Language_Parts tend to have_VS._Parts generally have

MODERATOR PROBE: Tell me more about that reaction. Did anyone else have that reaction? A different reaction?

19. Vote for which wording you like better. Use the labels on the image to vote for either image: Language_Tend to have or Language_Generally have (Poll Question 4)

1. Image with “Tends to have” language (image labeled Language_Tend to have)

2. Image with “Generally have” language (image labeled Language_Generally have)

20. Who wants to tell me which wording you prefer? Tell me more about why you prefer that wording.

MODERATOR PROBE: Who has a different preference? Remember everyone’s opinion is a good opinion. Yes, tell me why you prefer that wording.

Instruction to Interviewer/Moderator:

Interviewer shares image (Language_Research_VS._Studies) with participants.

Moderator asks Question 21 and shares they will read the text beneath body sides of the image (Language_Research_VS._Studies) out loud and invites participants to read along silently. Moderator asks the participants to enter their responses directly into poll on their tablet or computer. The response to the question will show each version of the text and the participant will select the one that they prefer. Moderator follows up by asking the group why they prefer the text they chose and captures responses by taking qualitative notes.

Only the Moderator can view the responses.

Language_Research_VS._Studies

21. Vote for which wording you like better. Use the labels on the image to vote for either image with “Studies” language and Diagram with “Research” language. (Poll Question 5)

1. Image with “Research” language (image labeled Language_Research)

2. Image with “Studies” language (image labeled Language_Studies)

22. Who wants to tell me which wording you prefer? Yes, tell me more about why you prefer that wording.

MODERATOR PROBE: Who has a different preference? Remember everyone’s opinion is a good opinion. Yes, tell me why you prefer that wording.

Instruction to Interviewer/Moderator:

Interviewer shares the image (Language_Tend to have_VS._Generally have_VS._ Research_VS._Studies) with participants.

The interviewer reads each question aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Language_Tend to have_VS._Generally have_VS._Research_VS._Studies

Final Preference:

23. On your screen, you see all four wording options we’ve looked at. To help you remember the differences are circled in for each. Vote for which one option is the easiest to understand of all four options. (Poll Question 6)

1. Image with “Tends to have” language (image labeled Language_Tend to have)

2. Image with “Generally have” language (image labeled Language_Generally have)

3. Image with “Research” language (image labeled Language_Research)

4. Image with “Studies” language (image labeled Language_Studies)

24. And last poll, which one option is the hardest to understand of all four options. (Poll Question 7)

1. Image with “Tends to have” language (image labeled Language_Tend to have)

2. Image with “Generally have” language (image labeled Language_Generally have)

3. Image with “Research” language (image labeled Language_Research)

4. Image with “Studies” language (image labeled Language_Studies)

25. Who wants to tell me why one of those options was the easiest or hardest to understand? Do any of these wordings change the way you think about which parts of the fish are safe to eat?
    
    
    

MODERATOR PROBE: If yes, which wording? Tell me what changed in terms of your way of thinking about which parts that are safe to eat?

Is that change important?

What does anyone else have to say? Is there a change?

Instruction to Interviewer/Moderator:

Interviewer shares the image (Language_assessed_VS._evaluated) with participants.

The interviewer reads questions 26 and 27 aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

`

Language_assessed_VS._evaluated

26. What is your first reaction when you read the wording in the gray boxes?

MODERATOR PROBE: Tell me more about that reaction. Did anyone else have that reaction? A different reaction?

27. Vote for which wording you like better and share why after everyone has voted. Use the labels on the image to vote for either image with “assessed” language and image with “evaluated” language. (Poll Question 8)

1. Image with “assessed” language (image labeled Language_assessed)

2. Image with “evaluated” language (image labeled Language_evaluated)

Instruction to Interviewer/Moderator:

Interviewer shares the image (Language_Little_VS._Limited) with participants.

The interviewer reads question 28 aloud to the participants. For questions asked via online poll, the participants enter their response directly into their tablet or laptop computer. For all other questions, the moderator will capture the responses qualitatively.

Only the Moderator can view the responses.

Language_Little_VS._Limited

28. Vote for which wording you like better and share why after everyone has voted. Use the labels on the image to vote for either image with “Little” language and image with “Limited” language. (Poll Question 9)

3. Image with “Little” language (image labeled Language_little)

4. Image with “Limited” language (image labeled Language_limited)

Information Sources/Channels:

Interviewer/Moderator Instruction

Read questions 29 and 30 aloud and capture responses by taking qualitative notes. Participants will not be asked to use the online Zoom poll in this section.

You are doing great. I just have a few more questions. Thinking about the images that we discussed today,

29. Where would you expect to see information like this?

30. What would be a good way to make sure that people who prepare and eat fish see this information?

1. For freshly caught fish
   MODERATOR PROBE: Tell me why this is the best way for freshly caught fish?

2. For store bought fish
   MODERATOR PROBE: Tell me why this is the best way for store-bought fish?

Interviewer/Moderator Instruction

Moderator asks Questions 31-35. Moderator asks the participants to enter their responses directly into the tablet or computer. Interviewer captures responses to Questions 36 and 37 via qualitative notes.

31. Before participating in this project, had you ever heard of the Environmental Protection Agency or EPA?

1. Yes

2. No [Skip to Question 27]

3. Don’t know

4. Prefer not to answer

32. How much do you disagree or agree with this statement: I trust the EPA as a source of health information?

1. Strongly disagree

2. Disagree

3. Neither disagree nor agree

4. Agree

5. Strongly agree

33. What people or organizations would you trust to provide health information about eating freshly caught fish?

34. What federal, state, local or tribal governmental agencies do you trust to provide information about the health impacts to people of eating fish?

35. I trust information about the health impacts of eating fish provided by the government agency or agencies mentioned in the previous question.

1. Strongly disagree

1. Disagree

2. Neither disagree nor agree

3. Agree

4. Strongly agree

You are finished with the interview. Thank you for your time. Your feedback has been very helpful.

EPA Form: 6100-082 6