Form 0920-0621 Youth Cognitive Testing Interview Guide – Users

User Protocol

Form Approved

OMB No. 0920-0621

Expiration Date: 04/30/2021

Attachment Q1: Cognitive Interview Protocol - Users

Understanding Use of E-Cigarettes among Youth

April 3, 2020

Note: The cognitive interviewer will use this protocol. Participants will see the same questions on a computer screen, but will not see any probes (shown in textboxes) or skip pattern instructions.

Welcome and thank you for participating in today’s interview.

• I work for NORC, a nonprofit research organization. This interview is being sponsored by the Centers for Disease Control and Prevention (also known as the CDC).

• During today’s interview, we will show you survey questions. We’re interested in getting your feedback on the survey questions themselves. Your feedback will help us improve the survey questions and our project.

• The interview is designed to last 60 minutes so that we have time to talk about your answers. At the end of our discussion, you will receive $40 to cover time and transportation costs.

• Anything that you say today will be kept private. Your name and other identifying information will not be used in any reports that result from our discussion today.

• We’ll be audio recording today’s discussion to help with a report we are going to write after we finish talking with everyone. Audio recordings will be stored on a password-protected computer and deleted at the end of the project.

• [My colleagues are sitting behind the mirror taking notes/a few of my colleagues are on the phone taking notes].

• I want you to know that I did not write the survey questions that we are going to look at today, so I will not be offended if you criticize them. Your honest feedback is very important. If something is confusing, it means that the writer did not do a good job writing the question and/or instructions.

INFORMED ASSENT

Before we get started, I’m going to give you an informed consent form on paper that describes in more detail what you’ll be asked to do. You can take this paper home with you. Please take a few minutes to read the form and let me know if you have any questions or concerns.

[After the participant has read the document and all questions have been answered.]

Do you agree to participate in this project?

Do you agree to be audio recorded?

[If yes, begin interview]

Let’s get started. I’m going to ask you to read out loud every question and then repeat the question using your own words. After you repeat the question, please answer it as you understand it. Please remember that we are not going to use your answers to the survey questions. Your answers will help us improve the question itself.

Nomenclature

[1]

For this first question, we will begin by looking at an image. Please look at this image while I ask you a few questions. [HAND THE PARTICIPANT APPENDIX A].

[NYTS 2020]

• What do you call these devices shown above?

• What have you heard other people call them?

• What devices do you think of when you hear electronic cigarettes or e-cigarettes?

  • What about when you hear vapes?

  • What about when you hear mods?

  • What about when you hear vape-pens?

• When you think about the devices above, what kind of substances are you thinking about?

  • Are you thinking about nicotine? Marijuana?

  • What about various flavors?

  • Other substances?

[2]

The next several questions are about electronic cigarettes or e-cigarettes, such as JUUL, SMOK, Suorin, Vuse, or blu. You also may know them as vapes, mods, e-cigs, e-hookahs, or vape-pens. For the rest of this survey, these products will be called e-cigarettes.

E-cigarettes are battery powered devices that usually contain a nicotine-based liquid that is vaporized and inhaled.

When I say “e-cigarette,” I mean any of the devices in this picture [SHOW APPENDIX A].

[NYTS 2020, adapted from INSTRUCT_3]

• What do you think of the examples of e-cigarettes– JUUL, SMOK, Suorin, Vuse, or blu?

• Would you add any other brands to the list?

• Are any other terms (rather than e-cigarettes) that we should be using?

• Is there anything unclear or confusing about the definition of an e-cigarette?

Reasons to use

[3a]

What are the reasons that you first used an e-cigarette? [Open-ended]

[NYTS 2020, adapted from Q16]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question? How did you choose your answer(s)?

• I’m going to show you a list of reasons why some people first start using e-cigarettes and ask you to select response options from the list.

[To interviewer: ask the participant to click to next page to see Q3 with response options]

[3b]

What are the reasons that you first used an e-cigarette?

1. A friend used them

2. A family member used them

3. To try to quit using other tobacco products, such as cigarettes

4. They cost less than other tobacco products, such as cigarettes

5. They were easier to get than other tobacco products, such as cigarettes

6. I’ve seen people on TV, online, or in movies use them

7. They are less harmful than other forms of tobacco, such as cigarettes

8. They were available in flavors, such as menthol, mint, candy, fruit, or chocolate

9. I could use them unnoticed at home or at school

10. I could use them to do tricks

11. I was curious about them

12. I was feeling anxious, stressed, or depressed

13. To get a high or buzz from nicotine

14. I used them for some other reason (specify: _____________________)

[NYTS 2020, adapted from Q16]

• What responses options would you pick from this list?

  • How did you choose your answer(s)?

  • What do you think of the answer choices?

  • Are any of these responses confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[4a]

What are the reasons that you currently use e-cigarettes? [Open-ended]

[NYTS 2020, adapted from Q16]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question? How did you choose your answer(s)?

• I’m going to show you a list of reasons why some people currently use e-cigarettes and ask you to select response options from the list.

[To interviewer: ask the participant to click to next page to see Q4 with response options]

[4b]

What are the reasons that you currently use e-cigarettes?

1. A friend uses them

2. A family member uses them

3. To try to quit using other tobacco products, such as cigarettes

4. They cost less than other tobacco products, such as cigarettes

5. They are easier to get than other tobacco products, such as cigarettes

6. I’ve seen people on TV, online, or in movies use them

7. They are less harmful than other forms of tobacco, such as cigarettes

8. They are available in flavors, such as menthol, mint, candy, fruit, or chocolate

9. I can use them unnoticed at home or at school

10. I can use them to do tricks

11. I am curious about them

12. Because I feel anxious, stressed, or depressed

13. To get a high or buzz from nicotine

14. I use them for some other reason (specify: _____________________)

[NYTS 2020, adapted from Q16]

• What responses options would you pick from this list? How did you choose your answer(s)?

  • What do you think of the answer choices?

  • Are any of these responses confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Nicotine perception in e-cigarettes – past 30 day use

[5]

Did any of the e-cigarettes that you used in the past 30 days contain nicotine?

A. Yes

B. No

C. Don’t know

[NYTS 2020, Q14]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• Do you know what nicotine is? If yes, what is it?

• From what you’ve heard or seen, do e-cigarettes have nicotine in them?

• How do people know if an e-cigarette does/doesn’t have nicotine in it?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

E-cigarette sources

[6]

During the past 30 days, how did you get your e-cigarettes? Select all that apply.

1. I bought them myself

2. I had someone else buy them for me

3. I asked someone to give me some

4. Someone offered them to me

5. I got them from a friend

6. I got them from a family member

7. I took them from a store or another person

8. I got them in some other way (specify: ___________________________________)

[NYTS, Q69]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• What does it mean to “get” an e-cigarette?

• How would you answer this question?

  • How did you choose your answer(s)?

  • What do you think of the answer choices?

  • Are any of these responses confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[7]

During the past 30 days, where did you buy your e-cigarettes? Select all that apply.

1. I did not buy e-cigarettes during the past 30 days

2. I bought them from another person (a friend, family member, or someone else)

3. A gas station or convenience store

4. A grocery store

5. A drugstore

6. A mall or shopping center kiosk/stand

7. A vending machine

8. On the Internet (such as a product website or store website like eBay or Facebook Marketplace)

9. Through the mail

10. Through a delivery service (such as DoorDash or Postmates)

11. A vape shop or tobacco shop

12. I don’t remember the place

13. Some other place not listed here (specify):_______________________________

[NYTS 2020, adapted from Q70]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How is “buying” an e-cigarette different than “getting” an e-cigarette?

• How would you answer this question?

  • How did you choose your answer(s)?

  • What do you think of the answer choices?

    • When you hear “vape shop,” what comes to mind? Is that different from a tobacco shop? How about a smoke shop?

    • When you hear “delivery service,” what comes to mind? Had you heard of DoorDash or Postmates before today?

  • Are any of these responses confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Flavors in e-cigarette

Were any of the e-cigarettes that you used in the past 30 days flavored to taste like tobacco, menthol, mint, clove or spice, alcoholic drinks, candy, fruit, chocolate, or any other flavor?

A. Yes

B. No

C. Don’t know

• IF YES, PROCEED TO [8]

• IF NO OR DON’T KNOW, SKIP TO [9]

[8]

What flavors were the e-cigarettes that you have used in the past 30 days? (Select one or more)

1. Tobacco

2. Menthol

3. Mint

4. Clove or spice

5. Fruit

6. Chocolate

7. Alcoholic drinks (such as wine, margarita, or other cocktails)

8. Candy, desserts, or other sweets

9. Some other flavor not listed here (Specify:

[NYTS 2020, adapted from Q60]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

    • What’s the difference between menthol and mint?

    • Have you heard of ‘tobacco’ flavored? What comes to mind when you hear ‘tobacco’ flavored?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Device type

[9]

During the past 30 days, which of the following e-cigarette product(s) did you get or buy from another person? (Select one or more)

1. A new e-cigarette device

2. A pod, cartridge, or e-liquid refill

3. A hit or a drag from another person’s e-cigarette device

4. Something else (specify:___________________________________________)

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

    • What does a “hit” or a “drag” mean to you?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Purchasing in school

[10]

Have you ever purchased an e-cigarette device, pod, cartridge, single hit, or e-liquid refill while at school or on school property?

1. Yes

2. No

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Nicotine perception in e-cigarettes – general

[11]

Do you think that e-cigarettes…

1. Never contain nicotine

2. Rarely contain nicotine

3. Sometimes contain nicotine

4. Usually contain nicotine

5. Always contain nicotine

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Social media

[12]

The next several questions ask about e-cigarettes and social media (such as YouTube, Instagram, Snapchat, Twitter, Facebook, Reddit, or TikTok).

Do not think about cigarettes or other tobacco products when answering these questions.

[New]

• Are we using the most relevant social media examples here?

• Would you add any other social media platforms to the list?

• Would you remove any from the list?

• Is there anything unclear or confusing about this text?

Do you use social media?

1. Yes

2. No

• IF YES, PROCEED TO Q13

• IF NO OR DON’T KNOW, SKIP TO Q14

[13]

When you use social media, how often do you see posts or content (pictures, videos, or text) related to e-cigarettes?

1. Never

2. Rarely

3. Monthly

4. Weekly

5. Daily

[NYTS 2020, adapted from Q106]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• What do you think about the examples of content (pictures, videos, or text)?

• Is anything missing in the examples of content?

• When the question says “posts,” are you also thinking of ads? What about sponsored content?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Use in school

The next question asks about being around other people’s e-cigarette vapor.

Do not include exposure to smoke from cigarettes or other tobacco products.

[14]

Have you ever seen anyone using an e-cigarette, such as JUUL, SMOK, Suorin, Vuse, or blu in any locations in or around your school? (Select one or more)

1. No

2. Yes, inside a school bathroom or locker room

3. Yes, inside a classroom

4. Yes, inside some other area of the school (hallway, cafeteria)

5. Yes, outside of the school, such as in the parking lot, sidewalk, sports field, or other school grounds

6. Yes, somewhere else not listed here [SPECIFY: ________________________]

[NYTS 2020, adapted from Q113]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • (if participant answers “Yes”)

    • How could you tell they were using an e-cigarette? Were they holding the e-cigarette? Did you see them take a hit or drag? Did you see the cloud? Smell something?

    • Were you thinking of someone ever using an e-cigarette?

    • Who were you thinking of when you answered this question? Were you thinking of classmates, or other people too?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Heated tobacco products

[15]

For the next question, we will begin by looking at an image. Please look at this image while I ask you a few questions. [HAND THE PARTICIPANT APPENDIX B].

[NYTS 2020]

• What do you call these devices shown above?

• What have you heard other people call them?

• What devices do you think of when you hear heated tobacco products?

[16]

The next section is about “heated tobacco products” such as iQOS, glo, and Eclipse. You may know them as heated cigarettes or “heat-not-burn” tobacco products. Heated tobacco products heat tobacco sticks (“heatsticks”) or capsules to produce a vapor. They are different from e-cigarettes, which heat a liquid to produce a vapor. When I say “heated tobacco product,” I mean any of the devices in this picture [SHOW APPENDIX B].

DO NOT THINK ABOUT E-CIGARETTES WHEN ANSWERING THESE QUESTIONS.

[NYTS 2020, adapted from INSTRUCT_9]

• In your own words, what is a heated tobacco product?

• What do you think is the difference between a heated tobacco product and an e-cigarette?

• What do you think about the examples of heated tobacco products? Is anything missing from the list?

• Is there anything confusing about this explanation? Is there anything we should change?

[17]

Before today, have you heard of “heated tobacco products”?

A. Yes

B. No

C. Don’t know/Not Sure

[NYTS 2020, Q55]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

SO/GI

The next questions ask you to describe yourself.

[18]

Which of the following best describes you?

A. Heterosexual (straight)

B. Gay or Lesbian

C. Bisexual

D. Not Sure

[NYTS 2020, Q116]

• Can you repeat the question in your own words?

• What do you think “best describes” means?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

    • Why do you think some people might say “not sure”?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[19]

Some people describe themselves as transgender when their sex at birth does not match the way they think or feel about their gender. Are you transgender?

A. No, I am not transgender

B. Yes, I am transgender

C. I am not sure if I am transgender

D. I do not know what this question is asking

[New]

• Can you repeat the question in your own words?

• What do you think about the definition of transgender? In your own words, how would you define transgender?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[CLOSING]

[20]

Were you comfortable answering these questions? Were any questions hard to answer?

[21]

Is there anything else about the survey questions that you would like to share with me?

I would like to thank you for coming here today and participating in this interview. CDC would like to thank you for sharing your opinions as they will be very useful in helping them to understand people’s reactions and thoughts about the survey questions we have talked about. CDC would like to remind you that there is no safe tobacco product, including the products we have talked about today. There are some fliers on the table that have more information. Please take one.

We appreciate your time and thoughtful comments.

APPENDIX A: E-CIGARETTE IMAGE

APPENDIX B: HEATED TOBACCO PRODUCT IMAGE

1