0920-0621 Youth Cognitive Testing Interview Guide (Non-users)

Non-User Protocol

Form Approved

OMB No. 0920-0621

Expiration Date: 04/30/2021

Attachment Q2: Cognitive Interview Protocol - Non-users

Understanding Use of E-Cigarettes among Youth

April 3, 2020

Note: The cognitive interviewer will use this protocol. Participants will see the same questions on a computer screen, but will not see any probes (shown in textboxes) or skip pattern instructions.

Welcome and thank you for participating in today’s interview.

• I work for NORC, a nonprofit research organization. This interview is being sponsored by the Centers for Disease Control and Prevention (also known as the CDC).

• During today’s interview, we will show you survey questions. We are interested in getting your feedback on the survey questions themselves. Your feedback will help us improve the survey questions and our project.

• The interview is designed to last 60 minutes so that we have time to talk about your answers. At the end of our discussion, you will receive $40 to cover time and transportation costs

• Anything that you say today will be kept private. Your name and other identifying information will not be used in any reports that result from our discussion today.

• We’ll be audio recording today’s discussion to help with a report we are going to write after we finish talking with everyone. Audio recordings will be stored on a password-protected computer and deleted at the end of the project.

• [My colleagues are sitting behind the mirror taking notes/a few of my colleagues are on the phone taking notes].

• I want you to know that I did not write the survey questions that we are going to look at today, so I will not be offended if you criticize them. Your honest feedback is very important. If something is confusing, it means that the writer did not do a good job writing the question and/or instructions.

INFORMED ASSENT

Before we get started, I’m going to give you an informed consent form on paper that describes in more detail what you’ll be asked to do. You can take this paper home with you. Please take a few minutes to read the form and let me know if you have any questions or concerns.

[After the participant has read the document and all questions have been answered.]

Do you agree to participate in this project?

Do you agree to be audio recorded?

[If yes, begin interview]

Let’s get started. I’m going to ask you to read out loud every question and then repeat the question using your own words. After you repeat the question, please answer it as you understand it. Please remember that we are not going to use your answers to the survey questions. Your answers will help us improve the question itself.

Nomenclature

[1]

For this first question, we will begin by looking at an image. Please look at this image while I ask you a few questions. [HAND THE PARTICIPANT APPENDIX A].

[NYTS 2020]

• What do you call these devices shown above?

• What have you heard other people call them?

• What devices do you think of when you hear electronic cigarettes or e-cigarettes?

  • What about when you hear vapes?

  • What about when you hear mods?

  • What about when you hear vape-pens?

• When you think about the devices above, what kind of substances are you thinking about?

  • Are you thinking about nicotine? Marijuana?

  • What about various flavors?

  • Other substances?

[2]

The next several questions are about electronic cigarettes or e-cigarettes, such as JUUL, SMOK, Suorin, Vuse, or blu. You also may know them as vapes, mods, e-cigs, e-hookahs, or vape-pens. For the rest of this survey, these products will be called e-cigarettes.

E-cigarettes are battery powered devices that usually contain a nicotine-based liquid that is vaporized and inhaled.

When I say “e-cigarette,” I mean any of the devices in this picture [SHOW APPENDIX A].

[NYTS 2020, adapted from INSTRUCT_3]

• What do you think of the examples of e-cigarettes – JUUL, SMOK, Suorin, Vuse, or blu?

• Would you add any other brands to the list?

• Are any other terms (rather than e-cigarettes) that we should be using?

• Is there anything unclear or confusing about the definition of an e-cigarette?

Health care interactions

The next several questions are about visits to a doctor, dentist, nurse, or other health professional during the past 12 months.

Have you visited a doctor, dentist, or nurse in the past 12 months?

A. Yes

B. No

C. Don’t know

• IF YES, PROCEED TO Q3

• IF NO OR DON’T KNOW, SKIP TO Q5 (NEXT SECTION)

Please think about e-cigarettes, only, when answering the next two questions.

[3]

During any of these visits to a doctor, dentist, nurse, or other health professional, were you asked if you used e-cigarettes?

A. Yes

B. No

[NYTS 2020, adapted from Q76]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• What do you think about the examples of health professionals? Are any missing from the list?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[4]

During any of these visits, were you advised to not use e-cigarettes?

A. Yes

B. No

[NYTS 2020, Q77]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Nicotine perception in e-cigarettes – general

[5]

Do you think that e-cigarettes…

1. Never contain nicotine

2. Rarely contain nicotine

3. Sometimes contain nicotine

4. Usually contain nicotine

5. Always contain nicotine

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• Do you know what nicotine is? How do people know if an e-cigarette does/doesn’t have nicotine in it?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Injunctive norms

[6]

Please complete the following sentence. My friends think that e-cigarette use is…

1. Completely acceptable

2. Mostly acceptable

3. Mostly not acceptable

4. Not acceptable

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

    • What does “friend” mean to you? How would your answer change if the questions asked if your “closest” friends or “friends you hang out with the most”?

    • What does “acceptable” mean do you?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

E-cigarette advertisements

The next question asks about e-cigarette advertisements.

Do not think about cigarettes or other tobacco products when answering these questions.

[7]

When you watch TV or streaming services (such as Netflix, Hulu, or Amazon Prime), or go to the movies, how often do you see people or characters using e-cigarettes?

A. I do not watch TV or streaming services, or go to the movies

B. Never

C. Rarely

D. Sometimes

E. Most of the time

F. Always

[NYTS 2020, adapted from Q101 and Q105]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• When you think about watching TV, what kind of devices are you thinking of? Were you also thinking about watching TB on phones, tablets, or computers?

• What do “streaming services” mean to you? What do you think about the examples of streaming services? Are there any missing from the list?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Social media

[8]

The next several questions ask about e-cigarettes and social media (such as YouTube, Instagram, Snapchat, Twitter, Facebook, Reddit, or TikTok).

Do not think about cigarettes or other tobacco products when answering these questions.

[New]

• Are we using the most relevant social media examples here?

• Would you add any other social media platforms to the list?

• Would you remove any from the list?

• Is there anything unclear or confusing about this text?

Do you use social media?

1. Yes

2. No

• IF YES, PROCEED TO Q9

• IF NO OR DON’T KNOW, SKIP TO Q10

[9]

When you use social media, how often do you see posts or content (pictures, videos, or text) related to e-cigarettes?

1. Never

2. Rarely

3. Monthly

4. Weekly

5. Daily

[NYTS 2020, adapted from Q106]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• What do you think about the examples of content (pictures, videos, or text)?

• Is anything missing in the examples of content?

• When the question says “posts,” are you also thinking of ads? What about sponsored content?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Use in school

The next question asks about being around other people’s e-cigarette vapor.

Do not include exposure to smoke from cigarettes or other tobacco products.

[10]

Have you ever seen anyone using an e-cigarette, such as JUUL, SMOK, Suorin, Vuse, or blu in any locations in or around your school? (Select one or more)

1. No

2. Yes, inside a school bathroom or locker room

3. Yes, inside a classroom

4. Yes, inside some other area of the school (hallway, cafeteria)

5. Yes, outside of the school, such as in the parking lot, sidewalk, sports field, or other school grounds

6. Yes, somewhere else not listed here [SPECIFY: ________________________]

[NYTS 2020, adapted from Q113]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • (if participant answers “Yes”)

    • How could you tell they were using an e-cigarette? Were they holding the e-cigarette? Did you see them take a hit or drag? Did you see the cloud? Smell something?

    • Were you thinking of someone ever using an e-cigarette?

    • Who were you thinking of when you answered this question? Were you thinking of classmates, or other people too?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

Heated tobacco products

[11]

For the next question, we will begin by looking at an image. Please look at this image while I ask you a few questions. [HAND THE PARTICIPANT APPENDIX B].

[NYTS 2020]

• What do you call these devices shown above?

• What have you heard other people call them?

• What devices do you think of when you hear heated tobacco products?

[12]

The next section is about “heated tobacco products” such as iQOS, glo, and Eclipse. You may know them as heated cigarettes or “heat-not-burn” tobacco products. Heated tobacco products heat tobacco sticks (“heatsticks”) or capsules to produce a vapor. They are different from e-cigarettes, which heat a liquid to produce a vapor.

When I say “heated tobacco product,” I mean any of the devices in this picture [SHOW APPENDIX B].

DO NOT THINK ABOUT E-CIGARETTES WHEN ANSWERING THESE QUESTIONS.

[NYTS 2020, adapted from INSTRUCT_9]

• In your own words, what is a heated tobacco product?

• What do you think is the difference between a heated tobacco product and an e-cigarette?

• What do you think about the examples of heated tobacco products? Is anything missing from the list?

• Is there anything confusing about this explanation? Is there anything we should change?

[13]

Before today, have you heard of “heated tobacco products”?

A. Yes

B. No

C. Don’t know/Not Sure

[NYTS 2020, Q55]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

SO/GI and PHQ-4

The next questions ask you to describe yourself.

[14]

Which of the following best describes you?

A. Heterosexual (straight)

B. Gay or Lesbian

C. Bisexual

D. Not Sure

[NYTS 2020, Q116]

• Can you repeat the question in your own words?

• What do you think “best describes” means?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

    • Why do you think some people might say “not sure”?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[15]

Some people describe themselves as transgender when their sex at birth does not match the way they think or feel about their gender. Are you transgender?

A. No, I am not transgender

B. Yes, I am transgender

C. I am not sure if I am transgender

D. I do not know what this question is asking

[New]

• Can you repeat the question in your own words?

• What do you think about the definition of transgender? In your own words, how would you define transgender?

• Is there anything unclear or confusing about the question?

• How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

[16]

During the past two weeks, how often have you been bothered by any of the following problems?

	Not at all	Several Days	More than half of the days	Nearly every day
Little interest or pleasure in doing things	A	B	C	D
Feeling down, depressed, or hopeless	A	B	C	D
Feeling nervous, anxious, or on edge	A	B	C	D
Not being able to stop or control worrying	A	B	C	D

[New]

• Can you repeat the question in your own words?

• Is there anything unclear or confusing about the question?

• Let’s look at the first row. What does “little interest or pleasure in doing things” mean to you? What might that look like?

  • How would you answer this question?

  • How did you choose your answer?

  • What do you think of the answer choices?

  • Are any of these answer choices confusing to you?

  • Do you think any response options are missing?

  • Should any responses be removed from the list?

• Let’s look at the second row. What does “feeling down” mean to you? How about feeling depressed? How about feeling hopeless?

  • How would you answer this question?

  • How did you choose your answer?

• Let’s look at the third row. What does “feeling nervous” mean to you? How about feeling anxious? How about feeling on edge?

  • How would you answer this question?

  • How did you choose your answer?

• Let’s look at the last row. What does “not being able to stop or control worrying” mean to you? Is the last row different from the third row? How?

  • How would you answer this question?

  • How did you choose your answer?

[CLOSING]

[17]

Were you comfortable answering these questions? Were any questions hard to answer?

[18]

Is there anything else about the survey questions that you would like to share with me?

I would like to thank you for coming here today and participating in this interview. CDC would like to thank you for sharing your opinions as they will be very useful in helping them to understand people’s reactions and thoughts about the survey questions we have talked about. CDC would like to remind you that there is no safe tobacco product, including the products we have talked about today. There are some fliers on the table that have more information. Please take one.

We appreciate your time and thoughtful comments.

APPENDIX A: E-CIGARETTE IMAGE

APPENDIX B: HTP IMAGE

1