Testing Message Evaluation Measures Using Youth-focused Vaping Prevention Messages

OMB# 0910-0810

Exp. 12/31/2024

U.S. Food and Drug Administration

Center for Tobacco Products

Message Testing Screener and Survey Instrument

PARENTAL PERMISSION

[INTRO] Thank you for your interest in this teen survey. To get started, are you willing to answer a few questions to see if your child is qualified to participate in the study?

1. Yes

2. No

[IF INTRO≠1, TERMINATE AND DISPLAY]

Thank you again for your time.

Are you a parent or guardian of at least one child who lives in your household who is…?

[USE SCROLLING LIST]

1. Yes

2. No

[P1] Younger than 8 years old?

[P2] 8 years old through 12 years old?

[P3] 13 years old through 17 years old?

[IF P3≠ 1, TERMINATE AND DISPLAY]

We’re sorry, but you are not eligible to participate in this study.

Thank you for your time.

[INSERT PARENTAL PERMISSION]

Do you agree to allow your child (aged 13-17) to take the survey?

1. Yes, I agree to allow my child to take the survey [GO TO P_INTRO]

2. No, I do not agree to allow my child to take the survey [TERMINATE AND GO TO END]

P_INTRO

It is important that your child be allowed to answer the questions in privacy. From this point on, your child should be able to read and answer all questions on his or her own. Press “Continue” when your child is ready to begin. [GO TO YOUTH ASSENT]

END

Thank you for your time.

YOUTH ASSENT

[INSERT YOUTH ASSENT]

Do you agree to participate in this study?

1. Yes, I agree to participate in this study. [GO TO SCREENER]

2. No, I do not agree to participate in this study. [TERMINATE AND GO TO END]

END

Thank you for your time.

[Disable back button]

SCREENER

[QUESTIONS IN SCREENER CANNOT BE SKIPPED]

Thank you for your interest in this survey. We have asked your parent to give you privacy to answer the survey questions. Please be sure you are in a private and comfortable place before you begin answering the questions below.

To get started, we first need to ask you a few questions to see if you are eligible to take the survey.

[ASK ALL]

[AGE] How old are you (in years)?

_________________years old [NUMERIC FIELD, WHOLE NUMBERS ONLY, MIN=3, MAX = 110]

[TERMINATE IF AGE < 13 OR > 17]

[VAPE_INTRO] These next few questions are about vaping products or vapes. You may also know them as electronic cigarettes, e-cigarettes, vape pens or mods. Some common brands are JUUL, Vuse, NJOY, Bidi Stick, Suorin, SMOK, and Puff Bar.

Please do NOT include vapes with marijuana, THC, or CBD when answering these questions.

[ASK ALL]

[EVER_VAPE] Have you ever vaped, even one time? [SOURCE: FDA EXPECTT SURVEY]

1. Yes

2. No

[ASK if ever_vape = 1]

[curr_VAPE] During the past 30 days, on how many days did you vape? [Adapted from path wave 6]

___________days [NUMERIC, WHOLE NUMBERS ONLY. RANGE = 0-30]

[ASK if ever_vape = 1]

[VAPE_NIC] When you vape, do you usually vape nicotine?

1. Yes

2. No

3. Don’t know

[ask IF CURR_VAPE = 0]

[Yr_VAPE] Do you think you will vape in the next year? [adapted from STRONG ET AL., 2015]

1. Definitely yes

2. Probably yes

3. Probably not

4. Definitely not

[ask if curr_vape = 0]

[Soon_VAPE] Do you think that you will try vaping soon? [adapted from STRONG ET AL., 2015]

1. Definitely yes

2. Probably yes

3. Probably not

4. Definitely not

[ask IF CURR_VAPE = 0]

[Friend_VAPE] If one of your best friends were to offer you a vaping product, would you try it? [adapted from STRONG ET AL., 2015]

1. Definitely yes

2. Probably yes

3. Probably not

4. Definitely not

[ask IF CURR_VAPE = 0]

[CURIOSITY] Have you ever been curious about vaping? [from STRONG ET AL., 2015]

1. Definitely yes

2. Probably yes

3. Probably not

4. Definitely not

[ASK ALL]

[SMK_100] This next question is about cigarette smoking. Have you smoked at least 100 cigarettes in your entire life?

1. Yes

2. No

[ASK all]

Which of these best describes your racial and/or ethnic background? (Select all that apply)

[RACE_ETH_1] American Indian or Alaska Native

[RACE_ETH_2] Asian

[RACE_ETH_3] Black or African American

[RACE_ETH_4] Hispanic, Latin(a/o), Latinx

[RACE_ETH_5] Native Hawaiian or other Pacific Islander

[RACE_ETH_6] White

[RACE_ETH_PNTA] Prefer not to answer [EXCLUSIVE]

[ASK IF RACE_ETH_4 = 1]

[HISPANIC_orgn]

Which of the following best describes you? (Select all that apply)

1. Mexican/Mexican American/Chicano/a

2. Puerto Rican

3. Cuban

4. Salvadoran

5. Dominican

6. Other Hispanic, Latin(a/o), Latinx origin or ancestry (Specify:________________)

7. Prefer not to answer [EXCLUSIVE]

[ASK all]

[gender] What is your gender identity? (Select all that apply) [SOURCE: NEW ITEM]

1. Woman/girl

2. Man/boy

3. Non-binary or gender non-conforming

4. Transgender woman/girl

5. Transgender man/boy

6. Another gender identity

7. Prefer not to say

[ASK ALL]

[STATE] Please indicate your state of residence.

[INSERT LIST WITH 50 US STATES + DC]

[TEST_SND] To take part in this survey, you must have sound enabled on your device. What sound is being played? Note: you may need to adjust your volume.

[INSERT CLASSICAL MUSIC AUDIO; RANDOMIZE RESPONSE OPTIONS]

1. Cat

2. Dog

3. Cow

4. Bird

5. Horse

6. Car horn

7. Airplane

8. Telephone

9. Classical music

10. None/couldn’t hear sound [ANCHOR]

[TEST_VID] To take part in this survey, you must be able to watch videos on your device. What animal do you see in this video?

[INSERT SQUIRREL VIDEO; RANDOMIZE RESPONSE OPTIONS]

1. Cat

2. Dog

3. Cow

4. Bird

5. Horse

6. Tiger

7. Bear

8. Lion

9. Squirrel

10. None/couldn’t see video [ANCHOR]

ELIGIBILITY AND QUOTAS

TERMINATE IF:

• YR_VAPE = 4 AND SOON_VAPE = 4 AND FRIEND_VAPE = 4 OR

• SMK_100 = 1 OR

• TEST_SND ≠ 9 OR

• TEST_VID ≠ 9

CATEGORIZE RESPONDENTS ACCORDING TO THE CRITERIA IN TABLE 1. GENERATE VARIABLE FOR TOBACCO USE GROUP (TOBGRP)

TERMINATE IF RESPONDENT DOES NOT MEET CRITERIA FOR ANY GROUP LISTED IN TABLE 1.

TABLE 1. AGE AND TOBACCO USE GROUPS

TOBACCO USE GROUP	CRITERIA
Susceptible Non-Triers	EVER_VAPE = 2 AND [(YR_VAPE = 1, 2, OR 3) OR (SOON_VAPE = 1, 2, OR 3) OR FRIEND_VAPE = 1, 2, OR 3)]
Susceptible Lifetime USERS	EVER_VAPE = 1 AND CURR_VAPE = 0 AND [(YR_VAPE = 1, 2 OR 3) OR (SOON_VAPE = 1, 2, OR 3) OR (FRIEND_VAPE = 1, 2, OR 3)]
Current USERS	CURR_VAPE > 0

QUOTAS:

MONITOR AND SET SOFT QUOTA TARGETS FOR THE FOLLOWING:

• MAX 65% WHITE, NON-HISPANIC (RACE_ETH_6 = 1 AND ALL OTHER RACE_ETH_* CATEGORIES ≠ 1)

• MAX 65% WOMAN/GIRL (GENDER_1 = 1 AND ALL OTHER GENDER_* CATEGORIES ≠ 1)

• EQUAL DISTRIBUTION ACROSS 4 U.S. REGIONS (MAX 40% IN ANY ONE REGION)

• EQUAL DISTRIBUTION ACROSS AGES 13, 14, 15, 16, 17 (MAX 35% IN ANY ONE AGE)

[IF TERMINATE, DISPLAY]

We’re sorry, but you are not eligible to participate in this study.

Thank you for your time.

[IF ELIGIBLE, DISPLAY INTRO]

[INTRO] Thank you for answering those questions. Based on your responses, you are eligible to participate in this survey. Please go to the next screen to begin.

[INCLUDE THE STATEMENT BELOW IN SMALLER GREY FONT AT THE BOTTOM OF THE END OF THE SCREENER]

Paperwork Reduction Act Statement: The public reporting burden for this information collection has been estimated to average 3 minutes to complete this screener (the time estimated to read and respond to questions). Send comments regarding this burden estimate or any other aspects of this information collection, including suggestions for reducing burden, to [email protected].

[ALL QUESTIONS FROM THIS POINT ON WILL ALLOW SKIPS. IF A PARTICIPANT SKIPS A QUESTION, THE FOLLOWING POP-UP REMINDER WILL SHOW: “There is 1 unanswered question on this page. Would you like to continue?” Options include: “Continue without answering” or “Answer this question”.]

SECTION 1: ACTUAL MESSAGE EFFECTIVENESS (WAVE 1)

[THIS SECTION WILL BE REPEATED IN SECTION 5. GENERATE VARIABLES WITH SUFFIX TO DENOTE WAVE (E.G. “GOOD_1” = ITEM “GOOD” FOR WAVE 1; “GOOD_2” = ITEM “GOOD” FOR WAVE 2]

[RANDOMIZE ORDER OF ITEMS IN THIS SECTION. RANDOMIZE WITHIN EACH SCROLLING MATRIX BUT KEEP EACH MATRIX QUESTIONS TOGETHER IN THE SECTION RANDOMIZATION.]

We’d like to ask you some questions about vaping.

Please do NOT include vapes with marijuana, THC, or CBD when answering these questions.

Vaping is … [SOURCE: NOAR ET AL., 2020]

[USE SCROLLING LIST. RANDOMIZE ORDER OF ITEMS]

[GOOD] Very bad (1)	2	3	4	Very good (5)
[ENJOY] Unenjoyable (1)	2	3	4	Enjoyable (5)
[POS] Negative (1)	2	3	4	Positive (5)

[CONF] How confident are you that you can stay away from vaping if you want to? [ADAPTED FROM: ZHAO & NAN, 2010]

Not at all sure I can (1)

2

3

4

Completely sure I can (5)

[FRNDVAPE] Out of every 10 people your age, how many do you think vape?

0. 0

1. 1

2. 2

3. 3

4. 4

5. 5

6. 6

7. 7

8. 8

9. 9

10. 10

[APPROVE] Most people my age…

Disapprove of vaping (1)

2

3

4

Approve of vaping (5)

If you were to vape every day, how likely is it that you would…[ADAPTED FROM SANGALANG ET AL., 2019; BRENNAN ET AL., 2017; O’BRIEN ET AL., 2021]

[RISKLU]…harm your lungs?

[RISKBR]…harm your brain?

[RISKAD]…become addicted to vaping?

[RISKSM]…start smoking cigarettes?

[RISKSO]…harm your relationships?

Not at all likely 1

2

3

4

Extremely likely 5

[INT] How likely is it that you will… [SOURCE: NOAR ET AL., 2020]

1. Vape soon

2. Vape in the next year

3. Be vaping years from now

Not at all likely (1)

2

3

4

Extremely likely (5)

SECTION 2: MESSAGE EXPOSURE

[IN THIS SECTION, EACH RESPONDENT WILL SEE A SINGLE AD, RANDOMLY SELECTED FROM THE SET OF 89 ADS]

Next, you'll view a vaping prevention ad. You may watch each video clip as many times as you like by clicking the "Play" icon in the video.

Please make sure that the volume on your device is turned on.

When you are ready, please click “Continue” to view the ad.

SHOW AD [FORCE VIEWING OF AD TO COMPLETION]

SECTION 3: DEMOGRAPHICS

[ASK IF RACE_ETH_5 = 1]

[COUNTRY] Where were your biological parents born? (Please remember that this information will remain confidential.)

1. Both parents born outside the U.S.

2. Both parents are U.S. born (including American territories)

3. One parent is U.S. born, one parent born outside the U.S.

4. One parent is U.S. born, unsure of other

5. One parent born outside the U.S., unsure of other

6. Don’t know/Not sure

[ASK IF RACE_ETH_5 = 1]

[LANG] What language(s) do you usually speak with your family? [SOURCE: NEW ITEM]

1. Only Spanish

2. Spanish more than English

3. Spanish and English equally

4. English more than Spanish

5. Only English

[ASK all]

[INCOME] How much money does your family have? [SOURCE: NEW ITEM]

1. Not enough to get by

2. Just enough to get by

3. Only have to worry about money for fun or extras

4. Never have to worry about money

[ASK all]

[sexor] Which of the following best represents how you think of yourself? (Select all that apply) [SOURCE: NEW ITEM]

1. Straight or heterosexual

2. Bisexual

3. Gay or lesbian

4. Something else (_______________)

5. I have not figured out or am in the process of figuring out my sexuality [EXCLUSIVE]

6. Prefer not to answer

SECTION 4: TOBACCO USE

ASK IF curr_VAPE > 0

For the next few questions, please indicate how often you do the following. [ADAPTED FROM MOREAN ET AL., 2018]

Please DO NOT INCLUDE vapes with marijuana, THC, or CBD when answering these questions.

[USE SCROLLING LIST. RANDOMIZE ORDER OF QUESTIONS]

1. Never

2. Rarely

3. Sometimes

4. Often

5. Almost always

[addict_vape1] I find myself reaching for my vaping product without thinking about it.

[addict_vape2] I vape more before going into a situation where vaping is not allowed.

[addict_vape3] When I haven’t been able to vape for a few hours, the craving gets intolerable.

[addict_vape4] I drop everything to go out and get vape products or e-juice.

ASK IF EVER_VAPE = 1

[LIFE_VAPE] How many times have you vaped nicotine in your entire life? [SOURCE: FDA EXPECTT SURVEY]

1. 0 times

2. 1 time, even just a few puffs

3. 2 to 10 times

4. 11 to 20 times

5. 21 to 50 times

6. 51 to 99 times

7. 100 or more times

The next questions are about your experience with cigarettes and other tobacco products

[ASK ALL]

[EVER_SMK] Have you ever tried cigarette smoking, even one or two puffs? [SOURCE: PATH Wave 6]

1. Yes

2. No

[ASK IF EVER_SMK = 1]

[CURR_SMK] During the past 30 days, on how many days did you smoke cigarettes? [SOURCE: NYTS 2020]

___________days [NUMERIC, WHOLE NUMBERS ONLY. RANGE = 0-30]

[ASK ALL]

For each of the tobacco products below, please indicate if you use the product every day, some days, rarely, or not at all

[USE SCROLLING LIST. RANDOMIZE ORDER OF ITEMS.]

1. Every day

2. Some days

3. Rarely

4. Not at all

[CURR_SLT] Chewing tobacco, snuff, or dip

[CURR_CIGAR] Cigars, cigarillos, or little cigars

[CURR_HOOK] Tobacco out of hookahs or water pipes

SECTION 5: ACTUAL MESSAGE EFFECTIVENESS (WAVE 2)

[REPEAT SECTION 1. GENERATE VARIABLES WITH SUFFIX TO DENOTE WAVE (E.G. “GOOD_1” = ITEM “GOOD” FOR WAVE 1; “GOOD_2” = ITEM “GOOD” FOR WAVE 2]

[RE-RANDOMIZE ORDER OF ITEMS IN THIS SECTION. RANDOMIZE WITHIN EACH SCROLLING MATRIX BUT KEEP EACH MATRIX’S QUESTIONS TOGETHER IN THE SECTION RANDOMIZATION.]

SECTION 6: MESSAGE EVALUATION

[EACH RESPONDENT WILL SEE AND RESPOND TO QUESTIONS ABOUT A TOTAL OF 3 ADS IN THIS SECTION, RANDOMLY SELECTED FROM THE SET OF 89. EXCLUDE THE AD SHOWN IN THE MESSAGE EXPOSURE SECTION—RESPONDENTS SHOULD SEE 1 UNIQUE AD IN PREVIOUS MESSAGE EXPOSURE SECTION AND 3 UNIQUE ADS IN THIS SECTION, FOR A TOTAL OF 4 UNDUPLICATED ADS. ADS SHOULD ALSO BE UNIQUE WITHIN CONCEPT (NO RESPONDENT SHOULD SEE MORE THAN 1 AD FROM ANY CONCEPT).

ADS AND CONCEPTS ARE NAMED ACCORDING TO THE FOLLOWING CONVENTION:

• VIDEOID_CONCEPTID_LENGTH

FOR EXAMPLE, AD 01_C01_30S REFERS TO AD#1 FROM CONCEPT #1 THAT IS 30S IN LENGTH.

REPEAT SECTION 6 (MESSAGE EVALUATION) ITEMS FOR EACH OF 3 ADS SHOWN.]

RANDOMIZE ORDER OF ADS SHOWN. REPEAT SECTION 6 FOR EACH AD. FOR EACH ITEM IN THIS SECTION, GENERATE A VARIABLE WITH A NUMERIC SUFFIX TO DENOTE THE CORRESPONDING AD (E.G., “REMEMBER_7” = ITEM “REMEMBER” FOR AD #7)]

Next, you'll view a few more vaping prevention ads. You may watch each video clip as many times as you like by clicking the "Play" icon in the video.

Please make sure that the volume on your device is turned on.

When you are ready, please click “Continue” to view the first ad. After each ad, there will be a few questions that ask about your opinions of the ad.

SHOW AD [FORCE VIEWING UNTIL AD COMPLETION]

The “Continue” button will appear after you finish watching the video.

For the next few questions, please tell us how much you agree or disagree with the following statements.

[USE SCROLLING LIST. RANDOMIZE ORDER OF QUESTIONS]

1. Strongly disagree

2. Disagree

3. Neither agree nor disagree

4. Agree

5. Strongly agree

[REMEMBER] This ad is worth remembering [SOURCE: DAVIS, ZHAO]

[ATTENTION] This ad grabbed my attention [SOURCE: DAVIS, ZHAO]

[POWER] This ad is powerful [SOURCE: DAVIS]

[INFORM] This ad is informative [SOURCE: DAVIS, ZHAO]

[MEANING] This ad is meaningful [SOURCE: DAVIS]

[CONVINCE] This ad is convincing [SOURCE: DAVIS, ZHAO]

[ANNOY] This ad annoys me [SOURCE: HALL REACTANCE]

MANIPULATE] This ad is trying to manipulate me [SOURCE: HALL REACTANCE]

[OVER] The negative effect in this ad is overblown [SOURCE: HALL REACTANCE]

[NEW] This ad told me something new

[INTEREST] This ad is interesting

[IMPORTANT] This ad is important

[BELIEVE] This ad is believable

[CLEAR] This ad is clear

[COMMON] The information in this ad is common knowledge [SOURCE: HALL NOVELTY (ADAPTED)]

[HEARD] I’ve heard the information in this ad a million times [SOURCE: HALL NOVELTY (ADAPTED)]

[KNEW] I already knew about the harms in this ad [SOURCE: HALL NOVELTY (ADAPTED)]

How much does this ad...

[WORRY] Make you worry about what vaping will do to you? [SOURCE: NOAR]

[BADIDEA] Make you think vaping is a bad idea? [SOURCE: NOAR]

[DISCOURAGE] Discourage you from vaping? [SOURCE: NOAR]

1. Not at all

2. Very little

3. Somewhat

4. Quite a bit

5. A great deal

People sometimes have different emotional reactions when they see advertisements. On a scale from 1 to 5, where 1 means “not at all” and 5 means “very much,” please indicate how much this ad made you feel:

Not at all 1

2

3

4

Very much 5

[SAD] Sad

[FEAR] Afraid

[ANGER] Angry

[DISGUST] Disgusted

[AMUSE] Amused

[SURPRISE] Surprised

[HOPE] Hopeful

If you saw this advertisement, on a scale from 1 to 5, where 1 is not at all likely and 5 is extremely likely, how likely would you be to do each of the following:

Not at all likely 1

2

3

4

Extremely likely 5

[SEEKON]…look for more information online?

[SEEKWEB]...visit the ad campaign’s website or social media page?

[SHAR]...mention it on social media?

SECTION 7: AD RANKING

[ADRANK] Next, we'd like you to rank the ads you just watched, from 1 (Best) to 4 (Worst). Click on the box below each ad image in the order you want to rank the ads from 1 (Best) to 4 (Worst).

[PIPE IN THUMBNAIL IMAGES CORRESPONDING TO ADS SHOWN]

[RANDOMIZE ORDER OF IMAGES]

1. Best

4. Worst

FINAL SCREEN

Thank you for taking this survey. If you are a tobacco user, or have a friend or family member who is a tobacco user, and you would like information on how to quit, please visit https://smokefree.gov/.

If you have any questions about this survey, you can contact the Study Coordinator, Susana Peinado, at (919) 316-3190 or [email protected]. If you have a question about your rights as a study participant, you can call RTI’s Office of Research Protection toll-free at (866) 214-2043.

[INCLUDE THE STATEMENT BELOW IN SMALLER GREY FONT AT THE BOTTOM OF THE END OF THE SURVEY]

Paperwork Reduction Act Statement: The public reporting burden for this information collection has been estimated to average 15 minutes to complete this survey (the time estimated to read and respond to questions). Send comments regarding this burden estimate or any other aspects of this information collection, including suggestions for reducing burden, to [email protected].

26