OMB No. 0910-0810
Exp. Date: 12/31/2024
The Real Cost Campaign (W3): Online Quantitative Study of Reactions to Rough-Cut Advertising Designed to Prevent Youth Tobacco Use
B. Statistical Methods
1. Respondent Universe and Sampling Methods
The respondent universe for this study is youth aged 13 to 17 who either:
1) are at risk of/susceptible to initiating use of ENDS products, or
2) have experimented with ENDS products.
Susceptibility to initiating use of ENDS is determined by an adapted Pierce Susceptibility Scale (1995). The Pierce susceptibility scale is a 3-item scale that assesses susceptibility by asking the participant if they will try the product soon, in the next year, or if one of their best friends offered the product. If respondents answer anything other than “definitely not” to at least one of the three items, they are defined as being susceptible to that product.
As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample is designed primarily to gain insight into youth perceptions of ENDS and cigarettes and reactions to draft advertising concepts. Additionally, generating a representative sample of the size necessary for this study, using random digital dialing or another similar method, would be cost prohibitive. The study will use convenience samples rather than probability samples. The results of the copy testing study will be used to inform specified recommendations around the U.S. Food and Drug Administration (FDA) public education programs’ impact and effectiveness in reducing tobacco-related death and disease.
Collection of detailed demographic information, including race/ethnicity and gender identity are necessary in order to assess disparities in tobacco use and possible differences in campaign impact across different populations. Decades of research has shown significant disparities in tobacco use by race/ethnicity (e.g., Harlow et al., 2019; Odani et al, 2018) and gender identity (e.g., Johnson et al, 2019; Delahanty et al, 2019). Therefore, collecting detailed information on these demographic characteristics will allow us to measure these differences with the goal of reducing these disparities. As multiple studies of youth and young adults have reported approximately 12-15% of the sample identified as gender non-conforming/non-binary (e.g., The Human Rights Campaign 2018 LGBTQ+ Youth Report; The Trevor Project 2020 National Survey on LGBT Youth Mental Health; CTP’s evaluation of This Free Life campaign), including gender non-conforming/non-binary response options is necessary to identify and assess message effectiveness among this population. Gender identity questions with genderqueer/gender non-conforming/non-binary response options have been approved by OMB for ExPECTT (0910-0753) and for RESPECT (0910-0808).
2. Procedures for the Collection of Information
This section describes the procedures for survey data collection. Online data collection will be completed by youth independently, on electronic devices, such as a mobile phone, tablet, or personal computer.
All surveys will be conducted using a self-administered, online survey. To be eligible, participants must give their assent, and provide their tobacco use status to be classified as either: 1) at risk of initiating use of ENDS products, or 2) having experimented with ENDS products. Those eligible will be sent directly to the full online survey, which will take up to 20 minutes to answer. At the end of the survey, participants who complete the survey will receive a $10 token of appreciation at the email address they provide. Specifically:
This study will recruit youth participants through their parents from online panels (via Marketing Workshop).
Parents of youth in the target age range will receive an initial email invitation informing them of the study and opportunity for their child(ren) to participate. The email invitation will also include a link that details the purpose of the study. After reading this invitation, the parent has three options:
If the parent determines that he or she wants the child to participate in this study, he or she can use instructions in the Opt-out Form to forward the study link to their child’s email address, thereby actively opting in and providing permission for their child to participate in the study.
The child cannot participate from their parent's email account for the survey to be accepted as valid.
The parent can do nothing, in which case the invitation dies in place and that parent's child will never receive information about the study.
If the parent does not want their child to participate in the study, he or she can use instructions in the Opt-out Form included in the email to indicate this. Parent reminders to participate will cease if the parent opts-out.
The same email invitation is sent a second time (reminder invitation) to all invitees who have not responded 48 hours after initial invitation. If an invitee does not respond after the reminder is sent, then he or she is considered unavailable and is not re-approached.
From the email forwarded by their parents, youth will access the study link that details the purpose of the study and includes study materials (i.e., screener, assent form, and the survey).
Prior to beginning the screener, participants must choose yes or no to assent to the screening process. The screener is a list of questions to determine eligibility. Responses to the screening questions determine eligibility to complete the full survey. Only participants who choose yes will continue with the screener.
Eligible youth participants, defined as those that meet screening criteria, will view a full assent form, which he or she must complete prior to the survey.
Following completion of the screener, the survey will be presented for the participant to complete if the participant is eligible. If the participant is ineligible, a termination notice will appear, and the online system will close.
After completing the survey, the participant will be thanked, and participation will end.
For the purposes of estimating statistical power, we assumed a sample size of 300 participants with equal allocation (n = 150 assigned to the ad condition, and n=150 assigned to the control condition.
Statistical analyses will be conducted to address the study’s primary research questions. Following the recommendation of the U.S. Food and Drug Administration’s (FDA’s) Center for Tobacco Products’ (CTP’s) Office of Science (Yang, 2017), analytic procedures will be based on nonparametric tests of statistical significance.
The ability to obtain the cooperation of potential respondents in the survey will be important to the success of this study. We will minimize the non-response rate by employing the following measures:
Use the online panels provided by Marketing Workshop to best reach the desired sample
Send reminders to all invitees who have not responded 48 hours after initial invitation.
Provide a $10 e-gift in the form of a voucher or points as a token of appreciation to participants who complete and submit the survey
As a token of appreciation, participants recruited who complete and submit the survey will receive a $10 e-gift in the form of a voucher or points. Such tokens are commonly used by research agencies to recruit participants efficiently and effectively; parents/guardians or individual members who choose to be a part of these online panels have an expectation that they will be compensated for their time. We estimate that the survey will take up to 20 minutes to complete (20 minutes for the ad-viewing participants and 10 minutes for the non-ad viewing participants). The token amount not only reflects the burden of time to participate, but it will also ensure that the respondent pool is recruited within a tight timeframe. Smaller token amounts are associated with slower pace of recruitment.
4. Tests of Procedures or Methods
KDH Research & Communication will conduct rigorous internal testing of the online survey instruments prior to fielding. Survey testers will review the online test version of the instrument that we will use to verify that instrument skip patterns are functioning properly, delivery of the token of appreciation is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by IRB and OMB.
5. Individuals Involved in Statistical Consultation and Information Collection
The following individuals inside the agency have been consulted on the design of the study plan, audience survey development, or intra-agency coordination of information collection efforts:
Emily Peterson, PhD
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
9200 Corporate Boulevard
Rockville, MD 20850
E-mail: [email protected]
Andrea Malterud, PhD
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-743-1750
E-mail: [email protected]
The following individuals outside of the agency have been consulted on survey development:
Kristen Holtz, Ph.D.
KDH Research & Communication
145 15th Street NE, Suite 831
Atlanta, GA 30309
Phone: 404-395-8711
E-mail: [email protected]
Dimas Adiwiyoto
Senior Vice President
FCB
111 W 33rd Street
New York, NY 10120
Reference
Pierce JP, Choi WS, Gilpin EA, Farkas AJ, Merritt RK. Validation of susceptibility as a predictor of which adolescents take up smoking in the United States. Health Psychol. 1996;15(5):355–361.https://doi.org/10.1037/0278-6133.15.5.355.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Chapman, LeTonya |
| File Modified | 0000-00-00 |
| File Created | 2023-08-18 |