SS part A - The Real Cost Campaign (W3): Online Quantitative Study of Reactions to Rough-Cut Advertising Designed to Prevent You

OMB No. 0910-0810

Exp. Date: 12/31/2024

The Real Cost Campaign (W3): Online Quantitative Study of Reactions to Rough-Cut Advertising Designed to Prevent Youth Tobacco Use

0910-0810

Generic IC Supporting Statement: Part A

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

In support of the U.S. Food and Drug Administration’s (FDA) efforts to refresh campaign messaging, the Center for Tobacco Products (CTP) will conduct a quantitative study to inform the development of appropriate messaging for FDA’s The Real Cost campaign.

In 2019, about 6.2 million U.S. middle and high school students were current users of some type of tobacco product (Wang, Gentzke & Creamer, 2019). Specifically, 5.8% high school students used cigarettes and 27.5% used e-cigarettes and 13.3% of middle and high school students used both cigarettes and e-cigarettes (Wang, Gentzke & Creamer, 2019). Young people continue to represent a priority population when it comes to prevention messaging. Among young adults, 7.8% currently smoke cigarettes (Creamer, Wang & Babb, 2019), and 99% of smokers start smoking by age 26 (U.S. Department of Health and Human Services, 2014). As a way to reduce the enormous public health burden of tobacco, the Family Smoking Prevention and Tobacco Control Act has given the FDA the authority to take action to protect children, encourage smokers to quit, and reduce tobacco-related disease and death. The law also enables FDA to educate the public, especially young people, about the dangers of tobacco products. Research shows that public education mass media campaigns can be used to change attitudes and beliefs about tobacco use and reduce smoking prevalence. In fact, the Centers for Disease Control and Prevention (CDC) considers mass media campaigns to be a “best practice” for tobacco control (CDC, 2014)

To develop appropriate messaging to inform youth about the risks of using tobacco products it is important for the FDA to conduct research to gain insight into youth perceptions of (electronic nicotine delivery systems) ENDS and cigarettes and reactions to draft advertising concepts. Information obtained through this study will be used to develop and refine messaging related to preventing ENDS use among youth aged 13 to 17 who are at risk of initiating or who have experimented with ENDS products.

The study will be conducted using self-administered online surveys on personal computers or mobile devices. All participants will complete an online screener to determine eligibility. Qualified participants will be invited to complete a full survey, where they will be randomly assigned to either the ad-viewing condition, where they will view one ad, or the control condition, where they will not view any ads. Each participant in the ad-viewing group will take approximately 24 minutes to complete the study (2-minute screener, 2-minute assent, 20-minute survey). Participants in the non-ad viewing group will take approximately 14 minutes to complete the study (2-minute screener, 2-minute assent, 10-minute survey).

2. Purpose and Use of the Information

CTP has contracted with FCB and FCB’s subcontractors KDH Research and Communication and Marketing Workshop to conduct online surveys to assess ad performance of draft advertising concepts. Recruitment will be conducted mostly online, with some telephone recruitment if needed. Data collection will consist of a national, online self-administered survey of 300 youth (ages 13-17).

The results of this survey will be used to inform specific recommendations around FDA’s public education programs’ impact and effectiveness in reducing tobacco-related death and disease. Information gathered will not be used for the purpose of substantially informing influential policy decisions. The information gathered is also not intended to yield results that are statistically projectable, nationally representative, or precise estimates of population parameters.

The research will explore the following questions:

• To what extent does the advertisement provide an understandable and engaging message about the harms of ENDS use?

• To what extent does the advertisement have any potential adverse or unintended consequences related to beliefs around the harms of ENDS use?

3. Use of Information Technology and Burden Reduction

This study will rely on online survey data collection to collect primary data. Online data collection reduces burden on the participants and on the contractors. Participants are more likely to answer candidly using online surveys compared to other types of data collection methods, especially when it is clear their answers will remain private. In addition, using an online survey will allow for more participants to respond in a cost-effective and timely manner. The self-administered, online survey makes data processing and analysis quicker, including coding and data entry. Data are transmitted electronically, rather than by mail. These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. An added benefit is increased data protection by limiting the amount of personally identifiable information (PII) collected from participants, reducing the risk of data security issues. Finally, as noted above, this technology permits respondents to complete the survey in privacy. The use of a more private data collection method makes reporting potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) less threatening and enhances response validity and response rates. Only Marketing Workshop will ever have data with PII. FDA, FCB, and KDH Research & Communication will not have access to PII.

4. Efforts to Identify Duplication and Use of Similar Information

The types of tobacco and vaping products on the market change quickly, and it is important for advertising to reflect the current state of vaping use to be salient and have the best chance of affecting change in tobacco use. In designing the proposed data collection activities, we took several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets already address the proposed study questions. We carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address FDA’s need for this information. Data sources we examined for this purpose include ongoing national surveillance systems such as the National Youth Tobacco Survey (NYTS), the Youth Risk Behavior Surveillance System (YRBSS), the National Health Interview Survey (NHIS), and the Population Assessment of Tobacco and Health (PATH). We also reviewed data collected to evaluate other national tobacco-focused media campaigns such as CDC’s Tips from Former Smokers and FDA’s The Real Cost, as well as other CTP surveillance mechanisms. We concluded that these data sources do not include the measures, nor do they test CTP’s draft advertising concepts.

5. Impact on Small Businesses or Other Small Entities

Respondents in this study will be members of the general public, specific subpopulations or specific professions, not business entities. No impact on small businesses or other small entities is anticipated.

6. Consequence of Collecting the Information Less Frequently

Respondents to this data collection will answer only once to ensure the participant burden is as low as possible. Without the data collection requested for this study, it would be difficult to determine the most effective messages to use in upcoming tobacco prevention campaigns. Failure to collect these data could reduce effectiveness of the FDA’s messaging, and therefore reduce the benefit of the messages for youth in the United States.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2). The message testing activities fully comply with the guidelines in 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside

Agency

The following individuals inside the agency have been consulted on the design of the study, instrument development, or intra-agency coordination of information collection efforts:

Emily Peterson, PhD

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

9200 Corporate Boulevard

Rockville, MD 20850

Phone: 301-633-4223

E-mail: Emily.P[email protected]

Andrea Malterud, PhD

Office of Health Communication & Education

Center for Tobacco Products

Food and Drug Administration

10903 New Hampshire Avenue

Silver Spring, MD 20993

Phone: 240-743-1750

E-mail: [email protected]

FDA collaborates with other federal government agencies that sponsor or endorse health communication projects, such as the Centers for Disease Control and Prevention, Office on Smoking and Health (CDC/OSH), the Substance Abuse and Mental Health Services Administration (SAMHSA) and the National Institutes of Health National Cancer Institute (NIH/NCI). These affiliations serve as information channels, help prevent redundancy, and promote use of consistent measures of effectiveness. Coordination activities include:

• Review of proposed messages for advertisements;

• Review of surveys for testing purposes;

• Sharing data; and

• Standardizing survey tools where at all possible.

The following individuals outside of the agency have been consulted on survey development.

Kristen Holtz, Ph.D.

KDH Research & Communication

145 15^th Street NE, Suite 831

Atlanta, GA 30309

(404) 395 8711

Dimas Adiwiyoto

Senior Vice President

FCB

111 W 33rd Street

New York, NY 10120

(646) 504 8586

9. Explanation of Any Payment or Gift to Respondents

As a token of appreciation, participants recruited who complete and submit the survey will receive a $10 e-gift in the form of a voucher or points. There is no token of appreciation for completing the screener.

Such tokens are commonly used by research agencies to recruit participants efficiently and effectively; parents/guardians or individual members who choose to be a part of these online panels have an expectation that they will be compensated for their time. We estimate that the survey will take up to 20 minutes per respondent. The token amount not only reflects the burden of time to participate, but it will also ensure that the respondent pool is recruited within a tight timeframe. Smaller token amounts are associated with slower movement on recruitment.

Paying the token of appreciation as a $10 e-gift ensures that all participants are compensated equally, which reflects our human subjects’ commitment to equity in research participation.

Tokens of appreciation are important to research for many reasons:

• Tokens of appreciation ensure efficient and effective recruitment, which saves the government time and money: Without such tokens, samples are substantially slower to recruit, have higher abandonment rates (e.g. incomplete data), and generally poorer quality than samples offered a token.

• Tokens of appreciation boost recruitment: Research finds that most participants will not participate without financial compensation and it is one of the main reasons why participants participate in research studies (Resnik, 2015). The recruitment strategy relies on the effective placement and reach of the advertisements used to recruit participants of the target populations into the study. This placement and reach on social media platforms are determined by the platforms’ systems for targeting the populations necessary for participation in the study. These systems price and place advertisements using an “auction” system, setting prices and prioritizing advertisement placement based in-part on the rate that potential participants engage with the advertisements. Due to the way these advertisement systems function, increasing potential participant engagement with the advertisements results in increased reach of subsequent advertisements among the target populations Not providing tokens of appreciation may hinder the research study, thus delaying scientific progress and unethically exposing participants to burden (Largent & Fernandez Lynch, 2017).

• Tokens of appreciation reflect human subjects protections: Providing tokens allows for a fair distribution of burden and benefit across participants, resulting in a diverse sample and creating trust between investigators and participants (Largent & Fernandez Lynch, 2017).

• Tokens of appreciation are associated with a more balanced and unbiased sample: Evidence suggests that participants who are offered a token of appreciation have significantly lower non-response rates and less missing data, potentially providing reduction in bias resulting from these factors (Singer, 2002).

• Tokens of appreciation are important to recruit at-risk and multicultural populations: Such populations are important to this research because of the risk factors for tobacco use; those population are also historically difficult to recruit and retain in health research (Hooven et al., 2011; Murthy et al., 2004; Post et al., 2012; Siddiqui et al., 1996; Zand et al., 2006). The use of tokens of appreciation is associated with more diverse sample populations (Booker et al., 2011; Caldwell et al., 2010; Martinson et al., 2000; Walter et al., 2013).

CTP has previously used this $10 token approach in a study with a similar participants (youth ages 13-17 who had experimented with ENDS use or at-risk of ENDS use) and study design; that study had a diverse sample and timely data collection (OMB control number 0910-8010). In that study, the response rate was about 19% (1,660 completed out of 8,837 screened). This past data show that this is a difficult population to recruit and enroll, and an incentive rate lower than $10 would severely negatively impact the response rate, costing the government additional time and money. Other CTP studies have used similar tokens of appreciation for surveys of similar length and with similar target populations (e.g., Monthly Monitoring Study, OMB control number 0910-0810). 

10. Assurance of Confidentiality Provided to Respondents

OMB Control Number 0910-0810 is covered underneath a Privacy Impact Assessment that has been approved by the Department of Health and Human Services (PIA Unique Identifier: P-9008729-198376). Concern for privacy and protection of respondents’ rights will play a central role in the study implementation, storage and handling of data, and data analysis and reporting. The Institutional Review Board (IRB) of KDH Research & Communication, the research organization contracted to manage data collection, reviewed and approved the protocols for the survey. The primary concern of IRB is protecting respondents’ rights, one of which is maintaining the privacy of respondent information.

As part of this study Marketing Workshop, is collecting and maintaining personally identifiable information (PII) about participants who complete the online screener and the online surveys. The only PII that will be collected is email address, IP address, and zip code. This information will be stored separately from each other and from survey responses. We are not collecting any Protected Health Information, defined as “Personally identifiable information that relates to a person's health, medical treatment or payment, and which was obtained from a "covered entity" (health care provider, health plan, or healthcare clearinghouse), as defined by HIPAA (Health Insurance Portability and Accountability Act) regulations.” Survey data will always be kept separate from PII. Only Marketing Workshop will ever have data with PII. FDA, FCB, and KDH Research & Communication will not have access to any PII.

The following procedures will be used to ensure participant privacy before, during, and after fielding:

1. PII in the form of participants’ email addresses, zip codes, and IP addresses will be stored separately from screening-related data and survey data;

2. email addresses, zip codes, and IP addresses will be deleted after survey completion;

3. datasets and reports will not contain any PII; and

4. respondents’ information will not be tied to their individual responses and all analyses will be conducted in the aggregate (i.e., any data used in reporting will not be attributed to individual participants). All datasets and reports delivered to FDA, FCB (FDA Contractor) and KDH Research & Communication (FDA sub-contractor) will not include PII.

PII will be collected in the form of email addresses for the purposes of data quality assurance. Zip codes will be collected to verify that participants are within determined geographic targets for the study. IP Address will be collected automatically to reduce fraudulent activity and verify participant country of origin. No additional personal identifiers (e.g., full name, phone number, social security number) will be collected aside from basic demographic information (e.g., gender, age, and race). PII will be stored separately from any survey responses.

This study is funded by the FDA, a Department of Health and Human Services supported agency, and is covered by a Certificate of Confidentiality (CoC). Section 2012 of the 21st Century Cures Act includes significant amendments, to the previous statutory authority for such protections, to enhance privacy protections for individuals who are the subjects of federally funded research, under subsection 301(d) of the Public Health Service Act (42 U.S.C. 241). Specifically, the amended authority requires the FDA to issue a CoC to investigators or institutions engaged in research funded by the Federal government to protect the privacy of individuals who are subjects of this research. We will notify participants in the assent form of the protections that the Certificate provides.

11. Justification for Sensitive Questions

The majority of questions asked will not be sensitive, but some will ask about specific health behaviors such as tobacco product use. Some questions about tobacco use are potentially sensitive because tobacco use among adolescents under age 18 is illegal in a few states, and sales to individuals under age 21 are illegal nationwide. These sensitive questions are essential to the objectives of this data collection, specifically to understand how draft messages impact beliefs around the harms of ENDS use and are necessary for measuring participant characteristics. Although we do not anticipate any risks from these health questions, some participants may perceive them to be sensitive. Questions about messages concerning lifestyle (e.g., tobacco product use) and some demographic information, such as race and ethnicity could also be considered sensitive, but not highly sensitive.

To address any concerns about inadvertent disclosure of sensitive information, participants will be fully informed of the applicable privacy safeguards. The informed assent protocol will notify participants that these topics will be covered in the survey. This study includes a number of procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including the following:

• Participants will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer;

• Online surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses; and

• Participants will be provided with a phone number and email address for the Principal Investigator should they have any questions or concerns about the study.

12. Estimates of Annualized Burden Hours and Costs

12 a. Annualized Hour Burden Estimate

An estimated one-time reporting burden for this collection will be approximately 160 hours (Table 1). This includes the time burden associated with the screener and informed assent. We will obtain a final sample size of 300 youth (13- 17).

Table 1. Estimated Annual Reporting Burden

Type of Respondent	Activity			Number of Respondents	Number of Responses per Respondent		Total Responses	Average Burden per Response		Total Hours¹
Parent Email Invitation/Reminder Invitation and Notification Form
Parents of youth aged 13–17	Youth Recruiting			750	1		750	4 minutes		50
Screening
Youth aged 13–17		Youth Recruiting and Screening	750			1	750		2 minutes		25
Informed Assent
Youth aged 13–17		Youth Assent	300			1	300		2 minutes		10
Survey
Youth aged 13–17	Self-administered online Survey (ad-view condition)			150	1		150	20 minutes		50
Youth aged 13-17	Self-administered online Survey (control condition)			150	1		150	10 minutes		25
Total Annualized Hours										160

12b. Annualized Cost Burden Estimate

Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. Marketing Workshop has conducted many smoking-related surveys of similar length among youth and adults. We have examined diagnostic data from prior surveys and estimate that data collection for this study will take approximately 2 minutes per respondent for screening, 2 minutes per respondent for assenting, and up to 20 minutes per respondent for the online surveys.

To calculate the estimate annual cost, the mean hourly wage of $7.25 was used for youth. The youth price represents the minimum wage from the U.S. Department of Labor Bureau of Labor Statistics (May 2020 data). There are no direct costs to respondents associated with participation in this information collection. Marketing Workshop has conducted many smoking-related surveys of similar length among youth. Thus, assuming an average hourly wage of $7.25 for youth and $28.43 for parents of youth, the estimated one-year annualized cost to participants will be $2,222. The estimated value of respondents’ time for participating in the information collection is summarized in Table 2.

Table 2. Estimated Annual Cost

Type of Respondent	Activity	Annual Burden Hours	Hourly Wage Rate	Total Cost1
Parent of youth aged 13-17	Youth Recruiting	50	$28.43	$1,422
Youth aged 13–17	Youth Recruiting and Screening	25	$7.25	$182
	Youth Assent	10	$7.25	$73
	Self-administered online Survey (ad-view condition)	50	$7.25	$363
	Self-administered online Survey (control condition)	25	$7.25	$182
Total				$2,222

¹ Cost was rounded up to the next dollar.

13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers

There are no capital, start-up, operating, or maintenance costs associated with this data collection.

14. Annualized Cost to the Federal Government

This information collection is funded through a contract with FCB New York. The estimated costs attributable to this data collection are $51,585 per year. (Table 3). There are additional contract-funded activities occurring before and after this data collection that include project planning and data analysis. Other activities outside this data collection include coordination with FDA, instrument development, reporting, KDH Research & Communication IRB, project management and progress reporting. This information collection will occur for one year.

Table 3. Itemized Cost to the Federal Government

Government Personnel	Time Commitment	Average Annual Salary	Total1
GS-12	5%	$86,335	$4,317
GS-13	10%	$102,663	$10,266
GS-13	10%	$102,663	$10,266
		Total Annual Salary Costs	$24,849
Annual Contract Cost			$51,585
Total Annual Cost			$105,600

¹ Cost was rounded up to the next dollar.

15. Explanation for Program Changes or Adjustments

This is a new individual generic data collection.

16. Plans for Reporting and Project Time Schedule

Data from this information collection will enable the FDA to gain insight into youth perceptions of ENDS and cigarettes and reactions to draft advertising concepts. This activity will allow the FDA to develop and refine messaging related to preventing ENDS and cigarette use among youth aged 13 to 17 who are at risk of initiating or who have experimented with tobacco products. Findings from these analyses will be used to inform FDA CTP health communication strategy and messaging.

Reporting

At the end of the study, a final report containing background information on the project objectives, scope and methodology, and key findings and conclusions will be completed. The approximate dates for completing project tasks are listed in Table 4.

Table 4. Approximate Project Schedule

The project schedule is shown in Table 4. Future development and research activities are dependent on the timely completion of the present study.

Table 4. Project Schedule

Project Activity	Approximate Date
Data Collection	November 2022
Data Delivery	December 2022
Draft Reporting Deliverables	January 2023
Final Reporting Deliverables	February 2023

17. Reason(s) Display of OMB Expiration Date is Inappropriate

FDA is not requesting an exemption for display of the OMB expiration date and is also not seeking OMB approval to exclude the expiration date for this information collection. The OMB approval and expiration date will be displayed on the relevant materials associated with the study.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

These information collection activities involve no exception to the Certification for Paperwork Reduction Act Submissions.

References

Booker, C., Harding, S., & Benzeval, M. (2011). A systematic review of the effect of retention methods in population-based cohort studies | BMC Public Health | Full Text. https://bmcpublichealth.biomedcentral.com/articles/10.1186/1471-2458-11-249

Caldwell, P., Hamilton, S., Tan, A., & Craig, J. (2010). Strategies for Increasing Recruitment to Randomised Controlled Trials: Systematic Review. https://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1000368

Creamer, M. R., Wang, T., Babb, S., & Cullen, K. (2019). Tobacco Product Use and Cessation Indicators Among Adults—United States, 2018. MMWR. Morbidity and Mortality Weekly Report, 68. https://doi.org/10.15585/mmwr.mm6845a2

Hooven, C., Walsh, E., Willgerodt, M., & Salazar, A. (2011). Increasing Participation in Prevention Research: Strategies for Youth, Parents and Schools. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3546549/

Largent, E. A., & Fernandez Lynch, H. (2017). Paying Research Participants: Regulatory Uncertainty, Conceptual Confusion, and a Path Forward. Yale Journal of Health Policy, Law, and Ethics, 17(1), 61–141.

Martinson, B. C., Lazovich, D., Lando, H. A., Perry, C. L., McGovern, P. G., & Boyle, R. G. (2000). Effectiveness of monetary incentives for recruiting adolescents to an intervention trial to reduce smoking. Preventive Medicine, 31(6), 706–713. https://doi.org/10.1006/pmed.2000.0762

Murthy, V. H., Krumholz, H. M., & Gross, C. P. (2004). Participation in cancer clinical trials: Race-, sex-, and age-based disparities. JAMA, 291(22), 2720–2726. https://doi.org/10.1001/jama.291.22.2720

Post, A., Gilljam, H., Bremberg, S., & Galanti, M. R. (2012). Psychosocial determinants of attrition in a longitudinal study of tobacco use in youth. TheScientificWorldJournal, 2012, 654030. https://doi.org/10.1100/2012/654030

Resnik, D. B. (2015). Bioethical Issues in Providing Financial Incentives to Research Participants. Medicolegal and Bioethics, 5, 35–41. https://doi.org/10.2147/MB.S70416

Siddiqui, O., Flay, B. R., & Hu, F. B. (1996). Factors affecting attrition in a longitudinal smoking prevention study. Preventive Medicine: An International Journal Devoted to Practice and Theory, 25(5), 554–560. https://doi.org/10.1006/pmed.1996.0089

Singer, E. (2002). The use of incentives to reduce nonresponse household surveys—WebSM. http://www.websm.org/db/12/15616/Web%20Survey%20Bibliography/The_use_of_incentives_to_reduce_nonresponse_household_surveys/

Walter, J. K., Burke, J. F., & Davis, M. M. (2013). Research Participation by Low‐Income and Racial/Ethnic Minority Groups: How Payment May Change the Balance. Clinical and Translational Science, 6(5), 363–371. https://doi.org/10.1111/cts.12084

Wang, T., Gentzke, A., Creamer, M., Cullen, K., Holder-Hayes, E., & Sawdey, M. (2019). Tobacco Product Use and Associated Factors Among Middle and High School Students—United States, 2019. MMWR. Surveillance Summaries, 68. https://doi.org/10.15585/mmwr.ss6812a1

Zand, D., Thomson, N., Budan, M., & Braun, J. (2006). Predictors of Retention in an Alcohol, Tobacco, and Other Drug Prevention Study—Debra Zand, Nicole Renick Thomson, Mary Dugan, James A. Braun, Pat Holterman-Hommes, Patricia L. Hunter, 2006. https://journals.sagepub.com/doi/10.1177/0193841X05281160



1