Supporting Statement Part A - Flavors Phase 2 Study 2022

U.S. FOOD AND DRUG ADMINISTRATION

Study of How Consumers Use Flavors to Make Inferences About Electronic Nicotine Delivery System (ENDS) Product Qualities and Intentions to Use (Phase 2)

OMB Control No. 0910-NEW

SUPPORTING STATEMENT Part A. Justification

1. Circumstances Making the Collection of Information Necessary

Electronic nicotine delivery systems (ENDS), also called electronic cigarettes, e-cigarettes, and vaporizers are deemed tobacco products and fall under the U.S. Food and Drug Administration’s regulatory scope. FDA has the authority under the Family Smoking Prevention and Tobacco Control Act (Pub. L. 111–31, H.R. 1256), to regulate and restrict the marketing of tobacco products. However, given the recency of ENDS products to the market, limited research exists to inform the regulation of certain aspects of their marketing. Research to understand “marketing influences on youth experimentation, initiation, use and cessation of tobacco products” is a regulatory priority for the FDA Center for Tobacco Products (CTP). ¹

Flavors are a unique and important aspect of ENDS. ENDS use a liquid (‘e-liquid’ or ‘e-juice’) that can span a diverse range of flavors, from tobacco flavor, menthol, mint, fruit flavors, non-fruit sweet flavors (e.g., crème brulee, gummi bears), spices (e.g., cinnamon, vanilla), alcohol (e.g., strawberry daiquiri, bourbon, irish crème), and ‘concept’ flavors. Flavors are a regulatory area of interest, and FDA has issued an advance notice of proposed rulemaking (ANPRM; Docket No. FDA-2017-N-6565) “to obtain information related to the role that flavors play in tobacco products”, with a specific interest in how flavors spur youth product initiation.

This project addresses FDA’s “high-priority” topic of tobacco products, which includes tobacco-related communications, and specifically addresses CTP’s research priority of understanding marketing influences and the “impact of potential marketing restrictions on youth experimentation, initiation, and use.” The primary goal of the current study is to understand whether flavor-related imagery, descriptors and flavor name modifiers affect product appeal, curiosity about the product, interest in using the product, and product perceptions among youth and young adults. FDA expects that the results of this research will produce evidence regarding the potential effect of restricting use of flavor-related imagery, descriptors and flavor name modifiers on youth ENDS use. This project’s results might inform FDA’s thinking regarding possible rulemaking, but it will not provide sole support for any rulemaking.

2. Purpose and Use of the Information Collection

The information collected in this study is relevant to CTP’s potential future rulemaking regarding the marketing and presence of flavor features in ENDS. CTP will review the final report produced by Johns Hopkins University (JHU) and take findings into consideration for the potential development of regulations. For example, if we find that the inclusion of flavor imagery significantly increases appeal, curiosity and interest in using the product among youth, CTP could consider implementing a rule restricting use of this marketing tactic on the grounds that it is a form of youth targeting.

3. Use of Improved Information Technology and Burden Reduction

Automated information technology will be used in the collection of information for this study. One hundred percent (100%) of participants will self-administer the online survey, which will record responses, and which has skip patterns for questions already programmed in so that the appropriate question set will be automatically displayed to participants. Burden will be reduced by recording data on a one-time basis for each participant and by not containing any written responses (all responses are multiple choice). Administration of the survey using web methods will help to contain costs, allowing for a sample that is geographically diverse without driving up interviewer costs for travel during data collection. The online screener and survey permit greater expediency with respect to data processing and analysis, minimizing back-end steps such as coding and data entry. These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. Finally, this technology permits respondents to complete the survey questions in private. Providing the respondent with a methodology that improves privacy reduces the potential for experiencing embarrassment or stigmatization when reporting on tobacco use behaviors and enhances response validity and response rates.

4. Efforts to Identify Duplication and Use of Similar Information

This study contributes to literature on the use of flavors in ENDS marketing and contributes to FDA’s ability to regulate ENDS marketing tactics – specifically, the use of flavor name modifiers, imagery, and descriptors – that target youth. Our prior work^2,3 analyzed three years’ worth of ENDS advertisements to identify the key ways that flavor was communicated in the ads. The current study uses the results of that work to test the effect of three key ways flavor is communicated. No existing data could have been used to address this need. We have conducted a literature review of the use of flavors in ENDS marketing via searches on PubMed and Google Scholar. No studies have produced information similar to that which will be obtained in the current data collection. No prior or currently on-going FDA data collection has focused on this topic. The current data collection is not duplicative of efforts on-going in other government agencies (i.e., CDC, NIH).

5. Impact on Small Businesses or Other Small Entities

No small businesses are involved in this data collection. All respondents are individuals. We do not anticipate any impact on small businesses or other small entities in terms of data collection burden.

6. Consequences of Collecting the Information Less Frequently

This is a one-time data collection.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CFR 1320.5(d)(2). The research activities fully comply with the guidelines in 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60 day notice for public comment in the FEDERAL REGISTER of March 3, 2021 (86 FR 12468). FDA received nine comments, four of which were PRA related.

(Comment) One commenter supports FDA’s proposed collection of information and stated that research on the advertising of flavored e-cigarettes and its impact on the perceptions of nonusers, e-cigarette users, cigarette smokers, and dual users is important. The commenter also noted that the proposed study length is acceptable and comprises typical burden for respondents in this type of research.

(Response) FDA agrees with this comment and believes the study will contribute to our understanding of how consumers interpret flavor features on product labeling to make inferences about ENDS product qualities and intentions to use. We also believe the study’s burden estimate aligns with previous research studies of this kind.

(Comment) One commenter stated that the FDA should research the role of flavored non-combustible tobacco products in converting adult smokers from cigarettes.

(Response) This study focuses on appeal of the selected advertising tactics on youth and young adults. Expanding the sample to include older adults (or all adults) is beyond the scope of the study.

(Comment) FDA received a comment suggesting they consider separating underage individuals from those who are of legal age to purchase tobacco products.

(Response) The aim of this study centers around appeal of the selected advertising tactics on youth and young adults. The selection of the advertising tactics to be studied was grounded in research conducted when the federal legal age to purchase tobacco was 18 years of age. Thus, we intend to sample youth aged 13-17 and young adults aged 18-24. However, as resources allow, we will plan to conduct supplementary analyses to account for the new federal legal age (e.g., under 21 years vs. 21+ years).

(Comment) FDA received a comment suggesting they expand the sample to include tobacco users aged 25 and older.

(Response) This study focuses on appeal of the selected advertising tactics on youth and young adults. Expanding the sample to include older adults is beyond the scope of the study.

(Comment) FDA received a comment suggesting they include a range of flavor name modifiers.

(Response) The flavor name modifiers used in the study were selected based upon careful review of prior research analyzing the tactics that ENDS companies use to advertise flavor. Our assessment is that the selected name modifiers are consistent with that research.

(Comment) One commenter stated that using generalized data to support premarket determinations for specific products on specific applications is scientifically inappropriate. The commenter stated that the public should have the opportunity to provide comment on any proposed regulations. Additionally, the commenter stated any proposed de facto category-wide restriction on the manufacture, marketing, and distribution of tobacco products should undergo the appropriate notice and comment rulemaking procedures.

(Response) The primary goal of the study is to understand whether flavor-related imagery, descriptors, and flavor name modifiers affect product appeal, curiosity about the product, interest in using the product, and product perceptions among youth and young adults. This study will not produce product-specific data; thus, it would not form the sole basis for any premarket determinations, but the results could be taken into consideration more broadly as part of premarket review. Additionally, this study might inform FDA’s thinking regarding possible rulemaking but it will not provide sole support for any rulemaking. FDA’s consideration of any future rulemaking would follow the appropriate notice and comment rulemaking procedures, which would include an explanation of the scientific basis for the proposed rule. The scientific basis would consider all relevant science, not just the results of this one study. Lastly, this study does not indicate FDA’s intent to propose such a rule. The intent is to advance scientific knowledge broadly regarding the use of flavors in ENDS marketing.

(Comment) FDA received a comment expressing concern about exposing youth to ENDS advertisements.

(Response) Our study protocol includes measures to minimize risk of youth exposure to ENDS advertisements. Before participating in the study, participants are informed that they will be shown five ENDS advertisements. All participants are free to stop participation at any time and for any reason. At the end of the survey, participants will view a “debrief” screen containing information about the risks of ENDS and references to FDA and others’ ENDS education and prevention campaigns.

The Johns Hopkins Bloomberg School of Public Health’s Institutional Review Board reviewed and approved this study. We amended our recruitment process to further address this concern. We will also recruit youth aged 13-17 through their parent panelists (parents who are members of an existing online panel). Recruitment emails will be sent to parent panelists inviting them to have their child aged 13-17 participate in the study. Parents who are interested in having their child participate can have their child click the survey link in the recruitment email. This means that youth will be recruited to participate through two ways. First, we will recruit current youth panel members. Second, we will recruit youth through their parent panelists (parents who are members of the existing online panel).

(Comment) FDA received a comment expressing that the study does not provide data that would inform “conclusions regarding the role of flavors in youth attractiveness” and that the study does not distinguish between characterizing and non-characterizing flavors.

(Response) The objective of this study is to examine the effect of flavor advertising tactics on consumer product perceptions and intentions to use, not the effect of actual flavors and flavor use. Therefore, this comment is out of scope for the proposed study.

(Comment) FDA received a comment inquiring about whether “the survey will representatively sample/oversample for certain subpopulations – with a particular lens on race/ethnicity and other priority populations.”

(Response) The current sample was designed with a primary focus of sampling adequate numbers of youth and young adults across a variety of cigarette and ENDS use statuses (non-cigarette and non-ENDS users; cigarette users only; ENDS users only; dual ENDS and cigarette users), and we are not able to do additional oversampling given that some of these groups are of low frequency in the general population. However, we will be able to identify how our sample compares to national data, and our data will be weighted to be proportionally reflective of the U.S. population by race/ethnicity.

The following individuals inside the agency have been consulted on the study design and questionnaire development:

Izabella Zandberg, PhD

Social Scientist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

11785 Beltsville Drive
Beltsville, MD 20705

[email protected]

240-402-2527

Michael Sawdey, PhD

Epidemiologist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

11785 Beltsville Drive
Beltsville, MD 20705

[email protected]

301-796-3452

Sarah Trigger

Social Scientist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

11785 Beltsville Drive
Beltsville, MD 20705

[email protected]

240-402-8140

Lexie Perreras, MPH

Social Scientist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

11785 Beltsville Drive

Beltsville, MD 20705

[email protected]

301-796-4985

The following individuals outside the agency have been consulted on the study design and questionnaire development:

Meghan Moran

Associate Professor

Department of Health, Behavior & Society

Johns Hopkins Bloomberg School of Public Health

624 North Broadway

Hampton House, Room 729

Baltimore, MD 21205

[email protected]

410-614-6872

Ryan Kennedy

Assistant Professor

Department of Health, Behavior & Society

Institute for Global Tobacco Control

Johns Hopkins Bloomberg School of Public Health

2213 McElderry Street
4th Floor
Baltimore, Maryland 21205

[email protected]

410-955-3435

Lauren Czaplicki

Assistant Scientist

Department of Health, Behavior & Society

Institute for Global Tobacco Control

Johns Hopkins Bloomberg School of Public Health

2213 McElderry Street
4th Floor
Baltimore, Maryland 21205

[email protected]

410-614-5378

9. Explanation of Any Payment or Gift to Respondents

Respondents are compensated by having a cash equivalent in the form of points deposited into their accounts. These points can then later be redeemed for gift cards or can be cashed out for equivalent dollar amounts. Participants will receive points equal to ~ $1.50 for completing the survey. The vendor, SSRS, set the incentive rate. This amount is driven primarily by the length and burden of the survey and is in line with what panelists are typically compensated to complete surveys. The JHU team has compensated participants with this amount for similar studies and has encountered no issues regarding response rates. We have carefully considered this incentive rate and believe it is high enough to recruit the participants needed without being coercive.

10. Assurance of Confidentiality Provided to Respondents

Privacy Analysis & Design

In developing this study, CTP consulted the agency Privacy Officer to identify potential risks to the privacy of participants and other individuals whose information may be handled by or on behalf of FDA in the performance of this study. FDA designed the study to minimize privacy risks in keeping with the Fair Information Practice Principles (FIPPs) and applying controls selected from the National Institute of Standards and Technology (NIST), Special Publication 800-53, Security and Privacy Controls for Federal Information Systems and Organizations. CTP also identified privacy compliance requirements and coordinated with FDA’s Privacy Officer to ensure responsible offices in CTP satisfy all in accordance with law and policy. FDA submitted a Privacy Impact Assessment to the privacy office that has been approved by the Department of Health and Human Services (PIA Unique Identifier: P-5981280-768484).

Privacy Act Applicability

The information collection is not subject to the Privacy Act of 1974. Hence, no Privacy Act Statement is required to be displayed on the form, website, mobile application or other point at which individuals submit their information.

PII Collection

As part of this study, neither FDA nor JHU will directly collect or maintain personally identifiable information (PII). A third-party vendor, SSRS, and their opt-in non-probability panel partner will collect PII for respondent enrollment and compensation purposes. SSRS and its panel provider will not share PII with JHU or FDA.

PII will be collected on an as needed basis during the enrollment process and to compensate individuals for participation. For example, e-mail addresses may be collected for contacting respondents regarding enrollment details (e.g., providing the survey link). PII collected as part of respondent enrollment and compensation will not be maintained or linked to other study information. Contractors and subcontractors that collect data will not pass along any PII to FDA or JHU. For this collection, FDA doesn’t have any systems where PII is maintained or retrieved.

Notice and Transparency

Neither FDA nor direct contractors, including third parties, will share PII gathered via this collection with any other individuals or entities.

All subjects are provided notice regarding the collection and use of the information they submit. SSRS’s panel provider may collect IP addresses when participants register for the panel, but FDA nor JHU will receive IP addresses. SSRS will notify participants if IP addresses are recorded. FDA sponsorship is explained to participants when appropriate (in some cases, FDA sponsorship will not be made known to respondents prior to data collection out of concern for the potential introduction of bias to study results; in such cases, FDA sponsorship will be made known after the data are collected.). Participants will be informed that their participation is voluntary at all times.

Prior to collecting any information, these methods will all be approved by FDA’s Research Involving Human Subjects Committee (RIHSC) and Johns Hopkins Bloomberg School of Public Health’s Institutional Review Board (JHSPH IRB). The primary concern of the RIHSC and JHSPH IRB are to protect participants’ rights, one of which is maintaining the privacy of participant information to the fullest extent of the law.

Individual Participation and Control

Participation in this study is completely voluntary. Respondents will be informed of this in the study’s assent/consent form. Respondents also will be advised of the following: the nature of the activity; the purpose and use of the data collected; FDA sponsorship (when appropriate); and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection or to any particular questions.

Affirmative assent/consent will be obtained by having participants click “I agree to participate” at the bottom of a text box containing our assent/consent form, which is approved by the JHSPH IRB. Individuals have the opportunity to not participate in the study at all, by not clicking the link to the survey they receive or by not agreeing to participate after reading the assent/consent form. Participants may also discontinue participation at any time by closing the browser window to the survey.

Third-Party Accountability

Contract agreements include clauses that hold prime and their service providers to federal standards and the laws and policies specifically applicable to this study. FDA reviewed the privacy policies of all third parties to confirm that it does not conflict with HHS/FDA and/or is superseded by contract content.

Data Security

Contractors and third-party vendors are required to maintain appropriate administrative, technical, and physical safeguards to ensure the security and confidentiality of records. User roles and responsibilities will determine the type and content of data and information necessary for job function (both PII and non-PII). Role-based access will determine and control who will have access to PII on an as-needed basis. Only personnel from the third-party vendor conducting the information collection will have access to PII. All project staff from the third-party vendor conducting the information collection must take required measures to ensure the privacy and anonymity of data. PII will be limited to information that may be required in the process of respondent enrollment and compensation. PII will not be linked to study data. All PII collected will be destroyed following data collection at the completion of the study.

Neither FDA employees nor any Federal employee of any other agency will have access to this information.

All electronic data will be maintained securely throughout the information collection and data processing phases. While under review, electronic data will be stored in locked files on secured computer. No hard copy data will be collected. As a further guarantee of privacy and anonymity, all presentations of data in reports will be in aggregate form, with no links to individuals. Reports will be used only for research purposes and for the development of communication messages.

Before data are collected, FDA researchers will obtain an exemption for all research from FDA’s RIHSC.

The research team understands that the security of online transmissions is not guaranteed due to the risk of interception by third parties or the possibility of monitoring software installed on research participants’ electronic devices. All participants will be assured that the information will be used only for research purposes and will be kept private to the extent allowable by law and the technology used. The survey assent/consent form will include information explaining this to respondents. Participants will be assured that their names and e‑mail addresses will not be shared outside of the vendor (SSRS) and identifying information will not be associated with any response data. Participants will be told that the information obtained from all the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

The Internet panel includes a privacy policy that is easily accessible from any page on the site. A link to the privacy policy will be included on all survey invitations. The panel complies with established industry guidelines and states that members’ personally identifiable information will never be rented, sold, or revealed to third parties except in cases where required by law or with the consent of the panel members. These standards and codes of conduct comply with those set forth by American Marketing Association, the Council of American Survey Research Organizations, and others. All SSRS employees and contractors are required to take yearly security awareness and ethics training, which is based on these standards.

Data security provisions will involve the following:

• All data collection activities will be conducted in full compliance with FDA regulations to maintain the privacy of data obtained from participants and to protect the rights and welfare of human research participants as contained in their regulations. Participants will receive information about privacy protections as part of the informed assent/consent process.

• All data is encrypted at rest on the Web-based survey system, as well as encrypted during transmission to and from the Web-based survey system with the use of an SSL certificate.

• Participants will be given a unique alphanumeric variable, and no identifiable information will be collected from participants in the survey.

• All participants will be assured that the information they provide will be maintained in a secure manner and will be used only for the purpose of this research. Participants will be assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Participants will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

The JHSPH IRB has approved this study, including its procedures to protect the privacy of participants.

11. Justification for Sensitive Questions

The majority of questions are not sensitive. We will not ask for a participant’s Social Security Number (SSN). It is necessary to ask some questions that may be sensitive in nature, specifically questions regarding tobacco use. It is necessary to ask these questions in order to meet the objectives of the study, i.e., to understand whether ENDS advertising affects tobacco users and non-users differently and to ensure the sample contains sufficient numbers of users and non-users. Questions about demographic information, e.g., race, education and income level, could be considered sensitive but are not highly sensitive. These questions are necessary to assess the diversity of the sample and to ensure participants are effectively randomized across conditions.

If a respondent does not take precautions to keep his or her answers confidential when completing the survey, it is possible that someone else in the household or other space within viewing distance of the respondent’s computer could view the respondent’s answers on the computer while the survey is in progress (e.g., if the participant walks away from the computer while completing the survey), which may make some participants feel uncomfortable.

To address other concerns about inadvertent disclosure of sensitive information, potential participants will be fully informed of the applicable privacy safeguards. This study includes a number of procedures and methodological characteristics designed to minimize potential negative reactions to these types of questions, including the following:

1. Participants will be informed that they do not need to answer any question that makes them feel uncomfortable, or that they do not wish to answer.

2. Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses.

3. In the Informed Assent/Consent, participants will be provided with a phone number and email address (linking directly to the JHSPH IRB Office) to contact if they have a question or concern about the sensitive issue. In the debriefing at the end of the survey, participants will also be provided with a phone number and an email address through which they can contact the JHSPH IRB Office.

Finally, as with all information collected, these data will be presented with all identifiers removed.

12. Estimates of Annualized Burden Hours and Costs

Approximately 250,000 respondents from an internet panel will be recruited via an email invitation, which is estimated to take 1 minute to read and respond. An estimated 7,500 (3,750 youth and 3,750 young adults) respondents will provide assent and consent and be screened to yield the desired sample size of 5,000 total (2,500 youth and 2,500 young adults) participants. The consent/screening process is estimated to take an average of 7 minutes per respondent. Participants that qualify for the study will be automatically directed to begin the online survey which is estimated to take an average of 20 minutes per respondent.

The total estimated burden to respondents is 6,712 hours, as summarized in Exhibit 1.

12a. Exhibit 1. Estimated Annual Burden Hours

Exhibit 1. Estimated Annual Reporting Burden
Participant Subgroup	No. of Respondents	No. of Responses per Respondent	Total Annual Responses	Average Burden per Response	Total Hours
Number to read the survey invitation
Youth (aged 13-17)/Parent panelists of youth (aged 13-17)	125,000	1	125,000	0.016 (1 min)	2,084
Young adults (aged 18-24)	125,000	1	125,000	0.016 (1 min)	2,084
Total	250,000				4,168
Number to complete the assent/consent and screener
Youth (aged 13-17)	3,750	1	3,750	0.116 (7 min)	438
Young adults (aged 18-24)	3,750	1	3,750	0.116 (7 min)	438
Total	7,500				876
Number to complete main study
Youth (aged 13-17)	2,500	1	2,500	0.333 (20 min)	834
Young adults (aged 18-24)	2,500	1	2,500	0.333 (20 min)	834
Total	5,000				1,668
Total Hours	6,712

12b. Annualized Cost Burden Estimate

To calculate the estimate annual cost, the mean hourly wage of $7.25 was used for youth and $26.95 was used for young adults. The youth price represents the minimum wage, and the young adult costs represent the mean hourly wage for other occupation earnings from the U.S. Department of Labor Bureau of Labor Statistics (May 2020 data). There are no direct costs to respondents associated with participation in this information collection. Thus, assuming an average hourly wage of $7.25 and $26.95 (youth and young adult), the estimated one-year annualized cost to participants will be $114,775.20. The estimated value of respondents’ time for participating in the information collection is summarized in Exhibit 2.

Exhibit 2. Estimated Annual Cost

Type of Respondent	Total Burden Hours	Hourly Wage Rate	Total Respondent Costs
Youth	3,356	$7.25	$ 24,331.00
Young adults	3,356	$26.95	$ 90,444.20
Total			$114,775.20

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

There are no capital, start-up, operating or maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

This project is funded by FDA through the Johns Hopkins Center for Excellence in Regulatory Science. The total estimated cost of this data collection is $319,393. This includes JHU’s costs to develop the survey and stimuli, obtain approval from the JHSPH IRB, manage the study, coordinate with FDA, analyze data and write a report. These costs include JHU staff and student time and supplies. This amount also includes costs to SSRS, the survey vendor, to recruit participants, program the survey, and compensate participants. The cost also includes FDA staff time to design and manage the study, to analyze the resultant data, and to draft a report ($34,915; 640 hours over the life of the study).

Exhibit 3. Itemized Cost to the Federal Government

Government Personnel	Time Commitment	Average Annual Salary	Total
GS-13	15%	$100,203	$15,030
GS-14	15%	$114,590	$17,189
GS-15	2%	$134,789	$2,696
Total Salary Costs			$34,915
Contract Cost			$284,478
Total			$319,393

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

We will use repeated measures ANOVA and mixed effect regression to explore overall differences in the outcomes of interest across conditions. We will conduct similar exploratory sub-group analyses by participant ENDS/cigarette use status and other participant characteristics of interest. We will use between-subjects ANOVA and regression models to conduct analyses by flavor and brand sub-group (e.g., analyzing the effect of the flavor features among each flavor or brand individually). Exploratory analyses will probe for effect modification by flavor type and by brand, testing for interactions between ad condition x flavor and ad condition x brand.

We expect data collection to commence in June 2022 following OMB clearance. We expect data collection to be complete in about one month. Data cleaning and analysis will begin upon completion of data collection, anticipated to start in August 2022 and lasting about 6 weeks. Write up of findings and reporting to CTP is anticipated to start by November 2022. See above for analysis plan.

2022

2023

Activity/deliverable

M

J

J

A

S

O

N

D

J

F

M

A

OMB clearance

X

Data collection

X

X

Data cleaning

X

Data analysis

X

X

Manuscript write-up

X

X

X

FDA manuscript clearance

X

X

Manuscript submission

X

Report of findings to CTP

X

17. Reason(s) Display of OMB Expiration Date is Inappropriate

All data collection instruments will display the expiration date for OMB approval of the information collection.

17. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

1https://www.fda.gov/tobacco-products/research/research-priorities

2 Moran, M.B., *Czaplicki, L., Lagasse, L., *Cino, S., Trigger, S., Engstrom, M.C., Zandberg, I., Sawdey, M.D. & Kennedy, R. An analysis of Juul advertising strategy compared to Vuse. Poster presented at the 2019 meeting of the Society for Research on Nicotine & Tobacco.

3 Kennedy, R.D., *Czaplicki, L., Lagasse, L., Clawson, C., Trigger, S., Zandberg, I., Sawdey, M., & Moran, M.B. (2018). The use of flavors in business-to-business ENDS advertising. Oral flash presentation given at the NIH/FDA Tobacco Regulatory Science Meeting.

5