Ssa

SUPPORTING STATEMENT FOR THE

New

Emerging Tobacco Products Communication Initiative

Pre-testing of Ads

(OMB No. 0920-0910, Exp. Date 01/31/2024)

Part A: JUSTIFICATION

Submitted by:

Office on Smoking and Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

Department of Health and Human Services

Refer questions to:

Program Official/Contact

Sarah Lewis, MPH, CHES

Office on Smoking and Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

4770 Buford Highway, NE MS F-79

Atlanta, Georgia 30341

770-488-5582

FAX: 770-488-5939

Email: [email protected]

July 20, 2021

TABLE OF CONTENTS

1. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

2. Purpose and Use of Information Collection

3. Use of Improved Information Technology and Burden Reduction

4. Efforts to Identify Duplication and Use of Similar Information

5. Impact on Small Businesses or Other Small Entities

6. Consequences of Collecting the Information Less Frequently

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

8. Comments in Response to the Federal Register Notice & Efforts to Consult Outside Agency

9. Explanation of Any Payment or Gift to Respondents

10. Protection of the Privacy and Confidentiality of Information Provided by Respondents

11. Institutional Review Board (IRB) and Justification for Sensitive Questions

12. Estimates of Annualized Burden Hours and Costs

13. Estimates of Other Total Annual Cost Burden to Respondents and Record Keepers

14. Annualized Cost to the Federal Government

15. Explanation for Program Changes or Adjustments

16. Plans for Tabulation and Publication and Project Time Schedule

17. Reason(s) Display of OMB Expiration Date is Inappropriate

18. Exceptions to Certification for Paperwork Reduction Act Submissions

LIST OF ATTACHMENTS

Attachment 1: Focus Group Recruitment Screener

Attachment 2: Limelight Privacy Policy

Attachment 3: Focus Group Consent Form

Attachment 4: Focus Group Advance Logistics Letter

Attachment 5: Focus Group Moderator Guide

Attachment 6: Survey Questionnaire

Attachment 7: Lucid Privacy Policy and Terms of Service

Attachment 8: Survey Questionnaire Email Invitation

Attachment 9: Survey Questionnaire Eligibility Screener

Attachment 10: Survey Questionnaire Consent Form

Attachment 11: Survey Questionnaire Sample Screenshots

Attachment 12: RTI International Institutional Review Board (IRB) Approval

Notes on Excluded Attachments. In this generic information collection request (under the existing generic clearance package #0920-0910), CDC outlines a plan to qualitatively and quantitatively test ads with content that may be considered sensitive. The draft ads to be tested are not included because the ads have not been approved for public distribution by HHS/Assistant Secretary for Public Affairs (ASPA). To support adequate review of this GenIC by OMB, Centers for Disease Control and Prevention requests permission to provide OMB with a secure link to the draft ads.

• Goal of the proposed project: The goal of this proposed project is to qualitatively and quantitatively test draft ads designed to encourage educators to speak with middle and high school students about e-cigarette prevention and cessation. The resulting information will be used to develop and refine the ads used for an Emerging Tobacco Products Communication Initiative.

• Intended use of the resulting data: Once collected data are analyzed, the results will yield information needed to produce clear, credible, and effective ads that motivate educators to speak with youth about avoiding and/or quitting e-cigarette use.

• Methods to be used to collect data: Mixed methods will be used to collect data on draft ads. Qualitative testing will gather participants’ in-depth reactions to three different creative themes for the initiative, which will then be used to inform the development of ads that will be tested with quantitative methods.

Qualitative data will be collected through eight 90-minute virtual focus groups via the ZOOM™ platform conducted with a total of 48-64 participants who will be recruited using a combination of direct outreach to schools and postings to professional organization listservs, newsletters, and social media sites. Qualitative testing will explore reactions to the three creative themes, each presented as a written description, a mood board, and a sample ad in the form of a static storyboard. Participants will be asked to share thoughts on the messaging, visual design, and proposed distribution channels, which will help us develop relevant and appealing ads for the subsequent quantitative testing.

Quantitative data will be collected through a 20-minute online survey (which includes a screener, informed consent, and questionnaire) of 600 respondents who will be recruited from an existing online survey panel. The survey will compare participants’ reactions to four draft ads—including perceived effectiveness, perceived relevance, and perceived effect on self-efficacy—and will enable us to identify any differences in reactions by key demographic and professional characteristics (e.g., educator role, years of professional experience, race, ethnicity). Collectively, this qualitative and quantitative testing will allow us to refine and strengthen the ads prior to implementation. RTI International (RTI) will conduct all data collection on behalf of CDC, and there will be no overlap in participants between the focus groups and the survey.

• Populations to be studied: The study population will be educators of middle and high school students in the U.S., including teachers, coaches, and school administrators (e.g., principals, assistant principals, guidance counselors, etc.).
  
  

• How data will be analyzed: Qualitative data will be analyzed using thematic analysis. Focus group responses will be transcribed, read thoroughly, coded using software (e.g., NVivo), and analyzed for themes and patterns of response. Quantitative data will be analyzed using statistical techniques and aggregate measures, such as percentages and means. Analyses will focus on whether participants’ assessments of clarity, credibility, and effectiveness differ across materials and how the materials can be improved.

Supporting Statement: Summary

Part A. Justification for Information Collection

A.1 Circumstances Making the Collection of Information Necessary

E-cigarettes entered the U.S. marketplace in 2007, and they have been the most commonly used tobacco product among U.S. youth since 2014 (Miech et al., 2019; U.S. Department of Health and Human Services, 2016). Between 2017 and 2018, the number of youth who used e-cigarettes increased by 1.5 million, leading the Surgeon General to declare e-cigarette use among U.S. youth an epidemic (U.S. Department of Health and Human Services, 2018). In 2020, about 3.6 million U.S. youth, including nearly 1 in 5 high school students and 1 in 20 middle school students, reported currently using e-cigarettes (Wang et al., 2020). Additionally, frequency of use has also increased among high school students who use e-cigarettes in recent years, which is reflective of increasing dependence among youth nationally (Creamer, et al., 2020).

Most e-cigarettes contain nicotine, which can harm the developing adolescent brain by impacting attention, learning, and memory (U.S. Department of Health and Human Services, 2016). Nicotine is also highly addictive and can prime the adolescent brain for addiction to other drugs (National Academies of Sciences, Engineering, and Medicine, 2018; U.S. Department of Health and Human Services, 2016). In addition, a growing body of science suggests that youth who use e-cigarettes are more likely to smoke cigarettes in the future (Wang, et. al., 2019; U.S. Department of Health and Human Services, 2016). Other harmful chemicals have also been identified in e-cigarettes, including heavy metals, volatile organic compounds, and ultrafine particles that can be inhaled deeply into the lungs (U.S. Department of Health and Human Services, 2016).

To address the urgent public health concern of e-cigarette use among youth, the Centers for Disease Control and Prevention (CDC) is developing and implementing an Emerging Tobacco Products Communication Initiative to raise awareness about risks of youth e-cigarette use among youth influencers, or key adults involved with and influential in the lives of youth aged 11 to 17 years old. The primary audience is educators in U.S. middle and high schools and school-adjacent programs, including teachers, coaches, and school administrators (e.g., principals, assistant principals, guidance counselors) (Thomas, et al., 2015; Anand et al., 2015). Secondary audiences include school nurses and mental health counselors, district school boards, and parents of youth (Mogro-Wilson, 2017; Personal Communications with Kelly Sarmiento, CDC, 2021). The goals of the Emerging Tobacco Products Communication Initiative are to:

1. Prompt educators to discourage youth from trying e-cigarettes and other vaping products.

2. Motivate educators to speak with youth who currently use e-cigarettes about the importance of cessation.

CDC has implemented several communication activities to raise awareness among youth influencers about the risks of youth e-cigarette use. For instance, CDC developed the Know the Risks communication initiative to help promote the findings of the 2016 Surgeon General Report entitled “E-cigarette Use Among Youth and Young Adults.” The “Busting the Myth” public service announcement was also developed to increase awareness of youth e-cigarette risk among parents and other youth influencers. In addition, CDC developed communication materials to help disseminate key messages of the Surgeon General’s “Advisory on E-cigarette Use Among Youth” released in December 2018. During 2019, CDC’s Protect Young People from E-cigarettes communication initiative resulted in 37.7 million impressions to raise awareness about the risks of youth e-cigarette use among parents, educators, and health care providers. The initiative’s digital and social media activities resulted in 155,000 clicks to the CDC website, and more than 15,000 engagements online.

In 2021, CDC has implemented a media buy based on existing creative materials to increase awareness among educators about the risks of youth e-cigarette use. The national media placement consists of digital display, digital video, native/sponsored content, paid search, and social media activities (Facebook, Instagram, Twitter, Pinterest, and LinkedIn). By increasing knowledge and awareness among educators, these activities are laying the groundwork for the future communication initiative, which aims to shift attitudes and beliefs among educators and to motivate educators to speak with youth by discouraging them from trying or continuing to use e-cigarettes (Figure 1) (Bandura, 1986).

Figure 1. Conceptual Model for Educator Behavior Change

To help ensure the communication initiative resonates with educators and motivates them to act (e.g., talk to youth about the risks of e-cigarette use and the importance of avoiding or quitting e-cigarettes), pre-testing of draft ads is needed. Pre-testing of ads is a standard advertising research activity used in the development of health communication programs and initiatives. Additionally, pre-testing is a way to preemptively identify and address any unanticipated confusion, ambiguity, or lack of understanding of the communication initiative’s key messages and calls to action (Centers for Disease Control and Prevention, 2018; U.S. Department of Health & Human Services, 2004).

The objective of this proposed information collection is to pre-test ads among middle and high school educators who work in a variety of geographical settings (e.g., urban, suburban, and rural regions). Participants will include: (1) teachers, (2) coaches, and (3) administrators (e.g., principals, assistant principals, guidance counselors). To identify the ad design and content that is most motivating to this audience, approximately 48-64 participants will view three creative themes for the communication initiative (each of which will include a text description, a mood board, and a sample ad in the form of a static storyboard) during qualitative focus groups. The focus group participants will include 12-16 high school teachers and/or coaches, 12-16 middle school teachers and/or coaches, 12-16 high school administrators, and 12-16 middle school administrators. We will use the results from these focus groups to select one creative theme for the communication initiative and to refine and develop additional ads that align with that theme for quantitative testing.

To assess how the ads are perceived across a number of measures, including perceived effectiveness (PE) (Davis, Duke, Shafer et al., 2017), believability, visual design, and relevance, approximately 600 survey respondents (who did not participate in the qualitative focus groups) will view and react to four draft ads (e.g., print ad, GIF) that focus on the risks of youth e-cigarette use and the importance of talking with youth about prevention and cessation. All 600 survey respondents will review all four ads, although the order in which the ads are presented to respondents will be randomized. Section A.12 and Part B includes additional information on sample size calculations.

A.2 Purpose and Use of Information Collection

As stated previously, the overall purpose of this information collection is to determine which ads are perceived as most credible, comprehensible, and persuasive among middle and high school educators, and how those ads can be strengthened prior to full-scale dissemination. Pre-testing is a way to measure any unanticipated confusion, ambiguity, or lack of understanding of the intended message in the ads. If the qualitative data collection is not performed, CDC will not know whether the draft ads communicate effectively with the main target audience and will not be able to obtain important feedback on recommended changes, information gaps, and preferred dissemination channels. This could result in the production of ads that are less effective in encouraging educators to talk with youth about the importance of e-cigarette prevention and cessation. In addition, if the quantitative data collection is not conducted, CDC will not know which ads are perceived as most effective and relevant among different types of educators. The survey will help CDC understand how to improve the ads before they are finalized and distributed.

Qualitative: Web-based Virtual Focus Groups. The qualitative portion of data collection will involve conducting focus groups to gain in-depth information about participants’ reactions to the creative themes and sample ads. Focus groups will be conducted remotely using the web-based conferencing platform ZOOM™. RTI International (RTI) will be responsible for recruiting respondents for the focus groups using a combination of direct outreach to schools and postings to professional organization listservs, newsletters, and social media sites. For direct outreach, RTI will identify a subset of middle and high schools in urban, suburban, and rural areas within most states, will contact schools using publicly available contact information on school websites, and will ask them to disseminate study information to educators at the school. Educators who are interested in the study will be directed to an online survey hosted by Limelight Insights, who will review and invite eligible individuals using the eligibility screener (Attachment 1) provided by CDC. (RTI has used a similar approach on multiple federal education studies.) Participants will be recruited with the aim of meeting quotas to capture diverse perspectives across years of experience, race/ethnicity, gender, geographic setting (i.e., urban, suburban, or rural), and school setting. The identifiable information about the participants will be maintained in the proprietary records systems of the recruitment firm (Limelight Insights) and will not be released to or accessible by CDC or RTI (Attachment 2).

When scheduled, participants will receive an informed consent form (Attachment 3) to review and sign electronically. They also will receive an advance logistics letter (Attachment 4) with login credentials for joining the focus group and guidelines for protecting their privacy. At the beginning of the focus group discussion, the moderator will verbally re-review the consent form and participants’ rights and will invite participants to ask any questions. With participants’ permission, focus groups will be audio- and video-recorded. Participants’ names will not be connected to the recordings, and recordings will be stored on password-protected computers. Participants will be advised that CDC and contractor staff may observe the discussion virtually.

Following the informed consent process, the moderator will use the moderator guide (Attachment 5) to lead each focus group. The moderator guide includes a preamble that explains to the participants the purpose, sequence, and ground rules of the focus group. The guide begins with brief introductions, followed by a discussion about the three draft creative themes. Discussion questions will explore participants’ perceptions of relevance, comprehension, self-efficacy, motivation, and visual design of the mood board and draft ad for each creative theme. After providing feedback on each creative theme, participants will be asked to answer a poll identifying their favorite and least favorite theme, followed by a group discussion comparing the creative themes. The poll will help stimulate and focus the conversation but will not be used for statistical analyses. Part B includes additional information on the format of these creative themes in addition to how they will be viewed by participants. The goal of these discussions is to solicit participants’ in-depth reactions and feedback on the messaging, visual design, and desired dissemination channels. This will enable us to refine the ads by determining ways in which the ads can be made more compelling and motivate educators to talk with youth about the risks of e-cigarette use and the importance of prevention and cessation. Ultimately, this qualitative data will be used in combination with the quantitative survey data (see following section) to inform refinement of ads for the communication initiative. The focus group design and anticipated number of respondents are depicted in Figure 2.

Figure 2. Focus Group Design and Anticipated Respondents

Quantitative: Online Questionnaire. The quantitative portion of this project will collect data via an online questionnaire (Attachment 6). Potential participants will be recruited from two existing, online, convenience panels available to Qualtrics, specifically Lucid (see https://luc.id/legal/privacy-policy/ for more detail on this panel). This panel provider maintains demographic information about panelists in its proprietary database, which will not be shared with CDC or RTI (Attachment 7), and this information will be used to ensure that the invitation to participate in this project will target only individuals who are likely to be eligible. Specifically, the invitation (Attachment 8) will be sent to panel members. An online, project-specific screener (Attachment 9) will be used to assess respondent eligibility and their alignment with minimum quotas for key demographic and professional characteristics, such as gender, race/ethnicity, job type/role, years of experience, school type, and geography. In addition to confirming eligibility, the screening information will be used to assign participants to one of three strata: teachers (n=200), coaches (n=200), and administrators (n=200). Respondents will be recruited with the aim of meeting robust quotas to capture diverse perspectives across experience, race/ethnicity, gender, geographic setting (i.e., urban, suburban, or rural; Census region), and school setting.

Following the screening process, eligible respondents will complete the consent form (Attachment 10) and the online questionnaire. The purpose of the online questionnaire is to show participants the draft ads and collect quantitative information on their perceived relevance, acceptability, and effectiveness. Specifically, the questionnaire will measure reactions to the ads (e.g., perceived effectiveness (PE), personal relevance, confusion, believability, behavioral intentions); suggestions for improving the ads; and participant demographic characteristics. Draft ads, in the format of prototypes or mockups (e.g., display ads, GIFs), will be presented to all respondents. This design results in high internal validity, given that all respondents will be seeing the same ads and answering the same questions, even though external validity (generalizability) is limited, given that the underlying panel was not created using address-based sampling or other probability-based methods. High external visibility is not needed for this information collection because the intent is to inform the design and implementation of ads rather than to produce generalizable results for research purposes. The survey design and anticipated number of respondents are summarized in Figure 3.

Figure 3. Survey Design and Anticipated Respondents

All information collected in the online questionnaire will allow for study outcomes to be measured and confounding influences controlled for. Some of the key variables measured are summarized in the Table 1 below.

Table 1. Rationale for variables measured in online survey

Measure	Construct	Questions
Screener variables (to achieve diversity quotas)	Age, role/job, level of student contact, school type, gender, race/ethnicity, years of experience, school type, geography	Screener Questions S1, S2, S3, 26, S8, 29, S10, S11, S12
Tabulation variables	Role/job, school type (middle vs. high school), geography (urban vs. rural vs. suburban; Census region), gender identity, race/ethnicity	Screener Questions S2, S3, S6, S7, S9, S10, S11, S12
Stratification variables	Role: teacher, coach, and administrator	Screener Questions S2 and S3
Key Outcomes	Knowledge; attitudes; perceived effectiveness of material; self-efficacy, outcome expectation, and behavioral intentions after viewing material; credibility and relevance of material; visual design of material	Knowledge: Q4 and Q5 Attitudes: Q2 and Q3 Perceived effectiveness of material: Q10 Self-efficacy: Q14 Outcome expectation: Q15 Behavioral intention: Q6, Q16, Q17 Material credibility and relevance: Q18 Material visual design: Q19 *Note: the questions about the ads are repeated for each ad presented
Potential confounding variables	Moderators, confounders, or contextual variables	Personal e-cigarette use (Q60-61), personal tobacco use (Q60-61), subjects taught (Q54), age (S1), school type (public vs. private, middle vs. high school) (S6, Q57), geography (Q58)
Other contextual variables	Behavioral beliefs, Normative beliefs, control beliefs	Attitudes: Q2-3

A.3 Use of Improved Information Technology and Burden Reduction

During quantitative data collection, all information (from the screener and the questionnaire) will be collected electronically utilizing Qualtrics, an integrated web-based software platform (Attachment 11). Qualtrics will utilize the Lucid convenience panel. Web-based surveys are an especially convenient option for eliciting feedback on visual, audio, and textual stimuli such as the communication materials to be tested. The use of a web-based platform also offers a number of benefits for managing the quantitative data collection:

• First, use of an existing online panel will allow CDC to obtain information quickly so that needed adjustments to health messaging can be made expeditiously and communication development can progress rapidly. The panel used for this testing is very large (more than 100 million people in the U.S.), allowing quick selection of respondents from small subgroups of the population, such as teachers, coaches, and administrators. Samples from this panel are not designed to generate nationally representative samples or precise population parameters but rather are used as a highly efficient, low cost, and low burden method of data collection for pre-testing of ads.

• Second, when a respondent enters the screener for this project, the link to his or her identifiable information is severed (i.e., the link to the identifiable information maintained by the panel provider). None of the information collected through screening or the online questionnaire is identifiable, providing a secure environment for respondents.

• Third, this technology permits respondents to complete the instruments in private. Providing the respondent with a methodology that improves privacy makes reporting of potentially embarrassing or stigmatizing behaviors (e.g., e-cigarette use) less threatening and enhances response validity and response rates.

• Finally, the web-based software system includes embedded logic that will route respondents efficiently through the screener and onto the online questionnaire (or a “thank you” screen if the respondent is found to be ineligible). This approach can increase response rates (which decreases time and costs related to information collection procedures) by reducing the number of respondents needed to complete the screener in order to achieve the desired enrolled sample size (i.e., by reducing drop off between the screener and questionnaire).

Overall, the software supports an efficient assignment and routing process, as well as a smooth user experience that would be difficult to attain in other modes of data collection.

During the qualitative data collection, RTI will be responsible for recruiting respondents for the focus groups using a combination of direct outreach to schools and postings to professional organization listservs, newsletters, and social media sites. RTI will partner with Limelight Insights, a recruitment firm, to review and invite eligible individuals using the eligibility screener (Attachment 1) and target quotas. Focus group discussions will be moderated using ZOOM™, a high-definition video conferencing service. The use of this service will facilitate and expedite the focus group transcription and analysis process. Additionally, the virtual focus group process will protect participants’ physical health during the COVID-19 pandemic and support quality control measures, as CDC staff and other contractor staff will be able to remotely observe the focus group discussions. The secure link to the virtual focus group session will be available to staff to observe the live session.

A.4 Efforts to Identify Duplication and Use of Similar Information

This generic information collection request (under the existing generic clearance package #0920-0910) is designed to pre-test ads to inform the development of a future Emerging Tobacco Products Communication Initiative. To prepare for data collection, CDC reviewed existing published literature, unpublished research, and evaluation reports when they were available and also conducted a social media scan.

CDC collaborates with other federal government agencies that sponsor or endorse health communication projects, such as the Food and Drug Administration (FDA) Center for Tobacco Products (CTP). Staff members in CDC’s Office on Smoking and Health work closely with staff in CTP’s Office of Health Communication and Education. Regularly scheduled conference calls are held to review plans and share research findings of mutual interest. For instance, CDC met with FDA on March 11, 2021, May 6, 2021, and June 8, 2021, to discuss our overall communication strategy and approach to ad testing. These collaborations serve as information channels, help prevent redundancy, and promote use of consistent measures of effectiveness. Coordination activities also include the review of data collection instruments and other support materials for testing purposes.

FDA CTP is investing in a number of public education campaigns aimed at youth and young adults, such as The Real Cost, to educate them about the risks of e-cigarettes. In contrast, CDC’s Emerging Tobacco Products Communication Initiative will focus on adult audiences (e.g., educators) as key influencers of youth behavior.

Given that CDC and FDA are developing complementary but distinct messages to educate the public about the risks of youth e-cigarette use, CDC has shared a courtesy copy of data collection instruments. CDC will also share the findings of this information collection with FDA to ensure that future duplication of efforts is preempted.

Points of contact for this coordination are:

• CDC: Brian Armour, Associate Director for Science, Office of the Associate Director for Science, telephone (404) 498-3014, email [email protected]

• CDC: Elizabeth Courtney-Long, Health Scientist, Office of the Associate Director for Science, telephone (404-498-0264), email [email protected]

• CDC: Jane Mitchko, Deputy Chief, Health Communications Branch, telephone (770) 488-5752, email [email protected]

• CDC: Sarah Lewis, Health Communications Specialist, Campaign Development, Health Communications Branch, telephone (770) 488-7424, email [email protected]

• CDC: Michelle O’Hegarty, Health Communications Specialist, Campaign Development, Health Communications Branch, telephone (770) 488-5582, email [email protected]

• CDC: Lauren Boyle-Estheimer, Senior Health Communications Specialist, Health Communications Branch, telephone (404) 498-2283, email [email protected]

• FDA: Matthew Walker, Lead Health Scientist, Office of Health Communication and Education, telephone (240) 402-3824, email [email protected]

A.5 Impact on Small Business or Other Small Entities

RTI plans to use Limelight Insights to recruit and screen focus group participants.

A.6 Consequences of Collecting the Information Less Frequently

This is a one-time information collection request.

A.7 Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances that require data collection to be conducted in a manner inconsistent with 5 CFR 1320.5 (d) (2). The information collection fully complies with the guidelines in 5 CFR 1320.5.

A.8 Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

A.8.a Federal Register Announcement

A Notice was published in the Federal Register on October 13, 2020, volume 85, number 198, pp. 64467 - 64468. CDC received one comment from the tobacco industry about testing of messages among adult audiences to inform harm reduction communications. CDC provided a courtesy response.

A.8.b Consultations

CDC’s Emerging Tobacco Products Communication Initiative has been funded primarily from the Office on Smoking and Health Division budget authority funds. CDC did not consult outside of the agency on the ads to be tested.

A.9 Explanation of Any Payments or Gift to Respondents

For the qualitative data collection, we will provide participants a monetary gift of $75 to participate in a 90-minute focus group. This proposed token of appreciation of $75 is intended to convey appreciation for contributing to this important project. Such an appreciation is especially important for educators, who will be participating in a professional (rather than personal) capacity and whose workload has increased substantially with the onset of remote learning at many middle and high schools.

For the quantitative data collection, respondents will be drawn from established panels maintained by Lucid, and Qualtrics will provide each respondent who completes the survey in full with a $5 Amazon e-gift card (for a 20-minute survey) to respondents to encourage response. Immediately upon completion of the survey, each respondent will be provided with their token of appreciation. Studies have indicated that a monetary gift can increase response rates (Church, 1993; Greenbaum, 2000; Haveman, 2010).

A.10 Protection of the Privacy and Confidentiality of Information Provided by Respondents
This submission has been reviewed by staff in CDC’s National Center for Chronic Disease Prevention and Health Promotion, who determined that the Privacy Act does not apply. This determination is based on the fact that no personal identifiers will be collected in this proposed project to reduce the likelihood of identification or re-identification. CDC has contracted with RTI for this information collection. All data collected and delivered to CDC from RTI’s data collection will be in aggregate form only. Further, the information that will be reported to and maintained by CDC is not considered a record as defined by the Privacy Act: It will not include individuals’ education, financial transactions, medical history, and criminal or employment history and name, or the identifying number, symbol, or other identifier assigned to any individual, such as a finger or voice print or a photograph. Staff from CDC, RTI, and the two recruitment firms (Limelight Insights and Qualtrics) will participate in planning the information collection; staff from CDC and RTI will interpret data but will not receive any Personally Identifiable Information (PII) on the respondents. RTI’s Institutional Review Board (IRB) reviewed and approved this project (Attachment 12). The IRB’s primary concern is protecting respondents’ rights, one of which is maintaining the privacy of participant information to the fullest extent of the law.

Privacy and Confidentiality: Qualitative Data Collection

The identifiable information of the participants we recruit for the qualitative data collection will be maintained in a proprietary records system and will not be released to CDC or other contractors/subcontractors (Attachment 2). Although demographic information (e.g., age, gender, race/ethnicity, etc.) will be confirmed through screening, no direct personal identifiers (e.g., date of birth [including day, month, year], name, phone number, address, email address, social security number, photograph, biometric information, or any other unique identifier that can be linked to an individual) will be collected or maintained as part of the screening process. A system of records notice (SORN) is not required because (1) the information collected is not considered a record as defined by the Privacy Act and (2) the records are not retrieved using a personal identifier.

At the completion of the qualitative focus groups, Limelight will provide participant rosters to RTI and CDC. These participant rosters will not include any personally identifiable information such as full name or email address; instead, they will include information from the screener such as age, school location (i.e., urban, suburban, rural), race, and ethnicity.

Privacy and Confidentiality: Quantitative Data Collection

All information for the self-administered screening process and self-administered questionnaire will be collected electronically in a secure, web-based data collection system (as described in Section A.2 and Part B). The identifiable information about Lucid panelists is maintained in a pre-existing proprietary records system and is not released to CDC or other contractors/subcontractors (Attachment 7). Although demographic information (e.g., age, gender, race/ethnicity, etc.) will be confirmed through screening, no direct personal identifiers (e.g., date of birth [including day, month, year], name, phone number, address, email address, social security number, photograph, biometric information, or any other unique identifier that can be linked to an individual) will be collected or maintained as part of the screener or questionnaire (Attachments 9 and 6). A system of records notice (SORN) is not required because (1) the information collected is not considered a record as defined by the Privacy Act and (2) the records are not retrieved using a personal identifier.

When the respondent begins the questionnaire, all identifiable links to the existing system of records are severed. As such, because it does not exist, CDC will not have direct contact with or access to any PII about participants during this stage. Qualtrics does have access to the email address of panel subscribers, but no match back is possible with the survey response data. IP addresses will not be stored by the online questionnaire system, and no first- or third-party cookies will be stored during questionnaire completion. No link between the respondent’s email and the specific survey is made after the potential respondent clicks on the link to start the survey.

Data Security
All findings will be reported in aggregate form only. All information will be stored on password-protected databases to which only RTI employees and the survey vendor, Qualtrics, working on this project have access. RTI will keep the quantitative data in non-aggregate form for six months after information collection has been completed, and then the respondent-level data will be deleted from the password-protected databases. RTI will provide CDC with the de-identified data to be used for analyses. Only CDC and RTI employees involved in data analysis will have access to the data. CDC will handle the de-identified data in accordance with the record control schedule (maintained at least six years, but no longer than ten years). No desktop or laptop computer will contain any PII. To prevent unauthorized access to their data servers (such as “hacking”), RTI and Qualtrics are both currently certified and have achieved the distinguished ISO 27001 accreditation. With this achievement, RTI and Qualtrics data systems have assurance that all data will be managed in a secure environment. This means that RTI and Qualtrics have been formally audited and have been certified compliant with the standard ISO 27001 accreditation. CDC will retain and destroy records in accordance with the applicable CDC Records Control Schedule (Table 2).

Table 2. Access Controls
Technical Controls	Physical Controls	Administrative Controls
User identification Passwords Firewall Virtual Private Network (VPN)	Guards/Security Officers 24-hour maintenance of Video/Audio of all data centers and all offices Identification badges Key Cards	The system security plan for the information collection is that survey data and all identifying information about respondents will be handled in ways that prevent unauthorized access at any point during the proposed project. The contingency plan for this information collection is that the screeners will be kept only on password-protected computer files stored on Qualtrics and RTI servers. No directly identifying information will be transmitted to CDC (thus, the Privacy Act does not apply). Backup file storage: RTI has a redundancy system stored on a FedRAMP-certified server farm for data security and quality. Reports will not include any identifiable information. There will not be user manuals for this information collection effort. Personnel who use the system will be trained to protect the information being collected and maintained by adhering to a procedure that removes identifiers from response data. Contractors who are operating/using the system will include clauses in the contracts that adhere to privacy provisions and practices. Methods will be in place to ensure least privilege. Data and all identifying information about respondents will be handled in ways that prevent unauthorized access at any point during the proposed project. There are policies/guidelines in place regarding the retention and destruction of PII: PII will not be transmitted to CDC, and PII will not be linked to response data.

A.11 Institutional Review Board (IRB) and Justification for Sensitive Questions

IRB Approval

All procedures have been developed in accordance with federal, state, and local guidelines to ensure that the rights and privacy of participants are protected and maintained. RTI’s IRB has reviewed and approved this proposed project (Attachment 13).

Sensitive Questions

The majority of questions asked in the screeners (Attachments 1 and 9), moderator’s guide (Attachment 5), and online questionnaire (Attachment 6) will not be sensitive in nature, as perceived by CDC, RTI , and Qualtrics. Questions asked during the screening about demographic information (e.g., age, gender, race/ethnicity, job type/role, years of experience, school type, geography) could be considered sensitive although these items would not generally be considered highly sensitive. It will also be necessary to ask some questions considered to be sensitive in order to assess individuals’ perceptions and to test draft ads about the risks of youth e-cigarette use. These items are not generally considered highly sensitive either. Participants will be informed of the applicable privacy safeguards. Sensitive information will be requested only when necessary to describe sample characteristics (e.g., age). Such questions will include a “prefer not to answer” option. Participants in the qualitative portion of the project will be told in the informed consent form that they can refuse to answer any question or leave the focus group at any time, and they will still receive their token of appreciation. This proposed project also includes a number of procedures and methodological characteristics that will minimize potential negative reactions to potentially sensitive questions, including the following:

• The focus group moderators are trained to navigate sensitive topics.

• The online questionnaire is entirely self-administered and maximizes participant privacy by being conducted online, without the need to verbalize responses.

• Participants will be provided with a phone number and email for the principal investigator and for the IRB, should they have any questions or concerns about the proposed project or their rights as a participant.

A.12 Estimates of Annualized Burden Hours and Costs

In the qualitative portion of the project, the goal is for a total of 48-64 respondents to participate in eight web-based focus groups. See Part B for information on how the estimated sample size was calculated. To obtain this sample size, it is expected that 90 respondents will need to complete the screener and 64 respondents will need to be scheduled to account for possible cancellations and no-shows. The burden per respondent for completing the screener to participate in the focus group is two minutes. The total estimated burden for completing the screener is 3 hours (all screened individuals).

The focus groups will include six to eight participants per each of the eight focus groups; 64 participants will be invited to participate in the focus groups, anticipating that some will not attend. If more than six participants attend, the additional participants will be permitted to participate. All invited participants (n=64) will be asked to complete the informed consent form, estimated to take 5 minutes. As such, the total estimated burden for completing the informed consent form is 5.3 hours (all invited eligible individuals).

The burden per respondent for completing the focus group (n=48 to 64) is 90 minutes. The total estimated burden for completing the focus groups is 96 hours (all completed participants).

The quantitative data collection includes a two-minute screener and 20-minute online questionnaire. As described in Section A.1, the draft ads will be tested with approximately 600 respondents. See Part B for an explanation of how this sample size was calculated.

To obtain a final sample size of 600, approximately 9,000 respondents are anticipated to complete the online screener (Attachment 9). Part B explains the calculations behind this sample size, and the screener figure is based on prior experiences in the field, which indicate that roughly 84 percent of screener respondents (n=7,570) will be deemed ineligible for the study because of not meeting inclusion criteria by not having direct contact with students or not meeting our segment criteria (e.g., working as a coach). In addition, another 800 individuals are likely to be excluded because, although eligible, they do not meet the necessary demographic or professional quotas (e.g., geographic location, race). The burden per respondent for completing the screener is two minutes. The total estimated burden for completing the screener is 300 hours (all screened individuals).

Those who meet the inclusion criteria and demographic/professional quotas will be administered the online questionnaire (Attachment 6). Of those 630 deemed eligible and invited to participate, an estimated additional five percent (n=30) will start but not complete the questionnaire. The burden per respondent for completing the online questionnaire is 20 minutes. Those who start but do not complete the questionnaire are estimated to spend about one-half of that time (10 minutes) on the questionnaire. Thus, the total estimated burden for completing the online questionnaire is 205 hours (all completed participants).

As outlined in Table 3, the total estimated burden for the entire project is 609 hours.

Table 3. Estimated Annualized Burden to Respondents
Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in minutes)	Total Burden (in hours)
Educators in U.S. Middle and High Schools and School-Adjacent Programs	Recruitment Screener for Focus Groups (Attachment 1)	90	1	2/60	3
	Informed Consent (Attachment 3)	64	1	5/60	5
	Moderator’s Guide for Focus Groups (Attachment 5)	64	1	90/60	96
	Recruitment Screener for Online Questionnaire (Attachment 9)	9,000	1	2/60	300
	Online Questionnaire and Consent (Attachments 6 and 10)	600	1	20/60	200
		30	1	10/60	5
Total 609

The estimated cost of the time devoted to this information collection by respondents is $17,517.38, as summarized in Table 4. To calculate this cost, we used the mean hourly wage of $28.75, which represents the Department of Labor estimated mean for Education Instruction and Library Occupations earnings (Bureau of Labor Statistics, 2020). There are no direct costs to respondents associated with participation in this information collection.

Table 4. Estimated Annualized Cost to Respondents
Type of Respondent	Form Name	Number of Respondents	Number of Responses per Respondent	Average Burden per Response (in minutes)	Total Burden (in hours)	Hour Wage Rate	Total Cost
Educators in U.S. Middle and High Schools and School-Adjacent Programs	Recruitment Screener for Focus Groups (Attachment 1)	90	1	2/60	3	$28.75	$86.25
	Informed Consent (Attachment 3)	64	1	5/60	5.3	$28.75	$152.38
	Moderator’s Guide for Focus Groups (Attachment 5)	64	1	90/60	96	$28.75	$2,760.00
	Recruitment Screener for Online Questionnaire (Attachment 9)	9,000	1	2/60	300	$28.75	$8,625.00
	Online Questionnaire and Consent (Attachments 6 and 10)	600	1	20/60	200	$28.75	$5,750.00
		30	1	10/60	5	$28.75	$143.75
Total $17,517.38

A.13 Estimates of Other Annual Cost Burden to Respondents and Record Keepers

There will be no respondent capital or maintenance costs.

A.14 Annualized Cost to the Government

Approximately 6.25% of one full-time equivalent (FTE) and 1.9% of one senior manager will be required to oversee the information collection activities for one month. Responsibilities will include internal coordination and review of ads and reports and maintaining proper accounting of burden hours. The agency estimates that it will take a GS-13, at a wage rate of $46.67/hour, approximately 10 hours to manage the project, totaling about $466.70. It is estimated to take a GS-14, at a wage rate of $55.15/hour, approximately three hours to oversee the total project, totaling $165.45. The total average annualized cost to the government for CDC oversight is $632.15.

Contractors will conduct the majority of information collection and management activities on CDC’s behalf. The total cost of the data collection to contractors is $159,234, which includes consultation, instrument design and development, respondent tokens of appreciation, data collection and analysis, and final report. Activities are coordinated through a contract with RTI, a non-profit research organization that researchers, designs, implements, and evaluates media campaigns. The grand total cost for the project, including government and contractor cost, is $159,866 (Table 5).

Table 5. Total Project Costs
Government Personnel	Percent Time Commitment	Hour Time Commitment	Hourly Basic Rate	Total
GS-13	6.25%	10	$46.67	$466.70
GS-14	1.9%	3	$55.15	$165.45
Subtotal, Government Personnel Contract Costs Total Costs				$632.15
				$159,234
				$159,866

A.15 Explanation for Program Changes or Adjustments

This is a new data collection under an existing generic clearance package (#0920-0910).

A.16 Plans for Tabulation and Publication and Project Time Schedule

Data Tabulation Plans

The information will be used to inform the development of ads for a future Emerging Tobacco Products Communication Initiative. It is anticipated that information collection will begin October 17, 2021, so an OMB approval date of October 16, 2021 is requested. The resulting quantitative data will be analyzed using conventional tabulation techniques. These dates may be adjusted depending on the approval process of this package.

Publication and Dissemination Plans

The final ads will be placed as part of future media buys, and campaign materials will be disseminated to CDC partners. Additionally, a comprehensive formative evaluation report summarizing findings from this information collection will be provided to CDC.

Project Time Schedule

Table 6. Project Time Schedule

Table 6. Project Time Schedule
Activity	Time Schedule
Focus group recruitment for qualitative testing	1-12 days after OMB approval
Focus group data collection	2-6 weeks after OMB approval
Complete field work	30-45 days after OMB approval
Validation	45-55 days after OMB approval
Data analysis	55-65 days after OMB approval
Report writing	65-95 days after OMB approval
Email invitations sent to respondents for quantitative testing	75-95 days after OMB approval
Online questionnaire data collection	75-110 days after OMB approval
Complete field work	95-115 days after OMB approval
Validation	115-125 days after OMB approval
Data analysis	125-135 days after OMB approval
Report writing	135-160 days after OMB approval

A.17 Reason(s) Display of OMB Expiration is Inappropriate

An exemption to this requirement is not being requested. The expiration date of OMB approval will be displayed on all information collection instruments.

A.18 Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

References

Anand, V., McGinty, K. L., O'Brien, K., Guenthner, G., Hahn, E., & Martin, C. A. (2015). E-cigarette use and beliefs among urban public high school students in North Carolina. Journal of Adolescent Health, 57(1), 46-51.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, N.J: Prentice-Hall.

Creamer, M. R., Jones, S. E., Gentzke, A. S., Jamal, A., & King, B. A. (2020). Tobacco product use among high school students. Morbidity and Mortality Weekly Report, 69(1), s56-s63.

Miech, R., Johnston, L., O’Malley, P. M., Bachman, J. G., & Patrick, M. E. (2019). Adolescent vaping and nicotine use in 2017-2018: U.S. national estimates. New England Journal of Medicine, 380(2), 192-193.

Mogro-Wilson, C., Allen, E., & Cavallucci, C. (2017). A brief high school prevention program to decrease alcohol usage and change social norms. Social Work Research, 41(1), 53–62.

National Academies of Sciences, Engineering, and Medicine. (2018). Public health consequences of e-cigarettes. Washington, D.C.: The National Academies Press.

Thomas, R. E., McLellan, J., & Perera, R. (2015). Effectiveness of school-based smoking prevention curricula: Systematic review and meta-analysis. BMJ Open, 5(3), e006976.

U.S. Department of Health & Human Services. (2004). Making health communication programs work: A planner's guide, pink book. Washington, D.C.: U.S. Department of Health & Human Services, National Institutes of Health, National Cancer Institute.

U.S. Department of Health and Human Services. (2016). E-cigarette use among youth and young adults: A report of the surgeon general. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention.

U.S. Department of Health and Human Services. (2018). Surgeon general’s advisory on e-cigarette use among youth. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention.

U.S. Department of Health and Human Services. (2018). Best practices user guide: Health communications in tobacco prevention and control. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.

Wang, T. W., Neff, L. J., Park-Lee, E., Ren, C., Cullen, K. A., & King, B. A. (2020). E-cigarette use among middle and high school students in the United States. Morbidity and Mortality Weekly Report, 69(1), 1-3.

Wang., T. W., Gentzke, A. S., Creamer, M. R., Cullen, K. A., Hayes-Holder, E., Sawdey, M. D., Anice, G. M., Portnoy, D. B., Hu, S., Homa, D. M., Jamal, A. & Neff, L. J. (2019). Tobacco product use and associated factors among middle and high school students: United States. Morbidity and Mortality Weekly Report, 68(12), 12-28.



17 of 24