Fresh
Empire Campaign: Wave 2 Quantitative Study of Reactions to Rough-Cut
Advertising
Designed to Prevent Youth Tobacco Use
0910-0810
Supporting Statement: Summary
|
Data Collection Instruments
Attachment A: Screener (In-Person Electronic)
Attachment B: Screener (In-Person Paper)
Attachment C: Screener (Social Media Electronic)
Attachment D: Copy Testing Questionnaire
Assent & Consent Forms
Attachment E: Parental Opt Out Form (In-Person Recruitment)
Attachment F: Parental Consent Form (In-Person Recruitment)
Attachment G: Parental Consent Verbal Script (In-Person Recruitment)
Attachment H: Parental Opt Out Form (Social Media Recruitment)
Attachment I: Participant Assent Form (In-Person Recruitment)
Attachment J: Participant Assent Form (Social Media Recruitment)
Social Media Advertisements & Study Stimuli
Attachment K: Social Media Advertisements
Attachment L: Video Ad Stimuli
IRB Approval
Attachment M: IRB Approval Letter
Study Locations
Attachment N: School Recruitment Locations
Fresh Empire Campaign: Wave 2 Quantitative Study of Reactions to Rough-Cut Advertising Designed to Prevent Youth Tobacco Use
0910-0810
A. Justification
Circumstances Making the Collection of Information Necessary
On June 22, 2009, the Food and Drug Administration (FDA) was granted new authority to regulate the manufacture, marketing, and distribution of tobacco products and to educate the public about the dangers of tobacco use. Under the Family Smoking Prevention and Tobacco Control Act (Tobacco Control Act) (P.L. 111-31), FDA is responsible for protecting the public health and reducing tobacco use among minors. Section 1003(d)(2)(D) of the Food, Drug and Cosmetic Act (21 U.S.C. Section 393(d)(2)(D)) and Sections 2, 3, 105, 201, 204, 904, and 908 of the Tobacco Control Act support the development and implementation of FDA public education campaigns related to tobacco use. Accordingly, FDA will implement multi-strategy youth-targeted public education campaigns to reduce the public health burden of tobacco. FDA has contracted with Rescue Social Change Group (Rescue SCG) for the development of FDA’s Fresh Empire Campaign. This campaign will utilize a paid media campaign, community engagement activations, and a comprehensive social media effort targeted to the Hip Hop peer crowd in order to reach multicultural teens who are cigarette experimenters or at-risk non-triers with a tobacco prevention message.
Tobacco use is the leading preventable cause of disease, disability, and death in the United States (USDHHS, 2014). More than 480,000 deaths are caused by tobacco use each year in the United States (USDHHS, 2014). Each day, more than 2,600 youth in the United States try their first cigarette, and nearly 600 youth become daily smokers (SAMHSA, 2015). The FDA Center for Tobacco Products (CTP) was created to carry out the authorities granted under the 2009 Tobacco Control Act, to educate the public about the dangers of tobacco use and serve as a public health resource for tobacco and health information.
For FDA’s Fresh Empire Campaign, the Hip Hop peer crowd was selected as the target audience as they are a prominent youth peer crowd and demonstrate above average rates of smoking prevalence (Lee et al., 2014; van der Rijt et al., 2002).
Research conducted by campaign contractor Rescue SCG over the past decade has identified several prominent youth peer crowds across the US, with the Hip Hop peer crowd being one of the most common overall and among multicultural youth in particular. Research has consistently noted higher rates of tobacco use among youth influenced by the Hip Hop peer crowd as compared to other youth peer crowds (Lee et al., 2014; Rescue SCG research; van der Rijt et al., 2002). In one study, the odds of smoking were roughly twice as high for Hip Hop youth as compared to youth primarily influenced by the Mainstream peer crowd (OR=1.97, 95% CI: 1.03, 3.76) (Lee et al., 2014). The Hip Hop peer crowd, therefore, represents a multicultural peer crowd at high risk of smoking or initiation, and thus is in need of health education messaging.
The campaign targets youth who have historically been underserved by previous tobacco control efforts and are considered a hard-to-reach population. The Fresh Empire campaign complements the FDA’s general market at-risk youth education campaign, “The Real Cost,” which launched in February 2014.
A peer crowd can be defined as a subculture with which an adolescent identifies. Peer crowds share common influences, life experiences, media habits, fashion styles, and musical interests—even across geographic locations. The 2012 Preventing Tobacco Use Among Youth and Young Adults Surgeon General Report reported that there is “sufficient evidence to conclude that there is a causal relationship between peer group social influences and the initiation and maintenance of smoking behaviors during adolescence.” The FDA’s strategic approach for Fresh Empire leverages key insights from primary focus group research across multiple states with at-risk multicultural youth, as well as extensive secondary research on social influences in the hip-hop community.
While multicultural teens identify with multiple peer crowds, the FDA chose to target at-risk multicultural youth who identify with the hip-hop peer crowd because research estimates that multicultural hip-hop youth are approximately 50 percent more likely to use tobacco than mainstream multicultural youth (Lee et al., 2014). Other research supports findings of higher tobacco use among hip-hop youth (Fallin et al., 2015; Fuqua et al., 2012).
FDA formative research has found that while the hip-hop culture encourages positive values such as working hard to be successful and overcoming personal struggles, at times it can also promote imagery and messages that portray tobacco use as a desirable behavior. By including tobacco use as part of lyrics and modeling the behavior on music videos and magazines, some hip-hop influencers help establish tobacco use as a peer-crowd norm. As part of FDA’s Fresh Empire Campaign, FDA will implement paid media advertising that highlights the negative health consequences of tobacco use. Before these video ads may be aired, they must undergo copy testing to assess overall ad performance and the potential for unintended consequences related to viewing the ads. The objective of the proposed data collection is to measure perceived ad effectiveness and unintended consequences of viewing the ads among the campaign target audience of multicultural teens ages 12-17 who are cigarette experimenters or at-risk non-triers and are influenced by the Hip Hop peer crowd.
This study is designed to measure youth reactions to 4 video ads (Attachment L). The study will be conducted using an electronic Screener and Questionnaire. The study will recruit approximately 855 youth who are 12-17 years old, influenced by the Hip Hop peer crowd, and who have experimented with cigarettes (i.e. have reported smoking at least 1 puff of a cigarette, but no more than 99 cigarettes in their lifetime) or who are at-risk non-triers (e.g. would smoke if a friend offered them a cigarette). Participants will be randomly assigned to the control group, where they will not view any ads, or to the ad-viewing group, where they will be asked to view 2 randomly assigned ads and provide quantitative and qualitative feedback about the ads. All participants will be asked to answer questions about their knowledge, attitudes and beliefs about tobacco use. Approximately one-third of the sample will be chosen at random to be control participants. Differences in responses from the control group will be compared with those from the ad viewing group as a check for potential unintended consequences of viewing the video ads.
Participants will be recruited using two strategies: (1) in-person recruitment from middle and high schools in multiple metropolitan areas; and (2) social media recruitment using targeted advertisements.
In-Person Recruitment
In-person recruitment will occur in middle and high schools across the United States. Schools are selected based on density of target audience as well as building geographic diversity into the sample. Districts with schools with high proportions of multicultural youth are selected. CTP contractors complete district applications to conduct research in advance of approaching schools. The district application process includes sharing all IRB approvals with the school district as part of the application process. Generally district approvals are valid for one school year, and need to be submitted prior to the beginning of the school year when research will be conducted. Inclusion criteria for district selection includes districts with schools that have at least 50% non-Caucasian student populations.
Individual schools will then be approached and recruited between March, and May of 2016. The first step in the school recruitment process is to demonstrate to the school administration that we have obtained district level permission to conduct research in their school. . School recruiters, overseen by the District & School Recruitment Team Lead and In-Person Lead, will submit IRB approval documentation to principals in order to clarify the goals of the research and inform them of privacy assurances. School officials will be given assurance that school and student-level information will be protected and that data will only be reported in aggregate form (i.e. no data will be presented that can be traced back to a particular school or student). The District & School Recruitment Team Lead and In-Person Lead will also ensure that school officials are provided information regarding the Protection of Pupil Rights Amendment (PPRA) and their obligation to inform parents of research activities on campus. Final approval to conduct research at each school will be obtained through the school principal. At this time all district and school recruitment has been completed. The list of recruited schools can be found in Attachment N.
Once permission to conduct research at schools is established, researchers will schedule a site visit to recruit participants onsite during lunch periods. At each recruitment session, In-Person Data Collectors will introduce themselves and explain that they will be conducting a study regarding teen culture and health (see Screener Script). If the student is interested and available, the team member will explain that students are selected via a screening survey (Screener) that they can take immediately. In a non-random fashion, Data Collectors will sample as many students as possible from every area of the lunchroom. Data Collectors will never turn away youth who ask to fill out a Screener. Potential participants will be informed that any information they provide will be private and not shared with the school or their parents.
The Screener will be completed electronically on a password-protected tablet. A paper version of the survey will also be available in case of technical difficulties with a tablet (see [Screener – In-Person Electronic] and [Screener – In-Person Paper]).
The Screener will collect the following information in two phases:
• Phase 1
o Demographic information: age (for verification of 12-17 age range), race/ethnicity (to ensure that sample consists of youth from a variety of racial and ethnic backgrounds as set forth by campaign goals), and sex (to ensure that male and female participants are roughly equal);
o Self-reported cigarette use or susceptibility to cigarette use: number of cigarettes smoked per lifetime, a series of validated questions assessing susceptibility to cigarette use;
o Battery of non-tobacco related behavioral questions utilized as “distractors” to help minimize non-response due to perceived tobacco-related nature of research (not utilized as qualification criteria);
o Questions related to previous research participation and tobacco industry affiliation;
o Assessment of Hip Hop peer crowd influence.
• Phase 2
o Identifying information for recruitment coordination: youth name, last classroom on day of recruitment (to notify selected participants), last classroom on the study day (for final study reminder), cell or home phone number (to text message or call the evening of recruitment to confirm study participation and to remind students of the location the night before participation), and youth email address (to complete the Copy Testing Questionnaire on their own device if they do not attend the study session).
Youth will be notified at the end of Phase 1 if they did not qualify in order to prevent collection of unnecessary data from ineligible youth. If youth qualify after Phase 1, they will automatically continue on to Phase 2 questions. This approach ensures that identifying information is only collected if youth qualify to complete the Copy Testing Questionnaire.
In-Person Data Collectors will invite qualifying youth to attend an after school study session. Timing and location of questionnaire administration is school dependent. Study sessions are organized by CTP contractors in collaboration with the school’s administration, in accordance with the school schedule, and on school grounds. Youth will be notified of their qualification on the same day either during lunch or their last class period with a written notification (see Informational Packets), and reminded the day of the study session with a written note (see Student Reminders). Data Collectors will also text message potential participants with cell phones or call landlines the evening of recruitment and the night before the study session to remind youth about their invitation to participate. Eligible youth ages 13-17 who do not attend an in-person study session and instead complete the Copy Testing Questionnaire on their own device will be emailed the study link and up to 2 reminders as needed.
Inclusion/exclusion is based on the campaign target audience, and includes age, tobacco use status/susceptibility, peer crowd affiliation, and no familial conflict of interest.
The process for determining eligibility for youth recruited in-person is as follows:
Cigarette use status inclusion and exclusion criteria:
1. Youth who indicate in the Screener that they satisfy the criteria of an “experimenter” – that is, have smoked at least one puff of a cigarette but have smoked no more than 99 cigarettes in their lifetime – will qualify for study participation.
2. Youth who indicate in the Screener that they satisfy the criteria of an “at-risk non-trier” – that is, they have never used cigarettes in their lifetime, not even one puff of a cigarette, but answered with an affirmative response to any of the susceptibility questions (i.e., did not answer "definitely not" to all questions) – will qualify for study participation.
3. Youth who respond that they have never used cigarettes, not even taken a puff of a cigarette, and respond "definitely not" to all questions assessing susceptibility to future smoking will be defined as "non-at-risk non-trier" and be excluded from participation.
4. Youth who respond that they have smoked more than 99 cigarettes in their lifetime will be designated as established users because they have crossed the threshold of experimenter as per Mowery et al. (2004) and will be excluded from participation.
Peer crowd influence inclusion and exclusion criteria:
As the campaign is targeted to youth who are influenced by Hip Hop culture, potential participants will also be selected based on the extent to which they are influenced by Hip Hop, as assessed in the Screener using a proprietary peer crowd assessment tool established by Rescue SCG. This tool uses 64 pictures consisting of 32 male and 32 female peers unknown to youth (see Screener for pictures). Pictures are obtained through photo releases and the public domain. Male and female pictures are presented separately to youth in the Screener, and youth will select the top 6 teens (3 male, 3 female) that they think would best fit their main group of friends, and the 6 teens (3 male, 3 female) that would least fit their main group of friends. The assessment will be completed separately with the male and female pictures. Based on the ranking of selected pictures, youth will be assigned a Hip Hop score. A positive score indicates some Hip Hop influence whereas a negative score indicates no Hip Hop influence. The score is calculated based on the system summarized in Table 1. For example, if a youth chooses all Hip Hop pictures for best fit for both genders and no Hip Hop pictures for least fit, (s)he will earn a Hip Hop score of 12 (6 points for male selections plus 6 points for female selections). If a youth chooses all Hip Hop pictures for best fit and one Hip Hop picture as first in the least fit, then (s)he will earn a Hip Hop score of 9.
Table 1. Summary of Hip Hop Scoring
Table 1. Summary of Hip Hop Scoring |
||||
Best Fit Rank |
Hip Hop Score |
|
Least Fit Selection |
Hip Hop Score |
First |
3 |
|
First |
-3 |
Second |
2 |
|
Second |
-2 |
Third |
1 |
|
Third |
-1 |
Youth who have a total Hip Hop score of 4 or more (from a scale of -12 to 12 total for both male and female picture selections) will be qualified to participate in the research. Prior internal research has shown that a minimum score of 4 will ensure that all participants are sufficiently influenced by the Hip Hop peer crowd, while maintaining a threshold that will not negatively impact sample size goals. Youth who have a Hip Hop score of less than 4 will not qualify as research participants.
Other inclusion and exclusion criteria:
To prevent respondent bias from recent exposure to market research or from affiliation with the tobacco industry, respondents will answer two items asking about self/family/friend employment within various industries and personal participation in market research. Exclusion criteria will include: (1) indication that self/family/friend work for the tobacco industry, and/or (2) youth having participated in a research study within the last 6 months (i.e., “Yes, more than 6 months ago”, “No”, or “I don’t know” are the only acceptable responses for inclusion).
No identifying information collected via the Screener will be retained or included in analyses. All paper Screeners will be destroyed following data entry and no identifying information (such as participant name or contact information) will be retained. See “Assurance of Privacy Provided to Participants” section for information on the protection of personal and sensitive data.
Social Media Recruitment
In-school data collection can be time consuming and is dependent on the individual school’s schedule. For that reason, recruitment via Facebook is being conducted to supplement the in-school sample, as well as explore the feasibility of an alternative recruitment option that may be more cost effective. Differences between groups will be examined to ensure that there are no biases associated with recruitment methods, however; results from both the social media group and the in-school recruitment group will be analyzed the same. Social media recruitment will use Facebook ads, optimized using key words and demographic targeting. Advertising through social media platforms, such as Facebook, can help increase the diversity of the study sample and increase representation of traditionally underrepresented groups, including racial/ethnic minorities (Lane, Armin, & Gordon, 2015; Graham et al., 2008). Data also suggest that social media engagement among multicultural youth ages 12-17 is high. For example, a recent online survey found that 71% of US teens ages 13-17 used Facebook, including 75% of African American teens and 70% of Hispanic teens (Pew Research Center, 2015). For many social media platforms, ad targeting can be adjusted in real-time allowing researchers to react to shifting recruitment needs if a particular demographic is lacking in the overall Copy Testing sample. Social media advertisements may be deployed based on factors such as age, geographic location within or around a target campaign city, and interest in Hip Hop cultural pages or hashtags. Social media advertisements can be found in Addendum C. Respondents who click on any social-media sponsored ad will be redirected to the Screener splash page. Youth who complete the Screener and are identified as eligible will be asked to provide a parent’s email address, for parental opt out notification, and will receive a link via email and text message to the Questionnaire no less than 24 hours later to allow sufficient time for parental opt out.
Screening and qualification criteria for participants recruited online are the same as those listed above for in-person recruitment, with the following differences:
• Phase 1:
o Demographics:
Participant zip code will be collected. This will be used to ensure that participants are within determined geographic targets for the study. Post-hoc analysis may be conducted to understand potential regional variations.
Participants aged 12 will not qualify to complete the Copy Testing Questionnaire if they are recruited via social media, to comply with COPPA regulations.
o Email verification:
Youth email address will be collected to check against all current respondent data to avoid duplicates and reduce fraudulent activity.
• Phase 2:
o Identifying Information:
Youth name and last period class information will not be collected from youth who complete the Screener online.
Youth will be required to provide an email address for their parent. This will be used to email the Parental Opt-Out Form to parents of eligible youth. Parental email address will not be used for any other purpose.
Youth email address will be verified in order to email eligible youth a $5 pre-paid gift card incentive.
IP addresses will be collected automatically as part of the Screener completion process to avoid duplication (i.e. ensure that no entry with same IP address already exists) and as a verification of participant country of origin (should be within US).
• Incentive:
o Youth who complete the Screener and qualify will receive a $5 electronic gift card pre-paid incentive to demonstrate the legitimacy of the study and reduce drop off between recruitment and Questionnaire completion. This represents a split incentive strategy and will be implemented along with an additional $20 electronic gift card post-paid incentive that participants will receive after completing and submitting the Questionnaire. See “Incentives” section for additional information.
No identifying information collected via the Screener will be retained or included in analyses. See “Assurance of Privacy Provided to Participants” section for information on the protection of personal and sensitive data.
Procedure
In-person data collection will be conducted during a study session held in a classroom setting. Participants will complete all activities individually with minimal assistance from a Data Collector as necessary. Each participant will be assigned a set of headphones and a tablet or desktop computer station on which they will view ads and provide individual feedback. Participants will be asked not to interact with each other or discuss the study stimuli. A supervisor from the In-Person Team and up to 3 Data Collectors will facilitate each study session. Youth ages 13-17 who are invited to but do not attend a study session will receive an email with details on how to access the Copy Testing Questionnaire from their own device, including a unique link to the Copy Testing Questionnaire. Youth 12 years of age who are invited to but do not attend a study session will not be invited to complete the Copy Testing Questionnaire on their own device, to comply with COPPA regulations. Youth who do not attend their assigned study session and have not completed the Questionnaire within 24 hours after receiving the link via email will receive up to 2 reminder emails with the study link. Invitation and reminder emails for in-person recruitment with online Questionnaire completion can be found in Participant Email Communications - Onsite.
Online data collection will be completed by youth independently, on their own electronic devices, such as a mobile phone or home computer. Eligible youth will be emailed and text messaged a link to complete the Copy Testing Questionnaire on their own devices after a 24 hour waiting period to allow for parental opt out. Qualified participants whose parents do not opt them out, and who have not completed the Copy Testing Questionnaire within 24 hours after initially receiving the study link will receive up to 2 reminders. The reminders will be in the form of an email and text message containing the study link. Invitation and reminder emails for online recruitment with online Questionnaire completion can be found in Parent & Participant Electronic Communications - Social Media.
Table 2 indicates the variables to be assessed during the Copy Testing Questionnaire and the participant groups that will be exposed to these variables (see Copy Testing Questionnaire).
Table 2. Structure of the Copy Testing Process and Questionnaire
Table 2. Structure of the Copy Testing Process and Questionnaire |
|||
Action or Variable |
Description |
Presented to Ad-Viewing Participants |
Presented to Control Participants |
Ad exposure |
Each of the ad-viewing participants will view 2 unique video ads. |
X |
|
Tobacco use and peer tobacco use |
Items on household tobacco use, peer cigarette use, and participant past 30-day tobacco use. |
X |
X |
Perceived ad effectiveness |
Ad-viewing participants will be presented with items to assess ad effectiveness immediately following exposure to each video ad. |
X |
|
Tobacco-related attitudes, beliefs and risk perceptions |
Items tailored to align with the tobacco facts chosen for inclusion in the video ads. Items assessing participants’ attitudes, beliefs, and risk perceptions related to tobacco use. |
X |
X |
A more detailed description of these variables is below:
Ad exposure: Each of the ad-viewing participants will be exposed to two video ads. All advertisements will average 30 seconds in length. See Addendum B for ads to be tested.
Tobacco use and peer tobacco use: Prior to ad exposure, a short series of questions in the survey will assess the participant’s current use of any tobacco products, household use of any tobacco products, and peer cigarette use. These questions will be used to assess the effectiveness of the randomization process and ensure there are no confounding differences between groups.
Perceived ad effectiveness: Following exposure to an ad, ad-viewing participants will be presented with a series of questions designed to assess their initial reactions to the ad, including what they liked or disliked, how the ad made them feel (e.g. sad, angry, motivated, surprised), and, among other things, whether they felt the ad was interesting, powerful, informative, meaningful, funny, confusing, or worth remembering. Additionally, questions will be asked on whether the ad influenced the participant’s thoughts about smoking, the degree to which the claims in the ad were believable, and the actions they might take in response to the ad (e.g. sharing the ad, mentioning it to a friend). To protect against order effects, questions assessing emotional response and ad effectiveness will be presented in a randomized order.
Tobacco-related attitudes, beliefs, and risk perceptions: All groups will complete a series of questions assessing understanding of the health consequences of tobacco, whether smoking cigarettes is good or bad, enjoyable or not enjoyable, and other attitudes about tobacco use. These questions are used to examine for the presence of unintended consequences.
In-Person Team members will manage in-person incentive distribution. Youth who complete the Copy Testing Questionnaire in person at the study session and receive a gift card incentive will be asked to provide their initials on a Check-Out Form. This serves as confirmation of receipt and is necessary for financial reconciliation of study costs. The Check-Out Forms will be stored with other paper data from the study in a locked cabinet for a period of three years after the end of the study, and then will be destroyed by secure shredding.
Online incentive distribution will be managed by the Social Media Team Lead. Electronic receipts from the purchase of online gift card incentives for participants who complete the Copy Testing Questionnaire on their own device will be retained for financial reconciliation purposes. No identifying information such as participant name or email address will be retained on receipts or Check-Out forms. Electronic receipts will be stored on a password-protected computer for a period of three years after the completion of the study, and then will be securely destroyed.
Qualified youth will be invited to complete the Copy Testing Questionnaire. Participants will be randomly assigned to the control group, which will not view any ads, or the ad-viewing group, which will view 2 randomly assigned ads from the 4 video ads being tested. All participants will complete a short series of questions in the Questionnaire to assess participants’ current use of any tobacco products, household use of any tobacco products, and peer cigarette use. These questions will be used to assess the effectiveness of the randomization process and ensure there are no confounding differences between groups. Following exposure to an ad, ad-viewing participants will be presented with a series of questions designed to assess their initial reactions to the ad, including what they liked or disliked, how the ad made them feel (e.g. sad, angry, motivated, surprised), and, among other things, whether they felt the ad was interesting, powerful, informative, meaningful, funny, confusing, or worth remembering. Additionally, questions will be asked on whether the ad influenced the participant’s thoughts about smoking, the degree to which the claims in the ad were believable, and the actions they might take in response to the ad (e.g. sharing the ad, mentioning it to a friend). All participants will complete a series of questions assessing understanding of the health consequences of tobacco, whether smoking cigarettes is good or bad, enjoyable or not enjoyable, and other attitudes about tobacco use. Participants in the control condition will be compared to the exposure condition to assess unintended consequences based on responses to cigarette related knowledge, attitudes and beliefs. Randomization is used to prevent bias resulting from participant group selection and order effect. These questions are used to examine for the presence of unintended consequences by examining differences in responses to cigarette related knowledge, attitudes and beliefs between the exposure and control groups. Separately, randomization is used to assess bias resulting from participant group selection and order effect.
We anticipate data collection will take approximately 6 weeks. The outcome of the survey will be an understanding of teens’ receptivity and potential unintended consequences for viewers of video ads created for FDA’s Fresh Empire Campaign. Understanding teen perceptions of video ads created to reduce tobacco initiation and use among at-risk multicultural youth will help to guide the optimization of these ads for FDA’s Fresh Empire Campaign.
Purpose and Use of the Information Collection
The information obtained from the proposed data collection activities will be collected from youth ages 12-17 living in the United States, and will be used to inform FDA about youth’s reception of video ads and potential unintended consequences for FDA’s Fresh Empire Campaign. While not exhaustive, the list below illustrates a range of purposes and uses for the proposed information collection:
Optimize video ads for FDA’s Fresh Empire Campaign.
Inform future programs that may be designed for similar purposes.
Participants will be recruited using two strategies: (1) in-person recruitment from middle and high schools in multiple metropolitan areas; and (2) social media recruitment using targeted advertisements. Youth will be recruited in-person using a tablet-based Screener survey (Attachment A) or paper Screener only if there are technical issues with the tablets (Attachment B), completed in school during lunch periods by interested youth. Youth ages 13-17 recruited in-person will be provided a parental opt-out form (Attachment E), and youth ages 12 recruited in person will be provided a parental consent form (Attachment F). Rescue SCG understands that local jurisdictions and/or schools may have different requirements for consent procedures and will follow those local requirements. The after school study session will be held no less than 24 hours after recruitment to allow for parental opt out & consent to occur. Researchers will attempt to obtain verbal parental consent for youth ages 12 who do not return a signed consent form (Attachment G). Qualified youth whose parents do not opt them out (ages 13-17) and whose parents provide consent (age 12) will be invited to complete the Questionnaire (Attachment D) during an after-school study session held at their school. If a participant aged 13 to 17 who was recruited in-person cannot attend an in-person session, he or she will be emailed a link to the Questionnaire for online completion on their own device. Youth aged 12 recruited in-person who do not attend a study session will not be invited to complete the Questionnaire online on their own device, in order to comply with Children’s Online Privacy Protection Act (COPPA) regulations.
Youth ages 13-17 will also be recruited online using targeted social media advertisements (Attachment K) and an online Screener (Attachment C). Youth aged 12 will not qualify via the online Screener in order to comply with COPPA regulations. At the end of the Screener, qualified youth ages 13-17 will be asked to provide a parent/guardians’ email address, so that a parental opt-out form can be emailed to them (Attachment H). Qualified youth recruited online will be invited to complete the Questionnaire (Attachment D) on their own device, such as a mobile phone or home computer. Youth will receive a link to the Questionnaire no less than 24 hours after the parental opt out form is emailed to their parent/guardian, to allow sufficient time for parental opt-out to occur.
All participants, regardless of recruitment method, will complete a participant assent form before beginning the Questionnaire (Attachment I & Attachment J). The assent form will be presented electronically at the start of the Questionnaire. Youth will be prompted to read the form and provide assent using a radio button. Youth must complete the Participant Assent form to continue to the Questionnaire; if they do not provide assent, they will not be able to complete the Questionnaire.
Use of Improved Information Technology and Burden Reduction
The use of electronic Screener and Questionnaire surveys offers a number of benefits. First, computerized administration permits the instrument designer to incorporate into the instruments routings that might be overly complex or not possible using a paper-based survey. For example, Screener surveys can be programmed to implement skip patterns based on a respondent’s previous answers. Interviewer and respondent errors caused by faulty implementation of skip instructions are virtually eliminated. Second, electronic administration increases the consistency of the data. The electronic Screener and Questionnaire can be programmed to identify inconsistent responses and attempt to resolve them through respondent prompts. This approach reduces the need for most manual and machine editing, thus saving time and money. In addition, it is likely that respondent-resolved inconsistencies will result in data that are more accurate than when inconsistencies are resolved using editing rules. Third, electronic data collection permits greater expediency with respect to data processing and analysis (e.g., a number of back-end processing steps, including coding and data entry, will be minimized). These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. Finally, this technology permits respondents to complete the instruments in privacy. Providing the respondent with a methodology that improves privacy makes reporting of potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) less threatening and enhances response validity and response rates.
Both in-person and social media recruitment methods offer benefits in terms of burden reduction and increased efficiencies. Conducting in-person recruitment at schools provides a number of methodological benefits, including efficiency in accessing this specific target audience by reaching teens in an environment they frequent often, school; and reduced burden on study participants by making participation easy and convenient via study sessions hosted at participants’ schools. During in-person recruitment, screening will be completed via a self-administered Screener survey hosted on a tablet. Paper Screeners will only be utilized in the case of technical issues with the tablets. This will allow for more accurate data management and participant qualification as these tasks will be completed automatically by the survey program based on preprogrammed logic. Additionally, participants may provide more honest responses on an electronic Screener as their responses will not be visible to the researcher conducting recruitment.
During social media recruitment, participants who click on a social media ad will complete the Screener electronically on their own device such as a mobile phone or computer. This allows for more accurate data collection because respondents provide more honest responses, since it is clear that the answers will remain private. In addition, use of social media as a recruitment tool will cast a wider net to identify eligible study respondents who are members of this very specific population. Recruitment via social media will also help to contain costs, allowing for a sample that is geographically diverse without driving up researcher costs for travel during data collection.
A unique feature of this study is the ability to offer participants ages 13-17 recruited in-person who do not attend a study session the opportunity to complete the Questionnaire on their own device. This functionality, which relies on electronic data collection to provide accurate records of Questionnaire completion and participant contact information, will reduce burden by increasing participation rates, thereby reducing the number of youth needed to complete the Screener in order to achieve the desired sample size. Additionally, reducing drop off between the Screener and Questionnaire will decrease costs related to additional travel for in-person recruitment, and additional ad buys for social media recruitment. FDA estimates that 95% of the respondents will use electronic means to fulfill the agency’s requirement or request.
Efforts to Identify Duplication and Use of Similar Information
The video ads being tested in this study are original to FDA’s Fresh Empire Campaign and have not previously been copy tested or publically aired. As such, there are no existing datasets that can be used or modified to address FDA’s need for information on youth reactions to the video ads for FDA’s Fresh Empire Campaign. Additionally, FDA’s Fresh Empire Campaign is relatively new and targets a specific subpopulation, i.e. multicultural youth ages 12-17 who are cigarette experimenters or at-risk non-triers and who are influenced by the Hip Hop peer crowd. As such, there are no national level surveys or datasets that focus on this specific target audience. This type of focused recruitment of youth in the target audience is necessary to test the effectiveness of ads developed for FDA’s Fresh Empire Campaign. Therefore, we have determined that the proposed information collection does not duplicate previous efforts.
Impact on Small Businesses or Other Small Entities
Respondents in this study will be members of the general public, specific subpopulations, or specific professions, not business entities. No impact on small businesses or other small entities is anticipated.
Consequences of Collecting the Information Less Frequently
There are no legal obstacles to reduce the burden. Respondents to this collection of information will answer only once to ensure the participant burden is as low as possible. Without the information collection requested, it would be difficult to measure target audience reactions to video ads for FDA’s Fresh Empire Campaign. Failure to collect these data could reduce effectiveness of the FDA’s messaging, and therefore reduce the benefit of the messages for multicultural youth in the United States.
Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2). The message testing activities fully comply with the guidelines in 5 CFR 1320.5.
Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
The following individuals inside the agency have been consulted on the design of the copy testing plan, survey development, or intra-agency coordination of information collection efforts:
Tesfa Alexander
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002
Phone: 301-796-9335
E-mail: [email protected]
Gem Benoza
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-0088
E-mail: [email protected]
Matthew Walker
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-3824
E-mail: [email protected]
Leah Hoffman
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-743-1777
E-mail: [email protected]
Atanaska (Nasi) Dineva
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002
Phone: 301-796-4498
E-mail: [email protected]
Chaunetta Jones
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002
Phone: 240-402-0427
E-mail: [email protected]
The following individuals outside of the agency have been consulted on questionnaire development.
Dana Wagner
Rescue Social Change Group
660 Pennsylvania Ave SE, Suite 400
Washington, DC 20003
Phone: 619-231-7555 x 331
Email: [email protected]
Carolyn Stalgaitis
Rescue Social Change Group
660 Pennsylvania Ave SE, Suite 400
Washington, DC 20003
Phone: 619-231-7555 x 313
Email: [email protected]
Mayo Djakaria
Rescue Social Change Group
660 Pennsylvania Avenue SE, Suite 400
Washington, DC 20003
Phone: 619-231-7555 x 120
Email: [email protected]
Xiaoquan Zhao
Department of Communication
George Mason University
Robinson Hall A, Room 307B
4400 University Drive, 3D6
Fairfax, VA 22030
Phone: 703-993-4008
E-mail: [email protected]
Explanation of Any Incentive or Gift to Respondents
Youth who participate in a study session or complete and submit the Questionnaire online will receive a total of $25 in gift card incentives. If a participant completes the screener online and qualifies, they will receive $5 as an incentive to respond to the invite to complete the survey after the 24 hour opt-out period. Any participant who does not begin the survey will not receive the incentive. However, in accordance with IRB requirements, incentives will be provided to participants who begin the survey and choose not to complete the survey for whatever reason. The use of incentives treats participants justly and with respect by recognizing and acknowledging the effort they expend to participate. In this research, we are asking participants to provide thought-intensive, open-ended feedback on video ads that require a high level of engagement. This incentive amount is considered an adequate compensation for participating in the study, and not an inducement for participation. The incentives are intended to recognize the time burden placed on participants, encourage their cooperation, and convey appreciation for contributing to this important study and are similar to incentives that are offered for other surveys of this type. Numerous empirical studies have shown that incentives can significantly increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys (e.g., Abreu & Winters, 1999; Castiglioni, Pforr, & Krieger, 2008; Jäckle & Lynn, 2008; Shettle & Mooney, 1999; Singer, 2002). Incentives have been shown to bring traditionally underrepresented groups into the sample, such as the less educated, nonwhites, and those with lower incomes (Singer & Kulka 2002). For this reason, incentives can be used to improve sample composition. Research shows that survey incentives largely have a differential, positive impact upon response rates and survey costs when surveying ethnic and racial minorities. In a study among multicultural Medicaid enrollees, African Americans in the sample who received the $2 pre-paid incentive had a 10% higher response rate than those who did not receive the pre-paid incentive (Beebe et al, 2005) at a savings of $5.56 per completion. The pre-paid incentive also benefitted other groups including American Indian/ Alaska Natives, Somalis, and Hmong but did not have the same positive influence on Latino participation, instead the opposite. While more research is needed, there may be other factors influencing Latino participation. An average of 20 attempts were needed to successfully complete an interview with Latino participants compared to only 3 attempts per African American enrollee. Incentives have an impact on response rates and study attrition for households in the poverty stratum and significantly reduce item nonresponse rates among respondents within this population (Creighton et al, 2007; Clark, S.M. and Mack, S.P, 2009).
The use of modest incentives is expected to enhance survey response rates without biasing responses. A smaller incentive would not appear sufficiently attractive to participants. We also believe that the incentives will result in higher data validity as participants will become more engaged in the survey process and will be more likely to attend a study session or complete the Questionnaire online.
Youth recruited via social media ads will receive a split incentive. Youth who complete the Screener after clicking on a social media advertisement and qualify to participate will be emailed a $5 electronic gift card as a pre-paid incentive. Youth who return and complete the Questionnaire after the 24-hour parental opt-out period will receive a $20 electronic gift card within 72 hours after submitting the Questionnaire. In this way, participants recruited via social media will receive $25 total in electronic gift cards.
A pre-paid incentive will be used for youth recruited via social media to reduce anticipated drop-off between Screener qualification and Questionnaire completion due to the 24-hour parental opt-out waiting period. In a similar study, Rescue SCG recruited youth influenced by the Hip Hop peer crowd to complete an online survey via targeted social media advertisements. Youth completed a brief Screener survey, and qualified youth provided assent and began the full questionnaire immediately upon qualification. In this study, 22.4% of otherwise qualified youth met qualification criteria for Hip Hop peer crowd influence (Rescue SCG research). In the proposed study, youth recruited online will have to wait 24 hours before receiving a link to complete the Questionnaire, further reducing study completion rates and increasing the number of youth needed to complete the Screener and qualify. It is believed that the $5 pre-paid incentive may proactively address this anticipated retention issue as a pre-paid incentive has been shown to increase response rates for surveys (Messer et al., 2011; Coughlin et al., 2013; Gajic, Cameron, & Hurley, 2012; Dirmaier et al., 2007; Ulrich et al., 2005; Stevenson et al., 2011), and to be effective when used in conjunction with a post-paid incentive (OMB Control No. 0920-0805, Report on Incentives). In some cases, pre-paid incentives have also been demonstrated to increase response rates among racial and ethnic minorities (Beebe et al., 2005; Dykema et al., 2012). Additionally, a $5 pre-paid incentive seems to be effective at maximizing participation, compared to other incentive amounts (Warriner et al., 1996; Asch et al., 1998; Han et al., 2012; Montaquila et al., 2013; Dykema et al., 2012). Based on this information, it is believed that a $5 prepaid incentive for youth recruited online in this study is necessary to maximize completion of the Questionnaire after the 24-hour parental opt-out period among youth recruited using social media advertisements.
Pre-paid Incentive Sizes and Cost Efficiency
Strouse and Hall (1997) recommend that in order to be successful, promised incentives have to be in the $15 to $35 range, and many federally-funded surveys today provide incentives ranging from $20 to $125. Pre-paid incentives have ranged from $1-$15 with many studies using $2 and $5. The choice of an incentive amount depends largely on the survey burden, including the survey length and other tasks that may be required of the respondent, and the survey topic. That said, pre-paid incentives at the $5 level seemed to perform well overall.
The National Household Education Survey (NHES, U.S. Department of Education) tested $2 and $5 pre-paid incentives for their mail screener, which was expected to take 2-8 minutes to complete depending on the survey version used. The study also tested the effectiveness of $5 and $15 pre-paid incentives to respondents who screened eligible for the topical survey. The study found that the larger pre-paid incentive amounts ($5 vs. $2, $15 vs. $5) achieved higher response rates. The NHES also offered a $5 promised incentive to a subset of respondents to encourage them to participate in the topical survey by telephone. Although higher response rates (6-8%) were achieved with the $5 promised incentive, none of the observed differences were statistically significant (Tubman and Williams, 2010).
Warriner et al. (1996) found that a $5 pre-paid incentive resulted in a $0.40 savings per case and was 10% higher response compared to the $2 pre-paid incentive. Asch, Christakis, and Ubel (1998) found similarly.
In a two phase sampling study for the 2011 NHES field test, both $2 and $5 cash incentives were used at the screening stage. The $5 incentive resulted in a significantly higher screener response rate than the $2 incentive (71.0% and 66.5%, respectively), but this did not carry over to the topical survey response rate (73.9% and 71.9%, respectively). However, the higher response rate to the initial screener (42.8% for the $5 incentive group compared to 36.3% for the $2 incentive group) resulted in saved cost associated with nonresponse follow-up mailings (Han et al., 2012). A separate experiment was also conducted with the 2011 NHES field test for the topical survey incentives, including $5, $10, $15, and $20 cash incentives. Findings from the study indicate that incentives greater than $10 did not increase the response rate compared to the $5 level ($5: 79.3%; $10: 75.6%; $15: 78.8%; $20: 78.3%) (Montaquila et al., 2013).
A similar study was conducted in Wisconsin to determine how incentives affected response rates on a paper survey and whether a second incentive increases the response rate (Dykema et al., 2012). The Survey of Health of Wisconsin was conducted among 2,608 households in Wisconsin. Households were randomly assigned to receive a cash incentive of $2 or $5 in the initial survey mailing. The group that received the $5 incentive had a significantly higher response rate than the group that received the $2 incentive (60.9% and 53.4%, respectively). Respondents who received the $5 second incentive had a response rate of 69.5%, which was higher than the response rate for respondents who received the second $2 incentive (64.2%). However, this observed difference is not significantly different.
One objective of a study of alcohol use among young adults in Wisconsin (N = 7,200) was to determine if small cash incentives perform differently in web-based surveys compared to mail surveys (Stevenson et al., 2011). Respondents were randomly assigned to be in the pre-paid mailing group or pre-paid web-based group and either received a $1 or $2 cash incentive in the initial contact. Before the alternative survey mode was offered, the response rate for the pre-paid mailing group with a $1 incentive was 39.2% and 42.7% for the $2 incentive group. Similarly, the $2 pre-paid web-based group had a higher response rate than the $1 incentive group (29.7% and 25.8%, respectively). The final response rate for the pre-paid mailing group was 3.1% higher in the $2 incentive group and 5.1% higher in the $2 pre-paid web-based incentive group. These results are statistically significant.
Because the in-person Questionnaire will be completed at the school at which youth were recruited, this increases accessibility to the survey. Thus, youth who complete the Screener in-person will not receive a monetary incentive for completing the Screener. Participants recruited in-person who complete the Questionnaire during an in-person study session will be given a $25 Visa or American Express gift card. Participants will receive this gift card even if they have to leave the study session early, or if they choose not to answer questions on the Questionnaire. Youth ages 13-17 recruited in-person who do not attend a study session will be emailed a link to complete the Questionnaire online on their own device. Youth recruited in-person who complete and submit the Questionnaire online will receive a $25 electronic gift card by email after submitting their survey online. The gift card will be emailed to participants at the same email address at which they received the study link.
Table 3. Incentive Type and Amount
Type of Incentive |
Participant |
Amount/Value |
Social Media Screener survey completion |
Qualified participants recruited via social media advertisements |
$5 electronic gift card |
Social Media Questionnaire survey completion |
Participants recruited via social media who complete and submit the Questionnaire online |
$20 electronic gift card |
In-Person Questionnaire survey completion |
Participants recruited in-person who complete and submit the Questionnaire in-person or online |
$25 electronic gift card |
Procedures for Obtaining Assent and Parental Opt-out Consent
Informed consent and assent procedures will be different depending on the age of potential participants due to differences in maturity and autonomy. Procedures are described separately below. Rescue SCG understands that local jurisdictions and/or schools may have different requirements for consent procedures and will follow those local requirements.
Parental Consent (Participants Under 13 Years of Age – In-Person Only)
Parents/guardians will complete a consent form prior to youth participation in the Copy Testing Questionnaire, but after they have been selected for the study (see Parental Consent Form). Active parental consent will only be collected during in-person data collection, as 12 year olds will only be recruited and allowed to complete the Questionnaire in-person due to COPPA regulations.
After completing the Screener, potential participants under 13 years of age will be requested to complete the Parent Contact Form and informed that they will receive an Informational Packet. In the Informational Packet, potential participants will receive relevant study information (see Informational Packet [Parental Consent]) and Parental Consent Forms. The Parental Consent Form will inform parents of the study, clearly state the date and time of the study session, and provide parents the option to provide or reject consent in writing.
Potential participants will be given two days to return their signed Parental Consent Forms. Over the next two days, a Data Collector will be present at lunch to collect the forms and remind potential participants to bring the completed consent forms back. On the second day, if a Parental Consent Form has not been returned, a Data Collector will attempt to contact the parent via phone to acquire verbal consent using the information from the Parent Contact Form. A consent script will be followed and will clearly state the date and time of the study session. Any attempts to obtain verbal consent will be documented on the form; specifically, the researcher will record the parent’s name they tried to call, the call time, and the parent’s response (see Parental Consent Verbal Script). If verbal parental consent is received, the Data Collector receiving parental consent will fill out the verbal consent form, including youth name, parent/guardian name, relation to youth, their phone number for consent confirmation, date, time of call, and the researcher’s signature. Parents who provide verbal consent will be offered a mailed, emailed, or faxed copy of the consent form to keep for their records. Potential participants will be excluded from the study if the consent form is not returned and a Data Collector is not able to obtain parental consent via phone.
Both parents/guardians and potential participants will be given an opportunity to ask questions during the two-day consent period. The consent forms will provide a telephone number and email address for the PI, who can answer any questions or respond to concerns about the study. According to the Readability Test Tool available at www.read-able.com, the Parental Consent Form was determined to be at or under 9.0 Flesch Kincaid grade levels.
Potential participants will be added to the Approved Participant List for their school. Upon study session check-in, the Approved Participant List will be used to verify youth identity and receipt of consent documentation. Only potential participants who have all appropriate forms indicating parental consent will be allowed to participate in a study session. Signed copies of consent forms will be retained by Rescue SCG and kept in locked cabinets only accessible by research team members for a period of three years after the end of the study, and then will be destroyed by secure shredding.
Parental Opt-Out (Participants 13 Years of Age and Older)
Due to the target population of this study, traditional written parental consent procedures would screen out the very subjects most appropriate for the aims of this study. Many youth who smoke or are at-risk for smoking are unlikely to seek out parental consent or have parents who provide written consent for their children’s participation in prevention programs, making the evaluations of such programs problematic (Levine, 1995; Pokorny et al., 2001; Unger et al., 2004; Severson and Ary, 1983). Demonstrating this point, there is consistent evidence of quantifiable differences in the characteristics of youth who participate in smoking cessation research when traditional written consent is required compared to waived parental consent, including participant demographics and smoking history. For instance, Kearney et al. (1983) found that explicit written consent procedure produced a sample that was approximately half the size of the eligible population and over-represented White students while under-representing Blacks and Asian Americans. Anderman et al. (1995) found differences between 9th- and 12th-grade students with and without written parental consent for a sensitive health survey. Participants with written consent were more likely to be White, live in two-parent households, and have a grade point average of “B” or above. Cigarette smoking was also less prevalent in the written consent group. Severson and Ary (1983) found that youth participants who gained consent were more likely to be nonsmokers compared to those non-consent participants.
Because obtaining written consent for at-risk youth will result in a sample with different characteristics than the target group, a parental opt-out approach is being requested for participants 13 years of age or older (see Parental Opt-Out Form).
For youth recruited in-person, potential participants will receive an Informational Packet (see Informational Packet [Parental Opt-Out]). In the Informational Packet, potential participants will receive relevant study information and a Parental Opt-Out Form. The Parental Opt-Out Form will provide clear and simple instructions for how to opt-out of participation in the research study, including multiple forms of contact for the parent to exercise that option. Parents will be allotted at least 24 hours between recruitment and the study session to contact the PI to opt their children out of the study. Potential participants will be added to the Approved Participant List for their school. Upon study check-in, the Approved Participant List will be used to verify that potential participants’ parents have not contacted the PI to opt their child out of the study. Youth will not be eligible for participation in either the in-person study session or to receive a link to complete the survey online if their parent/guardian opts them out. All youth ages 13-17 who do not attend the study session and have not been opted out by their parents will be provided the opportunity to complete the Copy Testing Questionnaire online on their own device.
For youth recruited online, immediately after completing the Screener, qualifying youth will be prompted to provide their parent/guardian’s email address. This email address will be used to email an electronic copy of the Parental Opt Out Form to a potential participant’s parent or guardian. The Parental Opt-Out Form will provide clear and simple instructions for how to opt-out of participation in the research study, including multiple forms of contact to exercise that option. Parents/guardians will be allotted at least 24 hours after the Parental Opt-Out Form is emailed to them before a link to the study is emailed and text messaged to their child. This will ensure sufficient time for the parent/guardian to contact study staff and opt their child out of the study if they choose to do so. Youth whose parents contact study staff to opt them out of the study will not receive a link to complete the Copy Testing Questionnaire.
Participant Assent (all youth)
In order to ensure that all youth provide informed assent in the same manner, youth will review and provide assent via an electronic Participant Assent Form presented prior to the start of the Copy Testing Questionnaire (see Participant Assent Form). At the start of the Questionnaire, youth will be prompted to read the form and provide assent using a radio button. Youth must complete the Participant Assent form to continue to the Copy Testing Questionnaire; if they do not provide assent, they will not be able to complete the Copy Testing Questionnaire.
An electronic log of Participant Assent will be retained by Rescue SCG and kept on password-protected computers only accessible by research team members for a period of three years after the end of the study, and then will be destroyed by secure deletion.
Assurance of Confidentiality Provided to Respondents
Chesapeake Institutional Review Board (IRB) and FDA IRB have reviewed and approve the protocols and consent forms for this study. The letter of approval can be found n Attachment M. IRBs’ primary concern is protecting respondents’ rights, one of which is maintaining the privacy of respondent information to the fullest extent of the law.
Middle School/High School Institution Confidentiality
School name and administrator contact information will be kept in a password-protected folder only accessible and viewable to researchers who are recruiting schools and managing onsite research teams. Once the school research sites are known, the research team will assign a unique letter to the school (i.e., A-Z). This coding system will be used for data management purposes only. The school key will only be accessible to the research team on password-protected computers. These efforts are done to protect school anonymity. Upon completion of data collection, the document containing school and administrative contact information and the key to the school IDs will be destroyed.
Screener Survey
Youth will be informed that the information they provide on the Screener will only be viewed by members of the research team and that their name and any other identifying information they provide will not be connected with their responses to the Questionnaire. Only youth who qualify to participate will be asked to provide identifying information on the Screener. This approach will ensure that identifying information is only collected if youth qualify to complete the Questionnaire.
IP address and youth email address will be collected from online participants to manage potential duplicate entries and study completion from non-US based respondents. Participant cell phone number and email address will be collected during the screening process for contact purposes. Researchers will only contact participants via email or text message to remind them of their assigned study session (in-person recruitment), share the link to the Questionnaire (in-person and social media recruitment), and remind youth who have not completed the Questionnaire to do so (in-person and social media recruitment). Parent/guardian email address will only be collected from youth recruited via social media, to facilitate distribution of the Parental Opt-Out Form. IP address, participant cell phone number and email address, and parent/guardian email address will be deleted from the Screener database prior to merging with Questionnaire data. Thus, all data will be completely de-identified prior to analysis.
Recruitment method-specific procedures are outlined below:
In-Person Electronic Screener: If participants do not qualify for study inclusion after Phase 1 questions, they will be notified immediately that they will not be invited for study participation and instructed to return the tablet to a researcher. Only participants who do qualify after Phase 1 will automatically be prompted to move on to Phase 2 where they are asked to provide necessary contact information. Therefore, contact information will only be collected from eligible youth.
In-Person Paper Screener: If there are technical issues with tablets, paper Screeners may also be utilized for in-person recruitment. For youth taking the paper Screener, eligibility for study inclusion will be determined after the fact according to the same eligibility algorithm used for the in-person electronic Screener. Paper Screeners will only be accessible by research team members onsite. All paper Screeners will be securely destroyed at the conclusion of data collection at each school.
Social Media Electronic Screener: All respondents will be asked to provide their email address. Email address will be checked against all current respondent data to avoid duplicates and reduce fraudulent activity, and will serve as an accurate link between Screener and Questionnaire data. All youth will also be asked to provide their zip code, which will be used to determine geographic eligibility, and IP address will be automatically collected. Qualifying participants will also be asked to provide a cell phone number to receive communication about the study, including the link to the Questionnaire, via text message. Parent/guardian email will also be collected from all qualifying youth, to facilitate distribution of the parental opt-out form. All identifying information will be removed from the database before analysis.
During in-person recruitment, youth under the age of 13 will also be asked to complete a Parent Contact Form including youth name, parent name, and phone number to contact parents for verbal parental consent in case potential participants forget to return the Parental Consent Form but are interested in participating. Parent Contact Forms will only be accessible by the researchers onsite and will be securely shredded at the conclusion of data collection at each school.
Copy Testing Questionnaire
During in-person data collection, the Questionnaire will be completed individually during an after school study session. There will be no group discussion and participants will be encouraged not to share their thoughts with other participants.
Youth who complete the Questionnaire on their own device will be emailed and/or texted a web link that corresponds to their Unique ID to track responses. This option will be used for youth who qualify via the in-person Screener but are unable to attend the after school study session, and for youth who are recruited via social media. IP address and email address will be collected from all youth who complete the Questionnaire online.
The research team will not share personal information regarding participants with any third party unless it is required by law to protect their rights or to comply with judicial proceedings, a court order, or other legal process. All data will remain on a password-protected computer and/or in locked cabinets for a period of three years after the end of the study, and then will be shredded and/or destroyed. Aggregate data from this study may be used in future research and/or shared with other researchers.
For electronic data collected by Rescue SCG, all servers will be hosted on a vendor service such as Amazon Web Services Cloud environment using industry standard firewalls and security practices. The vendor ensures encryption of data, and all data is encrypted in transit using HTTPS. The only people with access to the servers and raw databases are the vendor’s engineering team and trained Rescue SCG team members. All servers are operated and maintained according to industry standard practices, and all software is validated using industry standard QA practices prior to production usage. Computer system errors are automatically logged and investigated in a triaged manner. All code changes are built and logged in an automated fashion by vendor technology. The vendor conducts daily database and code backups, which are stored for minimum of two weeks.
The potential risks to participants in this study are minimal. As with any research study, there is a chance that confidentiality could be compromised. However, great effort is taken to protect participant identity and all responses will be kept private to the extent permitted by law. No other risks are anticipated. Based on this study being of minimal risk to participants, the potential benefit of information gained outweighs any risks involved. While the research will not immediately benefit the participants directly, the research will contribute to the creation of an effective tobacco prevention communication program, which could foreseeably influence these participants and their friends to live tobacco-free. This outcome would significantly benefit their health, but cannot be guaranteed for all participants.
Data Management
Data collected from the Screener and Questionnaire will be housed in two separate databases. The databases must be merged into one for analysis, but all data will be completely de-identified prior to analysis. Once Screener and Questionnaire data are connected, any personally identifiable information (e.g., name, phone number, email address) will be permanently deleted. In order to ensure participant privacy before, during, and after the database merging process, the research team will follow the secure protocol outlined below:
All participants who qualify on the Screener will automatically be assigned a Unique ID.
For youth recruited in-person, upon arrival to a study session, each youth will receive a paper note with their Unique ID, which they will provide to an onsite research team member who will enter it into their tablet or computer to begin the Questionnaire. Youth names will not be utilized during Questionnaire data collection.
Youth who complete the survey online will receive an email and text message with a unique link to the Questionnaire, which will ensure that no identifying information is needed for youth to complete the survey. This unique link will serve the same function as entering the Unique ID into the tablet/computer for in-person data collection by allowing Screener and Questionnaire data to be linked without the use of participant name. The survey can only be submitted one time through the study link provided.
Once data collection for the study is over, all identifying information from the Screener database will be deleted, resulting in a de-identified dataset.
Screener and Questionnaire data will be linked through the Unique ID.
All data will be stored on a secure website and data will be managed by the research team on their password-protected computers.
Upon completion of in-person data collection at each school, Parent Contact Forms and paper Screeners will be destroyed by secure shredding.
Justification for Sensitive Questions
The majority of questions asked will not be of a sensitive nature. However, it will be necessary to ask some questions that may be considered to be of a sensitive nature in order to assess specific health behaviors, such as cigarette smoking. These questions are essential to the objectives of this information collection. Questions about messages concerning lifestyle (e.g., cigarette smoking behavior) and some demographic information, such as race/ethnicity, could be considered sensitive, but not highly sensitive. To address any concerns about inadvertent disclosure of sensitive information, respondents will be fully informed of the applicable privacy safeguards. The informed consent protocol will apprise youth that these topics will be covered during the survey. This study includes a number of procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including the following:
Respondents will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.
The Screener and Questionnaire are entirely self-administered and maximize respondent privacy without the need to verbalize responses.
Participants will be provided with a toll-free phone number for the Rescue SCG project manager and a toll-free phone number for the Chesapeake IRB hotline should they have any questions or concerns about the study or their rights as a study participant.
Estimates of Annualized Burden Hours and Costs
12 a. Annualized Hour Burden Estimate
An estimated one-time reporting burden for this collection will be approximately 1,531 hours (Table 4). This includes the time burden associated with the Screener.
To obtain a final sample of 855 participants, we estimate that we will need to screen approximately 13,175 potential respondents. This is because, based on previous research conducted with this target audience, we anticipate approximately 20.0% of youth screened via social media and 40.0% of those screened in-person will qualify. We also anticipate that approximately 25.0% of qualified youth recruited via social media will return to the Questionnaire after the 24-hour parental opt-out period, and approximately 74.0% of those will complete and submit the Questionnaire. For in-person recruitment, we estimate that approximately 72.5% of qualified youth will attend a study session or click on the emailed link, and complete and submit the Questionnaire.
Table 4. Estimated Annual Reporting Burden1
Type of Respondent |
Activity |
Number of Respondents |
Number of Responses per Respondent |
Total Responses |
Average Burden per Response (in hours) |
Total Hours |
Screened Youth |
Screener completion |
13,175 |
1 |
13,175 |
0.0672 |
878 |
Parents of Qualified Youth |
Parental consent or opt-out process |
4,425 |
1 |
4,425 |
0.0832 |
367 |
Participants |
Youth assent |
855 |
1 |
855 |
0.0832 |
71 |
Questionnaire completion (ad-viewing group) |
573 |
1 |
573 |
0.3332 |
191 |
|
Questionnaire completion (control group) |
282 |
1 |
282 |
0.0832 |
24 |
|
Total Annualized Hours |
|
|
|
|
|
1,531 |
1
The total number of
respondents is 13,175; for this study 855 represents the total number
of participants
[ad-viewing group: N=573; control group:
N=282].
2 This number represents are rounded figure.
12b. Annualized Cost Burden Estimate
Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. Rescue SCG has conducted many surveys of similar length and content among youth. Based on previous experience, we estimate that Screener completion will take approximately 4 minutes per youth. Questionnaire completion will take approximately 20 minutes for ad-viewing participants, and approximately 5 minutes for control participants. We have also allocated 5 minutes for parents to review the consent/opt-out forms, and for youth to provide assent before beginning the Questionnaire.
To calculate this cost, the mean hourly wage of $7.25 was used for youth and $22.33 for parents. The youth price represents the minimum wage, and the parental price represents the Department of Labor estimated mean for state, local, and private industry earnings. There are no direct costs to respondents associated with participation in this study. Thus, assuming an average hourly wage of $7.25 and $22.33 (youth and parent), the estimated cost to participants will be $16,634.11. The estimated value of respondents’ time for participating in the information collection is summarized in Table 5.
Table 5. Estimated Annual Cost
Type of Respondent |
Activity |
Annual Burden Hours |
Hourly Wage Rate |
Total Cost |
Screened Youth |
Screener completion |
878 |
$7.25 |
$6,365.50 |
Parents of Qualified Youth |
Parental consent or opt-out process |
367 |
$22.33 |
$8,195.11 |
Participants |
Youth assent |
71 |
$7.25 |
$514.75 |
Questionnaire completion (ad-viewing group) |
191 |
$7.25 |
$1,384.75 |
|
Questionnaire completion (control group) |
24 |
$7.25 |
$174.00 |
|
Total |
|
1,531 |
|
$16,634.11 |
Estimates of Other Total Annual Costs to Respondents or Record Keepers
There are no capital, start-up, operating, or maintenance costs associated with this information collection.
Annualized Cost to the Federal Government
This information collection is funded through a contract with Rescue SCG. The total estimated costs attributable to this data collection are $1,010,831 (Table 6). There are additional contract-funded activities occurring before and after this data collection that include project planning and data analysis. Other activities outside this data collection include coordination with FDA, data collection plan development, instrument development, reporting, IRB, and progress reporting and project management. This information collection will occur in 2016.
Table 6. Itemized Cost to the Federal Government
Government Personnel |
Time Commitment |
Average Annual Salary |
Total |
GS-12 |
5% |
$77,490 |
$3,874 |
GS-13 |
10% |
$92,145 |
$9,215 |
|
|
|
|
|
|
Total Salary Costs |
$13,089 |
Contract Cost |
$997,742 |
||
Total |
$1,010,831 |
||
Explanation for Program Changes or Adjustments
This is a new individual generic collection of information.
Plans for Tabulation and Publication and Project Time Schedule
The analysis will examine perceived effectiveness scores by ad, and knowledge, attitudes, and beliefs about tobacco use. Perceived effectiveness scores will be analyzed for all viewers of each ad, and may also be analyzed by completion method (i.e. study session or online) and demographic characteristics (i.e. gender, race/ethnicity). Responses to the knowledge, attitude, and belief questions will be compared between ad-viewing and control participants to identify any statistically significant differences. Findings from these analyses will be used to optimize video ads for FDA’s Multicultural Campaign.
Reporting
The reporting and dissemination mechanism will consist of one primary component: summary statistics (in the form of PowerPoint presentations and other briefings) on youth reactions to video ads and potential unintended consequences. The key events and reports to be prepared are listed in Table 7.
Project Activity |
Date |
Survey |
April 2016 to May 2016 (Approximate) |
Data analysis |
May 2016 to June 2016 (Approximate) |
Presentation of findings |
July 2016 (Approximate) |
Reason(s) Display of OMB Expiration Date is Inappropriate
We are not requesting an exemption to this requirement. The OMB expiration date will be displayed.
Exceptions to Certification for Paperwork Reduction Act Submissions
These information collection activities involve no exception to the Certificate for Paperwork Reduction Act Submissions.
References
Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. In Proceedings of the Survey Research Methods Section (pp. 533-538).
Anderman, C., Cheadle, A., Curry, S., Diehr, P., Shultz, L., & Wagner, E. (1995). Selection bias related to parental consent in schoolbased survey research. Evaluation Review, 19, 663–674.
Asch, D. A., Christakis, N. A., & Ubel, P. A. (1998). Conducting physician mail surveys on a limited budget: a randomized trial comparing $2 bill versus $5 bill incentives. Medical Care, 36(1), 95-99.
Beebe, T. J., Davern, M. E., McAlpine, D. D., Call, K. T., & Rockwood, T. H. (2005). Increasing response rates in a survey of Medicaid enrollees: the effect of a pre-paid monetary incentive and mixed modes (mail and telephone). Medical Care, 43(4), 411-414.
Castiglioni, L., Pforr, K., & Krieger, U. (2008, December). The effect of incentives on response rates and panel attrition: Results of a controlled experiment. In Survey Research Methods (Vol. 2, No. 3, pp. 151-158).
Clark, S. M., & Mack, S. P. (2009). SIPP 2008 Incentive Analysis.
Creighton, K. P., King, K. E., & Martin, E. A. (2007). The use of monetary incentives in Census Bureau longitudinal surveys. Survey Methodology, 2.
Coughlin, S. S., Aliaga, P., Barth, S., Eber, S., Maillard, J., Mahan, C., & Williams, M. (2013). The effectiveness of a monetary incentive on response rates in a survey of recent US veterans. Survey Practice, 4(1).
Dirmaier, J., Harfst, T., Koch, U., & Schulz, H. (2007). Incentives increased return rates but did not influence partial nonresponse or treatment outcome in a randomized trial. Journal of Clinical Epidemiology, 60(12), 1263-1270.
Dykema, J., Stevenson, J., Kniss, C., Kvale, K., Gonzalez, K., & Cautley, E. (2012). Use of monetary and nonmonetary incentives to increase response rates among African Americans in the Wisconsin Pregnancy Risk Assessment Monitoring System. Maternal and Child Health Journal, 16(4), 785-791.
Fallin A, Neilands TB, Jordan JW, Hong JS, Ling PM. Wreaking “Havoc” on smoking: Social branding to reach young adult “Partiers” in Oklahoma. Am J Prev Med. 2015;48(1):S78-S85.
Fuqua JL, Gallaher PE, Unger JB, et al. Multiple peer group self-identification and adolescent tobacco use. Subst Use Misuse. 2012;47(6):757-766.
Gajic, A., Cameron, D., & Hurley, J. (2012). The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. The European Journal of Health Economics, 13(6), 789-799.
Han, D., Montaquila, J. M., & Brick, J. M. (2012). An Evaluation of Incentive Experiments in a Two-Phase Address-Based Sample Mail Survey. In Proceedings of the Survey Research Methods Section of the American Statistical Association.
Jäckle, A., & Lynn, P. (2008). Offre de primes d’encouragement aux répondants dans une enquête par panel multimodes: effets cumulatifs sur la non-réponse et le biais. Techniques d’enquête, 34(1), 115-130.
Kearney, K.A., Hopkins, R.H., Mauss, A.L., & Weisheit, R.A. (1983). Sample bias resulting from a requirement for written parental consent. Public Opinion Quarterly, 47, 96102.
Levine, R.J. (1995) Adolescents as research subjects without permission of their parents or guardians: Ethical considerations. Journal of Adolescent Health, 17(5), 289-297.
Lee, Y. O., Jordan, J. W., Djakaria, M., & Ling, P. M. (2014). Using peer crowds to segment Black youth for smoking intervention. Health Promotion Practice, 15(4), 530-537.
Messer, B. L., & Dillman, D. A. (2011). Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opinion Quarterly, 75(3), 429-457.
Montaquila, J. M., Brick, J. M., Williams, D., Kim, K., & Han, D. (2013). A study of two-phase mail survey data collection methods. Journal of Survey Statistics and Methodology, 1(1), 66-87.
Pokorny, S. B., Jason, L. A., Schoeny, M. E., Townsend, S. M., & Curie, C. J. (2001). Do participation rates change when active consent procedures replace passive consent. Evaluation Review, 25(5), 567-580.
Racial and Ethnic Approaches to Community Health Across the U.S. (REACH U.S.) Evaluation. OMB CONTROL NUMBER: 0920-0805 Report on Incentive Experiments www.reginfo.gov/public/do/DownloadDocument?objectID=30165501. Accessed on February 1, 2016.
Severson, H.H., & Ary, D.V. (1983). Sampling bias due to consent procedures with adolescents. Addictive Behaviors, 8(4), 433–437.
Shettle, C., & Mooney, G. (1999). Monetary incentives in US government surveys. Journal of Official Statistics, 15(2), 231.
Singer, E., & Kulka, R. A. (2002). Paying respondents for survey participation. Studies of welfare populations: Data collection and research issues, 105-128.
Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. Survey nonresponse, 51, 163-177.
Stevenson, J., Dykema, J., Kniss, C., Black, P., & Moberg, D. P. (2011). Effects of mode and incentives on response rates, costs and response quality in a mixed mode survey of alcohol use among young adults. In annual meeting of the American Association for Public Opinion Research, May, Phoenix, AZ.
Strouse, R., Hall, J., Potter, F., Carlson, B. L., Cunningham, P., Pascale, J., & Stapulonis, R. (1998). Report on Survey Methods for the Community Tracking Study’s 1996–1997 Round One Household Survey. Technical publication, 15.
Substance Abuse and Mental Health Services Administration (SAMHSA). (2015). Results from the 2014 National Survey on Drug Use and Health: Detailed Tables. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Behavioral Health Statistics and Quality.
Tubman, C. & Williams, D. (2010). The Effectiveness of Incentives Used in the Second Phase of a Two Phase Survey. Paper presented at the American Association for Public Opinion Research, May 14, Chicago, IL.
Ulrich, C.M., Danis, M., Koziol, D., Garrett-Mayer, E., Hubbard, R., & Grady, C. (2005). Does it pay to pay? A randomized trial of prepaid financial incentives and lottery incentives in surveys of nonphysician healthcare professionals. Nursing Research, 54(3), 178-183.
Unger, J.B., Gallaher, P., Palmer, P.H., Baezconde-Garbanati, L., Trinidad, T.R., Cen, S., & Johnson, C.A. (2004). No news is bad news: Characteristics of adolescents who provide neither parental consent nor refusal for participation in school-based survey research. Evaluation Review, 28(1), 52-63.
U.S. Department of Health and Human Services (USDHHS). (2014). The Health Consequences of Smoking—50 Years of Progress: A Report of the Surgeon General. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health.
Van der Rijt, G. A., d’Haenens, L. S., & van Straten, P. (2002). Smoking and other substance use as distinct features of teenage subcultures. Journal of Adolescent Health, 31(5), 433-435.
Warriner, Keith, John Goyder, Heidi Gjertsen, Paula Hohner, and Kathleen McSpurren. 1996. “Charities, No, Lotteries, No, Cash, Yes: Main Effects and Interactions in a Canadian Incentives Experiment.” Paper presented at the Survey Non-Response Session of the Fourth International Social Science Methodology Conference, University of Essex, Institute for the Social Sciences, Colchester, UK.
Page
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Modified | 0000-00-00 |
| File Created | 2021-01-26 |