U.S. Food and Drug Administration
Center for Tobacco Products
The Real Cost Campaign: Media Tracking Study
Supporting Statement: Part B
B. Statistical Methods
1. Respondent Universe and Sampling Methods
The one-time actual burden figures are listed in the tables in the Part A Supporting Statement.
For this study, we will obtain a final yearly sample size of 2,000 youth (ages 13 – 17) through a survey that will be fielded cross-sectionally approximately quarterly for 6 months. We plan to obtain approximately 750 youth completes per round of data collection for up to 2 rounds, totaling up to 2,000 survey completes among youth ages 13-17 who either: 1) are at risk of initiating use of tobacco products, or 2) have experimented with tobacco products. Each sample will be freshly recruited, but respondents will be eligible to participate again six months after initial participation.
As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample is designed primarily to monitor reactions to in-market and new advertising; validate changing knowledge, attitudes, beliefs, and behaviors to tobacco products; and explore motivations and psychographics that lead to tobacco use. The results of the tracking study will be used to inform specified recommendations around the U.S. Food and Drug Administration (FDA) public education programs’ impact and effectiveness in reducing tobacco-related death and disease.
Sampling Methods
The study sample will be comprised of participants between the ages of 13 and 17 years who either: 1) are at risk of initiating use of tobacco products, or 2) have experimented with tobacco products. This study is considered part of formative research for campaign development and planning, and these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample is designed primarily to provide information ad reactions and the perceptions and trends in tobacco use.
This study will recruit youth and young adult participants through an online panel. Online panels will be composed of youth and parents of youth in the United States with children ages 13-17 who have been pre-screened for their willingness for their child to participate in online surveys.
All panelists are double opt-in, meaning that they must request to join the panel by signing up, and then verify their intent to participate by responding to a confirmation email. Every respondent goes through a rigorous and extensive set of security procedures before joining the panel. Information about the participants’ demographics is collected during the initial sign-up process and then updated and verified through survey participation to maintain a current and accurate database.
Parents of children on the panel within the target age range (13-17) will receive an initial email
Invitation to have their child participate in a new study. The email invitation will also include a link that details the purpose of the study. If the parent determines that they would like their child to participate in the study, they will be asked to forward the study link to their child, thereby actively opting in and providing permission for their child to participate in the study (see Appendix A and B).
From the email forwarded by their parents, youth will access the study link that details the
purpose of the study and includes study materials (i.e., the online youth assent form, screener, and survey; see Appendix C, D, and E). Youth will be directed to review the purpose of the study and complete the screener to provide basic demographic information and establish eligibility. This screener is expected to take approximately 2 minutes.
The majority of questions asked will not be sensitive, but some will ask about specific health behaviors such as tobacco product use. These sensitive questions are essential to the objectives of this data collection. Although we do not anticipate any risks from these health questions, some participants may perceive them to be sensitive. Questions about messages concerning lifestyle (e.g., tobacco product use) and some demographic information, such as race and ethnicity could also be considered sensitive, but not highly sensitive. Demographic information will be collected to assess variations in tobacco knowledge, attitudes, beliefs, and behaviors, media use, and awareness and receptivity to CTP public education campaign advertisements by demographic subgroup in order to more effectively design and deliver campaign messages. To address any concerns about inadvertent disclosure of sensitive information, participants will be fully informed of the applicable privacy safeguards.
Reminders are sent to all invitees who have not responded 48 hours after initial invitation. If an invitee does not respond after the reminder is sent, they are considered not available and are not re-approached.
After completion of the screener, qualified youth will be granted access to the
survey where the youth assent form will be embedded.
Participants will be required to review and complete the assent form prior to
beginning the survey. If a participant decides they no longer want to participate, they will
not have access to the survey and the survey link will expire.
Sample Size
Sample of 2,000 youth ages 13–17.
2. Procedures for the Collection of Information
This section describes the procedures for survey data collection. Online data collection will be completed by youth independently, on their own electronic devices, such as a mobile phone, tablet, or home computer.
All surveys will be conducted using a self-administered, online survey. To be eligible, the youth (ages 13–17) must give their assent (Appendix C) and participants must provide their tobacco use status to be classified as either: 1) at risk of initiating use of tobacco products, or 2) having experimented with tobacco products. The screener is included in Appendix D. The survey instrument is in Appendix E.
OMB Control Number 0910-0810 is covered underneath a Privacy Impact Assessment that has been approved by the Department of Health and Human Services (PIA Unique Identifier: P-9008729-198376).
FDA IRB will not be the Institutional Review Board of record and will defer to KDH Research & Communication IRB’s policies and oversight. This study has been reviewed and approved by KDH Research & Communication Institutional Review Board and will be conducted under their oversight. The study team is committed to the protection of human subjects and the privacy and security of participant data. For this study, we are not collecting any data, including personally identifiable data, from parents. We consulted with the KDH Research & Communication’s IRB and based on the content of the online survey and study design, received confirmation that it is permissible to contact participants ages 13 to 17 after implied parental permission (e.g., parent must forward invitation on to the youth). Eligible youth who are 13 to 17 years old will encounter the assent language once they are routed to the survey. We recognize that some of the behaviors we are asking about are not legal for some respondents (e.g., respondents under 21 who use tobacco).
The central purpose of this study, which is to provide FDA with advertising tracking information as it relates to ENDS product use among youth, cannot be achieved without directly recruiting and surveying youth ages 13 to 17. Since the surveys focus on perceptions and trends in tobacco use, it is anticipated that the parents of some eligible participants would refuse to grant permission for their child to participate. Further, the youth may not answer questions accurately if they feel their parents are somehow observing or approving their participation and requiring parental permission would likely yield a lower response rate and impede the ability to carry out this study. Because obtaining active parental permission for at-risk youth will result in a sample with different characteristics than the target group, a parental opt-out approach is being requested for all potential youth participants (see Appendix B). The youth participants’ responses will not be linked to their names or email addresses. KDH Research & Communication IRB allowed us to forgo obtaining written parental permission based on the nature of this study, the lack of linkage between youth respondents’ answers to the online survey and their names, as well as the need for accurate and truthful responses from this population.
Consistent with the Federal Children’s Online Privacy Protection Act (COPPA), we will screen out youth under the age of 13. This study poses only a minimal risk to participants and would not adversely affect their rights or welfare in any foreseeable way.
Summary of Protocol
The list of study procedures is as follows:
Tracking Study participants recruited through online panel
Youth (13-17)
This study will recruit youth participants through an online panel (via Marketing Workshop). The online panel provider may use social media platforms to recruit additional participants as needed to augment their existing panels.
Parents will receive an initial email informing them of the study and opportunity for their child(ren) to participate. The email invitation will also include a link that details the purpose of the study. After reading this invitation, the parent has three options:
If the parent determines that they would like their child to participate in this particular study, they will be asked to forward the study link to their child, thereby actively opting in and providing permission for their child to participate in the study
The parent can do nothing, in which case the invitation dies in place and that parent’s child never receives information about the study.
If the parent does not want their child to participate in the study, they can use instructions in the opt-out form included in the email to indicate this. Parent reminders to participate will cease if the parent opts-out.
Reminders are sent to all invitees who have not responded 48 hours after initial invitation. If an invitee does not respond after the reminder is sent, they are considered not available and are not re-approached.
From the email forwarded by their parents, youth will access the study link that details the purpose of the study and includes study materials (i.e. the online screener, assent form and survey).
Youth participants will complete a screener, which is a list of questions to determine eligibility. Responses to the screening questions determine eligibility to complete the full survey. Once determined eligible, the participant is then presented an assent form that must be completed prior to having access to and completing the survey.
Following completion of the assent form, the survey will be presented for the participants to complete.
After completing the survey, the participant will be thanked, and participation will end.
Tokens of Appreciation
As a token of appreciation, participants recruited who complete and submit the survey will receive a Visa gift card worth $5. Such tokens are commonly used by research agencies to recruit participants efficiently and effectively; parents/guardians or individual members who choose to be a part of these online panels have an expectation that they will be compensated for their time. We estimate that the survey will take about 17 minutes to complete, with the entire study taking 21 minutes (including a 2-minute screener and 2-minute assent process). This amount not only reflects the burden of time to participate, it will also ensure that the respondent pool is recruited within a tight timeframe. Smaller token amounts are associated with slower movement on recruitment.
Email addresses/IP addresses, and zip codes are PII that is collected in this study. These pieces of
information play an important role in study implementation and quality assurance as described
below. FDA will not have access to any PII.
All participants will be asked to provide their email address, and IP address will be collected automatically. (1) Email addresses will be checked against all current respondent data to avoid duplicates and reduce fraudulent activity. If multiple emails have the same IP address, researchers will review the data, retain the first recorded response, and remove duplicates from the final analytical dataset. (2) Participants’ email addresses will also be collected when providing assent to participate in the study to send participants a record of the completed assent form. (3) Researchers may also contact participants in the unlikely case that there is a breach of confidentiality, for example, as a result of hacking or an issue with delivery of the token of appreciation. This information will be collected to confirm the validity of participant responses.
All participants will be asked to provide their zip code. Zip codes will be used to verify residency in the US (48 contiguous states and District of Columbia). Participants completing the screener from ineligible zip codes will be screened out.
Email addresses, IP addresses and zip codes will never be included in the dataset used for analysis. The dataset used for analysis will be de-identified. FDA/CTP and FCB will never receive a dataset with PII.
All data collection activities will be conducted in full compliance with FDA regulations to maintain the privacy of data obtained from respondents and to protect the rights and welfare of human research subjects as contained in their regulations.
Confirmit will be the software used for data collection and storage. Confirmit is a trusted survey
tool used by both researchers and marketing companies alike. Confirmit is provided as an online
application, accessed using a modern Internet browser where data are stored in a single secure
data center. The database servers that store respondent and response data are placed behind two
tiers of firewalls, and data can only be accessed through the Confirmit applications. No
application users have direct database access, the servers are only accessible for database
administrators. Remote server access is only available to our system administrators through
network controls and secure VPN tunnels. If outside the corporate network, dual factor
authentication is required to establish a secure VPN tunnel into the corporate network (in order
to access the production VPN through a hop-server), and only computers that are under
Confirmit’s control are allowed to connect to the VPN. Confirmit surveys do not require any
user-identifiable information to be transmitted between page submissions. Surveys use a combination of hidden form fields and system generated identifiers to identify the respondent.
Survey pages include meta code to prevent them from being cached on the client. No information is stored on a respondent’s computer when the browser is closed. To further prevent caching, all surveys are available over HTTPS. Data will be stored on dedicated servers with a redundant multi-tier network security infrastructure.
Any online panel used in this study will follow the Standard of Good Practice (SoGP,
https://www.securityforum.org/ research/thestandardofgoodpractice2016/) security practices
which emphasize security management, safe business application protocol, safe computer
installations, network fidelity, awareness of systems development requirements, and safety of the
end-user environment. Following these guidelines, the monitoring of data and sensitive
information will take place following the SoGP security practices such as limiting access to
information and data encryption. Members of the research team will receive de-identified data
that will be in an encrypted format when it is transmitted. This data transfer will occur via an
encrypted and secure broadband connection.
Only KDH Research & Communication and Marketing Workshop will ever have data with PII. FCB and FDA will not have access to PII. All PII except for informed assent will be destroyed no later than three months after the last survey is completed. Assents will be destroyed in three years. The data are backed up on a daily basis. No one can access the data unless it is required by law to protect participant rights or to comply with judicial proceedings, a court order, or other legal process. Data will be maintained on secure servers for a period of three years and then destroyed securely using best practices.
When the raw data are downloaded from the Confirmit, the data will be in two datasets paired by participant (1) screener responses, and (2) survey responses. The respective screener and survey datasets will be merged into a participant data set using unique identifiers that do not contain PII. KDH Research and Communication and Marketing Workshop will clean the data using SPSS. The datasets will go through a thorough review and data cleaning process to remove unfinished, duplicate, and/or fraudulent surveys, and a de-identified dataset will be created for analysis. Only members of the research team (i.e. FDA CTP, KDH Research and Communication, and FCB) will have access to the de-identified raw data files. FDA/CTP and FCB will never receive any PII.
Unusual Problems Requiring Specialized Sampling Procedures
No specialized sampling procedures are involved.
Use of Periodic Data Collection Cycles to Reduce Burden
The ability to obtain the cooperation of potential respondents in the survey will be important to the success of this study. We will minimize the non-response rate by employing the following measures:
Use the online panels provided by Marketing Workshop to best reach the desired sample
Send reminders to all invitees who have not responded 48 hours after initial invitation.
Provide a $5 gift card as a token of appreciation to participants who complete and submit the survey
As a token of appreciation, participants recruited who complete and submit the survey will receive a Visa gift card worth $5. Such tokens are commonly used by research agencies to efficiently and effectively recruit participants; parents/guardians or individual members who choose to be a part of these online panels have an expectation that they will be compensated for their time. We estimate that the survey will take about 17 minutes to complete, with the entire study taking 21 minutes (including a 2-minute screener and 2-minute assent process). This amount not only reflects the burden of time to participate, it will also ensure that the respondent pool is recruited within a tight timeframe. Smaller token amounts are associated with slower movement on recruitment.
4. Tests of Procedures or Methods
KDH Research & Communication will conduct rigorous internal testing of the online survey instruments prior to fielding. Survey testers will review the online test version of the instrument that we will use to verify that instrument skip patterns are functioning properly, delivery of the token of appreciation is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.
5. Individuals Involved in Statistical Consultation and Information Collection
The following individuals inside the agency have been consulted on the design of the study plan, audience survey development, or intra-agency coordination of information collection efforts:
Morgane Bennett, DrPH, MPH
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-402-9281
E-mail: [email protected]
Atanaska (Nasi) Dineva, MS
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 301-796-4498
Email: [email protected]
The following individuals outside of the agency have been consulted on survey development:
Kristen Holtz, Ph.D.
KDH Research & Communication
145 15th Street NE
Suite 831
Atlanta, GA 30309
Phone: 404-395-8711
E-mail: [email protected]
Dimas Adiwiyoto
Senior Vice President
FCB
111 W 33rd Street
New York, NY 10120
(646) 504 8586
References
Singer, E. (2002). The use of incentives to reduce nonresponse in household surveys. In: Wiley, ed. Survey nonresponse, 51, 163-177.
Shettle, C., & Mooney, G. (1999). Monetary incentives in US government surveys. Journal of Official Statistics, 15(2), 231.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Chapman, LeTonya |
| File Modified | 0000-00-00 |
| File Created | 2023-08-28 |