OMB No. 0910-0810
Exp. Date: 12/31/2024
U.S. Food and Drug Administration
Center for Tobacco Products
Increasing Understanding of Digital Advertising:
Awareness, Receptivity, and Comprehension (ARC) Experiment
Supporting Statement: Part A
|
Study Materials (attached):
Supporting
Statement A
Supporting Statement B
Attachment A: Screener and Survey
Attachment B: Informed Consent
Attachment C: Sample Email Prompt
Attachment D: Digital Survey Recruitment Ads
Attachment E: Digital Survey Recruitment Ads FAQ
Attachment F: Privacy Policy
Video Ads
RTI IRB Approval Letter (Correspondence Letter)
Increasing Understanding of Digital Advertising:
Awareness, Receptivity, and Comprehension (ARC) Experiment
Supporting Statement: Part A
A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
To inform recommendations around the FDA’s public education programs’ impact and effectiveness in reducing tobacco-related death and disease, more research is needed to understand the potential for digital anti-tobacco advertising to influence tobacco-related beliefs. Digital marketing has clear value in the commercial space; advertising research on thousands of online advertising campaigns shows that digital advertising drives brand and ad awareness and positive attitudes toward brands (Twose & Elmore, 2015). However, there is still much to be learned about the use of digital advertising for public education campaigns, such as those discouraging tobacco use. Digital media metrics can reveal a lot about consumer experiences in aggregate: how often consumers were exposed to a video, how much of the video they watched, whether they shared it with their friends, and whether they reacted positively or negatively. We have used some of these metrics to develop a better understanding of the relative effectiveness of digital and broadcast advertising in generating ad/campaign awareness in the context of FDA CTP campaigns. However, many questions remain that can only be addressed in experimental studies or prospectively with campaign implementation approaches that have sufficient flexibility to manipulate media buying. Specifically, it is unclear how closely young adults attend to public health messages related to tobacco delivered via social media and streaming media services. This is a critical question as traditional broadcast television audiences are decreasing and shifting their media consumption to digital sources (Guttmann, 2023).
2. Purpose and Use of the Information
RTI International (RTI) will conduct a field experiment and online survey administered at two time points with young adults in 9 media markets in the United States. The experiment will randomly assign 9 media markets to receive one of 3 doses of media (i.e., low, medium, or high) and one of 3 media mixes that varies the share of the media on social media and streaming media services. In addition, there will be a control condition drawn from non-adjacent media markets where campaign advertising will not be airing. To measure study outcomes, we will recruit young adults through social media (i.e., Facebook, Instagram) and Dynata’s online survey panel to assess awareness of and receptivity to the study video ads and agreement with specific beliefs about hookah. We will survey approximately 4,035 young adults. Approximately 1,418 young adults (aged 18 to 24-years) in the treatment markets will be surveyed 4 weeks after the advertisements begin to air in the 9 media markets. An additional 1,418 young adults in the treatment markets will be surveyed 8 weeks after the advertisements begin to air. A control sample of an additional 1,200 young adults drawn from non-adjacent media markets will be collected once, 8 weeks after the advertisements begin to air. The control sample will allow us to measure the extent of false awareness reported by young adults. It is possible that recruitment will not close the instant the target sample is obtained. Therefore, up to 4,435 respondents could be recruited.
The primary objectives of the survey are to:
Measure awareness and comprehension of and receptivity to video ads running in the 9 media markets, and
Measure knowledge, attitudes, and beliefs about hookah use.
The results of the ARC study will be used to inform the implementation of public education campaigns designed to reach youth and young adults that rely on digital media advertising.
3. Use of Information Technology and Burden Reduction
This study will rely on web-based survey data collection to collect primary data to understand young adults’ response to digital video advertisements. Using an online survey allows the respondent to be candid with their responses. This increases accuracy of the data because respondents provide more honest responses than when other types of data collection methods are employed. In addition, using an online survey is more cost-effective than in-person, mail, or telephone surveys and will allow for more participants to respond in a timely manner. The self-administered, web-based survey permits greater expediency with respect to data processing and analysis (e.g., coding and data entry). Data are transmitted electronically, rather than by mail. These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. An added benefit is increased data protection by limiting the amount of personally identifiable information (PII) collected from participants, reducing the risk of data security issues. Finally, as noted above, this technology permits respondents to complete the survey in privacy. The use of a more private data collection method makes reporting potentially embarrassing or stigmatizing behaviors (e.g., tobacco use) less threatening and enhances response validity and response rates.
4. Efforts to Identify Duplication and Use of Similar Information
In designing the proposed data collection activities, we took several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets would address the proposed study questions. We reviewed the published literature to confirm an experiment with this design had not yet been conducted. In addition, we carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address FDA’s need for information on beliefs about various tobacco products. We reviewed data collected to evaluate other national tobacco-focused media campaigns such as CDC’s Tips From Former Smokers® campaign and FDA’s The Real Cost. We also examined ongoing national surveillance systems such as the National Youth Tobacco Survey (NYTS), the Youth Risk Behavior Surveillance System (YRBSS), the National Health Interview Survey (NHIS), and the Population Assessment of Tobacco and Health (PATH) to see if they included survey items that would address our research questions. Although these data sources measure tobacco use and beliefs, they do not provide adequate samples sizes in the selected media markets, nor do they include measures of awareness of the study’s video ads or related beliefs around hookah use.
5. Impact on Small Businesses or Other Small Entities
Respondents in this study will be members of the general public, not business entities. No impact on small businesses or other small entities is anticipated.
6. Consequence of Collecting the Information Less Frequently
This study consists of one online survey administered at either 4 or 8 weeks following the launch of the experiment to measure ad awareness and receptivity, message comprehension, and tobacco-related beliefs.
7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CRF 1320.5(d)(2).
8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside
Agency
The following individuals inside the agency have been consulted on the design of the study, instrument development, or intra-agency coordination of information collection efforts:
Elizabeth Petrun Sayers
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-750-5297
E-mail: [email protected]
Lindsay Pitzer
Office of Health Communication & Education
Center for Tobacco Products
Food and Drug Administration
10903 New Hampshire Avenue
Silver Spring, MD 20993
Phone: 240-620-9526
E-mail: [email protected]
FDA collaborates with other agencies that sponsor or endorse health communication projects. These affiliations serve as information channels, help prevent redundancy, and promote use of consistent measures of effectiveness. Coordination activities include:
Review of questionnaires for testing purposes;
Sharing data; and
Standardizing survey tools where at all possible.
The following individuals outside of the agency have been consulted on questionnaire development.
Jamie Guillory
RTI International
64 Dame Street
Dublin D02RTY72, Ireland
Phone: 919-695-9264
Email: [email protected]
Kim Hayes
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: 919-541-1215
E-mail: [email protected]
Jessica Sobolewski
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: 919-541-6657
Email: [email protected]
Vaughn Armbrister
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: 919-248-8521
Email: [email protected]
9. Explanation of Any Payment or Gift to Respondents
As a token of appreciation, participants who complete a survey and are deemed non-fraudulent or non-duplicate will receive a $5 token of appreciation. Participants recruited through the online panel will receive panel points. Each token of appreciation amount reflects the burden of spending an average of 15 minutes taking the survey. There is no incentive for completing the web screener.
A token of appreciation treats participants justly and with respect by recognizing and acknowledging the effort they expend to participate. The token of appreciation must be high enough to equalize the burden placed on participants with respect to their time and cost of participation. Too low of an amount may result in a difficult and lengthy recruitment process and/or respondents who agree to participate and then drop out early. Empirical studies show that incentives can increase response rates in cross-sectional surveys and reduce attrition in longitudinal surveys within some respondent populations (Singer & Ye, 2013; LeClere et al., 2012; Cantor et al., 2003; Singer, 2002; Singer et al., 1998) Several studies have shown the effects of incentives within the context of web-based surveys. For example, a 2006 meta-analysis of 32 studies indicates that incentives increase the odds that potential respondents will begin a web survey, and a second meta-analysis of 26 studies shows that incentives increase the odds of completing a web survey once respondents have begun it (Goritz, 2006).
A reasonable token of appreciation is standard practice for research and is suggested by organizations that set the standards for conducting ethical industry-led research among human subjects (Code of Ethics and Standards for Market Research and Data Analytics, 2021; Insights Association, 2021).
A
previous “Promising Themes” study (OMB control number
0910-0810) of a similar length took 13 weeks to recruit 2,622 youth
and young adults of a targeted 3,000 completes with while offering a
$5 token of appreciation. Studies that focused on similar products
and participants, such as the Monthly Monitoring Studies (OMB control
number 0910-0810), had success recruiting 1,501 participants when a
$5 token of appreciation was offered. In addition, the Hookah Beliefs
Survey (OMB control number 0910-0810) had success recruiting 1,200
participants when a $5 token of appreciation was offered.
10. Assurance of Confidentiality Provided to Respondents
OMB Control Number 0910-0810 is covered underneath a Privacy Impact Assessment that has been approved by the Department of Health and Human Services (PIA Unique Identifier: P-9008729-198376).
PII Collection
As part of this study, RTI, the contractor acting on behalf of FDA, will be collecting and maintaining PII about participants who complete an online screener/online survey. The only PII RTI will be collecting is email address, IP address, zip code, and birthdate; each of these pieces of information will be stored separately from one another and from survey responses (except for 48 hours after download while the fraud detection procedures are completed). We are not collecting any Protected Health Information, defined as “Personally identifiable information that relates to a person's health, medical treatment or payment, and which was obtained from a "covered entity" (health care provider, health plan, or healthcare clearinghouse), as defined by HIPAA (Health Insurance Portability and Accountability Act) regulations.” Survey data will be kept separate from PII and/or stored on the Federal Information Processing Standards (FIPS) 199 except for the 48-hour period after download when the combined dataset is stored temporarily on the study share drive so that the fraud detection procedures can be conducted. FDA will not be intervening or interacting with study participants and will not have access to any personally identifiable information (PII) collected from study participants at any time.
This study is funded by the FDA, a Department of Health and Human Services supported agency, and is covered by a Certificate of Confidentiality (CoC). The study involves Human Subjects as defined by 45 CFR Part 46.
Overview of Data Collection System
All information will be collected electronically through a self-administered survey instrument hosted in a secure, web-based data collection system. Approximately half of participants will be recruited via social media, and half of participants will be recruited through an online survey panel. All participants will be screened for eligibility prior to administration of the survey instrument. The screener is designed not to reveal specifically why a respondent is ineligible. All respondents, regardless of age, gender, race/ethnicity, tobacco use behavior, and residence will complete the full screener. Respondents must complete all screener questions to find out whether they can move on to the survey. To recruit for the survey, RTI will place ads on social media (i.e., Facebook, Instagram). As much as possible, these ads will target potentially eligible respondents who are thought to be age 18-24. In addition, we will partner with Dynata to recruit participants via online survey panel.
Each participant will answer items about their hookah use behavior; hookah-related KABs; awareness of, receptivity to, and comprehension of messaging; and demographic information. Participants will complete the survey at the time of their choosing. There is no website content directed at children younger than 13 years of age.
Overview of How Information will be Shared and for What Purposes
All data will be downloaded from Qualtrics (which requires a password) and stored in databases only on RTI’s Federal Information Processing Standards (FIPS) 199 moderate network, which is only accessible by RTI study staff trained in human subjects. At the completion of data collection, the data will be deleted from RTI’s Qualtrics account and remain only on RTI’s FIPS 199 moderate network.
Overview of the Impact the Proposed Collection will have on the Respondent’s Privacy
The following procedures will be used to ensure participant privacy before, during, and after fielding: (1) PII in the form of participants’ IP address, email address, zip code, and birthdate will be stored separately from screening-related data and survey data, and IP and email addresses, zip code, and birthdate will be deleted after study completion; (2) datasets of screener and survey responses and reports will not contain any PII; and (3) respondents’ information will not be tied to their individual responses and all analyses will be conducted in the aggregate (i.e., any data used in reporting will not be attributed to individual participants). All datasets and reports delivered to FDA will not include PII.
RTI will collect email addresses for the purposes of distributing the token of appreciation and birthdate to confirm age. IP addresses will be collected to verify participants are taking the survey from within the United States. No additional personal identifiers (e.g., full name, phone number, social security number) will be collected aside from basic demographic information (e.g., gender, age, and race). PII will be stored separately from any survey responses.
Overview of Voluntary Participation
Potential participants will be advised of the nature of the survey, the length of time it will require, and that participation is voluntary. Participants will be assured that they will incur no penalties if they wish not to respond to the data collection as a whole or to any specific questions. Participants will have the option to decline to respond to any item in the survey for any reason and may drop out of the survey at any time. These procedures conform to ethical practices for collecting data from human participants.
Overview of Data Security
E-mail addresses, zip code, and birthdate will each be collected separately in the Qualtrics survey platform and stored in separate isolated datasets that will contain an RTI-assigned unique ID. IP address will be collected in the survey platform in an isolated dataset that contains IP address, RTI-assigned unique ID, and screener responses. Responses to the body of the survey will be collected in the Qualtrics survey platform and stored in an isolated survey. IP address, e-mail address, zip code, and birthdate will not be collected in the same file.
All survey data files (i.e., separate files for IP address, e-mail, zip code, birthdate, and screener/survey responses) will be downloaded separately from Qualtrics (which requires a password). Since the Federal Information Processing Standards (FIPS) 199 does not permit access to the internet (and downloading the data from Qualtrics requires an internet connection), the files will be downloaded to the secure RTI study share drive and stored on the study share drive for no more than 48 hours after download. RTI study staff will be given as-needed access to the data files on the share during that 48-hour period to conduct fraud detection procedures, at which point data from the individual will be combined to check for fraudulent responses.
At the end of data collection, a member of the RTI project staff will export the data from the survey and out of the FIPS 199 moderate network, saving them directly onto the project share drive. Only RTI project staff directly involved in programming, sampling, recruitment, or analysis will have access to the survey data or sampling frame. No respondent identifiers will be contained in reports to FDA, and results will only be presented in aggregate form.
11. Justification for Sensitive Questions
Most questions asked will not be of a sensitive nature. However, it will be necessary to ask some questions that may be of a sensitive nature in order to assess tobacco use among respondents under 21 years old (the minimum age for legal purchase of tobacco products). These questions are essential to the objectives of this data collection. Although we do not anticipate any risks from these health questions, some participants may perceive them to be sensitive. Questions about lifestyle (e.g., tobacco product use) and some demographic information, such as race and ethnicity and gender identity and sexual orientation, could also be considered sensitive, but not highly sensitive. Collection of these data is necessary in order to assess disparities in tobacco use and possible differences in campaign impact across different populations.
Decades of research has shown significant disparities in tobacco use by race/ethnicity (e.g., Harlow et al., 2019; Odani et al, 2018), gender identity (e.g., Johnson et al, 2019; Delahanty et al, 2019), and sexual orientation (e.g., Johnson et al, 2019; McCabe et al., 2018). Therefore, collecting detailed information on these demographic characteristics will allow us to measure these differences with the goal of reducing these disparities.
As multiple studies of youth and young adults have reported approximately 12-15% of their samples identified as gender non-conforming/non-binary (e.g., The Human Rights Campaign 2018 LGBTQ+ Youth Report; The Trevor Project 2020 National Survey on LGBT Youth Mental Health; CTP’s evaluation of This Free Life campaign), including gender non-conforming/non-binary response options is necessary to identify and assess tobacco use and campaign effectiveness among this population. Along with the extensive and increasing body of literature showing tobacco use disparities among LGBTQ+ populations, the White House issued the Executive Order on Advancing Equality for Lesbian, Gay, Bisexual, Transgender, Queer, and Intersex Individuals which includes obligations for federal agencies to collect SOGI data. The order states that, “advancing equity and full inclusion for LGBTQI+ individuals requires that the Federal Government use evidence and data to measure and address the disparities that LGBTQI+ individuals, families, and households face.” It also states that federal agencies must “describe disparities faced by LGBTQI+ individuals that could be better understood through Federal statistics and data collection” (White House, 2022). A gender identity item that includes responses such as “non-binary” or “another gender identity” along with “male,” “female,” and “transgender” has been recommended by the White House Office of the Chief Statistician (2023).
To address any concerns about inadvertent disclosure of sensitive information, participants will be fully informed of the applicable privacy safeguards. This study includes several procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including the following:
Participants will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer;
Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses; and
Participants will be provided with an email address to contact should they have any questions or concerns about the study.
Survey respondents will have the option to only answer the items that they select to answer. They may end the end a survey at their discretion. We will take every measure to ensure that respondents are aware of their choices and provide assent or consent to participate.
Before requesting any information from participants, RTI will provide information about data security and ways to request data removal from RTI databases. RTI will also inform participants that they can select “prefer not to answer” to any question and will recommend that participants complete the survey where no one can see their answers. RTI will provide study information through informed consent and conclude the survey immediately for participants who state that they do not want to take the survey after reading the study information. The project team will not conduct or report on statistical analysis for demographic groups for which there is insufficient statistical power.
12. Estimates of Annualized Burden Hours and Costs
12 a. Annualized Hour Burden Estimate
An estimated one-time reporting burden for this collection will be approximately 1,936 hours annually (Table 1). This includes the time burden associated with the screener and informed consent. We will obtain a final sample size of 4,435 young adults aged 18-24, collected at 4 weeks and 8 weeks after the media launch.
Table 1. Estimated Annual Reporting Burden
Type of Respondent |
Activity |
Number of Respondents |
Number of Responses per Respondent |
Total Responses |
Average Burden per Response (in hours) |
Total Hours |
Young Adult aged 18-24 |
Recruiting and Screening |
12,105 |
1 |
12,105 |
0.05 |
605 |
Young Adult aged 18-24 |
Consent |
4,435 |
1 |
4,435 |
0.05 |
222 |
Young Adult aged 18-24 |
Online Survey |
4,435 |
1 |
4,435 |
0.25 (15 minutes) |
1,109 |
Total Annualized Hours |
|
|
|
|
|
1,936 |
12b. Annualized Cost Burden Estimate
Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. RTI has conducted many tobacco-related surveys of similar length among young adults. We have examined diagnostic data from prior surveys and estimate that data collection for this study will take approximately 6 minutes per respondent for screening and informed consent, and approximately 15 minutes per respondent for the online survey.
To calculate the estimated annual cost, we used the mean hourly wage $29.76 from the U.S. Department of Labor Bureau of Labor Statistics (May 2022 data). There are no direct costs to respondents associated with participation in this information collection. RTI has conducted many tobacco-related surveys of similar length among young adults. We have examined diagnostic data from each of these prior surveys and estimate that data collection for this study will take, on average, 3 minutes per respondent for screening, 3 minutes per respondent for informed consent and 15 minutes per respondent for the online survey. Thus, assuming an average hourly wage of $29.76, and doubling this to account for benefits and overhead, yielding an hourly wage rate of $59.52, the estimated one-year cost to participants will be $115,232. The estimated value of respondents’ time for participating in the information collection is summarized in Table 2.
Table 2. Estimated Annual Cost
Type of Respondent |
Activity |
Annual Burden Hours |
Hourly Wage Rate |
Total Cost1 |
Young adult aged 18–24 |
Recruiting and Screening |
605 |
$59.52 |
$36,010 |
Consent |
222 |
$59.52 |
$13,214 |
|
Online Survey |
1,109 |
$59.52 |
$66,008 |
|
Total |
|
|
|
$115,232 |
1 Cost was rounded up to the next dollar.
13. Estimates of Other Total Annual Cost Burden to Respondents or Record Keepers
There are no capital, start-up, operating, or maintenance costs associated with this data collection.
Annualized Cost to the Federal Government
The total estimated cost to the Federal Government for this study is $169,371, as shown in Table 3. Contractor costs attributable to this information collection are $138,659. This includes costs to program the survey, recruit the sample, and collect the data. Other contractor activities outside this data collection estimate include coordination with FDA to develop the instrument and deliver the final data set and reporting deliverables.
Table 3. Itemized Cost to the Federal Government
Government Personnel |
Time Commitment |
Average Annual Salary |
Total1 |
GS-13 |
15% |
$122,8452 |
$18,427 |
GS-13 |
10% |
$122,8452 |
$12,285 |
Total salary costs |
|
|
$30,712 |
Contractor costs |
|
|
$138,659 |
Total |
$169,371 |
||
1 Cost was rounded up to the next dollar.
2 Represents the median GS-13 pay in Washington-Baltimore-Arlington, DC-MD-VA-WV-PA in 2022.
15. Explanation for Program Changes or Adjustments
This section in not applicable as this is a new study under the generic data collection package.
16. Plans for Reporting and Project Time Schedule
Data from this information collection will be used to enable the FDA to assess the dose-response relationship between digital media and self-reported hookah education video ad awareness and the relative effectiveness of social and streaming media in promoting ad awareness. In addition, this study will assess whether hookah-related beliefs among young adults aged 18-24 years change as a result of exposure to small dose of digital media. Findings from these analyses will be used to inform future experiments to test the potential for digital advertising to influence tobacco-related beliefs.
Reporting
At the end of the study, a draft report and a final report containing background information on the project objectives, scope, methodology, key findings, and conclusions will be completed. The approximate dates for completing project tasks are listed in Table 4.
Table 4. Approximate Project Schedule
Project Activity |
Date |
Survey |
July 2023 (Approximate) |
Preparation of analytic data file |
Approximately 1–2 weeks after completion of data collection |
Data Analysis |
Approximately 3–5 weeks after completion of data collection |
Report Writing |
Approximately 6-8 weeks after completion of data collection |
17. Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable. All data collection instruments will display the expiration date for OMB approval of the information collection.
18. Exceptions to Certification for Paperwork Reduction Act Submissions
These information collection activities involve no exception to the Certification for Paperwork Reduction Act Submissions.
References
Bandiera FC, Arguelles W, Gellman M, et al. Cigarette smoking and depressive symptoms among Hispanic/Latino adults: Results from the Hispanic Community Health Study/Study of Latinos (hchs/Sol). Nicotine Tob Res.
Cantor D et al. (2003). Comparing promised and pre-paid incentives for an extended interview on a random digit dial survey. In Presentation for the Annual Meeting of the American Association for Public Opinion Research, Nashville, TN.
Göritz AS. (2006). Incentives in web studies: methodological issues and a review. Int J Internet Science. 1(1), 58-70.
Guttmann, A. (2023, January 9). Time spent with digital vs. traditional media in the U.S. 2011-2023 [Graph]. Statista. https://www.statista.com/statistics/565628/time-spent-digital-traditional-media-usa/
Kaplan RC, Bangdiala SI, Barnhart JM, et al. Smoking among US Hispanic/Latino adults: The Hispanic Community Health Study/Study of Latinos. Am J Prev Med. 2014;46(5):496-506. doi:10.1016/j.amepre.2014.01.014
LeClere F et al. (2012). Household early bird incentives: leveraging family influence to improve household response rates. In American Statistical Association Joint Statistical Meetings, Section on Survey Research.
Office of the Chief Statistician of the United States. (2023). Recommendations on the best practices for the collection of sexual orientation and gender identity data on federal statistical surveys. https://www.whitehouse.gov/wp-content/uploads/2023/01/SOGI-Best-Practices.pdf
Parrinello CM, Isasi CR, Xue X, et al. Risk of cigarette smoking initiation during Adolescence among US-Born and non-US-born Hispanics/Latinos: The Hispanic Community Health Study/Study of Latinos. Am J Public Health. 2015;105(6):1230-1236. doi:10.2105/AJPH.2014.302155
Singer E. (2002). The use of incentives to reduce nonresponse in household surveys. In Survey Nonresponse, eds. Groves RM et al. 163-78. New York, NY: Wiley.
Singer E et al. (1998). Does the payment of incentives create expectation effects? Public Opin Q. 62, 152-64.
Singer E & Ye C. (2013). The use and effects of incentives in surveys. The Annals of the American Academy of Political and Social Science. 645(1), 112-141.
Slobig Z. Latino College Health Initiative: A Study of Tobacco-Related Health Disparities In Hispanic/Latino Subpopulations. Truth Initiative; 2014:44. Accessed January 14, 2021. https://truthinitiative.org/sites/default/files/media/files/2019/03/Latino_College_Health_Initative_2014.pdf
Twose, D., & Elmore, M. (2015, April). Online advertising: Not wearing out its welcome. Warc Exclusive, Millward Brown.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Chapman, LeTonya |
| File Modified | 0000-00-00 |
| File Created | 2023-08-28 |