Information Collection #7:
National Tobacco Prevention and Control Public Education Campaign:
Digital Media Copy Testing for
Click-to-Call Ads
Submitted for approval under CDC generic approval #0920-0910
Message Testing for Tobacco Communication Activities
Submission of this GenIC has been approved by
HHS/Assistant Secretary for Planning and Evaluation (ASPE)
Supporting Statement: Part B
__________________________________________
Data Collection Instruments
Other Attachments
Notes on Excluded Attachments
To support adequate review of this GenIC by OMB, CDC requests permission to provide OMB with a secure link to the draft materials.
__________________________________________
B.1 Respondent Universe and Sampling Methods
CDC’s data collection contractor, Toluna, will recruit respondents from the opt-in Toluna Panel, a multimillion-member panel of cooperative online respondents. It is one of the largest databases of individual opt-in respondents for market research in the world. Individuals in the Toluna Panel have agreed to periodically participate in online surveys. Toluna’s panel is highly profiled with both behavioral and demographic characteristics. Panelists have joined the Toluna Panel through hundreds of different sources. Many diverse methods are leveraged to acquire panelists, including co-registration offers on partners’ Websites, targeted emails sent by online partners to their audiences, graphical and text banner placement on partners’ Websites, trade show presentations, targeted postal mail invitations, TV advertisements, member referrals, and telephone recruitment of targeted populations. Toluna has developed multiple proprietary panels across the globe. Since it is known that people who want to join an online panel are often similar to one another, Toluna improves the quality and representative nature of its online sample by incorporating participants from online communities, social networks, and Websites of all types. Participants are recruited to Toluna’s sample via banners, invitations and messaging of all types, and then they go through rigorous quality controls before being included in any sample or on any panel.
In addition, Toluna can potentially access anyone online via a network of relationships with Websites, panels, communities and social media groups. Toluna’s goal is to provide access to people to give their opinions wherever they are in the way that best suits the needs of the research project. For Terms of Use and Conditions for Toluna, please see Attachment 4.
The sample for respondents in this survey will be drawn from the Toluna panel based on the populations of interest – the proposed information collection will involve testing of digital ads among smokers ages 18-54 of low SES who own smartphones. The legal age of consent is 18 years in all states in the United States except for Alabama and Nebraska, where the age of majority is 19. Toluna’s opt-in process for this survey is designed to screen out minors and anyone less than 18 of age, except in Alabama and Nebraska. In Alabama and Nebraska, anyone aged less than 19 will not be able to participate. In all cases, the screeners will also screen out individuals older than 54. The sample plan is essentially a convenience sample but will be based on demographic variables to ensure a reasonable degree of diversity in key demographic characteristics, such as age, gender, and region of residence, race/ethnicity, and income. As this study is considered part of formative research for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of messages from digital ads targeted at 18-54 year old smokers of low SES who own smartphones.
We will test six draft ads.
300X250 Call for Free Help
300X250 Click to Call Free Help
300X250 Black Background
300X50 1-800-Quitnow
300X50 Click to Call
300X50 Click for Info
Each respondent will view only one of the six draft ads. The sample size recommendation is based on a calculation that determines number of respondents needed to assess differences between groups. We will need to screen approximately 2,200 individuals who are smokers, own smartphones, and are of low SES from the Toluna panel to obtain 2,000 complete responses. Each of the 2,000 respondents will view one ad, resulting in approximately 330-335 total views (2,000/6) per ad. To adjust for order effects within each group, we will also balance the presentation order of the ads by randomization of the ads, thus possibly resulting in a small difference of base size per ad.
As this is an ad test we need a certain number of people to view each ad to compare receptivity across ads. In order for us to test differences between ads by different behavioral and demographic groups, it is important that the smallest cell size for a specific attribute (low-SES smokers) to be at minimum of 125 to 150. Given the number of response categories of some items range up to 10, having 333 respondents view each ad will allow, for specific dependent variables, a minimum base size of 140-150, if those response categories are grouped and summarized, a level of aggregation acceptable for this specific digital test, especially given the previous screening criteria of the entire sample.
Only smokers will be interviewed, and these smokers also have to be of low SES and own a smartphone. Toluna has profiled its panel in terms of smartphone ownership, and smoking behavior and as such can target and identify respondents who are pre-identified with this screening criteria, of legal age and younger than 55, to the survey. Other screened for and profiled characteristics of the Toluna panel are those who are low SES (as determined by education classification). It is anticipated that the likelihood of respondents who do not qualify will be in the 5-8% range. A small percentage (2%) is anticipated to decide to opt-out of the survey once started.
During the data collection period, we will review the distribution of the qualified respondents who have participated and select additional panel members, as needed, to receive targeted email invitations to ensure the appropriate balance of respondents.
B.2 Procedures for the Collection of Information
The survey will be hosted on Toluna’s server farm, managing Toluna’s 4.7-million member panel. All interviews will be conducted using a self-administered, online questionnaire via proprietary, Web-assisted interviewing software. The screening questions are presented in Attachment 1a and the main questionnaire is included in Attachment 1b. The selected panelists will receive an initial invitation that indicates they have been invited to participate in a new survey (see Attachment 2). The email invitations will also state the length of the survey and incentive they receive if they qualify and complete the survey. The invitations that are currently utilized have been developed based on continual testing of the content, including but not limited to types of subject line, topic description, survey details, incentive description, format (html vs. text) that elicits the most favorable response rates.
Once the invitation is opened, respondents then will link to the survey URL, with an individual, unique and secure link, and complete the survey. Due to identity protection technology, it will not be possible for anyone to enter the survey without being recruited, or for a respondent to complete the survey more than once. In addition, the same-worded invitation will be sent at regular intervals after the original invitation is sent to those respondents who have not yet responded.
B.3 Methods to Maximize Response Rates and Deal with Nonresponse
The project methodology attempts to maximize response rates at two points in the data collection process. The survey invitation is targeted at pre-identified individuals who share the screening characteristics. The survey invitation itself is developed to elicit a broad response to maximize the number of respondents that “click” on the survey link. The survey will be available over mobile (smartphone and tablet) and desktop Internet devices. The respondent email invites will include information regarding the survey length, the incentive for participation and a secure and protected link to the survey.
Response rates are closely monitored during the field period and, if needed, multiple reminders and new replicates are also sent to potential survey respondents.
B.4 Test of Procedures or Methods to be Undertaken
None.
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data
George Terhanian, PhD
Toluna USA
21 River Road
Wilton, CT 06897
Phone (646) 430-3420
Email: [email protected]
David L. Vannette, PhD Candidate
Stanford University
Stanford, CA 94305
Phone: (616) 502-4828
E-mail: [email protected]
Carol Sue Haney
Toluna USA
21 River Road
Wilton, CT 06897
Phone (203) 846-5838
Email: [email protected]
John Bremer
Toluna USA
21 River Road
Wilton, CT 06897
Phone (203) 846-5838
Email: [email protected]
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | bkf4 |
| File Modified | 0000-00-00 |
| File Created | 2021-01-28 |