SUPPORTING STATEMENT B FOR OMB CLEARANCE
National Youth Anti-Drug Media Campaign
Copy Testing
OMB Control Number
3201-0006
Prepared by:
352 Park Ave. South
New York, New York 10010
On behalf of:
Office of National Drug Control Policy
Executive Office of the President
750 17th St. NW
Washington, DC 20503
March 22, 2013
SUPPORTING STATEMENT B FOR OMB CLEARANCE
NATIONAL YOUTH ANTI-DRUG MEDIA CAMPAIGN
COPY TESTING
COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
1 Respondent universe and sampling methods 3
2 Procedures for the collection of information 5
3 Methods for maximizing response rates 7
4 Describe any tests of procedures or methods to be undertaken 7
5 Contact information for data collection and analysis 7
Copy Testing of Messages for ONDCP
National Youth Anti-Drug Media Campaign
Supporting Statements
Ads are tested for communications effectiveness with members of its stated target audience. The target audience for most ads is stated rather broadly (e.g., youth ages 14 to 16, parents of tweens and teens); however, some ads are targeted to a more specific segment as specified by age, gender, geographic location, and/or racial or ethnic group.
Recruitment quotas will include gender, age, and race/ethnicity as appropriate, in order to adequately reflect the target audience. In addition, samples will reflect variety in geographic density (i.e., urban, suburban, rural) and region of the country. For youth, the majority of convenience sample of test participants will be recruited from existing online research panels. For adults, the convenience samples of test participants will be recruited entirely from existing online research panels.
Online panels are used by online market research companies in order to gain efficiencies in recruiting respondents. Panel participants are pre-screened based on specific recruitment criteria, including but not limited to demographic, psychographic, behavioral and/or geographic parameters. Panels contain a large pool of potential respondents from which a specific survey can be disseminated. The pool is typically developed via website or banner solicitation. Respondents agree to be part of the panel, and also agree to participate in each specific survey.
Ads will also be tested for unintended negative effects with other audience of concern, as appropriate. Specifically, ads targeting older children (e.g., high school) may have unintended negative consequences on younger children (e.g., middle school age youth). Therefore, as appropriate, youth-targeted ads will be tested for unintended negative effects with both the intended target audience as well as younger target audiences.
Ads directed to adults will typically be tested only among adults.
Youth Advertising Copy Tests
For each youth ad that we copy test, we will recruit 150 youth (male and female) who are 14-16 years old: 50 White, 50 Hispanic and 50 African American. We will also recruit 150 youth (male and female) who are 11-13 years old: 50 White, 50 Hispanic and 50 African American. The majority of subjects will be recruited from existing online panels across the U.S. To ensure we reach quotas among Hispanic and African American youth, supplemental youth respondents we will be recruited using the place-based recruitment methods currently used. No more than 30% of African American and Hispanic respondents will be recruited via the place-based intercept method.
Based on past experience, we expect a response rate among age-appropriate youth of over 80%. The table below shows the universe of potential youth respondents, and our sample.
NUMBER OF YOUTH LIVING IN USA, 11-16 YRS. OLD
(Approximate, based upon Simmons Data, 2007)
AFRICAN TOTAL
WHITE HISPANIC AMERICAN UNIVERSE
11-13 YRS. 9.4 million 2.1 million 1.9 million 13.4 million
14-16 YRS. 9.5 million 2.2 million 2.1 million 13.8 million
11-13 YRS. 50/cell 50/cell 50/cell 150/cell
14-16 YRS. 50/cell 50/cell 50/cell 150/cell
TOTAL 100/cell 100/cell 100/cell 300/cell
Adult Advertising Copy tests
For each adult ad that we copy test in the test/control method, we will typically recruit 150 adults: 50 Caucasian, 50 Hispanic, and 50 African American; for general communications checks, we will typically recruit 300 adults: 100 Caucasian, 100 Hispanic, and 100 African American. Subjects will be recruited exclusively from existing online panels. We expect a qualification rate among parents to be at or above 80%. The table below shows an example of the universe of potential adult respondents, specifically parents of 11-16 year olds, and our sample.
NUMBER OF HOUSEHOLDS WITH YOUTH 11-16 YRS. OLD, LIVING IN USA
(Approximate, based upon Simmons Data, 2007)
AFRICAN TOTAL
WHITE HISPANIC AMERICAN UNIVERSE
Universe 10.8 million 2.8 million 2.2 million 15.8 million
Test (or) Control 50/cell 50/cell 50/cell 150/cell
General 100/cell 100/cell 100/cell 300/cell
A research design incorporating a randomly assigned control group will be used. Graphical representations of this design for TV ads are shown below in Figure 1.
For
youth and adult, qualified respondents answer questions within a
self-administered survey using a personal computer. The survey is
taken independently and at the time and place of the respondents
choosing. Although the online survey will be without an official
interviewer present, respondents will have access to help via email
should questions arise; for youth in the place-based supplement, an
attendant will be available to answer questions as needed. The
interview requires approximately 15 minutes to complete. Ads will be
tested in an uncluttered monadic test situation
(i.e., test and
control ads will not be embedded in program content).
Respondents in the test group are exposed to advertising (two times) in addition to answering questions. Those in the control group only answer questions. After exposure to advertising, respondents in the test group are questioned about the main idea (open-ended), likeability, clarity, and believability (open- and closed-ended) of the ad, along with relevant ad diagnostic questions, including but not limited to, credibility, relevance, trustworthiness, strength, persuasiveness, realism, and willingness to pay attention to the ad.
Respondents in both the test and control groups are asked about key beliefs (based on advertising communications objectives/main idea) and intentions to use drugs in the future. Statistical comparisons are made on the basis of these responses in order to determine if the advertising viewed by the test group successfully moved beliefs and/or intentions in the desired direction relative to the control group that was not exposed to messaging. This is currently the major test of ad effectiveness.
Test Condition
Pre-test:
---------- First presentation of test ad ----------
---------- Second presentation of test ad ----------
Post-test:
|
Control Condition
Pre-test:
Post-test:
|
Areas used in group comparisons are boxed. |
|
In each advertising copy test, our goal is to determine if exposure to the anti-drug ad significantly alters subjects’ drug-related beliefs or behavioral intentions relative to a control condition, which involves no ad exposure. Further, we want to know if the causal effects of the anti-drug ad are contingent on ethnicity (e.g. White vs. Hispanic vs. African American) or for youth on age (e.g. 11-13 yrs. vs. 14-16 yrs.). We have adopted the standard criteria for determining statistical significance in scientific advertising copy testing: an alpha of .05 and a power of .95. Based on our past advertising copy tests that were similar in nature, we estimate the within-group variance to be .6. According to standard power analyses, our target sample size of 100 per ethnic group (see above) will be adequate to meet our goal of detecting a 10% change (test vs. control) in beliefs or intentions, given the desired alpha and power, and the estimated variance. Variables that could have an impact on results but should not be considered in the analysis are used as a covariate. Among these are sensation seeking for youth and age of children for parents. Controlling for these variables in the analysis prevents them from confounding the results by causing groups to be unbalanced in their responses.
Multiple ads that are executed against the same target audience and strategy will be tested in a single testing cycle. An equal number of research participants will be randomly assigned to each test condition or to the control condition. Using this approach, a single control condition can serve as the comparison for each of the test ads in a given cycle, helping to control costs.
As is customary, a reimbursement will be given to respondents. Online panels use multiple resources to aggregate online participants and each successful respondent will be given a reimbursement in the monetary equivalent of $5.00 to be redeemable through each panel’s proprietary reward program. This basic reimbursement increases responses from individuals. Given the anti-drug focus of the research, there is particular need to ensure that participation is drawn from a wide array of individuals. The response rates that we have attained in the past and expect to attain in the future (81% among youth, 76% among adults) are standard for advertising copy testing, and are considered to be more than adequate for the intended purposes. These response rates reflect percentages of individuals who have met the screening criteria and then go on to complete the study. This is not reflective of the number of individuals who are initially approached by the researcher.
Describe Any Tests of Procedures or Methods to be Undertaken
No more than two rounds of creative concepts each for youth and adults will be tested simultaneously.
Contact Information for Data Collection and Analysis
Data collection will be conducted by a vendor, managed by [email protected], that specializes in online copy testing research.
| File Type | application/msword |
| File Title | Copy-Testing of Messages for ONDCP |
| Author | TaWhite |
| Last Modified By | EOP |
| File Modified | 2013-04-10 |
| File Created | 2013-03-22 |