Supporting Statement B

Information Collection #4:

National Tobacco Prevention and Control Public Education Campaign:

Rough Cut Testing of Print and Radio Ads

Submitted for approval under CDC generic approval #0920-0910

Message Testing for Tobacco Communication Activities

Draft #4: Submitted November 28, 2012

­­­­Section B: Statistical Methods

__________________________________________

Data Collection Instruments

• Attachment 1. Screener for General Population and American Indians/Alaska Natives

• Attachment 2(e). Screener for Spanish Speaking Hispanics (in English)

• Attachment 2(s). Screener for Spanish Speaking Hispanics (in Spanish)

• Attachment 3 (e). Main Questionnaire (in English)

• Attachment 3 (s). Main Questionnaire (in Spanish)

Other Attachments

• Attachment 4(e). Emails to Participants (Initial Email Invitation, Instructions Email, Instructions Reminder Email)

• Attachment 4(s). Emails to Participants (Initial Email Invitation, Instructions Email, Instructions Reminder Email) (in Spanish)

• Attachment 5(e). Privacy Policies (Harris, Survey Sampling International)

• Attachment 5(s). Privacy Policies (Harris, Survey Sampling International)

• Attachment 6(e). Terms (Terms & Conditions; Terms of Use)

• Attachment 6(s). Terms (Terms & Conditions; Terms of Use)

__________________________________________

B.1 Respondent Universe and Sampling Methods

Harris Interactive will recruit respondents from the Harris Poll Online panel, a multimillion-member panel of cooperative online respondents. It is one of the largest databases of individual opt-in respondents for market research in the world. Individuals in the Harris Poll Online panel have agreed to periodically participate in online surveys. Panelists have joined the Harris Poll Online through over 100 different sources. Many diverse methods are leveraged to acquire panelists, including co-registration offers on partners’ websites, targeted emails sent by online partners to their audiences, graphical and text banner placement on partners’ websites, trade show presentations, targeted postal mail invitations, TV advertisements, member referrals, and telephone recruitment of targeted populations. A panel maintained by Survey Sampling International (SSI), LCC will be used to recruit Spanish-speaking Hispanics. SSI has developed multiple proprietary panels across the globe. Since SSI’s research shows that only certain types of people want to join an online panel, SSI improves the quality and representative nature of its online sample by incorporating participants from online communities, social networks, and websites of all types. Participants are recruited to SSI’s sample via banners, invitations and messaging of all types, and then they go through rigorous quality controls before being included in any sample or on any panel.

SSI can potentially access anyone online via a network of relationships with websites, panels, communities and social media groups. SSI’s goal is to provide access to people to give their opinions wherever they are in the way that best suits the needs of the research project. For Terms of Use and Conditions for both Harris Interaction and SSI, please see Attachment 6.

The sample for English speaking respondents in this survey will be drawn from the Harris panel based on the populations of interest – nationwide, 18-54 year olds, smokers and non-smokers. The sample for Spanish Speaking Hispanics will be drawn from the SSI panel. The legal age of consent is 18 years in all states in the United States except for Alabama and Nebraska, where the age of majority is 19. The screener (see Attachments 1-2) is designed to screen out minors and anyone less than 18 of age or over the age of 54, except in Alabama and Nebraska. In Alabama and Nebraska, the screener will screen out anyone aged less than 19 and over the age of 54. The stratified sample plan is essentially a convenience sample but will be based on demographic variables to ensure a reasonable degree of diversity in key demographic characteristics, such as age, gender, region of residence, education, and income. As this study is considered part of formative research message testing for campaign development and planning, these methods are not intended to generate nationally representative samples or precise estimates of population parameters. The sample drawn here is designed primarily to provide information on the perceived effectiveness of messages from television, radio and print ads for 18-54 year olds, including examining message effectiveness for smokers and non-smokers, specific racial/ethnic groups, individuals with certain disease conditions, and groups of differing sexual orientation, as these groups may differ in their assessment of smoking-related messages.

The sample size recommendation is based on a statistical calculation that determines the differences between sample sizes that would be needed to gauge significant differences between groups. The overall English-language sample per ad would have a precision level of +/-4.9% based on a 95% confidence level, the 25 to 54 year old sample would be +/5.6% and the 18 to 24 year old sample would be +/-9.8%.   The total sample size for English-language survey is 6,600. We are estimating a response rate of 60% for the General Population 18-54 year olds, American Indian/Alaska Native and people with diabetes and a response rate of 20% for the smoker oversample. Based on this information, we will need to screen approximately 13,500 individuals.

The overall Spanish-language sample per ad would have a precision level of +/-9.8% based on a 95% confidence level. Assuming a response rate of 50%, we will need to screen approximately 1,200 individuals. The lower incidence rate for the Spanish-speaking Hispanic sample is based on the additional requirements related to level of comfort reading/writing Spanish as well as the amount of time consumer Spanish media. The total sample size for Spanish-language survey is 600.

Smokers will be oversampled to allow for a more robust analysis of this group. In addition to the expected 20-21% natural fall-out from the general population sample, an additional 100 smokers per ad will be included (the oversample is limited to 100 per ad in order to maintain the ability to merge the samples together without applying heavy weight factors). For ads in which Diabetes is the core message focus, an oversample of 100 people with Diabetes will be included. A small oversample of 40 per ad for the American Indian/Alaskan Native population will be included, as will an oversample of 100 people with Diabetes for only ads in which Diabetes is the core focus. In both these cases, the oversample would require extreme weighting if combined with the GenPop sample, thus we will treat these segments as a separate population.

During the data collection period, we will review the distribution of the qualified respondents who have participated and select additional panel members, as needed, to receive the email invitation to ensure the appropriate balance of respondents.

B.2 Procedures for the Collection of Information

The survey will be hosted on Harris Interactive’s server, using Harris Poll Online^SM. All interviews will be conducted using a self-administered, online questionnaire (see Attachment 3) via proprietary, web-assisted interviewing software. The selected panelists will receive an initial invitation that indicates they have been invited to participate in a new survey. The email invitations will also state the length of the survey and compensation they receive if they qualify and complete the survey (see Attachment 4). The invitations that are currently utilized have been developed based on continual testing of the content, including but not limited to types of subject line, topic description, survey details, incentive description, format (html vs. text) that elicits the most favorable response rates. As quantitative research such as this relies on the ability to appeal to a general population audience, our initial invitation is broad to reduce self-selection bias that may occur if the subject or sponsor is revealed at this stage.

Once the invitation is opened, respondents then will link to the survey URL, with an individual, unique password protected link, and complete the survey. Due to password protection, it will not be possible for anyone to enter the survey that has not been recruited, or for a respondent to complete the survey more than once. In addition, a reminder invitation will be sent roughly two days after the original invitation (see Attachment 4).

B.3 Methods to Maximize Response Rates and Deal with Nonresponse

The project methodology attempts to maximize response rates at two points in the data collection process. The survey invitation is developed to elicit a broad response to maximize the number of respondents that “click” on the survey link. In addition, to encourage participation and to remind respondents of closing dates for completing the survey, the contractors will send follow-up/reminder emails (see Attachment 4), beginning two days after distribution of the initial email. The follow-up/reminder email includes information regarding the survey length, the incentive for participation and a password-protected link to the survey.

Response rates are closely monitored during the field period and, if needed, second reminders are also sent to potential survey respondents.

B.4 Test of Procedures or Methods to be Undertaken

None.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

	Statistical/ methodological consultants	Data collection	Data analysis
Jennifer Cantrell , DrPh, MPA Assistant Director, Research and Evaluation LEGACY 202-454-5798 [email protected]	X		X
Donna Vallone, PhD Senior Vice President for Research and Evaluation, LEGACY 202-454-5555, [email protected]	X
Michelle Murphy, Vice President Harris Interactive 585-214-7515 [email protected]	X	X	X
Joanna Allenza, Research Manager Harris Interactive 585-214-7225 [email protected]	X	X	X

4