SS Part B 0920-05PI Extended Evaluation of TPCPEC_091015

TIME SENSITIVE

Extended Evaluation of the National Tobacco Prevention and Control Public Education Campaign

Supporting Statement: Part B

Centers for Disease Control and Prevention

National Center for Chronic Disease Prevention and Health Promotion

Office of Smoking and Health

Health Communications Branch

September 10, 2015

Project Manager: Deesha Patel, MPH

Phone: 770-488-8503

Fax: 770-488-5939

Email address: [email protected]

Centers for Disease Control and Prevention

4770 Buford Highway, Mailstop F-79

Atlanta GA 30341

HHS/ASPE HAS APPROVED SUBMISSION OF THIS ICR TO OMB

Contents

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

B.2. Procedures for the Collection of Information

B.3. Methods to Maximize Response Rates and Deal with Non-response

B.4. Tests of Procedures or Methods to be Undertaken

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

List of Attachments

A-1. Public Health Service Act

A-2. Family Smoking Prevention and Tobacco Control Act

A-3. PPHF

B. 60-Day Federal Register Notice

C. Screening & Consent Questionnaire—English and Spanish

D. Waves A-D Smoker Survey—English and Spanish

E. Waves A-D Nonsmoker Survey—English and Spanish

F. KnowledgePanel Recruitment Procedures

G. RTI IRB Approval Notice

H. GfK Privacy and Security Procedures

I-1. ABS Sample Invitation Letter

I-2. KnowledgePanel Email Invitation and Reminders

I-3. ABS Sample Postcard Reminders

J. Source of Respondents for Each Wave of Data Collection

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS

B.1. Respondent Universe and Sampling Methods

Four waves of information collection (A—D) will be conducted over approximately 16 months in a 2-year clearance period to facilitate evaluation of Phase 4 of CDC’s National Tobacco Prevention and Control Public Education Campaign (The Campaign) and to help inform the evaluation of Phase 5 of The Campaign . This study is an extension of a longitudinal web-based survey under currently-approved ICR 0920-0923. .

The participants for these surveys will be recruited from two sources: (1) an online longitudinal cohort of adult smokers and nonsmokers, sampled randomly from postal mailing addresses in the United States (address-based sample, or ABS); and (2) the existing GfK KnowledgePanel, an established long-term online panel of U.S. adults. The ABS-sourced sample consisting of smokers and nonsmokers will serve as the core sample upon which estimates of key outcomes will be made. ABS-sourced participants will make up approximately 65% of the total sample between smokers and nonsmokers (35% will originate from KnowledgePanel). The use of the ABS-sourced sample addresses will increase the coverage of the core sample and will alleviate possible concerns over “panel conditioning”. This sample will be recruited by GfK, utilizing nearly identical recruitment methods that are used in the recruitment of KnowledgePanel (see Attachment F). The GfK KnowledgePanel will be used in combination with the ABS-sourced cohort to support larger sample sizes that will allow for more in-depth subgroup analysis, such as race/ethnicity, income, education, which is a key objective for CDC. All online surveys, regardless of sample source, will be conducted via the GfK KnowledgePanel Web portal for self-administered surveys.

Power analyses were conducted to determine the necessary number of respondents to detect anticipated changes in outcomes as a function of campaign exposure. These analyses were informed by previous studies of earlier phases of The Campaign. In addition, we examined existing evaluation literature and research to determine the expected effect sizes on the outcome of making a quit attempt (Farrelly et al., 2012, McAfee et al., 2013). Based on these analyses, we have powered the study to detect an underlying odds ratio of 1.18 between campaign exposure and the likelihood of a quit attempt. This power analysis is based solely on the ABS-sourced cohort sample size of 4,800 total respondents per wave. Previous media evaluations of earlier phases of The Campaign have quantified similar magnitude of impact on the likelihood of a quit attempt. We have conservatively powered the sample to detect this effect at approximately 86% power among ABS-sourced smokers in the sample. The sample is slightly overpowered to guard against the possibility of smaller effect sizes as The Campaign matures. For nonsmokers, we estimated statistical power based on recent data from RTI’s 2012 Tips evaluation showing The Campaign was associated with increased communica­tions about the dangers of smoking with an estimated odds ratio of 1.16 between Campaign exposure and the likelihood of nonsmokers’ communications (McAfee et al., 2013). We estimate that we will have approximately 80% power to detect an effect of this magnitude based on a maintained ABS nonsmoker cohort size of at least 3,000 respondents per wave. These required sample sizes reflect the total cohort size for ABS smokers and nonsmokers after attrition and replenishment at each wave. Our power analyses also account for an estimated intracluster correlation of 0.008 across markets, variation in market-level sample sizes, and a variance inflation factor of approximately 1.58, estimated from earlier waves of survey data. Exhibit 1 summarizes the detailed power calculations for smokers and nonsmokers.

Exhibit 1. Sample Power and Detectable Effects on Outcomes Among Smoker (n=4,800) and Nonsmoker (n=3,000) Cohorts

Odds Ratio for Relationship Between Campaign Exposure and Likelihood of Quit Attempt	Effect Size (Pre-Post % Point Change in Quit Attempts)	Estimated Power for Smokers (n = 4,800)	Estimated Power for Nonsmokers (n = 3,000)
1.18	3.8%	85.7%	----
1.17	3.7%	83.8%	----
1.16	3.6%	81.8%	79.5%

Planned sample sizes are based on minimum required sample sizes to detect Campaign effects on quit attempts among smokers and on communications about the dangers of smoking among nonsmokers as determined via the power analyses above. We estimate that a minimum total ABS sample size of 4,800 per survey wave is required to detect The Campaign effect on quit attempts. Based on previous data collections involving similar populations, we anticipate an initial retention rate of approximately 59% between Waves A and B among the ABS smokers at Wave A. This retention rate requires an initial ABS sample size of approximately 8,200 smokers at Wave A to yield 4,800 smokers at Wave B. Once panelists are enrolled, wave-by-wave retention is expected to be approximately 65% at Waves C and D. This retention rate will yield approximately 3,120 retained ABS smokers at each of Waves C and D, requiring approximately 1,680 new respondents for replenishment at each Waves C and D to meet the minimum sample size requirement of 4,800 ABS smokers at these waves. Similar processes of retention and replenishment are applied to the ABS nonsmoker sample in order to yield a minimum cohort of size of 3,000 respondents per wave. See Attachment J for a detailed summary of sample sizes, retention rates, and planned sample replenishment by survey wave.

As noted above, the KnowledgePanel sample will be used in combination with the ABS-sourced cohort to support larger sample sizes for stratified analyses of subgroups of interest including by race/ethnicity, income, and education. The planned KP sample sizes for smokers and nonsmokers (Attachment J) are designed to boost the overall sample sizes of these subgroups and yield reliable point estimates and descriptive statistics on all major outcome variables assessed in the evaluation of The Campaign. Exhibit 2 summarizes the expected sample sizes for these subpopulations among the combined ABS and KnowledgePanel samples (smokers and nonsmokers combined) at each survey wave. These estimates are based on demographic distributions available from the 2011 National Health Interview Survey.

Exhibit 2. Estimated Sample Sizes of Subpopulations of Interest by Survey Wave

Demographic Segment	National Incidence (%)	Sample Sizes of Adult Smokers and Nonsmokers
		Wave A (n=18,000)	Wave B (n=12,000)	Wave C (n=12,000)	Wave D (n=12,000)
Race/Ethnicity
White, Non-Hispanic	72.6	13,068	8,712	8,712	8,712
Black, Non-Hispanic	11.5	2,070	1,380	1,380	1,380
Hispanic	10.3	1,854	1,236	1,236	1,236
American Indian/Alaska Native	0.6	108	72	72	72
Education
Less than high school	18.9	3,402	2,268	2,268	2,268
High school or equivalent	36.5	6,570	4,380	4,380	4,380
Some College	32.1	5,778	3,852	3,852	3,852
4-year college degree or more	12.4	2,232	1,488	1,488	1,488
Income
Below poverty	21.5	3,870	2,580	2,580	2,580
At or above poverty	78.5	14,130	9,420	9,420	9,420

Survey Weighting

All data collected for this study will be weighted for analysis. GfK will weight all data to facilitate separate analysis of the ABS-sourced and KP-sourced samples as well as analysis of the combined samples. Weights for the sample will be calculated using a standard post-stratification weighting procedure that adjusts for survey non-response as well as non-coverage. This weighting procedure also applies a standard post-stratification adjustment based on national distributions of age, gender, race/ethnicity, and education among smokers from the most recent 2010–2011 Tobacco Use Supplement of the U.S. Census Bureau’s Current Population Survey (CPS). Benchmark distributions for Internet access used in this weight are obtained from the most recent (October 2009) special CPS supplemental survey measuring Internet access.

It should be noted that while the sample recruitment procedures are designed to maximize the degree of representativeness of this national sample, the limitations associated with online data collection require that all results from this information collection be reported with appropriate caution and interpretation. Specifically, although all participants (ABS-sourced and KnowledgePanel-sourced) must be invited to participate and cannot volunteer on their own, there may be systematic differences between individuals who choose to join internet surveys and the type of individuals who do not wish to participate in these types of studies over an ongoing timeframe. Therefore, evaluation results must be interpreted with appropriate caution regarding our ability to generalize the findings to the national population of smokers and nonsmokers.

B.2. Procedures for the Collection of Information

All surveys, regardless of sample source, will be conducted via the GfK KnowledgePanel Web portal for self-administered surveys. Surveys will be accessible to respondents any time of day for a designed period. Participants can complete each survey only one time. Like all previous survey waves, the additional information collection will rely on Web surveys to be self-administered on computers at home or in a location convenient to the respondent. The surveys will be fielded in English and Spanish and will occur from approximately September 2015 through December 2016. All participants will be re-contacted for follow-up at subsequent survey waves with new participants enrolled to replenish the sample and maintain sample size.

All respondents will be asked to complete an online screening and consent questionnaire (Attachment C) to verify eligibility and assign the respondent to the appropriate survey instrument (smoker or nonsmoker). However, because respondents will be drawn from two sources, procedures supporting initial recruitment and later follow-up are slightly different and are described separately below. ABS-sourced participants will be initially contacted by advance letter. GfK KnowledgePanel panel participants will be initially contacted by email.

ABS-Sourced Participants

Recruitment of the ABS-sourced sample will parallel recruitment methods used for the existing KnowledgePanel. Persons residing at randomly sampled addresses will be invited to join the study via a series of mailings. Specifically, the ABS sample will be sent an advance letter (Attachment I-1) that describes the study, the length of commitment to the cohort, available incentives, and the overall purposes of the study. CDC will be prominently identified as the sole sponsor of the survey effort in all recruitment materials to encourage study cooperation.

Invited households that receive the advance letter will be able to join the study by going to a designated study Website where the study screener can be accessed and completed. After initially accepting the invitation to join the study, respondents will then complete an online consent and screening survey (Attachment C) to initiate their cohort tenure. The consent and screening survey will require a PIN that will be supplied to the respondent in the advance letter. Households that do not respond to the advance letter will be mailed up to 2 postcard reminders about the study (Attachment I-3). Each postcard will contain brief information about the study, will remind invitees of the importance of responding, and will provide the aforementioned PIN and Website for accessing the survey online.

KnowledgePanel Participants

Sampled KP participants will receive email notification that the survey is available for completion. Nonrespondents will receive two e-mail reminders requesting their participation in the survey. See Attachment I-2 for study email notifications for the KnowledgePanel sample. The email notifications contain links to the online consent and survey screening questionnaire that is used to determine study eligibility (Attachment C). Informed consent will be sought from participants for participation in the Web survey. Participants will consent by selecting the appropriate link on the Web screen. A detailed description of KnowledgePanel recruitment methodology is provided with this submission (Attachment F).

B.3. Methods to Maximize Response Rates and Deal with Non-response

One of the primary purposes of the later follow-up surveys will be to track longer-term cigarette abstinence among smokers who initially report quitting as a result of The Campaign. This will be essential to properly estimating the impact of The Campaign on long-term successful quitting. Hence, long-term cohort maintenance will be critical to the success of the evaluation. We have developed a comprehensive recruitment and retention plan.

The following procedures will be used to maximize cooperation and participation in this study:

1. Incentive Plan:

   • Our incentive plan acknowledges that the ABS-sourced participants may need an additional incentive because they may not have access to Internet in their home. The incentives for the KP participants are consistent with their customary bonus points system.

     • Participants recruited to the ABS-sourced longitudinal cohort will be offered $20 for completion of each survey they participate in. An additional $20 per survey (for a total of $40) will be offered to ABS-sourced respondents who do not have internet capability and must seek out public computers or other types of internet access to complete the online surveys. This additional incentive for non-Internet households is meant to encourage their participation and appropriately acknowledge respondents’ time and effort.

2. Prompted Reminder System:

   • Postcard reminders (Attachment I-3) will be sent to all individuals sampled via ABS who do not respond to the initial advance letter. In addition, email reminders (Attachment I-2) will be sent to all sampled KP participants who do not complete their assigned survey within a given period of time after it is assigned. A second round of email reminders will be sent to KP non-responders who do not complete the survey once the initial email reminder is delivered.

3. Technical Assistance:

   • GfK will provide a toll-free telephone number to all sampled individuals and invite them to call with any questions or concerns about any aspect of the study.

   • GfK data collection staff will work with RTI project staff to address concerns that may arise.

   • We have ongoing communication with the contractor to identify and resolve barriers to full participation.

In addition to the above procedures for maximizing study cooperation, we will perform analysis of nonresponse at each survey wave to assess the potential for biases that may arise from nonresponse. The existence of non-response bias will be reported in two ways: 1) Differential non-response among qualified respondents who do not consent or otherwise respond to the survey; and 2) Differential wave-by-wave cohort retention. For the former, we will assess the extent to which qualified respondents who do not consent to the study are systematically different from those who do agree to participate. This will be accomplished using existing variables that are known prior to survey consent and response. This analysis will be broken down into differential non-response rates at the invitation stage (among KnowledgePanel participants with known profile data prior to receiving study invitations) and at the consent stage among screened participants with known profile data. For longitudinal non-response, patterns in wave-by-wave attrition will be assessed by estimating the relationship between the odds of completed follow-up surveys and demographic and other characteristics of all potential respondents at the initial wave. These analyses will help determine whether cohort dropout rates are disproportionate across various types of participants.

B.4. Tests of Procedures or Methods to be Undertaken

Prior to launching the Wave A surveys for smoker and nonsmokers, we will field an eight-case pretest of the survey instrument. This survey will be identical to the instrument that will be used in this evaluation and approved by OMB with the exception of a few additional alternative question formats and questions to assess overall clarity of instrument questions and respondents’ opinions on any aspects of the survey that were not clear. The purpose of the pilot test will be twofold: (1) to assess technical aspects and functionality of the survey instrument, and (2) to identify areas of the survey that were either unclear or difficult to understand. Once this pretest is completed, GfK will create a data file for analysis by RTI International. This data file will contain diagnostic data on average time of survey completion, survey completion patterns (e.g., are there any concentrations of missing data?), and other aspects related to the proper function of the survey. We will also examine data on pilot test measures that will be used to assess the clarity of item wording and ease of understanding. Although this pretest will be conducted, such pretests rarely result in changes to the instruments. Therefore, we do not expect or plan to have any changes made to the instruments.

In addition to the aforementioned eight-case pretest, RTI and CDC will conduct rigorous testing of the online survey instrument prior to its fielding. RTI and CDC researchers will have access to an online test version of the instrument that we will use to verify that instrument skip patterns are functioning properly, delivery of campaign media materials is working properly, and that all survey questions are worded correctly and in specification with instrument approved by OMB.

B.5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The following individuals inside the agency have been consulted on the design and statistical aspects of this information collection as well as plans for data analysis:

Tim McAfee, MD, MPH

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-5709

Email: [email protected]

Rebecca Bunnell, ScD, MEd

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-5592

Email: [email protected]

Shanna Cox, MSPH

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: 770-488-6477

Email: [email protected]

Diane Beistle

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-5066

Email: [email protected]

Ralph S. Caraballo, PhD, MPH

Office on Smoking and Health

Centers for Disease Control and Prevention (CDC)

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-5732

Email: [email protected]

Deesha Patel, MPH

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-8503

Email: [email protected]

Bob Rodes, MS, MBA, MEd

Office on Smoking and Health

Centers for Disease Control and Prevention

4770 Buford Highway NE, Mailstop F79

Atlanta, GA 30341

Phone: (770) 488-5748

Email: [email protected]

The following individuals outside of the agency have been consulted on the questionnaire development, statistical aspects of the design, and plans for data analysis:

Tesfa N. Alexander, PhD

FDA, Center for Tobacco Products

9200 Corporate Boulevard

Rockville, MD 20850

Phone: (301) 796-9335

Email: [email protected]

Amanda Berger, PhD

FDA, Center for Tobacco Products

9200 Corporate Boulevard

Rockville, MD 20850

Phone: (301) 796-9335

Email: [email protected]

Natalie Gibson

FDA, Center for Tobacco Products

9200 Corporate Boulevard

Rockville, MD 20850

Phone: 240-402-4095

Email: [email protected]

Kevin C. Davis, MA

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: (919) 541-5801

Email: [email protected]

Jennifer Duke, PhD

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: (919) 485-2269

Email: [email protected]

The following individuals will conduct data collection and analysis:

Kevin C. Davis, MA

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: (919) 541-5801

Email: [email protected]

Jennifer Duke, PhD

RTI International

3040 Cornwallis Road

Research Triangle Park, NC 27709

Phone: (919) 485-2269

Email: [email protected]

References

Abreu, D.A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.

Chang L. & Krosnick J.A. (2009). National surveys via RDD telephone interviewing versus the Internet: comparing sample representativeness and response quality. Public Opinion Quarterly. 74(4):641-678

Farrelly, M.C., Duke, J.C., Davis, K.C., Nonnemaker, J.M., Kamyab, K., Willett, J.G., Juster, H.R. (2012). Promotion of smoking cessation with emotional and/or graphic antismoking advertising. American Journal of Preventive Medicine, 43(5):475-482.

McAfee, T., Davis, K.C., Alexander, R.L., Jr., Pechacek, T.F., Bunnell, R. (2013). Effect of the first federally funded US antismoking national media campaign. Lancet, 382(9909):2003-2011.

Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231-250.

Southwell B, Barmada C, Hornik R, et al. (2002).Can we measure encoded exposure? Validation

evidence from a national campaign. J Health Commun, 7:445-453.

Yeager D.S., Krosnick J.A., Chang L., et al. (2011). Comparing the accuracy of RDD telephone

surveys and Internet surveys conducted with probability and non-probability samples. Public Opinion Quarterly. 75(4):709-747.