Evaluation of the National Tobacco Prevention and Control Public Education Campaign
Supporting Statement: Part A
Centers for Disease Control and Prevention
National Center for Chronic Disease Prevention and Health Promotion
Office of Smoking and Health
Health Communications Branch
February 16, 2012
Contract Officer Representative: Robert Alexander, Ph.D.
Health Communications Specialist
Phone: (770) 488-1212
Fax: (770) 488-5939
Email address: [email protected]
Centers for Disease Control and Prevention
4770 Buford Highway, Mailstop K-50
Atlanta GA 30341
Table of Contents
Section
A. Justification
A.1. Circumstances Making the Collection of Information Necessary
A.2. Purposes and Use of Information Collection
A.3. Use of Improved Information Technology and Burden Reduction
A.4. Efforts to Identify Duplication and Use of Similar Information
A.5. Impact on Small Businesses or Other Small Entities
A.6. Consequences of Collecting the Information Less Frequently
A.7. Special Circumstances
A.8. Comments in Response to the FR Notice and Consultation
A.8.a. Federal Register Notice
A.8.b Outside Consultation
A.9. Explanation of Any Payment or Gift to Respondents
A.10. Assurance of Confidentiality Provided to Respondents
A.11. Justification for Sensitive Questions
A.12. Estimates of Annualized Burden Hours and Cost
A.13. Estimates of Annualized Respondent Capital and
Maintenance Costs
A.14. Estimates of Annualized Cost to the Federal Government
A.15. Explanation for Program Changes or Adjustments
A.16. Plans for Tabulation and Publication and Project Time Schedule
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
A.18. Exceptions to the Certification Statement
List of Exhibits
Exhibit 1. Selected OMB-Approved Studies Using the Knowledge Networks Online Panel
Exhibit 2. Estimated One-Year Annualized Burden
Exhibit 3. Estimated One-Year Annualized Cost to Respondents
Exhibit 4. Project Schedule
List of Attachments
A-1. Public Health Service Act
A-2. Family Smoking Prevention and Tobacco Control Act
A-3. PPHF
B-1. Federal Register Notice
B-2. Summary of Public Comments and CDC Response
C-1. Smoker Baseline Questionnaire
C-2. Smoker Follow-Up Questionnaire
D-1. Non-smoker Baseline Questionnaire
D-2. Non-smoker Follow-Up Questionnaire
E-1. Screening and Consent Process
E-2. Email Notifications and Reminders
F-1. Knowledge Networks Panel Recruitment Procedures
F-2. Knowledge Networks Privacy and Confidentiality Procedures
G-1. RTI IRB Approval Notice
H-1 and H-2. Reference Articles on Knowledge Networks
I-1. Primary and Secondary Outcome and Survey Items
Abstract
A. Justification
The primary mission of the Health Communications Branch (HCB) of the Office on Smoking and Health (OSH) at the Centers for Disease Control and Prevention (CDC) is to serve as a public health resource for tobacco and health information. Through the HCB, OSH develops and distributes information about tobacco and health to the public, professionals, various branches of government, and other interested groups nationwide using a wide array of formats and media channels. OSH also maintains a reference library of tobacco-related communication materials, called the Media Campaign Resource Center (MCRC), and provides technical assistance to organizations so that MCRC materials can be customized for specific media applications. CDC is authorized to conduct information collection supporting these activities under the Public Health Service Act (41USC 241) Section 301 (Attachment A-1). OSH also collaborates closely with the Center for Tobacco Research (CTR) in the Food and Drug Administration (FDA). Since 2009, the FDA has gained broad authority to regulate tobacco product advertising through the Family Smoking Prevention and Tobacco Control Act (Attachment A-2).
The recent enactment of the Affordable Care Act (ACA) established the Prevention and Public Health Fund (PPHF) which contains essential disease prevention initiatives to help reduce the health and financial burden of tobacco use (Attachment A-3). One of these major initiatives includes the implementation of a national, evidence-based media campaign to increase awareness of the health consequences of tobacco use and exposure to secondhand smoke. This campaign is currently under development and will include evidence-based paid media advertising that highlights the negative health consequences of smoking. The Campaign’s primary target audience is adult smokers and will include paid advertisements that are aimed at providing support to smokers in the process of quitting, with information and other resources to increase smokers’ chances of success in their attempts to quit smoking. A primary objective of The Campaign is to increase smokers’ awareness of these messages as well as their knowledge about the harmful effects of smoking. A secondary audience for The Campaign is adult non-smokers. A key objective for the non-smoker audience is to encourage non-smokers to communicate with other smokers they may know (including family and friends) and encourage them to quit smoking. Additional campaign ads will focus on raising awareness of the harmful effects of secondhand cigarette smoke, particularly for children. In addition to television advertisements that will air nationally beginning in March 2012, The Campaign will include complementary ads in radio, Internet, print, outdoor, and other media formats. The goal of the proposed information collection is to evaluate the reach of The Campaign among intended audiences and to examine the effectiveness of these efforts in impacting specific cognitive and behavioral outcomes that are targeted by The Campaign.
Privacy Impact Assessment
Overview of the Information Collection
This study is designed to measure national awareness of and exposure to the National Tobacco Prevention and Control Public Education Campaign among smokers and non-smokers in the U.S. This study will rely on Web surveys to be self-administered at home on personal computers. This study will be conducted using the Knowledge Networks (KN) panel, a large online panel of the U.S. population. Specifically we will conduct a two-wave longitudinal survey of smokers and non-smokers from the KN panel to facilitate repeated measures on outcomes of interest. The first survey will be fielded during February and early March 2012 before the launch of The Campaign, serving as a baseline assessment of variables of interest. These include knowledge, attitudes, and beliefs related to smoking as well as intentions to quit and prior quitting behavior. Participants who complete the baseline survey will be surveyed again in a follow-up survey approximately three months post-baseline after The Campaign has aired. This timeframe for follow-up data collection correlates to the 3-month duration of The Campaign itself. We want to ensure that Campaign awareness is measured immediately after all media delivery from The Campaign has taken place, minimizing the potential for decay in awareness of The Campaign. This will facilitate analysis of relationships between individuals’ exposure to The Campaign and pre-post changes in outcomes of interest. This longitudinal design allows us to calculate baseline-to-follow-up changes in campaign-targeted outcomes for each study participant. We hypothesize that if The Campaign is effective, the baseline-to-follow-up changes in outcomes should be larger among individuals exposed to The Campaign more frequently (i.e., dose-response effects).
The primary outcome variable of interest for this evaluation study is self-reported campaign awareness. The KN panel will be the primary source of sample and will be used to generate a singular estimate of this outcome across a large, geographically and socio-demographically diverse sample. However, the KN panel alone may not be sufficient to generate hypotheses regarding potential differences in campaign awareness at more granular levels, particularly within smaller geographic areas. In order to generate such hypotheses, additional sample is needed, particularly for areas with relatively small population densities. CDC has an interest in knowing more about campaign awareness in these regions. To address this gap, we plan to utilize additional sample from outside the KN panel. An additional sample of smokers will be recruited from the Survey Sampling International (SSI) online panel, which is a separate online panel of U.S. residents who participate in online surveys on an ongoing basis. SSI is one of the leading providers of online sampling in the U.S. and around the world. These data will be used to boost the overall sample sizes of specific geographic regions of interest that would otherwise not be sufficiently represented in the KN panel alone.
The evaluation design used allows CDC to estimate the potential for this type of three-month national campaign to reach a large portion of the population, to gauge change in knowledge and immediate behaviors of smokers and nonsmokers, and to generate hypotheses about potential differences in responsiveness by geographical region. Study design limitations, however, decrease our capacity to draw nationally representative conclusions about either smoking-related knowledge and behavior or the impact of the campaign on long-term quit rates in sub-populations. However, the design is the best available solution to CDC’s evaluation objectives, within the time, cost, and feasibility constraints noted above.
Items of Information to be Collected
Each survey will include measures on knowledge, attitudes, and beliefs related to smoking and secondhand smoke, as well as behaviors related to smoking cessation (among the smokers in the sample) and peer-to-peer communication about smoking. The follow-up survey will include measures of audience awareness of and exposure to The Campaign advertisements as well as follow-up measures on the aforementioned outcome variables of interest. The baseline and post-campaign follow-up questionnaires for smokers are located in Attachments C-1 and C2. It should be noted that the baseline questionnaire for smokers will include a few additional questions that will be answered by only participants from the SSI sample, for weighting purposes. These items will include questions on language fluency, Internet access, and early adoption of technology. Because these variables already exist as part of KN’s panel profile data, they do not need to be asked among KN participants. The baseline and post-campaign follow-up questionnaires for non-smokers are located in Attachments D-1 and D-2. The screening instrument used for recruiting respondents is located in Attachment E-1.
Identification of Website(s) and Website Content Directed at Children Under 13 Years of Age
All participants will be 18 years of age or older. None of the intervention websites have content directed to children younger than age 13. In addition, the Web-based version of the follow-up survey will only be accessible to participants of the study.
A.2. Purposes and Use of Information Collection
The information obtained from the proposed data collection activities will be used to inform CDC, policy makers, adult smokers and non-smokers in the U.S., prevention practitioners, and researchers about the extent of adults’ exposure to The Campaign’s messages nationally and the extent to which exposure to these messages is associated with changes in outcomes targeted by The Campaign. Primary and secondary outcomes that will be estimated for this evaluation are discussed in detail in Section A.16 and are summarized in Attachment I-1. While not exhaustive, the list below illustrates a range of purposes and uses for the proposed information collection:
Provide critical data on the reach of The Campaign among adults in the U.S., particularly with estimates of the proportion of the population that was exposed to The Campaign.
Understand the influence of The Campaign on attitudes, knowledge, beliefs, and behaviors around tobacco use and smoking cessation behaviors.
Inform CDC, policy makers, and other stakeholders on the impact of The Campaign overall.
Inform the public through scientific reports, publications, and conference presentations about the impact of The Campaign and cost effectiveness of resources invested in The Campaign’s development and implementation.
Inform the health communication efforts related to the release of Surgeon General reports. For example, this information will generate a better understanding of The Campaign’s impact on key precursors to smoking cessation such as self-efficacy to quit.
Inform the health communication efforts undertaken by the Food and Drug Administration including upcoming campaigns that while intended for different audiences can benefit from knowledge of the approaches utilized for this study.
Inform future programs that may be designed for similar purposes.
Report to the U.S. Congress, via Congressional testimony or other communication formats, about the impact of The Campaign and the cost effectiveness of resources invested in The Campaign, under the authority granted by the Public Health Services Act.
All communications about the evaluation results via these uses of the information, including any and all Congressional testimony, will carefully enumerate and describe any underlying limitations of the study design and ensure that evaluation results are interpreted with appropriate care and caution. Specifically, we will include the following statement: “ Study design limitations decrease our capacity to draw nationally representative conclusions about either smoking-related knowledge and behavior or the impact of the campaign on long-term quit rates in sub-populations. However the design is the best available solution to CDC’s evaluation objectives, within the time, cost, and feasibility constraints noted above.”
A.3. Use of Improved Information Technology and Burden Reduction
This study will rely on Web surveys to be self-administered at home on personal computers. The primary Web panel we are using for this study is Knowledge Networks’ KnowledgePanel®. Knowledge Networks utilizes address-based sampling (ABS) for its panel recruitment. When KnowledgePanel® began over 10 years ago, panelists were recruited via RDD telephone surveys. At the time, RDD samples allowed access to over 90% of U.S. households. This is no longer the case due to marked declines in landline households, dramatic increases in cell-only households, the use of caller ID devices and call screening, answering machines, and do-not-call lists. Hence, a change was made in 2009 to begin recruiting entirely with the U.S. Postal Service’s Delivery Sequence File, which provides coverage to 97% of U.S. households. Under this recruitment procedure, randomly sampled addresses are invited to join KnowledgePanel® through a series of mailings and, in some cases, telephone follow-up calls to non-responders when a telephone number can be matched to the sampled address. Operationally, households invited to participate in the KnowledgePanel® have the option to join the panel one of several ways: (1) completing and returning a paper form in a postage-paid envelop; (2) calling a toll-free hotline maintained by Knowledge Networks; or (3) going to a dedicated Website and completing an online recruitment form. Once these recruitment procedures are completed, invited participants become empaneled and are available to begin participating in specific online surveys. All KN panelists complete their surveys online. See Attachment F-1 for additional details on Knowledge Networks’ panel recruitment methodology.
Utilization of this online panel provides a number methodological advantages including increased accuracy in measurement of key variables of interest, attractive sample characteristics, and reduced burden on study participants. This approach also yields significant cost efficiencies compared to other modes of data collection such as telephone surveys. These advantages include but are not limited to:
Increased privacy (compared to telephone interviewing) reduces vulnerability to socially desirable survey responses, particularly on sensitive subjects such as tobacco use. Surveys are self-administered in a private setting and respondents do not speak to human interviewers as they would with telephone surveys.
Coverage of non-Internet households - Households are provided with access to the Internet and hardware if needed (free Netbook laptop and free internet service). Thus unlike Internet convenience panels, also known as “opt-in” panels, that include only individuals with Internet access who volunteer themselves for research, KnowledgePanel recruitment covers households with and without Internet access.
ABS provides coverage to cell-phone only households.
Flexible and timely data collection – Because KN does not involve human interviewers and all ensuing requirements for interviewer training and quality control, it is easier and cheaper to launch surveys very quickly.
Significant cost savings over traditional telephone surveys (due to lack of human interviewers and interviewer training).
Allows for inclusion of any campaign media material including video streaming of campaign ads, streaming of radio ads, and presentation of print materials all within the survey. This significantly enhances the ability to accurately measure awareness of and exposure to campaign ads. By comparison, telephone surveys do not allow for direct exposure to campaign messages and stimuli, preventing more accurate measurement individual awareness and recall of campaign ads. It has been demonstrated that the use of visual cues to prompt ad recognition (which is only possible with Web surveys) is a superior method for measuring encoded ad exposure compared to telephonic surveys that must rely on verbal cues from human interviewers to prompt ad recognition (Southwell et al., 2002).
KnowledgePanel® utilizes an unbiased general topic recruitment protocol that is free of self-selection biases related to pre-existing interests in specific research topics.
Finally, KnowledgePanel® has been used for a number of similar evaluation studies, including CDC media evaluation studies led by RTI. Exhibit 1 lists selected OMB-approved studies that have utilized Knowledge Networks’ KnowledgePanel®.
Exhibit 1. Selected OMB-Approved Studies Using the Knowledge Networks Online Panel
Sponsoring Agency |
OMB Approval Number |
Approval Date |
Study Name |
Data Collection Contractor |
Contact Person |
HHS-OPA |
0990-0345 |
9/9/2009 |
Evaluation of the Parents Speak Up National Campaign: National Media Tracking Surveys |
RTI International |
Kevin C. Davis |
HHS-OPA |
0990-0325 |
8/15/2008 |
Evaluation of the Parents Speak Up National Campaign: Children’s Study |
RTI International |
Kevin C. Davis |
HHS-CDC |
0920-0752 |
8/24/2007 |
Examining the Efficacy of the HIV Testing Social Marketing Campaign for African American Women |
RTI International |
Kevin C. Davis |
HHS-OPA |
0990-0311 |
6/7/2007 |
Evaluation of the National Abstinence Media Campaign |
RTI International |
Kevin C. Davis |
HHS-CDC |
0920-0565 |
8/19/2002 |
Reactions to Canadian Style Cigarette Warning Labels |
RTI International |
Carol Prindle and Paul Mowery |
Environmental Protection Agency |
2090-0024 |
1/22/2004 |
Estimating the Value of Improvements to Coastal Waters—A Pilot Study of a Coastal Valuation Survey |
RTI International |
George L. Van Houtven |
Environmental Protection Agency |
2060-0502 |
2/19/2003 |
Eliciting Risk Tradeoffs for Valuing Fatal Cancer Risks |
RTI International |
George L. Van Houtven |
Environmental Protection Agency |
2010-0031 |
10/2002 |
Water Quality in America Pretest Round 1 |
Harvard University, Law School |
Kip Viscusi |
Environmental Protection Agency |
2010-0031 |
2/2003 |
Water Quality in America Pretest Round 2 |
Harvard University, Law School |
Kip Viscusi |
Environmental Protection Agency |
2010-0031 |
4/2003 |
Water Quality in America Pretest Round 3 |
Harvard University, Law School |
Kip Viscusi |
Environmental Protection Agency |
2010-0031 |
4/2004 |
Water Quality in America Main Interview |
Harvard University, Law School |
Kip Viscusi |
United States Department of Agriculture |
0536-0062 |
12/16/2003 |
Estimating Consumer Benefits of Improving Food Safety |
Harvard University, Center for Risk Analysis, Department of Health Policy and Management |
James K. Hammitt |
United States Department of Agriculture |
0536-0062 |
3/11/2005 |
Estimating Consumer Benefits of Improving Food Safety |
University of Wyoming, Department of Economics and Finance |
Jason F. Shogren |
National Oceanic and Atmospheric Agency |
0648-0531 |
11/16/2005 |
Coral Reef Economic Valuation Pretest |
Stratus Consulting |
David Chapman |
A.4. Efforts to Identify Duplication and Use of Similar Information
The CDC’s National Tobacco Prevention and Control Public Education Campaign is a new media campaign. To date, there has been no in-depth evaluation of this campaign in a real-world setting and there are no existing data sources that contain measures on awareness of and exposure to The Campaign. This proposed information collection therefore does not duplicate previous efforts. In designing the proposed data collection activities, we have taken several steps to ensure that this effort does not duplicate ongoing efforts and that no existing data sets would address the proposed study questions. We have carefully reviewed existing data sets to determine whether any of them are sufficiently similar or could be modified to address CDC’s need for information on the effectiveness of The Campaign with respect to promoting attempts to quit smoking among smokers. We investigated the possibility of using existing data to examine our research questions, such as data collected as part of ongoing national surveillance systems; evaluations of current or past state-level campaigns describing the harms of tobacco; the National Adult Tobacco Survey; and the Behavioral Risk Factor Surveillance System. Due to the early 2012 timing of The Campaign, none of these existing data sources will be able to facilitate pre- and post-test data on The Campaign, none will include the necessary in-depth survey questions on awareness of individual ads and other campaign materials, and none contain all of the necessary outcome variables specific to The Campaign’s messages. It should also be noted that while the FDA conducts health communications work, this information collection is not duplicative since the CDC is the specific Operating Division that has been tasked with implementing and evaluating the aforementioned campaign.
A.5. Impact on Small Businesses or Other Small Entities
Respondents in this study will be members of the general public, specific subpopulations or specific professions, not business entities. No impact on small businesses or other small entities is anticipated.
A.6. Consequences
of Collecting the Information Less Frequently
The evaluation efforts proposed in this study are required by the authorizing legislation as follows (Attachment A-3): “The Secretary shall ensure that the campaign implemented under paragraph (1) is subject to an independent evaluation every 2 years and shall report every 2 years to Congress on the effectiveness of such campaigns towards meeting science-based metrics.” While there are no legal obstacles to reduce burden, any lack of information needed to evaluate the National Tobacco Prevention and Control Public Education Campaign may impede the Federal government’s efforts to improve public health. Without the information collection requested for this evaluation study, it would be difficult to determine the value or impact of The Campaign on the lives of the people it is intended to serve. Failure to collect these data could reduce effective use of CDC’s program resources to benefit smokers and non-smokers in the U.S. Careful consideration has been given to how frequently The Campaign’s intended audience should be surveyed for evaluation purposes. We believe that the proposed two-wave longitudinal survey will provide sufficient data to effectively evaluate The Campaign.
A.7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
There are no special circumstances that require data collection to be conducted in a manner inconsistent with 5 CRF 1320.5 (d)(2). There are no special circumstances. The message testing activities fully comply with the guidelines in 5 CFR 1320.5.
A.8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency
A.8.a. Federal Register Notice
A 60-day Federal Register Notice was published in the Federal Register on Tuesday July 12, 2011, Volume 76, No. 133, pages 40914-40915 (Attachment B-1). Two public comments were received. One public comment was an expression of opinion that received a courtesy reply. The other comment was a request for additional information, which CDC provided (Attachment B-2, Summary of Public Comments and CDC Response).
A.8.b. Consultation
The following individuals inside the agency have been consulted on the design of The Campaign evaluation plan, audience questionnaire development, or intra-agency coordination of information collection efforts:
Tim McAfee, MD, MPH
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-5709
Email:
[email protected]
Terry Pechacek, PhD
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-5592
Email: [email protected]
Diane Beistle
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-5066
Email: [email protected]
Robert L. Alexander Jr., PhD, MPH, CHES
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-1212
Email: [email protected]
Jami L. Fraze, PhD, MEd, CHES
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-5186
Email: [email protected]
Bob Rodes, MS, MBA, MEd
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-5748
Email: [email protected]
Karen Debrot, DrPH, MNS, RD
Office on Smoking and Health
Centers for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K50
Atlanta, GA 30341
Phone: (770) 488-1037
Email: [email protected]
Jeffrey McKenna, MS
National
Center for Chronic Disease Prevention and Health Promotion
Centers
for Disease Control and Prevention
4770 Buford Highway NE, Mailstop K40
Atlanta, GA 30341
Phone: (770) 488-5131
Email: [email protected]
The following individuals outside of the agency have been consulted on the audience questionnaire development. CDC OSH holds regular by-weekly meetings with FDA CTR where updates on The Campaign are included as a standing agenda item. Additionally input has been solicited and received from FDA on the design of this study including participation by FDA on a call with OMB on November 17, 2011 (Message Testing for Tobacco Communication Activities (MTTCA) Previous Title: “Testing and Evaluation of Tobacco Communication Activities”).
Kevin C. Davis, MA
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: (919) 541-5801
Email: [email protected]
Jennifer Duke, PhD
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
Phone: (919) 485-2269
Email: [email protected]
Donna Vallone, PhD
Legacy Foundation
1724 Massachusetts Avenue, NW
Washington, DC 20036
Phone:
(202) 454-5783
Email:
[email protected]
April Brubach
FDA, Center for Tobacco Products
9200 Corporate Boulevard
Rockville, MD 20850
Phone: (301) 796-9214
Email: [email protected]
A.9. Explanations of Any Payment or Gift to Respondents
Participants from the KN panel will be offered 5,000 Knowledge Networks bonus points (equivalent to $5 cash) for completion of the baseline survey and 15,000 bonus points (equivalent to $15 cash) for completion of the follow-up survey. Thus, a total incentive of $20 will be offered to KN panelists who complete the entire 2-wave study. Participants from the SSI supplemental sample will be offered $1 for completion of the baseline survey and $3 for completion of the follow-up survey. These incentive structures are customary for each panel and are known by participants ahead of time. KN panel incentives are generally higher because KN maintains a longer-term panel with fewer participants. Therefore, they offer higher incentives for fewer surveys in order to maintain consistent participation among their panelists over time and to minimize panel attrition. The lower SSI panel incentive reflects the “opt-in” nature of this panel, as opt-in panels are comprised of volunteer participants and generally involve shorter-term participation but more frequent surveys. Alterations to each respective incentive structure would not be advised as this would de-stabilize each sample vendor’s ability to estimate sample response and project the yield of completed interviews.
The incentives are intended to recognize the time burden placed on participants, encourage their cooperation, and to convey appreciation for contributing to this important study and are similar to incentives that are offered for most surveys among participants in each respective panel. Numerous empirical studies have shown that incentives can significantly increase response rates (e.g., Abreu & Winters, 1999; Shettle & Mooney, 1999). The decision to use incentives for this study is based on the need to ensure high retention from baseline to follow-up in order to retain the necessary analytic power of the longitudinal sample. The higher incentive ($15 for KN panel, $3 for SSI) for the follow-up survey is implemented particularly to ensure sufficient sample retention between baseline and follow-up.
The use of modest incentives is expected to enhance survey response rates without biasing responses. A smaller incentive would not appear sufficiently attractive to participants. We also believe that the incentives will result in higher data validity as participants will become more engaged in the survey process. This will also enhance overall response to both the baseline and follow-up surveys. It is crucial that the baseline survey be completed prior to the launch of The Campaign as this is central to our planned data analyses. The use of incentives will help ensure that baseline data collection is completed in a timely manner. The specific amounts of the proposed incentives are based on several previous projects conducted by RTI and Knowledge Networks, which found that use of similar incentives increased response rates among adults, particularly for retention in longitudinal studies. All respondents in the KN panel are given free hardware (if needed), free Web access (if needed), free e-mail accounts (if needed), and ongoing technical support as pre-incentives by Knowledge Networks. Because all selected individuals may not be eligible for the study, we want to ensure sufficient project spending and only provide bonus point incentives to respondents after they are determined to be eligible.
A.10. Assurance of Confidentiality Provided to Respondents
All procedures have been developed, in accordance with federal, state, and local guidelines, to ensure that the rights and privacy of participants are protected and maintained. The RTI Institutional Review Board (IRB) reviewed and approved all instruments, informed consent materials, and data collection and management procedures (see RTI IRB approval notice in Attachment G).
Privacy Act Determination
This submission has been reviewed by CDC’s National Center for Chronic Disease Prevention and Health Promotion and CDC’s Information Collection Review Office, which have determined that the Privacy Act does not apply. Although identifiable information about respondents will be used to facilitate initial contact and follow-up, the identifying information is maintained in a secure, pre-existing records system owned by Knowledge Networks. The response data transmitted from Knowledge Networks to RTI International, the data analysis contractor, will be de-identified prior to transmission and analysis.
Safeguards
To ensure data security, all RTI and Knowledge Networks project staff are required to adhere to strict standards and to sign a non-disclosure agreement as a condition of employment on this project. RTI maintains restricted access to all data preparation areas (i.e., receipt and coding). All data files on multi-user systems will be under the control of a database manager, with access limited to project staff on a “need-to-know” basis only. Knowledge Networks has developed a secure transmission and collection protocol, including the use of system passwords, and two separate sets of firewalls to prevent unauthorized access to the system. Neither questionnaires nor survey responses are stored onto the Knowledge Networks-provided laptops; questionnaires are administered dynamically over the Internet. Survey responses are written in real-time directly to Knowledge Networks’ server and are then stored in a local Oracle database. The database is protected primarily through firewall restrictions, password protection, and 128-bit encryption technology. Individual identifying information will be maintained separately from completed questionnaires and from computerized data files used for analysis. A detailed description of Knowledge Networks privacy safeguards is provided with this submission (Attachment F-2). No respondent identifiers will be contained in reports to CDC and results will only present data in aggregate.
Consent
All respondents will be assured that the information they provide will be maintained in a secure manner and will be used only for the purpose of this research. Please refer to the assurances and study descriptions that are included in the Screener and Consent Process (Attachment E-1). Respondents will be assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Respondents will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.
Nature of Participation
Respondents will participate on a voluntary basis. The voluntary nature of the information collection is described in the introductory section of the Screener and Consent Process (Attachment E-1) and the initial contact email (Attachment E-2).
A.11. Justification for Sensitive Questions
The majority of questions asked will not be of a sensitive nature. There will be no requests for a respondent’s Social Security Number (SSN). However, it will be necessary to ask some questions that may be considered to be of a sensitive nature in order to assess specific health behaviors, such as cigarette smoking. These questions are essential to the objectives of this information collection. Questions about messages concerning lifestyle (e.g., messages about smoking, current smoking behavior, attempts to quit smoking, etc.), and some demographic information, such as Race, Ethnicity and Income could be considered sensitive, but not highly sensitive. To address any concerns about inadvertent disclosure of sensitive information, respondents will be fully informed of the applicable privacy safeguards. The informed consent protocol (Attachment E-1) will apprise respondents that these topics will be covered during the survey. This study includes a number of procedures and methodological characteristics that will minimize potential negative reactions to these types of questions, including:
Respondents will be informed that they need not answer any question that makes them feel uncomfortable or that they simply do not wish to answer.
Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses.
Participants will be provided with a specific toll-free phone number (linking directly to the RTI IRB Office) to call in case there is a question or concern about the sensitive issue.
Finally, as with all information collected, these data will be presented with all identifiers removed.
A.12. Estimated Annualized Burden Hours and Cost
A.12.a. Estimated Annualized Burden to Respondents
Information will be collected through on-line interviews involving adult smokers and non-smokers in the U.S., ages 18-54. Information will be collected at baseline (pre-campaign) and again approximately three months later (post-campaign). The target number of complete pre-/post-campaign interviews for smokers is 5,000. The target number of complete pre-/post-campaign interviews for non-smokers is 2,000.
To recruit the target number of complete pre-/post-campaign interviews, we estimate that a total of 34,660 respondents must be contacted through the initial screening and consent process (Attachment E-1), the estimated burden per response is two minutes.
Because we anticipate respondent attrition between the baseline and post-campaign (follow-up) surveys, we must collect enough interviews at baseline to yield the desired sample sizes at follow-up. Based on data from previous longitudinal studies we have conducted, we estimate that 11,600 smokers must be recruited to complete the Smoker Baseline Questionnaire (Attachment C-1) in order to yield 5,000 completed post-campaign Smoker Follow-Up Questionnaires (Attachment C-2).
We estimate that 2,666 non-smokers must be recruited to complete the Non-smoker Baseline Questionnaire (Attachment D-1) in order to yield 2,000 completed post-campaign Non-smoker Follow-up Questionnaires (Attachment D-2).
For both respondent groups, the estimated burden per response is 25 minutes for each baseline questionnaire. In addition, the estimated burden per response is 25 minutes for each post-campaign (follow-up) questionnaire.
All respondents who are non-smokers will be drawn from the Knowledge Networks panel. Respondents who are smokers will be drawn from both the Knowledge Networks panel and the SSI panel. The same data collection instrument will be used all respondents who are smokers. Additional information about the KN panel, SSI panel, and estimated response rates is provided in Section B.1.
This data collection will take place in 2012 only. Thus the one-year annual response burden is estimated at 10,015 hours. Exhibit 2 provides details about how this estimate was calculated. The Web self-administered surveys will be designed to maximize ease of response (at home on personal computers) and thus decrease respondent burden.
Exhibit 2. Estimated Annualized Burden Hours
Type of Respondent |
Form Name |
Number of Respondents |
Number of Responses per Respondent |
Average Burden per Response (in Hours) |
Total Burden (in Hours) |
General Population |
Screening and Consent Process |
34,660 |
1 |
2/60 |
1,155 |
Adults, ages 18-54 in the U.S. |
Smoker Baseline Questionnaire |
11,600 |
1 |
25/60 |
4,833 |
Smoker Follow-Up Questionnaire |
5,000 |
1 |
25/60 |
2,083 |
|
Non-Smoker Baseline Questionnaire |
2,666 |
1 |
25/60 |
1,111 |
|
Non-Smoker Follow-up Questionnaire |
2,000 |
1 |
25/60 |
833 |
|
|
Total |
10,015 |
|||
A.12.b. Estimated Annualized Burden Costs
Respondents participate on a purely voluntary basis and, therefore, are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. RTI has conducted many smoking-related surveys of similar length among smokers and non-smokers with Knowledge Networks. We have examined diagnostic data from each of these prior surveys and estimate that data collection for this study will take approximately 25 minutes per respondent. According to the U.S. Department of Labor (DOL) Bureau of Labor Statistics as of March 2011 the national average hourly wage is $22.89. Thus assuming an average hourly wage of $22.89, the estimated one-year annualized cost to participants will be $229,243. The estimated value of respondents’ time for participating in the information collection is summarized in Exhibit 3.
Exhibit 3. Estimated One-Year Annualized Cost
Type of Respondent |
Form Name |
Number of Respondents |
Total Burden Hours |
Hourly Wage Rate |
Total Cost |
General Population |
Screening and Consent Process |
34,660 |
1,155 |
$22.89 |
$26,438 |
|
Smoker Baseline Questionnaire |
11,600 |
4,833 |
$22.89 |
$110,627 |
Smoker Follow-Up Questionnaire |
5,000 |
2,083 |
$22.89 |
$47,680 |
|
Non-Smoker Baseline Questionnaire |
2,666 |
1,111 |
$22.89 |
$25,431 |
|
Non-Smoker Follow-up Questionnaire |
2,000 |
833 |
$22.89 |
$19,067 |
|
|
|
Total |
$229,243 |
||
A.13. Estimate of Other Total Annual Cost Burden to Respondents or Record Keepers
None.
A.14. Estimates of Annualized Cost to the Federal Government
This information collection is funded through a contract with RTI International. The total estimated costs attributable to this data collection are $803,406. There are additional contract-funded activities occurring before and after this data collection that include project planning and data analysis. Other activities outside this data collection include coordination with CDC and its media contractor; evaluation plan development; instrument development; reporting; RTI IRB and progress reporting and project management. This information collection will occur in 2012, thus the annual cost to the Federal government is estimated to be $846,609 ($803,406 + $43,203 CDC cost). Three CDC health communications specialists are responsible for overseeing the content of this information collection, overall project management, and coordination with other CDC activities.
Itemized Cost to the Federal Government |
|||
CDC Staff Member |
Annual Salary |
% Allocation (Annualized |
Cost |
GS-12 |
$86,280 |
25% |
$21,570 |
GS-13 |
$108,299 |
15% |
$16,245 |
GS-14 |
$107,770 |
5% |
$5,389 |
|
|
Subtotal, CDC Personnel |
$43,203 |
Contractual Costs for Data Collection and Management (RTI) |
|
Subtotal, Contractual Costs |
$803,406 |
|
|
Total |
$846,609 |
A.15. Explanation for Program Changes or Adjustments
There are no changes, as this is a new information collection.
A.16. Plans for Tabulation and Publication and Project Time Schedule
Data from this information collection will be used to estimate awareness of and exposure to The Campaign among a geographically and demographically varied population of smokers and non-smokers in the U.S.. These estimates will take the form of self-reported ad recognition and recall that assess basic exposure as well as frequency of ad exposure. These estimates will also be calculated separately for each specific Campaign advertisement. The KN panel will be the primary source of sample for describing campaign awareness for a large, varied national sample. However, the KN panel alone may not be sufficient to assess awareness at more granular levels, particularly within smaller geographic areas. In order to describe campaign awareness among smokers in areas with relatively small population densities we will add sample. The added sample will be obtained from a separate internet panel. Specifically, an additional 2,000 smokers will be recruited from the SSI panel to participate. This additional sample will be concentrated, to the extent possible, in geographic areas with less coverage by the existing KN panel.
Data from this information collection will also be used to examine statistical associations between exposure to The Campaign and pre-post changes in specific outcomes of interest. There are three domains of primary outcomes for this evaluation – (1) campaign awareness and receptivity, (2) awareness of cessation services, and (3) cessation-related information-seeking. Based on previous empirical research, these outcomes are expected to change within a relatively short amount of time (within approximately 3 months), commensurate with the short duration of The Campaign. There are also three domains of secondary outcomes. While it is uncertain whether secondary outcomes will change during the brief timeframe of The Campaign, they will be examined to determine their relationships to campaign exposure. These include (1) cessation-related communication with smokers, (2) smoking and cessation-related knowledge, attitudes, and policies, and (3) cessation-related intentions and behaviors. As discussed elsewhere in this information collection request, national estimates of these outcomes will generally be derived from the KN sample whereas separate estimates for smaller geographic regions may be derived from the combined KN and SSI samples. See Attachment I-1 which enumerates all of the specific outcome measures under each of these broad domains, the instrument items that will be used to measure those outcomes, and the timeframe for expected change.
Analysis of the relationship between exposure to The Campaign and changes in these outcomes will be accomplished with the use of multivariate models that estimate follow-up measures of each relevant outcome as a function of prior self-reported exposure to The Campaign, controlling for baseline measures of each outcome as well as baseline individual characteristics that may confound the relationship between Campaign exposure and changes in outcomes. These models will generally take the form of logistic (or logit) regressions for dichotomous outcomes and ordinary least squares (OLS) regressions for any continuous outcomes that are measured. The primary outcomes of interest among smokers will be awareness of The Campaign, knowledge, attitudes and beliefs related to smoking, intentions to smoke, and making an attempt to quit smoking. The primary outcomes of interest among non-smokers will include awareness of The Campaign and encouragement of family or friends who smoke to quit smoking, as well as knowledge, attitudes, and beliefs related to smoking and secondhand smoke. We hypothesize that there should be larger changes in outcomes among individuals who are exposed to The Campaign more frequently (i.e., dose-response effects).
We will also utilize measures of market-level campaign intensity, which will be constructed with available data on campaign gross ratings points (GRPs) for each market covered by this survey. GRPs are the advertising industry’s standard for quantifying reach and frequency of exposure to an advertising campaign in a given media market. GRPs are based on broadcast ratings in the markets where the commercials air and provide an estimate of the percentage of the target audience that is exposed to the ads and the frequency with which they are exposed. For example, if 80% of the target audience in a media market saw a commercial four times in a given time period, this would translate into 320 GRPs (i.e., 80 × 4) for that period. GRPs are calculated for each of the 210 designated market areas (DMAs) in the U.S. DMAs are a standard geographic unit comprised of U.S. counties that are grouped together within television market viewing areas. U.S. counties are assigned to only one DMA. Because these data represent averages within each media market, GRPs represent individual’s potential exposure to The Campaign. An individual’s actual exposure may be more or less than the average represented by the GRP based on TV viewing habits.
GRP data serves two purposes. First, it allows us to examine in more detail the reach of The Campaign across media markets in the U.S. Second, it provides an alternative measure of exposure to The Campaign that is exogenous to the individual. This helps address the potential limitations of selective attention bias that may be present with self-reported awareness whereby smokers that already are already in the process of quitting may also be more attentive to campaign ads and thus more likely to indicate ad recognition. Thus we are able to use this data to check the robustness of campaign effects across different measures of exposure and to examine the validity of individual self-reported measures of exposure. That is, market-level campaign intensity should be correlated with individual-level self-reported exposure.
The GRP data will be merged to the survey data based on DMA geographic identifiers which are known for each survey participant via pre-existing panel profile information for both KN and SSI. That is, each individual in the survey will be assigned a value of campaign GRPs equal to the total GRP that was delivered over the course of The Campaign in the media market where they reside. In this way, the GRPs serve as an exogenous and naturally-varying measure of campaign “dose” among our survey participants. This will allow us to analyze the relationship between the market-level delivery of The Campaign and actual levels of awareness in each sample that is collected. This will also facilitate further analyses of the relationship between exogenous market-level measures of campaign dose and changes in the aforementioned outcome variables of interest.
It should be noted that while the KN panel’s recruitment procedures are designed to approximate a nationally representative sample, the limitations associated with the panel decrease our capacity to draw nationally representative conclusions about either smoking-related knowledge and behavior or the impact of the campaign on long-term quit rates in sub-populations. Although KN panelists must be invited to participate and cannot volunteer on their own, there may be systematic differences between individuals who choose to join an ongoing internet panel and the type of individuals who do not wish to participate in either an internet panel and/or over an ongoing committee. In addition, while Knowledge Networks goes to great lengths to ensure that persons who are not web-users are included in the sample, the web medium may introduce some bias towards panelists more comfortable with web-based communication. Furthermore, our estimates for smaller geographic areas are even more limited with respect to representativeness because we will only be adding smokers who agreed to be in a separate ongoing panel, and we anticipate the response rate will be about 1/3 of that expected for the KN panel (i.e., 25% for the supplemental panel and 75% for the KN panel). Therefore, evaluation results must be interpreted with appropriate caution regarding our ability to generalize the findings to the national population of smokers and nonsmokers.
The reporting and dissemination mechanism will consist of three primary components: (1) summary statistics (in the form of Power Point presentations and other briefings) on individual awareness of and reactions to The Campaign; (2) a comprehensive evaluation report summarizing findings from this information collection and (3) at least 3 peer-reviewed journal articles that document the relationships between Campaign exposure and changes in the aforementioned outcomes of interest. In recognizing the aforementioned data limitations, all communications about the evaluation results via these publication formats will carefully enumerate and describe those data limitations and ensure that evaluation results are interpreted with appropriate care and caution. The key events and reports to be prepared are listed in Exhibit 4.
The Campaign is scheduled to begin in March, 2012. Baseline information collection must be completed before the launch of The Campaign. OMB approval is requested as soon as possible in February 2012.
Project Activity |
Date |
Baseline data collection |
February/March 2012 |
Follow-up data collection |
May /June 2012 |
Preparation of analytic data file |
2-4 weeks after completion of data collection (Apx. June/July 2012) |
Data analysis |
Apx. July-September1, 2012 |
Report writing and dissemination |
Apx. September-December, 2012 |
A.17. Reason(s) Display of OMB Expiration Date is Inappropriate
Not applicable. All data collection instruments will display the expiration date for OMB approval of the information collection.
A.18. Exceptions to the Certification Statement
Not applicable. No exceptions to the certification statement are being sought.
References
Abreu, D.A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association.
Chang L. & Krosnick J.A. (2009). National surveys via RDD telephone interviewing versus the Internet: comparing sample representativeness and response quality. Public Opinion Quarterly. 74(4):641-678
Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15, 231-250.
Southwell B, Barmada C, Hornik R, et al. (2002).Can we measure encoded exposure? Validation
evidence from a national campaign. J Health Commun, 7:445-453.
Yeager D.S., Krosnick J.A., Chang L., et al. (2011). Comparing the accuracy of RDD telephone
surveys and Internet surveys conducted with probability and non-probability samples. Public Opinion Quarterly. 75(4):709-747.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Center for Disease Control and Prevention and |
| Author | Angela Ryan |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |