Supporting Statement Part B:
Collection of Information
Comparing the Effectiveness of Traditional Evidence-Based Tobacco Cessation Interventions to Newer and Innovative Interventions Used by Comprehensive Cancer Control Programs
August 11, 2011
Contact Persons:
Behnoosh Momin, MS, MPH
Project Director
Scientific and Clinical Translation Team
Comprehensive Cancer Control
Centers for Disease Control and Prevention
4770 Buford Highway, NE, MS K-57
Atlanta, GA 30341-3717
Phone: (770) 488-3112
Fax: (770) 488-4335
E-mail: [email protected]
Susan Henderson, MD, MPH
Technical Monitor
Centers for Disease Control and Prevention
Division of Cancer Prevention and Control
4770 Buford Highway, NE, MS K-57
Atlanta, GA 30341-3717
Phone: (770) 488-3111
Fax: (770) 488-4335
E-mail: [email protected]
TABLE OF CONTENTS
Section Page
B. Collection of Information Employing Statistical Methods 1
B.1 Respondent Universe and Sampling Methods 1
Quitline Promotional Activities Component 1
Cessation Intervention Component 1
B.2 Procedures for the Collection of Information 2
Quitline Promotional Activities Component 2
Cessation Intervention Component 4
B.3 Methods to Maximize Response Rates and Deal with Nonresponse 7
Quitline Promotional Activities Component 7
Cessation Intervention Component 8
B.4 Test of Procedures or Methods to Be Undertaken 9
Quitline Promotional Activities Component 9
Cessation Intervention Component 10
B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data 10
Exhibits
Number Page
Exhibit 1. Metrics and Data Sources for the Quitline Promotional Activities Component 3
Exhibit 2. Cessation Intervention Component: Proposed Seven-Month Follow-Up Survey Data Collection Flow 7
LIST OF APPENDICES
Appendix 5a. Intake Survey Data for Quitline Clients
Appendix 5b. Intake Survey Data for Web-based Service Clients
Appendix 6a. Follow-up Survey Data for Quitline Clients
Appendix 6b. Follow-up Survey Data for Web-based Service Clients
Appendix 11. IRB documentation
The Centers for Disease Control and Prevention (CDC) and RTI will ask up to 50 state Tobacco Control Programs (TCPs) to participate in the Quitline Promotional Activities Component. To conduct this component, investigators will compile existing data from each TCP to create a comprehensive dataset. All data will be obtained from TCP staff, their contractors (including media vendors and service providers), and the National Quitline Data Warehouse (NQDW), a CDC-sponsored repository of quitline data from all 50 states. New, original data will not be collected as part of this study.
Investigators will collect information from a total sample of 8,000 smokers who enroll in either quitline or Web-based cessation services across approximately four states for up to a 24 month period (dependent upon how quickly participant recruitment and follow-up takes place). Although the primary consideration in this study is sample size, a secondary goal of the study is to maximize the variety of states while obtaining sufficient comparison information in each state. Discussions with states with both a Comprehensive Cancer Control (CCC) program and a TCP have indicated approximately four states have programs that would meet the eligibility criteria and who wish to participate (see Appendix 9).
Sample Size and Statistical Power
Investigators used power and sample size calculations based on criteria published by Rosner (Rosner, 1995). Calculations assumed an alpha of 0.05, power of 0.80, equal sample sizes, and an oversample of 50% anticipating an approximate participation rate of 50%. This oversampling percentage is conservative and approximate as loss to follow up rates varied greatly among previous phone-based cessation studies from 4% (Katz et al. 2004) to 55% (Rabius et al., 2004; McAlister et al., 2004) as reported in Stead et al. 2006. Both An et al. and Zbikowski et al. have reported quitline / Web-based quit rates of 29.3% / 12.5% and 18% / 7% for each modality respectively (An et al., 2010; Zbikowski et al., 2008). Given these rates, investigators sought to have sufficient sample size to determine at least a 3.0 % difference between quitline and Web-based success rates. Although this degree of difference is less than that seen in the An and Zbikowski studies (16.8% and 11% respectively), requiring a larger sample size will allow investigators to undertake stratified and multivariate modeling practices that would otherwise have insufficient power at smaller sample sizes. Investigators calculated an estimated sample size of 4,000 participants in each modality using these criteria and assumptions.
The sample will be drawn from smokers who complete the intake questionnaire for either the quitline or Web-based intervention offered in each state. The number of clients of quitline and Web-based services within each state will determine whether a census (all participants) or random sample method will be used to identify sufficient numbers of participants from each state to obtain a sufficient sample size. Census methods will be used when the number of study participants in a state is expected to approximate the state’s quitline call volume and/or registrations for Web-based services during the study time period. Random sample methods will be used when the state’s quitline call volume and/or registrations for Web-based services during the study time period exceeds the requisite number of study participants. If participation rates are lower than expected for a state, investigators will determine the best method of increasing the sample size in other states to meet the total sample requirements. As one main goal of the study is to describe differences in user populations between the modalities, no efforts beyond the usual TCP-initiated recruitment methods will be made to influence participation in either arm. The multi-modal follow-up strategy of offering a Web-based or telephone follow-up survey should increase the likelihood of achieving the target response rate of 80% while decreasing intrusion on participant’s privacy.
Investigators will monitor, describe, and report on ongoing activities from all participating states to promote their quitline for up to a 24-month study period. Data on the type and timing of promotional activities occurring during the study period will be gathered from participating states as feasible (see Appendix 4). The impact of promotional activities will be measured by the associated quitline call volume data. Call volume will be summarized for the entire length of data collection for each state as they join the project for up to 24 months (see Appendix 3 for an overview of study design and data sources). No new, original data collection will be conducted as part of the Quitline Promotional Activities component.
Promotional activities may include traditional efforts (e.g., paid television, radio, print) or new, innovative efforts (e.g., online banner ads, search engine ads, social media, websites). Each of these promotional activities will be coded by investigators to identify activities as traditional versus innovative and for other qualitative characteristics such as type of message, theme, and target audience. Different metrics are associated with the various promotional activities implemented by the TCPs. For instance, level of exposure of television advertising is typically measured by Gross Rating Points (GRPs), whereas exposure associated with online banner ads is measured by Click-Through Rates (CTRs). Traffic to state websites that promote the quitlines will be monitored using the states’ preferred Web-analytics platform. If the state does not currently monitor Web traffic, investigators will assist the TCP in linking Google Analytics (a free Web-analytic tool) to their site.
Media purchase and placement data are typically monitored by TCP staff and/or their media vendors. Investigators will work with TCPs and/or their media vendors to ensure that the most appropriate metrics are obtained. In addition, investigators will work with each state to customize the most efficient mechanism and timeline for sharing all promotional activity data (e.g. quarterly automated Web traffic reports). Monthly call volume will be obtained from TCPs and de-identified quitline data will be obtained from the NQDW (OMB control no. 0920-0856, exp. 7/31/2011). By relying on automated reports of existing data, it is expected that this project will create a minimal response burden on the TCP staff. All data sharing procedures will be designed to maintain the privacy of the participants and quality of the dataset while minimizing burden to the TCPs and their service providers.
The requested metrics and their associated data sources are listed in Exhibit 1 (see also Appendix 3).
Exhibit 1. Metrics and Data Sources for the Quitline Promotional Activities Component
State Level Data Requests |
Examples
of Metrics |
Sources of Data |
QuitLine |
Weekly call volume, demographics, self-reported source of referral, confirmed awareness of promotion activities |
TCP and National Quitline Data Warehouse (NQDW)
|
Promotional Activities |
Media placement data (e.g., Type of media, theme/content/message, target audience, GRPs, impressions, click-thru-rates, cost) |
Program manager/media contractor |
Analytics for Web site/Mobile Phone Applications
|
Aggregate level weekly Web traffic – visits, unique visitors, average time spent on site, geographic region of visitors
|
Google Analytics or other Web analytic platform
(If available) Web site database of registered users |
Social Media Tracking |
Activity on social media platforms such as YouTube, Facebook, Twitter |
Free public programs (e.g., Facebook Insight, Twitter) |
Investigators aim to obtain information from approximately 8,000 participants (4,000 “Web-only” and 4,000 “phone-only” participants) in approximately four states. To be eligible for participation state TCPs must offer both Web-based and traditional telephone-based cessation services so that a comparison may be made. Intake (baseline), follow-up, and service utilization data will be used to characterize users of each intervention modality and to assess the comparative effectiveness of the interventions. Intake data and service utilization data are already being collected by the quitline or Web service providers using protocols established between each state and its service provider. Investigators will work closely with the TCPs to develop acceptable informed consent procedures and establish seamless data transfer schedules to obtain intake data from the quitline and Web-based program users collected by the TCP or their contracting service provider.
When possible, 7-month follow-up information regarding participation in cessation programs and current smoking status will be collected directly from state TCPs through previously established databases. Unfortunately this information is not always collected for each individual by all TCPs, as is required to meet the goals of the study, and it may be necessary for investigators to supplement this effort. As such, it will be necessary for investigators to collect and store – as well as ultimately destroy when done using – personally identifiable information. Identifiable information such as names, addresses, telephone numbers, and e-mail addresses that is needed to locate participants for the follow-up data collection will be stored in separate encrypted files as noted below. With consent, follow-up data obtained by investigators will be shared with the respondent’s respective state TCP. All data sharing procedures will be designed to maintain the privacy of the participants and quality of the dataset while minimizing burden to the TCPs, their service providers, and their clients.
For participants who solely engage in telephone-based counselling (i.e., the “phone-only” sample) the standardized intake questionnaire will be collected primarily by computer-assisted telephone interviews (CATI). CATI systems provide a script to interviewers and a data entry interface to help improve quality assurance of the data collection and reduce interviewer and respondent burden (Groves, 2009). Participants who solely engage in the Web-based smoking cessation program (the “Web-only” sample) will complete a standardized online survey as part of the registration process. All state quitlines include a standardized set of items, known as the North American Quitline Consortium’s (NAQC’s) Minimum Data Set (MDS), in their questionnaires (NAQC, 2009a). The intake questionnaires collect data related to demographics, current tobacco use, intention to quit, and experiences with smoking cessation. These topics are generally accepted as minimally sensitive information, as recommended by the MDS. While the intake measures included on Web-based programs have not been as formally standardized as the MDS, most of the intake questionnaires include a series of comparable, commonly used survey items formatted specifically for online data collection. MDS intake items are included in Appendix 5a.
Utilization data (e.g., number of calls to the quitline, length of calls, log-ins to the Web), metrics regularly collected by these programs, will be obtained from the quitline and Web service providers. These measures will be included in the intake questionnaire data files, or they will be stored in separate files linked by a participant ID number created and maintained throughout the data collection phase to ensure these repeated observations are linked to the individual.
Where available, investigators will work with TCPs to collect already-existing 7-month follow-up data that is concordant with study goals and timelines. Should this information not be available, investigators will need to conduct the data collection for the 7-month follow-up questionnaire and coordinate with TCPs and their service providers to replace or supplement their normal 7-month follow-up efforts (see Appendix 6a and 6b for the follow-up survey instruments for quitline and Web-based clients, respectively). This will be the only new, original data collection obtained in this study. This approach will minimize the burden of data collection on individuals, TCPs, and investigators while also ensuring sufficient information to address study questions is obtained.
CDC and its contractor, RTI have extensive experience in data collection and have safeguards in place to maintain the privacy of all collected data. The informed consent process will describe privacy safeguards to respondents. The primary data file with respondent’s answers will not include names, phone numbers, addresses or other identifying information that could be associated with respondents. All respondents will be assigned a unique study identification number, which will be the only means to distinguish respondents in the primary data file.
Names, addresses, email addresses, or phone numbers associated with respondents will only be used to contact respondents for the follow-up survey or send incentive payments to respondents who agree to provide this information. This contact information will be maintained in a separate data file from the primary data file so that respondents’ answers cannot be matched to identifying information. Access to identifying information will only be granted to project staff who will be required to use this information to fulfill the study protocol.
All data will be stored on a secure RTI network server which can only be accessed by authorized project staff. No hard copy forms with respondent information will be generated. Files that include identifying information will be password-protected so that only appropriate members of the project team can access these data. All identifying information will be destroyed at the conclusion of the project. In addition, investigators will follow protocols laid out by the National Center for Health Statistics regarding the minimum cell sizes for reporting findings to protect identifying participants due to small subsample sizes.
Should data collection be necessary for the follow-up survey, investigators have built mail, e-mail, and telephone prompting steps into the approach to maximize response rates. In states that are not collecting follow-up information it will be requisite that investigators work with the TCP and vendor to ensure participants recruited during intake are willing and prepared to participate in follow-up activities. A dual-purpose Web-based CATI system for the follow-up survey will allow respondents the option to participate by either Web or telephone. To maximize the advantages of the Web survey and reduce burden on respondents, the Web version of the questionnaire will be released to respondents two weeks before initiating any telephone attempts. This less expensive, less intrusive, and less burdensome to the individual data collection method can be used during the first two weeks of each sample release to collect data from cooperative individuals. Offering a Web-based version of the follow-up survey should increase the likelihood of achieving the target response rate by providing a data collection mode that matches the preferences of quitline users who enrolled via the Internet. In general, using multiple survey modes in a sequential manner, like investigators propose, can both control costs and reduce nonresponse error (Biemer and Lyberg, 2003; Groves et al., 2009).
Exhibit 2 provides an overview of data collection for the Web and telephone follow-up survey.
Exhibit 2. Cessation Intervention Component: Proposed Seven-Month Follow-Up Survey Data Collection Flow
Data Collection Phase |
Data Collection Stage |
Projected Maximum Number of Cases |
Phase 1 |
Mail lead letter |
16,000 |
Early 2-week response period for Internet self-administration only |
1,120 |
|
Phase 2 |
Reminder mailing |
14,880 |
Telephone self-administration |
5,600 |
|
Internet self-administration completed |
2,400 |
|
Total Interviews Completed |
8,000 |
|
Investigators will determine the appropriate avenues for inviting states to participate in the Quitline Promotional Activities Component. Investigators will present the study on at least one monthly CDC call held with state CCC and TCP staff. In advance of the call investigators will circulate a document outlining expectations of the states and benefits of participation. By providing comprehensive descriptions of the study and expectations for data sharing procedures it is hoped that states will be willing to participate due to the importance of the study and low burden on TCP staff. To maximize the number of states who agree to participate, investigators have created ways to ensure an appropriate and mutually beneficial project, including the following:
Working with states to ensure a feasible and comfortable level of participation. If a TCP is willing or able to share only certain pieces of information or data, investigators will work with the program to customize the level of collaboration.
Limiting any time burden or financial costs on the TCPs and their vendors as much as possible by creating mechanisms for automated reports. Investigators will incur any additional costs that may arise from obtaining media placement and service utilization data from the media vendors and service providers.
Creating a customized quarterly report that summarizes the information gathered by investigators specifically for each state with a benchmark metric comparing the state’s activity with an aggregate measure of all the other participating states.
Offering technical assistance in setting up analytic platforms or accessing other free resources that may help states monitor their promotional activities.
All quitline call volume and Web-traffic data will be obtained from existing data sources. For instance, call volume from TCPs and de-identified demographic quitline data will be obtained from the NQDW (OMB Control No: 0920-0856); therefore, maximizing response rates related to these existing data will not be of concern to this study.
The following procedures will be used to maximize cooperation and to achieve the desired high response rates within the Cessation Intervention component:
Participants will be offered $40 monetary honoraria for completing the follow-up survey, only. (see A.9.2 for more information)
A dual-purpose Web-based CATI system will be developed for the follow-up survey. This system will allow respondents the option to complete the follow-up survey by either Web or telephone. The multiple survey modes will be used in a sequential manner where the Web version will be released two weeks before undertaking any attempts to contact individuals by telephone. Multimodal strategies for data collection have been found to increase response rates. Offering a Web-based version of the follow-up survey should increase the likelihood of achieving the target response rate by providing a data collection mode that matches the preferences of quitline users who enrolled via the Internet. In addition, a Web-based survey can facilitate reaching participants who are hard to contact via the telephone.
Six trained RTI staff at the RTI Help Desk will provide assistance to participants using the self-administered follow-up survey once the Web follow-up option is released. The Help Desk will operate the same hours as RTI’s Call Center (8:30 a.m. to midnight EST, Monday through Friday; 8 hours each on Saturdays and Sundays). These hours may be adjusted during data collection if it is determined that extended hours would be beneficial to study participants. This coverage will allow for the immediate resolution of most requests for assistance.
Telephone interviewers will be trained intensively on active listening techniques so that they can directly address any concerns expressed by sample members.
Participants who do not respond to the Web-based follow-up survey will be contacted through mailings and telephone.
Given previous experience investigators anticipate that up to 10% of individuals will not be reached via their contact information. This may either be due to errors in contact information or recent changes such as moving to a new home or changing phone service. For these cases, RTI’s tracing staff will work to identify updated contact information, especially current telephone numbers. This staff has experience at interactively using available databases to recover accurate contact information for individuals.
The Quitline Promotional Activities Component will not collect any new, original data. Data will be extracted from already existing resources; therefore, it will not be necessary to test any study protocols or data collection instruments.
The 7-month follow-up questionnaire is primarily based on the NAQC MDS. The MDS items have been previously approved by the OMB as part of the NQDW (OMB Control No: 0920-0856). These items are commonly used in the tobacco control research field; therefore, investigators will only conduct additional cognitive testing of new survey items. New items will be tested among a small sample of less than nine smokers who have previously used quitline services.
Before data collection for the follow-up questionnaire has begun, the dual-purpose Web-based CATI system will be tested by programmers and staff to ensure that survey programming and logic are functional and appropriate. In addition, the survey will be tested to ensure high levels of usability for participants who opt to complete the self-administered Web version of the survey. Investigators will implement a training program for new and experienced telephone interviewers. During this training, interviewers will be given the opportunity to practice the interview and time the length of the survey.
Data Collection Jennifer Keeney, MA RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709 919-316-3525
Survey Development Vance Rabius, PhD The University of Texas MD Anderson Cancer Center Unit 1330 P.O. Box 301439 Houston, TX 77230-1439 713-745-4474
|
Data Analysis Nathan Mann, BA RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709 919-485-5584
Data Analysis Burton Levine, MS RTI International 3040 Cornwallis Road Research Triangle Park, NC 27709 919-541-1252
|
|
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Loraine Monroe |
| File Modified | 0000-00-00 |
| File Created | 2021-01-31 |