NQDW Part B_01302019

SUPPORTING STATEMENT FOR THE

NATIONAL QUITLINE DATA WAREHOUSE (NQDW)

(OMB No. 0920-0856, exp. 03/31/2019)

Extension

PART B: STATISTICAL METHODS

January 25, 2019

Submitted by:

Epidemiology Branch

Office on Smoking and Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

Department of Health and Human Services

Refer questions to:

Lei Zhang

Epidemiology Branch

Office on Smoking and Health

National Center for Chronic Disease Prevention and Health Promotion

Centers for Disease Control and Prevention

4770 Buford Highway, NE S107-7

Atlanta, Georgia 30341

(770) 488-1144

FAX (770) 488-5848

E-mail: [email protected]

TABLE OF CONTENTS

B. STATISTICAL METHODS

B.1 Respondent Universe and Sampling Methods 1

B.2 Procedures for the Collection of Information 2

B.3 Methods to Maximize Response Rates and Deal with No response 3

B.4 Tests of Procedures or Methods to be Undertaken 4

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or

Analyzing Data 5

LIST OF APPENDIX

A-1. Public Health Service Act

A-2. American Recovery and Reinvestment Act of 2009

A-3. Affordable Care Act/ Prevention and Public Health Funds

A-4. 60-Day Federal Register Notice

A-5. Public Comments and Responses

B. Instructions for Submitting Summary Caller Intake and 7-Month Follow-Up Data to CDC

C. NQDW Quitline Services Survey

D. Consultants on NQDW

E-1. NQDW Intake Questionnaire

E-2. NQDW Intake Questionnaire_subset

F-1. NQDW_ASQ Intake Questionnaire_Chinese

F-2. NQDW_ASQ Intake Questionnaire_Korean

F-3. NQDW_ASQ Intake Questionnaire_Vietnamese

F-4. NQDW Intake Questionnaire_subset_Chinese

F-5. NQDW Intake Questionnaire_subset_Korean

F-6. NQDW Intake Questionnaire_subset_Vietnamese

G-1. NQDW_ASQ 7-Month Follow-up Questionnaire_English

G-2. NQDW_ASQ 7-Month Follow-up Questionnaire_Chinese

G-3. NQDW_ ASQ 7-Month Follow-up Questionnaire_Korean

G-4. NQDW_ASQ 7-Month Follow-up Questionnaire_Vietnamese

H-1. Request Email for Submitting Summary Caller Intake Data to CDC

H-2. Reminder Email for Non-Respondents for Intake data submission

H-3. Request Email for Submitting NQDW ASQ 7-month Follow-up data

H-4. Reminder Email for Non-Respondents for ASQ 7-month Follow-up data

H-5. Request Email for the NQDW Quitlines Services Survey

H-6. Reminder Email for Non-Respondents the NQDW Services Survey

I. Sample Table shells

B. Statistical Methods

B.1 Respondent Universe and Sampling Methods

National Quitline Data Warehouse (NQDW) Intake Questionnaire

The NQDW Intake Questionnaire will be administered to the entire respondent universe of quitline callers from the 50 U.S. states, the District of Columbia, Guam, Puerto Rico, and the Asian Smokers’ Quitline (ASQ). Basic information about caller demographics, tobacco-use behaviors of callers, why callers are calling the quitline, and how callers heard about the quitline will be collected. Sampling methods are not applicable, as it is necessary to enumerate services provided to all callers for program planning and evaluation.

CDC anticipates call rates to1-800-QUIT-NOW and the other state-specific quitline telephone numbers, as well as the number of tobacco users registering to receive services from state quitlines, to be relatively similar to the annual average over the past few years. We estimate that approximately 502,998 tobacco users from the 50 U.S. states, the District of Columbia, Guam, Puerto Rico, and the ASQ will complete an NQDW Intake Questionnaire each year, based on the average annual adjusted NQDW Intake Questionnaire record counts from 2016 to 2017. This estimate of 502,998 persons only includes persons who at least began an intake process and provided at least a minimal level of information to the quitline; it does not include individuals who contact the surveyed quitlines but do not take the NQDW intake questionnaire. In the 2010 NQDW Reference Manual, states are provided with the following instruction regarding who needs to be included in the data submitted to CDC: “All callers who respond to the question ‘How can I help you? (Reason for Calling)’ during the intake questionnaire should be included in this data collection (i.e., we expect records for all of these individuals), regardless of whether the intake questionnaire was completed, counseling/services were provided or a follow-up survey was conducted.” Most states (48 states and the District of Columbia) comply with this directive and only collect and report data for callers who provide some response to the intake questionnaire; thus it is not possible to compute the percentage of individuals who call and hang up for these two states and time periods (West Virginia, 2017 Q2 & Q3). Two states however have submitted one or more intake data files to the NQDW that include data records for individuals who did not answer the “Reason for Calling” question. Data from these two states can be used to estimate the average proportion of persons who call the quitline and do not begin the intake questionnaire. Between 2015 and 2017, this percentage ranged from 11.6% in 2015, to 19.9% in 2016 (annual average of 16.4%). 

Of the estimated 502,998 quitline clients from the 50 U.S. states, the District of Columbia, Guam, Puerto Rico, and the ASQ who will respond to the NQDW Intake Questionnaire annually, an estimated 490,781 clients will call for themselves for help with quitting and other reasons not directly related to quitting, and an estimated 12,217 quitline clients will call for someone else (Appendices E-1 to E-2). Quitline callers to the ASQ use NQDW Intake questionnaires that have been translated into the following languages: Chinese, Korean, and Vietnamese (Appendices F-1 to F-6).

National Quitline Data Warehouse (NQDW) Seven-Month Follow-Up Questionnaire

In 2010-2011, CDC collected seven-month follow-up data for the NQDW from all states using the NQDW Seven-Month Follow-up Questionnaire. To reduce burden on states, CDC suspended ongoing data collection of the NQDW Seven-Month Follow-up Survey in 2012. Barring substantial changes in quitline services provided and tobacco users receiving services from quitlines, seven-month follow-up quit rates should remain relatively stable over time. CDC has recently awarded a cooperative agreement that involves collecting and reporting seven-month follow-up data from a single, national quitline service provider (University of California San Diego) that serves quitline callers whose preferred language is Chinese, Korean or Vietnamese (the Asian Smokers’ Quitline (ASQ)). Survey among those clients administered in Chinese, Korean, and Vietnamese languages. (Appendices G-2 to G-4, Appendix G-1 is provided in English language for reference). Should the need arise in the future to resume collecting seven-month follow-up data from all states, a revision request to this information collection request approval will be submitted to OMB. Sampling methods are not applicable.

National Quitline Data Warehouse (NQDW) Quitline Services Survey

CDC will request that health department personnel (i.e., state tobacco control managers or their designees) from the 50 state health departments, the District of Columbia, Puerto Rico, and Guam, as well as a representative from the Asian Smokers’ Quitline (ASQ) complete the NQDW Quitline Services Survey electronically each quarter. Sampling methods are not applicable (Appendix C).

B.2 Procedures for the Collection of Information

Data will be collected using the NQDW Intake Questionnaire (Appendix E-1 and E-2 for callers to U.S. or U.S. territories, quitlines and F-1 to F-6 for ASQ quitline callers), the NQDW ASQ Seven Month Follow-up Questionnaire (Appendices G-2 to G-4), and the NQDW Quitline Services Survey (Appendix C) by either state health department personnel who manage the quitline or their designee, such as contracted quitline service providers.

Telephone quitline specialists will continue to collect intake information at the beginning of their telephone interactions with callers, as part of the needs assessment process for determining appropriate counseling messages. Much of this information will be collected passively as clients naturally share information about their smoking/tobacco use and history. Telephone quitline specialists will actively ask questions as necessary, using a conversational style whereby questions will be woven into the conversation rather than asked in a highly structured format. The demographic questions are also asked at naturally-occurring and appropriate points in the conversation or at the conclusion of the conversation when the customer service questions about scheduling a repeated counseling call are made. Quitline experts agree that this approach is preferable for collecting the necessary information in a manner that is respectful of the reason the caller called the quitline – for assistance in quitting. This approach also prevents callers from becoming fatigued with the interview and discontinuing the call before they received their cessation counseling. State quitlines seek to prioritize high-quality services over data collection. Missing intake data has not been an issue for quitlines that are using this strategy to collect intake data from their callers using the MDS suggested intake questions. For the seven-month follow-up survey, individuals who received services from the Asian Smokers’ Quitline will be asked questions in a structured manner – question-by-question as listed on the NQDW ASQ Seven-Month Follow-up Questionnaire (Appendices G-2 to G-4) as this information, including information on caller satisfaction, is being collected for evaluation purposes.

A total of 54 respondents, including all 50 U.S. states, the District of Columbia, Guam, Puerto Rico, and the ASQ will be providing CDC with data collected from the NQDW Intake Questionnaire. The state Tobacco Control Manager or their designee, which might include representatives from the state’s quitline service provider (e.g. ASQ), is responsible for sending CDC a de-identified electronic data file containing caller intake data collected from the NQDW Intake Questionnaire on a quarterly basis. These electronic data files are compiled from the state’s quitline data system, which is supported by state funding sources supplemented by cooperative agreement assistance from CDC. In addition to the per respondent burden estimates for respondents completing the NQDW Intake Questionnaire, we also estimated the burden associated with compiling electronic data files for the NQDW Intake Questionnaire and submitting those to CDC. We estimate the burden for preparing and submitting NQDW Intake Questionnaire data to CDC to each of the 54 respondents, which consist of 53 state tobacco control program managers or their designees, such as quitline service providers, and 1 representative from the Asian Smokers’ Quitline, to be one hour for each quarterly submission of the electronic data files for the NQDW Intake Questionnaire. This estimate also includes time for uploading data files to CDC’s secure FTP server or saving the data to a CD/DVD and preparing it to be mailed to CDC.

The 50 U.S. states, the District of Columbia, Guam, and Puerto Rico are no longer submitting Seven-Month Follow-Up data to the NQDW. A representative from the ASQ will submit a de-identified electronic data file containing data collected from the Seven-Month Follow-up Questionnaire to CDC on an annual basis. We estimate the burden to the Asian Smokers’ Quitline for submitting the de-identified electronic data file for the NQDW Seven-Month Follow-up to be one hour per year.

The NQDW Quitline Services Survey (Appendix C) instruments are state-specific fillable form-style Microsoft Word documents that CDC prepares each quarter for states to complete. The survey consists of 17 questions, and the electronic form for the survey includes drop-down boxes, data entry fields, and checkboxes to help reduce data entry errors. The first part of the survey (questions 1-7) consists of collecting respondent contact information and survey questions regarding the state’s quitline call volume, the number of tobacco users served by the quitline, types of referral systems utilized by the state quitline, and number of referrals received by the quitline. The second part of the survey (questions 8-17) consists of questions regarding the services offered by the state’s quitline including: (a) the name of the state’s quitline; (b) the phone numbers used by the state’s quitline; (c) the quitline’s hours of operation; (d) available counseling languages offered by the quitline; (e) eligibility criteria for receiving counseling from the quitline; (f) amount of counseling offered by the quitline; (g) free quitting medications that are offered by the quitline; (h) eligibility criteria to receive free quitting medications from the quitline, and; (i) the amount of free quitting medications offered by the quitline. Quitline services typically do not change much from quarter to quarter for a given state, and consequently responses to these questions tend to be the same each quarter. To reduce burden on survey respondents, CDC pre-populates the responses to questions 8-17 with the information the state reported for the previous quarter. States are asked to review their previous responses to those questions and make edits if there were any changes in the services being offered by the quitline since the last time the state responded to the survey. CDC sends an email, with the survey included as an attachment, to state health department personnel (i.e., state health department tobacco control managers or their designee) from all 50 states, DC, Guam, Puerto Rico, as well as a representative from the ASQ, once each calendar quarter. As part of our ongoing technical assistance through the National Tobacco Control Program, we keep updated contact information for personnel at each state health department. The estimated burden per response for this submission is 20 minutes. This includes time for emailing the completed survey back to CDC.

CDC and created an online data submission tool (a secure FTP server) in which states have a unique user name and password. The server is checked several times each week for files and the files are quickly removed after being downloaded. The only problems reported by states with this system involve states not being able to access the FTP server because of firewall issues on their side. The CDC TAs have provided detailed instructions to states in the form of screen shots to aid states who may be unfamiliar with submitting data through this mechanism, and are available to “walk” states through this process on a phone call. We believe that most of the technological difficulties have been resolved, as evidenced by the fact that we have received fewer CD/DVDs via U.S. mail compared to the secure FTP server. After receiving data from states, NQDW TA’s return PDF files to states that present a formatted summary of the data reported by the state for the state to review and sign off on.

B.3 Methods to Maximize Response Rates and Deal with No response

Quitlines are voluntary, state-based treatment protocols initiated by tobacco users who either call the quitline or register with the quitline through a website. Unlike population surveys, which have a sampling frame and a response rate indicating the proportion of eligible respondents that participated, NQDW has no sampling frame, participation is initiated by the caller and is entirely voluntary, and data reported by NQDW to CDC is restricted to individuals who have provided some information during the call. The NQDW does not calculate a response rate for individuals responding to the NQDW Intake Questionnaire because all quitline clients must provide this information as part of the registration and counseling process. Callers who provide any information, at least as to why they are calling, are included in the intake data. In cases where callers refuse to provide any information about themselves to the quitline telephone operator or refuse to answer any questions, (e.g., individuals who abandon the call before providing any information), per protocol, these persons are automatically excluded from the data pool for NQDW intake data. With the exception of two states, most states do not maintain data for abandoned calls. Data from the seven states that have these data indicate that the annual average abandonment rate for calls is 16.9%. The NQDW defines the response rate for the NQDW Intake Questionnaire as the number of states/territories that provide Intake Questionnaire data files to the NQDW each quarter. With the addition of the ASQ in 2016, the quarterly response rate for the NQDW Intake Questionnaire data files collected from states remained at 98%, with 53 of 54 states/territories/quitline service providers submitting quarterly NQDW Intake Questionnaire data files to NQDW. Only Massachusetts does not currently provide NQDW Intake Questionnaire data files to NQDW based on privacy concerns. CDC has been working with Massachusetts to identify a subset of the de-identified NQDW Intake Questionnaire requested by the NQDW that Massachusetts would be comfortable sharing with the NQDW. If CDC is able to successfully work with Massachusetts to obtain NQDW Intake Questionnaire data files from them, the expected response rate for the quarterly NQDW Intake Questionnaire data files is expected to be 100%.

State health department tobacco control managers or their designee from all 50 states, DC, Guam and Puerto Rico, as well as a representative from the ASQ submit a summary file of these data to NQDW quarterly. A request email for submitting summary caller intake data and reminder email for non-respondents (within two weeks) are used to prompt submission of this information (Appendices H-1 and H-2).

The NQDW Seven-Month Follow-up Questionnaire will be administered to tobacco users who received a service from the Asian Smokers’ Quitline and collected by a single, national quitline service provider (University of California San Diego). A recent study indicated a response rate for follow-up surveys conducted with tobacco users who received services from the Asian Smokers’ Quitline as 82% (Cummins et al., 2015).  A request email for submitting annual summary Seven Month follow up caller intake data and reminder email for non-respondents (within two weeks) are used to prompt submission of this information (Appendices H-3 and H-4). 

CDC emails a request and copy of the state-specific reporting forms for the NQDW Quitline Services Survey to state health department personnel (i.e., state health department tobacco control managers or their designee) from all 50 states, DC, Guam and Puerto Rico, as well as a representative from the ASQ, for them to complete and return once each calendar quarter (Appendix H-5). CDC asks individuals who are responding to the NQDW Quitline Services Survey to complete and return the survey within four weeks. To maximize response rates for this survey, we will send a reminder email to states that have not completed the survey within two weeks and another reminder email 2 days before the survey completion deadline (Appendix H-6). If the survey has still not been completed, we will follow-up with emails and telephone calls to state contacts to ensure completion. For calendar years 2010 through 2017, we asked 53 states/territories to complete the quarterly NQDW Quitline Services Survey between 2010 and 2014 – a total of 32 quarters - resulting in a total of 1,696 possible responses (53 x 32 = 1,696). Additionally, the ASQ submitted Quitline Services Survey between 2015-2017 for a total of 12 quarters and thus 12 responses for a grand total of 1,708 responses to NQDW.

To support high response rates, CDC-OSH has provided extensive technical assistance (TA) to state programs and quitline service providers including specific information on the data files needed for the NQDW. CDC provides technical assistance for the 54 states/territories and the ASQ participating in the NQDW to minimize the need for formatting data for the NQDW and make data submission procedures more routine and easily integrated into their regular state procedures. Over the past three years, CDC increased the ease of submission by being flexible with the NQDW data formats and data submission requirements. The data reported to NQDW represent a core set of information reported by states. Additionally, states have the option of adding state-specific questions to their intake or follow-up surveys that CDC does not require to be submitted. CDC began accepting some quitline service providers’ secondary individual-level intake “as is” in 2013. For one particular quitline service provider that serves approximately 27 states the need to reformat data for submission was dramatically reduced. CDC substantially overhauled and simplified the NQDW Quitlines Services Survey in early 2015 by developing a version of the NQDW Quitline Services Survey that can be completed by filling out a form-style Word document that contains drop-down boxes, data entry fields, and checkboxes to make data entry easier and cut down on the potential for data entry errors. In addition, the instrument was reduced from over 50 questions respondents at state health departments or their designees had to answer to a total of 17 questions, including 7 questions which must be answered each quarter and an additional 10 questions that respondents are asked to review and update only if there have been any changes in the services offered by the state quitline since the previous quarter. Once respondents have completed the survey, using the electronic reporting form provided, they email it back to CDC where their responses are uploaded to the NQDW database. After receiving data from states, CDC returns PDF files to states that present a formatted summary of the data reported by the state for the state to review and sign off on. CDC-OSH has also responded to states’ requests for easier data submission of summary NQDW Intake Questionnaire data quarterly to CDC and created an online data submission tool (a secure FTP server) in which states have a unique user name and password. Feedback received from state programs indicate that the NQDW data submission procedures have become more routine and more easily integrated into their regular state procedures. State programs have also mentioned that they appreciate CDC-OSH’s flexibility regarding the form in which data are submitted and CDC-OSH’s effort to effectively and efficiently respond to state needs.

B.4. Tests of Procedures or Methods to be Undertaken

Burden estimates are based on the results of instrument pre-testing and experience with the NQDW data collection during the previous six years of OMB clearance (2012-2017). Since states and quitline service providers have been submitting data to the NQDW for the past six years, both states and quitline service providers are quite familiar with and accustomed to the process. The total burden hours for submitting intake (from all states and the Asian Smoker’s Quitline) has decreased since the initial OMB approval due to updates in the estimated number of respondents based on average tobacco users who completed an NQDW Intake Questionnaire per year from 2012-2017 for the 50 U.S. states, the District of Columbia, Guam, and Puerto Rico plus an additional estimated 1,587 average annual number of tobacco users who completed an intake with the Asian Smokers’ Quitline between 2016-2017 be requested from the Asian Smokers’ Quitline.

State health department personnel (e.g., tobacco control program manager, quitline manager) for each state, the District of Columbia, Puerto Rico, and Guam will be asked to complete the NQDW Quitline Services Survey electronically once each calendar quarter. The NQDW Quitline Services Survey instruments are state-specific fillable form-style Microsoft Word documents that CDC prepares each quarter. The survey consists of 17 questions, of which the first 7 questions have to be answered and the remaining 10 questions dealing with the quitline services offered by the state quitline will be pre-populated with the information reported by the state on their previous NQDW Quitline Services Survey. Survey respondents will be asked to review the pre-populated responses to those questions and will only have to modify the pre-populated responses if there has been a change to any of the services offered by the quitline since the last time the state reported data to the NQDW. Since the services offered by state quitlines do not change often from quarter to quarter for a given state, this section will typically not involve any revisions at all for most states. The survey reporting includes drop-down boxes, data entry fields, and checkboxes to facilitate easier data entry and to help reduce data entry errors.

B.5 Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

CDC has consulted with, and will continue to consult with, leading tobacco researchers and CDC partners including the North American Quitline Consortium (NAQC) as appropriate. In 2015, CDC and NAQC engaged in active alignment of the minimal data set (MDS). We have plans to communicate about data elements semi-annually and remain committed to this alignment in order to standardize the data elements.

During the first five years of the NQDW, CDC convened an evaluation workgroup consisting of quitline evaluators and representatives from quitline service providers, NAQC, and state tobacco control programs and other federal agencies. CDC hosted an in-person meeting with the NQDW evaluation workgroup and RTI International, CDC’s previous contractor providing technical assistance and evaluation support for the NQDW, in May 2013. This meeting provided CDC with an opportunity to obtain stakeholder feedback and expert opinion on using NQDW for evaluation, monitoring, and program improvement from NAQC and the quitline community. During the meeting, the workgroup discussed evaluation plans for the NQDW that included data analysis, quality assurance, and dissemination. Notes were taken during the meeting, and a summary of the items discussed and recommendations identified during the meeting was prepared.

Data management and analysis will be performed by the Office on Smoking and Health at CDC with contractor assistance from NORC at the University of Chicago. Starting in November 2013, CDC began sharing tabulations using the NQDW data collected from 2010-2017 publicly online through the CDC’s State Tobacco Activities Tracking and Evaluation (STATE) System website (http://apps.nccd.cdc.gov/statesystem/Default/Default.aspx) which contains a variety of current and historical state-level and national data on tobacco use prevention and control. CDC plans to continue sharing quarterly NQDW data tabulations from the NQDW data on the STATE system on an ongoing basis (Appendix I).

Study Design, Data Analysis, Data Collection and Reporting:

Lei Zhang

Telephone: 770-488-1144

Email Address: [email protected]

4770 Buford Highway, S107-7

Atlanta, GA 30341

Michael Latterner

NORC at the University of Chicago

Telephone: 312-759-4236

Email Address: [email protected]

55 E Monroe St #3000

Chicago, IL 60603

vii