Special Memo PATH Study Memo

TO: Office of Management and Budget (OMB)

Through: Reports Clearance Officer, DHHS

Project Clearance Chief, NIH

FROM: Kevin P. Conway, Ph.D.

SUBJECT: Review of the Population Assessment of Tobacco and Health (PATH) Study’s Request to Conduct Cognitive Interviews to Support Development of Questionnaire Items for Wave 3 (OMB Control Number 0925-0663) by HHS Agencies with Shared Interest in the PATH Study and in Other Tobacco-Related Information Collections

In response to guidance from OMB at the July 9, 2014 HHS Data Council Meeting, NIDA reached out on July 10, 2014 to meeting participants from four HHS agencies with shared interest in the PATH Study’s information collections. The purpose was to ask for their reviews of NIDA’s request, in partnership with FDA, to conduct cognitive interviews under OMB Control Number 0925-0663 (expiration date 11/30/15), Generic Clearance for Cognitive Testing of Instrumentation and Materials for the Population Assessment of Tobacco and Health (PATH) Study (NIDA).

This memorandum summarizes the reviews provided by the four HHS agencies, including the NIH’s National Cancer Institute (NCI), the Centers for Disease Control and Prevention (CDC) Office on Smoking and Health (OSH), the Substance Abuse and Mental Health Services Administration (SAMHSA), and the National Center for Health Statistics (NCHS).

Based on this feedback, the original the cognitive testing memo and the English testing protocols were updated; these two documents are attached and should replace the earlier versions of the same documents in the “package” sent to OMB.

The PATH Study planned to begin recruitment for this sub-study on July 22 and to conduct the cognitive testing from August 12 to September 9, 2014. Conducting the testing during this time would help ensure the availability of its findings in time to inform the PATH Study’s Wave 3 questionnaires, to be developed for inclusion in the 60-day Federal Register Notice for Wave 3 planned for publication in January 2015. The consequences of not conducting this cognitive testing are that its findings will not be available in time to meet the PATH Study’s needs for reliable and valid measures in its Wave 3 questionnaires.

NIDA and FDA respectfully request OMB’s approval of this cognitive testing sub-study so that the testing can be conducted in time to inform the development of the PATH Study’s Wave 3 questionnaire.

The memorandum is organized in two parts. The first part below provides the response of each reviewer group (NCI, CDC, SAMHSA, NCHS) to this question: “Will the PATH Study’s cognitive testing sub-study help to inform your cognitive testing data needs?” (Note that some of the responses are edited, with inserted text indicated by [text]).

Part I. Will the PATH Study’s cognitive testing sub-study help to inform your cognitive testing data needs?

NCI: Thank you for the opportunity to review the PATH Study’s Wave 3 cognitive testing sub-study. We [NCI] are very supportive and enthusiastic for this sub-study to move forward as planned and on time. The memo and set of questions to be asked seem to be comprehensive and should be very valuable to other HHS national survey efforts, including those that we lead/coordinate for NCI, including TUS-CPS, NHIS-CCS, and HINTS. This sub-study asks the most fundamental and often the most challenging questions that will not only inform PATH how to ask detailed aspects of the diverse and growing number of tobacco products, but also will provide the basis for other HHS surveys in their development of questions for their own purposes. Thus it will be efficient and further facilitate harmonization across surveys. We are especially enthusiastic about the information from the PATH sub-study that will be mined with respect to how best to ask about e-cigarettes or more generally “electronic nicotine products;” information about flavors; nicotine dependence with respect to non-cigarette tobacco products; and switching between products, as these are some of the same aspects that are needed to inform future TUS-CPS which captures this information via cross-sectional assessment nationally and within states.

CDC: [Although the PATH Study is not a surveillance study] the results from this cognitive testing will be very helpful for U.S. tobacco surveillance efforts, including joint FDA-CDC efforts such as NYTS. It will be helpful to discuss how the PATH Study will share its findings with CDC and others working on tobacco surveillance in the U.S.

SAMHSA: The NSDUH is in the process of establishing its measurement goals for e-cigarettes. This involves determining whether to measure the prevalence of any potential electronic delivery system or to focus on when respondents just use nicotine in these devices. The NSDUH also needs to determine if initiation, frequency measures, and recent-use questions can be included while keeping the total NSDUH questionnaire administration to about 60 minutes. Although these issues are yet to be resolved, we still believe we can learn from the PATH Study’s cognitive testing sub-study for its Wave 3 since it will help us in the process we are undertaking to determine how to establish meaningful measures in NSDUH. We also hope that there will be an opportunity to observe this cognitive testing in person, if this is possible.

NCHS: The National Health Interview Survey’s (NHIS) tobacco content is limited. That said, there are some issues [in the PATH Study’s cognitive testing sub-study] that are relevant to the NHIS and are addressed, and some that could be expanded to address NHIS measurement questions. Naturally, the PATH survey has exponentially greater needs for understanding the nuances of tobacco use compared with the NHIS and your testing protocol reflects this. Still, there may be small changes in your protocol that could benefit NHIS data collection. I have compiled some specific comments on the PATH protocol from the NHIS perspective. We recognize the complexity of such coordination—especially with such a tight deadline. We hope that our comments are helpful. We appreciate your offer to look over our tobacco-related submissions as well. Early collaboration benefits us all although it can be tricky to carry out, given all of our various timelines and data needs. The PATH Study will be an amazing resource for understanding tobacco use in a rapidly changing world of tobacco products. Congratulations on your work to date -- we hope that all goes well for the final stages of Wave 3 approval.

Part II. Summary of comments and recommendations by reviewers at NCI, CDC, SAMHSA, and NCHS of the PATH Study’s cognitive testing sub-study for its Wave 3 questionnaires.

The second part of this memo groups reviewer comments and recommendations into four themes, then briefly describes the action taken in response, including whether or how the comment/recommendation is reflected in the updated cognitive testing memo and English testing protocols. (Some of the comments and recommendations are edited, with inserted text indicated by [text]). The comments/recommendations address the following themes:

1. Justification for site selection of the English and Spanish interviews;

2. More probing on the “rarely” option for the ‘everyday/some days/not at all’ item;

3. More probing on electronic nicotine products (or ENDS); and

4. More probing on e-cigarette specific terminology.

a) Site Selection

Several reviewers commented on the site selections for the cognitive testing study, including questions about regional variation, particularly around the Spanish interviews.

(English Testing)

CDC: [Please explain the basis for] selection of these four locations as [we] have a concern that they may not adequately reflect the true regional variation that you’re trying to achieve. For example, state partners suggest that e-cigarettes are booming in the Midwest, which [may not] be captured in the St. Louis [site] where the trend [may not be] as prominent. [We] realize that multiple different products are being assessed, but it would seem important to ensure that your interview locations reflect areas where the use of these products is most prominent, thus increasing the potential that you’ll receive the most useful information on how questions about these products should be framed.

Action Taken – None, however the updated cognitive testing memo, attached, clarifies that the four testing sites were selected based on a number of factors, including having existing relationships with testing facilities at each location, the availability of cognitive testing staff, and the need to control costs and maximize efficiency. The PATH Study team recognizes that it is not possible to achieve ideal geographic diversity for the proposed testing with only four locations.

(Spanish Testing)

CDC: This location may not draw Spanish language speakers who are representative of Spanish speakers elsewhere in the country. Cognitive testing with Spanish speakers elsewhere might be helpful.

SAMHSA: Due to the dialects and language variations, it would be useful to do Spanish interviews in more than one area of the country, if possible.

Action Taken – None, however the updated cognitive testing memo now explains that the reason for conducting Spanish interviews in only one location is due to the PATH Study’s need to control costs. The subcontractor for this part of the cognitive testing is experienced in recruiting Spanish language participants from varied backgrounds. The diverse population of the Washington, D.C. area will help to ensure that the pool of respondents for this sub-study will be representative of Spanish speakers elsewhere in the country. Pending the availability of funding, future testing will include more geographically diverse sites for Spanish-language testing.

b) “Rarely” option for the ‘everyday/some days/not at all’ item

A second theme in the reviews was on the PATH Study’s probing of the “rarely” response option to the question: “Do you now smoke x….every day, some days or not at all?”

NCI: It is good to see that the PATH Study is exploring ways to capture infrequent or incidental use of some of the emerging tobacco products, such as the term “rarely” when asking about current use. [We suggest] that they not only include the probe about “differences between the terms “some-day” and “rarely,” but also follow-up, as appropriate, by asking what term the participant would have chosen if “rarely” was not a choice. For those responding to the current use question with a response of “some days,” “rarely,” “not at all,” or “DK/refused,” it would be useful, at least for some of the products (e.g., cigars, e-cigarettes and hookah), to ask these questions in conjunction with questions on the number of days within the last 30 days these products were used.

CDC: It is important to know what is meant by the term “rarely”. [We suggest that] asking follow-up questions specifically of respondents who answer “rarely” will help gauge what the frequency of use actually is. The follow-up questions should [probe] beyond just asking whether the [respondent] can differentiate between the two (e.g., NATS has shown that including the “rarely” option considerably increases prevalence estimates for all products). We need to know exactly what respondents mean when they choose the “rarely” response.

NCHS: Testing and quantifying the “rarely” response, and determining whether it adds to the knowledge base will be helpful. To better understand the distinctions respondents may make, consider probing for differences between “rarely” and “not at all,” in addition to “rarely” and “some days” for all products. NHIS would find it useful to test two “current use” questions together (e.g., “every day, some days, rarely, or not at all” and “past 30 day use”) to assess [differences] in the distributions of frequencies of use in the past 30 days among the four categories of use.

Action Taken – The PATH Study has incorporated additional probes to the “every day, some days, rarely, or not at all” responses across products. E.g., “If rarely wasn’t there, what option would you have chosen? Tell me more.” Although the PATH Study is not testing the past-30-day item for most products, it has incorporated this as a probe to inform and quantify what these options mean to the respondent. Note that this change is limited to the English protocol because testing the “rarely” option is not in the current scope of the testing in Spanish. Future testing in Spanish may include the “rarely” response option, however.

c) Electronic Nicotine Products (or Electronic Nicotine Delivery Systems (ENDS))

A predominant theme among the reviewers was on the extent to which the cognitive testing study probes respondents’ understanding and familiarity with electronic nicotine products or “ENDS.” Reviewers recommended more detailed probing on terminology, product description, and question wording for ENDS items, including probing respondents’ understanding of the terms used to refer to these products, if or how they differentiate between different types of ENDS, particularly e-cigarettes, and their patterns of both current and ever use of ENDS.

NCI: All other surveys will be expected to collect [information on e-cigarettes] in the future, at least to some extent, but none other than the PATH Study can examine it as extensively. Thus, [we] are happy to see that the PATH Study will include a global electronic nicotine product description/question for EVER use that combines the various forms of these products, building upon the question fielded in 2014-15 TUS-CPS and also slated for the 2015 NHIS-CCS. Those questions were originally derived from the PATH Study’s baseline questions and further modified based on current changes in these products. We recognize that the PATH Study’s proposed cognitive testing sub-study is to test questions as they are expected to be asked in the Wave 3 questionnaires, as is appropriate. However, if possible, it would be appreciated if the PATH Study includes probes for a combined question on e-cigs/electronic nicotine products for current use, as well as what it now includes for EVER use.

CDC: It will be important for this cognitive testing [study] to ascertain the most appropriate terms used to identify ENDS. Communication with state partners suggests that “electronic nicotine product” may not resonate with youth and that a term like “electronic vapor product” may be more appropriate. To that end, it may be good to specifically ask what the respondents call these products if the issue isn’t raised by the existing questions.

NCHS: The initial NHIS question for e-cigarettes includes examples of products that are used even more rarely than e-cigs (vape-pens, hookah-pens, e-hookahs, or e-vaporizers). We would like to know [what respondents think] these products are. The NHIS needs additional testing on the lengthy introductions to ENDS / electronic product questions. For example: are people differentiating between these types of products?  Do [respondents consider] “e-cigarettes” to mean a general category descriptor, or [do they prefer] another general category name (like ENDS or electronic nicotine products)? Are brand names important in these descriptions, especially given how many there are? Are there other descriptions/characteristics of these products that are device-independent (inhale nicotine vapor, not smoke; don’t need a lighter to ignite)?

Action Taken - In response to these comments and recommendations, the PATH Study has incorporated additional probes to garner respondents’ understanding of the PATH Study’s electronic nicotine product pictures and descriptions, the terminology used in referring to these products, and if and how respondents differentiate among the various electronic nicotine products. Items with probes were added to the English testing protocol to cognitively test use patterns, including current and ever use. Testing the ENDS items is not included within the scope of the current testing effort for Spanish; however, the PATH Study can consider this in a future cognitive testing study.

d) Specific to E-cigarettes

In addition to ENDS, reviewers also provided comments and suggestions for additional probes on respondents’ understanding of the terminology used in a subset of questions about e-cigarettes.

NCHS: What do people think “vapor” is; why do they use e-cigs rather than another product; why is [product] the product of choice; and do they think [e-cigs] are safer than conventional cigarettes? 

Action Taken – The PATH Study has added a probe about “vapor” after the ENDS product description. A probe is also included in the poly-use section of the English testing protocol on reasons for use, for respondents who use e-cigarettes and something else. “Reasons for use” for users of only e-cigarettes is outside the scope of this cognitive testing study. However, Wave 2 questionnaires capture this and were previously cognitively tested. They will be fielded when Wave 2 begins in October of this year.

CDC: Questions on tank systems could be strengthened. Although these are the fastest component of the market, assessment of respondent nomenclature around tank systems seems limited. It would be helpful to expand this [to help] ensure that patterns of use information will be captured, [given that] they are different from disposable e-cigarettes and [may] vary across youth and adults.

Action Taken – The PATH Study has added a probe in the English protocol for tank systems that will ask respondents to describe what they think ‘tank system’ means, in addition to probing on their familiarity and terminology they use. These items are not being tested in the current Spanish testing protocol, however, but they may be explored for testing among Spanish speakers in a future round.

CDC: It would seem reasonable to ask respondents whether they actually know what “voltage” means. It is unlikely that the term is readily understood by many people.

Action Taken – The PATH Study has added a probe to explore respondents’ understanding of the term “voltage”. As noted above, the probe has been added to the English protocol but not to the Spanish protocol given its more limited scope for this testing effort.

List of REVISED Attachments

PATH Study Cognitive Testing Sub-Study Wave 3 memo

Attachment F – PATH Study English Language Protocol

4

PATH Study Wave 3 Cognitive Testing