Supporting Statement_Part A

Supporting Statement PART A

0910-NEW

Food and Drug Administration’s

Investigation of Consumer Perceptions of Expressed Modified Risk Claims

A. Justification

1. Circumstances Making Collection of Information Necessary

Historically, tobacco product manufacturers have marketed some of their products using modified risk claims stating that the product is less risky or less harmful to users’ health than other tobacco products or exposes users to lower levels of harmful substances than other products. As stated in the Family Smoking Prevention and Tobacco Control Act (Pub.L. 111–31, H.R. 1256), “the costs to society of the widespread use of products sold or distributed as modified risk products that do not in fact reduce risk or that increase risk include thousands of unnecessary deaths and injuries and huge costs to our health care system,” and “the dangers of products sold or distributed as modified risk tobacco products that do not in fact reduce risk are so high that there is a compelling governmental interest in ensuring that statements about modified risk tobacco products are complete, accurate, and relate to the overall disease risk of the product.”

The Federal Food, Drug, and Cosmetic Act (FD&C Act; Section 911, 21 USC 387) prohibits tobacco product manufacturers from making modified risk claims about their products unless they have applied for, and been granted, a modified risk tobacco product (MRTP) marketing authorization order for the product. To be granted an MRTP marketing authorization order, a manufacturer must demonstrate that the MRTP, as marketed, would (1) significantly reduce the risk and harm to individual tobacco users, and (2) benefit the health of the population as a whole (when taking into account the effects on current tobacco users and on people who do not currently use tobacco products). The manufacturer must also demonstrate that any advertising or labeling for the MRTP will enable the public to comprehend the information concerning modified risk and to understand the relative significance of such information in the context of total health and in relation to all tobacco-related diseases. In other words, a manufacturer wishing to make a modified risk claim about its product is required to demonstrate not only that the claim is factually accurate, but also that the public can understand it and its relevance to their health.

To address the latter need in a modified risk tobacco product application (MRTPA), an applicant can conduct and submit consumer perception studies to the Food and Drug Administration (FDA). One FDA research priority is to develop measures and methods to support the robustness of such research, and this priority is supported by the current study.

The current study has one primary goal and two secondary goals. The primary goal is to develop and validate survey measures for assessing several specific risk perception constructs. Section 2 below describes the methods used to evaluate the measures’ validity (in terms of a range of metrics including criteria for content validity, known groups validity, discriminant validity, and convergent validity). If these measures of specific risk perception constructs are successfully validated, tobacco companies submitting applications to market new products or to market products with modified risk claims can use them in studies conducted to support these applications. This saves them from having to develop their own measures and can yield higher quality data for FDA to evaluate in reviewing their studies. Tobacco researchers and public health practitioners can also use these measures in their research.

This study also has two secondary goals that can inform the methods used in studies of modified risk tobacco products. The first secondary goal is to identify populations that may be susceptible to, or disproportionately influenced by, MRTP claims. We will do this by seeing which of several traits (e.g., uncertainty about the risks of using the type of tobacco product) moderates the effect of the modified risk claim on outcomes. This can help industry understand what populations they can consider oversampling in studies conducted to support applications to market tobacco products with modified risk claims, in-line with FDA draft guidance recommending “Oversampling of populations that are particularly likely to be affected, positively or negatively, by the marketing of the product” (p. 28 line 1105, https://www.fda.gov/media/122008/download).

The second secondary goal is to assess the performance of two approaches for debriefing participants to increase the likelihood that they will be fully informed about key aspects of study participation. Because this study will involve exposing participants to tobacco product ads or packages, some of which will contain modified risk claims that have not been authorized by FDA (note that no such claims have been authorized at this time), we will test two approaches (and a combination thereof) of providing debriefing information (i.e., information about the study and the risks of using tobacco products) in a way that participants will read and understand the information. The first approach is to simplify the debriefing from the standard debriefing text to make it a lower reading level, potentially making it easier to understand. The second approach is to group relevant pieces of debriefing information together in boxes to make it easier for participants to process (vs. reading a block of text). We will also test the combination of these approaches. The method that leads to the best knowledge scores (number of correctly answered items about the debriefing) can be recommended as an option for the tobacco industry and public health community to consider using in their own studies that involve presenting modified risk claims that have not yet been authorized by FDA.

2. Purpose and Use of the Information Collection

The primary goal of this study is to develop and validate measures of several specific types of risk perceptions that can be used in future studies of modified risk claims on tobacco product labeling and advertising. These specific risk perceptions and the items used to measure them are listed below.

(1) The risk of developing various diseases or conditions as a result of using certain e-cigarette or snuff products (questionnaire p. 12)
If you were to use [product name] every day, how likely is it that you would… [Response scale: 1 (not at all likely) to 5 (extremely likely)]
Harm your overall health Have a shorter life Get sick often Get cancer Get a life-threatening disease	Have heart problems Get addicted Have breathing problems Damage your teeth
(2a) The relative risk associated with using these products compared with use of cigarettes (questionnaire p. 14)
If you either used [product name] OR cigarettes every day, which product would make it more likely that you would… [Response scale: 1 (MUCH more likely with [product name]) to 5 (MUCH more likely with cigarettes)]
Damage your teeth Get mouth sores Get tooth decay Get gum disease Get stomach ulcers Get mouth irritation Get mouth cancer Get the common cold or flu Damage your lungs Get pain in your lungs Get lung disease Get asthma Get emphysema Get bronchitis Crave the product all the time Get lung cancer Get stomach cancer Cough often	Get pancreatic cancer Get hooked Irritate your throat Get shortness of breath Get high blood pressure Have a heart attack Have a stroke Get heart disease Have trouble stopping using the product Get diabetes Harm your overall health Have a shorter life Get sick often Get cancer Get a life-threatening disease Have heart problems Get addicted Have breathing problems
Imagine that a woman was pregnant and either used [product name] OR cigarettes every day. Which product would make it more likely that… [Response scale: 1 (MUCH more likely with [product name]) to 5 (MUCH more likely with cigarettes)]
The baby would be harmed before it is born The woman would have a miscarriage The baby would be born too early		The baby would be born too small The baby would have a birth defect
(2b) The relative risk associated with using these products compared with use of nicotine replacement therapy (NRT) (questionnaire p. 18)
Imagine you used either [product name] OR nicotine replacement therapy (NRT) every day, and no other nicotine or tobacco products. Which product would make it more likely that you would… [Response scale: 1 (MUCH more likely with [product name]) to 5 (MUCH more likely with NRT)]
Harm your overall health Have a shorter life Get sick often Get cancer Get a life-threatening disease	Have heart problems Get addicted Have breathing problems Damage your teeth
(2c) The relative risk associated with using these products compared with use quitting all tobacco (questionnaire p. 20)
Which situation would make it more likely that you would… [Response scale: 1 (MUCH more likely with [product name]) to 5 (MUCH more likely with quitting all tobacco)]
Harm your overall health Have a shorter life Get sick often Get cancer Get a life-threatening disease	Have heart problems Get addicted Have breathing problems Damage your teeth

A measure’s validity can be described as the extent to which the set of items comprising a measure accurately represents the underlying theoretical construct. In developing measures of psychosocial constructs such as risk perceptions, a particular measure’s validity is not something that is definitively demonstrated by a single particular analysis or benchmark. Rather, there are numerous types of psychosocial measurement validity that can be demonstrated a number of different ways. To assess the validity of each measure, we will use several different statistical methods commonly used in developing measures of psychosocial constructs (described in Crano, Brewer, & Lack, 2014; Devellis, 2016). These methods can be grouped into two categories: methods to assess the structure of a measure and systematically narrow down the list of items into a smaller set to represent the final scale; and methods to further assess the validity of the final scale. For each construct, we seek to develop a measure that consists of a subset of the items listed in the table above that have performed well in validity analyses listed in the table below.

Methods to narrow down set of items to a final measure
Method	Purpose
Content validity with Exploratory Factor Analysis (EFA)	Content validity can be described as the extent to which the items in a measure represent the full construct. EFA can help determine the underlying factor structure of the scale based on how items correlate with one another, i.e., whether items tend to cluster into different factors (subscales) or they are all part of the same main scale. EFA can also identify if one or several items do not correlate with the scale or any subscale and should be removed from the final measure.
Content Validity with Confirmatory Factor Analysis (CFA)	After determining the underlying structure in EFA, assess the extent to which the factor structure fits the data based on model fit indices. Rather than assessing what the structure is, this analysis tests to see if the structure we identified represents the data well.
Reliability analysis based on Cronbach’s alpha	For a measure to be valid, it must be reliable (i.e., internally consistent; to show items are tapping into the same construct). For either the overall scale (if it’s a one-factor scale based on factor analyses) or each factor of the scale (if it’s a multi-factor scale based on the factor analyses), we will assess the extent to which items correlate with one another. Cronbach’s alpha should be at least .70 to indicate good reliability (Crano et al., 2014; Devellis, 2016). Analyses can determine the reliability of the overall scale and whether it can be improved by removing certain items that may not correlate well with the other items.
Methods to assess the validity of the final measure
Method	Purpose
Known groups validity, t-tests comparing current product users to nonusers.	This type of validity is demonstrated when groups that theoretically should have different scores on the measure actually do have different scores on the measure. Because prior literature has established that users of a particular tobacco product think it is less risky than nonusers of the product, we expect higher mean risk ratings among product nonusers compared to users.
Discriminant validity, correlational analysis with skepticism about the harms of tobacco.	This type of validity is demonstrated when the measure of the construct differs from measures of constructs it should theoretically differ from. We will assess this by correlating scores from a valid measure of skepticism of the harms of tobacco (questionnaire p.10) with scores from each final measure. Validity will be demonstrated if correlations are moderate and negative, as people who are skeptical of the harms of tobacco should also hold low risk perceptions of tobacco.
Convergent validity, correlational analysis of single-item measure of harm perception.	This type of validity is demonstrated when the measure of the construct corresponds to other measures of the same construct. To assess this, we will correlate scores on each final measure with a single-item measure of harm perception of the product category (e-cigarettes or snuff) adapted from another government survey (questionnaire p. 8, Q25). Validity will be demonstrated if correlations are moderate and positive.

Sources:

Crano, W. D., Brewer, M. B., & Lac, A. (2014). Principles and methods of social research. Routledge.

DeVellis, R. F. (2016). Scale development: Theory and applications (Vol. 26). Sage publications.

This study also has two secondary goals that can inform the methods used in studies of modified risk tobacco products. The first secondary goal is to identify populations that may be susceptible to, or disproportionately influenced by, MRTP claims. To accomplish this, we will test whether the following variables moderate the effect of (i.e., interact with) study condition (control or modified risk information) in predicting the outcomes of risk perceptions (described above) or intentions to use the products (questionnaire p. 21):

1. Current tobacco product use (S2-S6)

2. Age (S1)

3. Race/ethnicity (Q51)

4. Whether or not they are currently serving in the U.S. armed forces (Q52)

5. Sex (Q55)

6. Interest in quitting smoking (Q5)

7. Whether or not participants have children under the age of 12 living in the house (Q49)

8. Educational attainment (S7)

9. Uncertainty about the risks of the product type (Q26-Q30)

These variables were selected because they relate to current tobacco use or previous research has found them to be associated with tobacco use (a-e), because they relate to interest in quitting smoking (f-g), because they may relate to difficulty understanding health information (h), and because they may indicate an openness to changing beliefs about product risk (i).

We will compute interaction terms between each of these variables and study condition to see whether they predict outcomes in linear regressions. This will assess whether these variables moderate the effect of seeing a claim on the outcomes. If significant, we will conduct separate linear regressions to assess the difference between the control and modified risk condition at different levels of the moderator (e.g., low, medium, and high level of certainty; for participants who do and do not have children under the age of 12 living in the house). This will allow FDA to see if the claim made a bigger difference in some levels of the moderator than others (e.g., perhaps modified risk claims make a bigger impression upon people who are uncertain about the harms of using the tobacco product). This can help industry understand what populations they can consider oversampling in studies conducted to support applications to market tobacco products with modified risk claims, in-line with FDA draft guidance recommending “Oversampling of populations that are particularly likely to be affected, positively or negatively, by the marketing of the product” (p.28 line 1105, https://www.fda.gov/downloads/TobaccoProducts/Labeling/UCM297751.pdf).

The second secondary goal is to assess the performance of two approaches for debriefing participants to increase the likelihood that they will be fully informed about key aspects of study participation (important because some participants will be exposed to modified risk claims on tobacco products that have not been authorized by FDA). To accomplish this, we will run a 2x2 ANOVA where we will assess the effect of simplifying the standardized debriefing, grouping relevant information in bubbles, and the combination of these strategies on debriefing knowledge. Debriefing knowledge will be assessed by number of correct responses to items asking about specific key concepts in the debriefing (items D1-D8 in the questionnaire). The method that leads to the best knowledge scores (number of correctly answered items about the debriefing) can be recommended as an option for the tobacco industry and public health community to consider using in their own studies that involve presenting modified risk claims that have not yet been authorized by FDA.

3. Use of Improved Information Technology and Burden Reduction

Automated information technology will be used in the collection of information for this study. One hundred percent (100%) of participants will self-administer the survey via a computer, which will record responses and provide appropriate probes when needed. In addition to its use in data collection, automated technology will be used in data reduction and analysis. Burden will be reduced by recording data on a one-time basis for each participant, and by keeping the written parts of surveys to 20 minutes or less per respondent on average in both the pretest and main study. Administration of the survey using web methods will help to contain costs, allowing for a sample that is geographically diverse without driving up interviewer costs for travel during data collection. Furthermore, the online screener and survey permit greater expediency with respect to data processing and analysis (e.g., a number of back-end processing steps, including coding and data entry, will be minimized). These efficiencies save time due to the speed of data transmission, as well as receipt in a format suitable for analysis. Finally, this technology permits respondents to complete the survey questions in privacy. Providing the respondent with a methodology that improves privacy reduces the potential for experiencing embarrassment or stigmatization when reporting on tobacco use behaviors and enhances response validity and response rates.

4. Efforts to Identify Duplication and Use of Similar Information

This study is designed to build on the existing body of literature about the effects of marketing claims on perceptions about tobacco products, and to complement, but not duplicate, other data collections. Specifically, it is one of three data collection efforts that support FDA’s ability to assess the effects of modified risk claims on (1) perceived harm, (2) perceived risk, and (3) perceived exposure to specific substances, which support FDA’s responsibilities as outlined in Section 911 of the FD&C Act. Because of the unique nature of the information to be collected, duplication of information is highly unlikely. One previous FDA data collection effort has been conducted on the effects of modified risk claims on perceptions of risk (OMB Control Number 0910-0819). This data collection tested whether expressed modified risk claims had different effects on consumers’ perceptions of risk depending on individuals’ current tobacco use status and, among current users of tobacco products, their preferred brands. FDA completed internal analyses and disseminated the results internally and externally within 24 months of approval. The contractor completed a final report in April of 2017. These results were presented internally in 2017, and were used to inform the Agency's understanding about how to implement Section 911 of the FD&C Act. Results were also presented externally at the February 2018 annual meeting of the Society for Research on Nicotine and Tobacco. Also, one other related FDA data collection is planned for the future. However, this other data collection will test the effects of implied (but not explicit) modified risk claims on perceptions of risk among individuals of different ages. Each of these data collection efforts is unique and will provide distinct information for use by FDA, companies, and tobacco researchers.

No other FDA data collections have focused on validating measures that can be used to evaluate the effects of explicit modified risk claims on perceptions of risk of e-cigarette or snuff use. Also, we have searched the published literature to assess whether researchers have already developed and validated measures in this area. We published the findings from our literature search (O’Brien, Persoskie, & Tam (2019). Multi-item measures of tobacco health perceptions: A review. American Journal of Health Behavior, 43, 266-278). We identified shortcomings in all previously published measures. When designing our measures for the proposed study, we built on the items published in prior research: We aggregated the items, eliminated duplicates, made additions, and reworded them where needed. For example, we added sets of items that allow participants to compare the risks from using e-cigarettes and snuff to the risks from using cigarettes and NRT. Based on our thorough review of the literature, we determined that the proposed study will contribute new and vital information to FDA, so FDA can provide guidance to industry on the design of consumer perception research in MRTPAs.

5. Impact on Small Businesses or Other Small Entities

Respondents in this study will be specific subpopulations of the public, not business entities. We do not anticipate any impact on small businesses or other small entities in terms of data collection burden. Results of the data collection may be useful to small and large businesses seeking to submit MRTPAs, if the studies provide evidence for the validity of measures of consumer perceptions of risk from using tobacco products.

6. Consequences of Collecting the Information Less Frequently

The proposed data collection is one-time only. There are no plans for successive data collections.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information that require the data collection to be conducted in a manner inconsistent with 5 CFR 1320.5(d)(2). The message testing activities fully comply with the guidelines in 5 CFR 1320.5.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60-day notice in the Federal Register on May 21, 2018 (83 FR 23464), requesting public comment on the proposed collection of information. FDA received four comments that were PRA related. Within those submissions, FDA received multiple comments which the Agency has addressed.

(Comment) Three of the comments were supportive of the usefulness and importance of the proposed data collection. These comments stated that validated measures of consumers’ health risk perceptions could be useful for FDA, researchers in the field, and industry--in particular, sponsors of modified risk tobacco product applications (MRTPAs). One of these comments expressed hope that the proposed study would be part of a more general effort by FDA to establish methods and standards for evaluating other aspects of MRTPAs.

(Response) FDA agrees with these comments to the extent they relate to this study.

(Comment) One of the comments was unsupportive of the proposed data collection, stating that it should not be undertaken for two reasons. The comment stated that the data are unneeded because U.S. consumers already understand the negative health effects of tobacco use and will not use a tobacco product if they are concerned about their health.

(Response) The proposed data collection focuses on consumer perceptions of modified risk tobacco products, which are products that are sold or distributed for use to reduce harm or the risk of tobacco-related diseases associated with commercially marketed tobacco products.

(Comment) A comment stated that the proposed data collection should not be undertaken because it would waste taxpayers’ money.

(Response) FDA believes this study will provide information important to its implementation of The Family Smoking Prevention and Tobacco Control Act. FDA also notes that the study is not funded by taxpayers’ money, but rather by industry user fees paid by regulated tobacco companies.

(Comment) One comment suggested that the proposed data collection should be guided by a theoretical approach.

(Response) The main objective of the data collection--developing and validating measures of consumer perceptions of tobacco health risks--is intentionally atheoretical. We intend for this aspect of the research to be data-driven rather than theory-driven. To accomplish this, we have created a large pool of risk perception items by aggregating items from all of the multi-item measures we could find in the published tobacco literature, putting them into the main categories of tobacco health effects that have been identified in prior health reviews, changing the wording of the items to put them in a common format, eliminating redundant or poorly worded items by consulting expert colleagues in medicine, epidemiology, and social science, and adding items to fill remaining gaps in terms of the main categories of tobacco health effects. When analyzing data from this proposed data collection, we plan to use factor analysis to identify the main dimensions underlying how U.S. consumers perceive tobacco product risks. Thus, overall, the goal of the proposed measurement development research is to comprehensively assess risk perceptions without overlaying our own preconceptions about how people may perceive these risks.

(Comment) One comment stated that the findings from our proposed analyses of moderation effects--in particular, the moderating effects of prior beliefs and the certainty with which those beliefs are held--should be considered exploratory, given that these effects are not well established in prior literature. Relatedly, another comment pointed out that the findings from these moderation analyses may only apply to moist snuff smokeless tobacco and e-cigarette products, given that these are the product types under study in this proposed data collection.

(Response) FDA agrees that the findings of these analyses will be novel in the tobacco literature, and we plan to encourage others to replicate and extend our findings. However, we also note that the measures used in this part of the study were adapted from measures developed and used previously in the attitude certainty literature, and the hypotheses about the potential moderating effects of belief certainty were developed based on prior studies of attitude certainty (Refs. 1 and 2). Thus, there is related literature that will help us interpret our findings on this topic.

(Comment) A comment encouraged FDA to consider how to account for participants’ prior beliefs when the tobacco product under study has not been previously marketed in the United States and is therefore unknown to U.S. consumers.

(Response) Our hypothesis would be that consumers may tend to be less certain about their beliefs about such unknown products, and therefore their beliefs about such products may be more susceptible to influence by modified risk information--but this is a hypothesis that has not been empirically tested. We agree that our findings from the proposed analyses of the moderating effects of prior beliefs will benefit from replication and extension by others.

(Comment) One comment suggested that we should consider making four changes to the proposed data collection methodology. First, this comment suggested modifying the study design to change it from a between-subjects design (i.e., in which participants are randomized to conditions and complete a posttest) to a mixed factorial design (i.e., in which participants complete a pretest, are randomized to conditions, and then complete a posttest). The comment stated that this modified design, described as a pretest-posttest-control-group design, would allow us to control for pretest scores, which would “explicitly minimize the potential threat to internal validity, namely, selection bias.”

(Response)

Including a pretest would not affect selection bias, which is when the sample is selected from the population in such a way that participants do not represent the population. If present, selection bias will already be an issue before participants start the survey even if the study uses a pre-post design. Selection bias does not affect differences between the experimental and control conditions as participants selected to participate in the study are randomly assigned to conditions and are expected to be equal at pretest (Crano et al., 2014).

Still, we considered the pros and cons of including a pretest. A potential upside to including a pretest is that it could allow us to control for within-person variance, increasing statistical power. However, because the survey is designed to be completed in one sitting, including a pretest would not work from a social science perspective. First, including a pretest would involve repeating all the outcome questions twice in one sitting—one in the pretest and once in the posttest. This would nearly double the length of the survey, which would greatly increase participant burden and would increase the dropout rate. Second, including a pre-test could have two different psychological effects, which could affect responses in two different ways. First, participants could anchor on their responses from the pretest and respond similarly in the posttest. Giving their opinion twice in a row, with an ad or package in between, would make the experimental manipulation transparent to participants (i.e., they would suspect that we are studying the effects of the ad or package on responses). Second, including a pretest introduces the problem of pretest sensitivity (Crano et al., 2014; Dimitrov & Rumhill, 2003). Specifically, people who spend a few minutes thinking through the specific health effects of using a product might respond differently to a package or ad that speaks to the health effects of using that product. This decreases the external validity of the study. For these reasons, we do not think the downsides of using a pretest outweigh the potential benefit (increased statistical power).

Crano, W. D., Brewer, M. B., & Lac, A. (2014). Principles and methods of social research. Routledge.

Dimitrov, D. M., & Rumrill Jr, P. D. (2003). Pretest-posttest designs and measurement of change. Work, 20(2), 159-165.

There are advantages and disadvantages to this alternative design type. Whereas the pretest-posttest-control-group design may help determine whether there is anything unusual about the sample that would reduce its representativeness of the target population (i.e., caused by biased selection), using this design would require participants to respond to the key measures twice within a short period of time. This would significantly lengthen the study, which is currently estimated to take approximately 20 minutes, and may influence how participants respond on the posttest (e.g., because of boredom or frustration with repetitive items, testing effects, or demand characteristics). Instead, we propose to use the original, between-subjects design and to conduct analyses to examine the sociodemographic and other characteristics of the sample to understand its representativeness of the U.S. population and to test the success of the randomization procedure.

(Comment) A comment suggested that we should consider using a newly developed measure of participants’ intentions to use tobacco products rather than the currently proposed intention items. The comment noted that the currently proposed items are based on prior research but stated that the new measure was developed and validated following procedures in FDA’s (2009) guidance on patient-reported outcome measures.

(Response) We appreciate this comment and support the continued development and validation of intention measures. However, at this time, we cannot use the commenter’s newly developed measure. Although the commenter stated that they used principles from an FDA document when developing and validating their own measure, the research evaluating the measure’s validity has not yet been published in a peer-reviewed journal.

(Comment) A comment suggested that this proposed data collection should assess many more of participants’ pre-existing beliefs and attitudes between the subjects receiving alternative package or advertising designs. As examples, the comment suggested assessing participants’ skepticism and perceived truthfulness of modified risk claims, stating that this would allow us to more fully capture the key constructs that explain why some people are more likely than others to recall and comprehend the claims.

(Response) As with the recommendations above, we appreciate this suggestion but propose not to assess these additional constructs in this data collection because of concerns about participant burden. The proposed data collection is not intended to comprehensively assess influences on consumer responses to modified risk claims. Rather, it is intended to achieve several specific goals such as developing measures and testing novel potential moderators of the effects of modified risk information. The constructs proposed in this comment have been studied in prior research, as have additional constructs such as brand loyalty (November 19, 2014 (79 FR 68888)). Assessing such constructs may be informative but is not required to achieve the goals of the current proposed data collection.

(Comment) To assist with this project’s aim of validating survey measures for assessing perceptions of risk, this comment recommended that the study should collect two types of evidence discussed in an FDA guidance on patient-reported outcome measures (FDA, 2009): evidence of the measures’ content validity, such as open-ended input from appropriate populations, and evidence of reliability, other aspects of validity, and sensitivity to detect change.

(Response) The proposed data collection is consistent with both these recommendations. As described above, to achieve content validity, we developed our initial pool of items to be as comprehensive as possible, consulting multi-item measures used previously in the tobacco literature, literature on the objective health effects of tobacco use, and expert colleagues. Additionally, we cognitively tested our pool of items in individual, qualitative interviews with tobacco users and non-users to evaluate their understanding of the items and beliefs about product risks. These interviews included open-ended questions, as recommended. Moreover, the proposed data collection is designed to test the performance of our measures on the criteria discussed in the comment, including internal consistency reliability, other aspects of validity (e.g., known groups, convergent, and discriminant validity), and sensitivity to detect changes (i.e., based on responsiveness to viewing advertisements with vs. without modified risk information).

(Comment) Lastly, one comment requested that we clarify how the proposed data collection will assist in measuring consumers’ understanding of modified risk information, in addition to their perceptions of health risk.

(Response) In our conceptualization, risk perceptions are a component of consumer understanding, which also includes other components. The goal of the present study is to develop and validate measures of understanding insofar as this construct includes people’s perceptions of absolute and relative health risks of using tobacco products.

The following individuals inside the agency have been consulted on the study design and questionnaire development:

Alexander Persoskie, PhD

Social Scientist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

10903 New Hampshire Avenue
Silver Spring, MD 20993

[email protected]

240-402-6864

Erin O’Brien, PhD

Social Scientist, Office of Science

Center for Tobacco Products

U.S. Food and Drug Administration

10903 New Hampshire Avenue
Silver Spring, MD 20993
[email protected]

240-402-2760

Donna N. Cheung

Regulatory Health Project Manager, Office of Science
Center for Tobacco Products

U.S. Food and Drug Administration

10903 New Hampshire Avenue
Silver Spring, MD 20993
[email protected]

240-402-5340

The following individuals outside of the agency have been consulted on study design and questionnaire development. Additionally, input has been solicited and received from FDA on the design of this study, including participation by FDA in meetings with OMB:

Jane A. Allen, MA

Center for Health Policy Science & Tobacco Research

RTI International

3040 E. Cornwallis Road
Research Triangle Park, NC 27709

[email protected]

919-597-5115

Sarah Parvanta, PhD, MPH

Center for Communication Science

Science in the Public Sphere Program

RTI International

3040 E. Cornwallis Road

Research Triangle Park, NC 27709

[email protected]

919-541-6045

James Nonnemaker, PhD

Center for Health Policy Science & Tobacco Research

RTI International

3040 E. Cornwallis Road
Research Triangle Park, NC 27709

[email protected]

919-541-7064

9. Explanation of Any Payment or Gift to Respondents

Respondents will receive compensation for survey completion in a non-monetary, virtual currency. Respondents can redeem their virtual currency for a large range of gift cards, points programs, and partner products or services. The vendor, Lightspeed, LLC, set the incentive rates. Respondents who complete the 20-minute survey will receive the virtual currency, the cash equivalent of which is between $0.25 and $7.00. Incentives vary from panel to panel and are primarily driven by the length of the survey, but also the incidence rate and screening criteria. The rates are consistent with what the same panels have offered in previous surveys conducted by the vendor. The lower end of incentives are awarded to those in high incidence rate (IR) targeted sample groups (targeted smokers) and the higher end to those we will sample from the general population at a lower IR to collect the necessary completes from the more challenging groups such as smokeless users. The contractor, RTI, has carefully considered the incentive plan and expects the incentive amounts to be high enough to recruit the participants needed (i.e., a sample of sufficient size with reasonable diversity in age, income and education), without being coercive.

10. Assurance of Privacy Provided to Respondents

In developing this study, CTP consulted the agency Privacy Officer to identify potential risks to the privacy of participants and other individuals whose information may be handled by or on behalf of FDA in the performance of this study. FDA designed the study to minimize privacy risks in keeping with the Fair Information Practice Principles (FIPPs) and applying controls selected from the National Institute of Standards and Technology (NIST), Special Publication 800-53, Security and Privacy Controls for Federal Information Systems and Organizations. CTP also identified privacy compliance requirements and coordinated with FDA’s Privacy Officer to ensure responsible offices in CTP satisfy all in accordance with law and policy. CTP submitted a privacy impact assessment which was approved by the FDA and HHS privacy offices. The PIA unique identifier number is P-4612305-112571.

Privacy Act Applicability

The information collection is not subject to the Privacy Act of 1974. Hence, no Privacy Act Statement is required to be displayed on the form, website, mobile application, or other point at which information is collected.

PII Collection

For respondent enrollment, PII will be collected on an as needed basis during the enrollment process. Mailing address or e-mail addresses may be collected for contacting the respondent regarding enrollment details (e.g., a survey link, directions). PII collected as part of respondent enrollment will not be maintained or linked to other study information. Contractors and subcontractors that collect data on behalf of FDA never pass along any PII to FDA. For these collections, we don’t have any systems where we maintain or retrieve PII.

All data collection activities will be conducted in full compliance with FDA regulations to maintain the privacy of data obtained from respondents and to protect the rights and welfare of human research subjects as contained in their regulations. All those who handle or analyze data will be required to adhere to the standard data security policies of RTI. The vendor – i.e., the subcontracted organization that will collect the data (Lightspeed, LLC) – will compile data from completed surveys into an SPSS data set and send to RTI for analysis, with no personally identifiable information (PII). The vendor will maintain all information that can identify individual respondents in a form that is separate from the data provided to RTI and FDA. RTI will keep the data it receives from the vendor in a secured fashion that will not permit unauthorized access. RTI will also share the data it receives from the vendor with FDA, which will also maintain the data in a secured fashion that will not permit unauthorized access. Confidentiality of the information submitted is protected from disclosure under the Freedom of Information Act (FOIA) under sections 552(a) and (b) (5 U.S.C. 552(a) and (b)), and by part 20 of the agency’s regulations (21 CFR part 20).

For study implementation, PII in the form of e-mail, IP addresses, or zip codes may be collected on an as needed basis for study implementation. This type of PII may be checked against respondent data to avoid duplicates and reduce fraudulent activity. If multiple e-mails have the same IP address, researchers will review the data, retain the first recorded response, and remove duplicates from the final analytical dataset. Researchers may also contact respondents to convey follow-up information about the study or if there is an issue with incentive delivery. PII collected as part of the study implementation will not be included in the dataset used for analysis and will not be sent to FDA. PII will only be used for primary study purposes (i.e., contacting, enrolling, and compensating study participants).

FDA has minimized the risk of unnecessary access, disclosure, use, or proliferation of PII about respondents. FDA’s subcontractor involved in the study collects and maintains study records containing PII only as long as required. PII will be removed before any data is sent to FDA. That PII may be linked to data by a code, only when necessary.

Notice and Transparency

Neither FDA nor direct contractors, including 3rd parties, share PII gathered via this collection with any other individuals or entities.

All subjects are provided notice regarding the collection and use of the information they submit. A panel provider may collect IP addresses when participants register for the panel, but FDA does not receive IP addresses. FDA and its contractors will notify participants if IP addresses are recorded. FDA sponsorship is explained to participants when appropriate (in some cases, FDA sponsorship will not be made known to respondents prior to data collection out of concern for the potential introduction of bias to study results; in such cases, FDA sponsorship will be made known after the data are collected.). Participants will be informed that their participation is voluntary at all times.

Prior to collecting any information, these methods will all be approved by FDA’s Research Involving Human Subjects Committee (RIHSC) and RTI’s Institutional Review Board (IRB). The primary concern of the RIHSC and IRB are to protect participants’ rights, one of which is maintaining the privacy of participant information to the fullest extent of the law.

Individual Participation and Control

While anonymity of respondents generally cannot be assured unless there is a statutory requirement associated with the information collection, information provided by respondents will be kept private and anonymous, to the extent allowable by law and the technology used as part of the informed consent process. See Attachment 5 for the informed consent text. This will be communicated to respondents by means of introductory letters, explanatory texts on the cover pages of questionnaires, telephone interviews, and consent forms. Respondents also will be advised of the following: the nature of the activity; the purpose and use of the data collected; FDA sponsorship (when appropriate); and the fact that participation is voluntary at all times. Because responses are voluntary, respondents will be assured that there will be no penalties if they decide not to respond, either to the information collection or to any particular questions.

Data Security

Contractors are required to maintain appropriate administrative, technical, and physical safeguards to ensure the security and confidentiality of records. User roles and responsibilities will determine the type and content of data and information necessary for job function (both PII and non-PII). Role-based access will determine and control who will have access to PII on an as-needed basis. Only personnel from the subcontractor conducting the information collection will have access to PII. All project staff from a contractor conducting the information collection must take required measures to ensure the privacy and anonymity of data. PII will be limited to information that may be required in the process of respondent enrollment. PII will be accessible to contractors on an as-needed basis and will not be linked to study data. All PII will be destroyed following data collection at the completion of the study.

Neither FDA employees nor any Federal employee of any other agency will have access to this information.

All electronic and hard copy data will be maintained securely throughout the information collection and data processing phases. While under review, electronic data will be stored in locked files on secured computers and hard copy data will be maintained in secure building facilities in locked filing cabinets. As a further guarantee of privacy and anonymity, all presentations of data in reports will be in aggregate form, with no links to individuals. Reports will be used only for research purposes and for the development of communication messages.

Interviews are typically considered exempt from the “Regulations for the Protection of Human Subjects” in accordance with 45 CFR 46.101(b)(3).

Before data are collected, FDA researchers will obtain either an exemption or a full approval for all research from FDA’s Investigational Review Board (IRB), the Research Involving Human Subjects Committee.

The research team understands that the security of online transmissions is not guaranteed due to the risk of interception by third parties or the possibility of monitoring software installed on research participants’ electronic devices. All participants will be assured that the information will be used only for research purposes and will be kept private to the extent allowable by law and the technology used. The survey consent form will include information explaining this to respondents. Participants will be assured that their names and e‑mail addresses will not be shared outside of the vendor (Lightspeed), and identifying information will not be associated with any response data. Participants will be told that the information obtained from all the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

The Internet panel includes a privacy policy that is easily accessible from any page on the site. A link to the privacy policy will be included on all survey invitations. The panel complies with established industry guidelines and states that members’ personally identifiable information will never be rented, sold, or revealed to third parties except in cases where required by law or with the consent of the panel members. These standards and codes of conduct comply with those set forth by American Marketing Association, the Council of American Survey Research Organizations, and others. All Lightspeed employees and contractors are required to take yearly security awareness and ethics training, which is based on these standards.

Data security provisions will involve the following:

• All data collection activities will be conducted in full compliance with FDA regulations to maintain the privacy of data obtained from participants and to protect the rights and welfare of human research participants as contained in their regulations. Participants will receive information about privacy protections as part of the informed consent process.

• All data entered via the Web-based survey system will be encrypted as the responses will be on a Web site with an SSL certificate applied. Data will be passed through a firewall at RTI and then collected and stored on a protected network share on the RTI Network. Only authorized RTI project staff members will have access to the data on the secure network share.

• Participants will be given a unique alphanumeric variable and will log onto Lightspeed’s secure server using a link provided by Lightspeed, with the result that no information about the respondent’s identity will be downloaded to or housed on RTI’s server.

• All participants will be assured that the information they provide will be maintained in a secure manner and will be used only for the purpose of this research. Participants will be assured that their answers will not be shared with family members and that their names will not be reported with responses provided. Participants will be told that the information obtained from all of the surveys will be combined into a summary report so that details of individual questionnaires cannot be linked to a specific participant.

Administrative safeguards include user training; system documentation that advises on proper use; implementation of Need to Know and Minimum Necessary principles when awarding access; and others. Technical Safeguards include use of multi-factor access authentication, firewalls, and network monitoring and intrusion detection tools. Physical controls include that all system servers are located at facilities protected by guards, locked facility doors, and climate controls. Other appropriate controls have been selected from the National Institute of Standards and Technology’s (NIST’s) Special Publication 800-53, as determined using Federal Information Processing Standard (FIPS) 199.

11. Justification for Sensitive Questions

The majority of questions asked will not be sensitive in nature. There will be no requests for a respondent’s Social Security Number (SSN). However, it will be necessary to ask some questions that may be considered to be sensitive in nature in order to assess specific health behaviors, specifically questions about tobacco use. While tobacco use may be a sensitive topic, it is important to ask respondents about their tobacco use because these questions determine participants’ eligibility for the study and assignment to study conditions (e.g., e-cigarette or snuff stimuli). Questions about behavior (e.g., smoking, attempts to quit smoking) and demographic information (e.g., race, income) could be considered sensitive but are not highly sensitive. The Screener is Attachment 4 and the Questionnaire is Attachment 6.

If a respondent does not take precautions to keep his or her answers confidential when completing the survey, it is possible that someone else in the household or other space within viewing distance of the respondent’s computer could view the respondent’s answers on the computer while the survey is in progress (e.g., if the participant walks away from the computer while completing the survey), which may make some participants feel uncomfortable. The consent form informs potential participants of this possibility.

To address other concerns about inadvertent disclosure of sensitive information, potential participants will be fully informed of the applicable privacy safeguards. This study includes a number of procedures and methodological characteristics designed to minimize potential negative reactions to these types of questions, including the following:

• Participants will be informed that they need not answer any question that makes them feel uncomfortable or that they do not wish to answer.

• Web surveys are entirely self-administered and maximize respondent privacy without the need to verbalize responses.

• In the Informed Consent, participants will be provided with a toll-free phone number (linking directly to the RTI IRB Office) to call if they have a question or concern about the sensitive issue. In the debriefing at the end of the survey, participants will be provided with a phone number and an email address through which they can contact FDA’s Research Involving Human Subjects Committee.

Finally, as with all information collected, these data will be presented with all identifiers removed.

12. Estimates of Annualized Burden Hours and Costs

12a. Annualized Hour Burden Estimate

As shown below, the survey invitation is expected to take no more than 1 minute per recipient, on average. The informed consent is expected to take no more than 2 minutes per respondent, on average, and the screener is expected to take no more than 4 minutes per respondent, on average, for a total of 6 minutes across the consent and screener. The questionnaire is expected to take no more than 20 minutes per respondent, on average. The Invitation is Attachment 3, the Screener is Attachment 4, the Informed Consent text is Attachment 5, and the Questionnaire is Attachment 6. This will be a one-time (rather than annual) collection of information. Exhibit 1 shows FDA’s estimates of the burden of this information collection. These estimates are based on FDA’s and the contractor’s experience with similar studies.

Exhibit 1. Estimated Annual Burden Hours

Participant Subgroup	No. of Respondents	No. of Responses per Respondent	Total Annual Responses	Average Burden per Responsea	Total Hoursa
Number to read the survey invitation
Adults (ages 26+)	29,000	1	29,000	0.02 (1 min)	580
Young adults (ages 18-25)	29,000	1	29,000	0.02 (1 min)	580
Total	58,000	1	58,000	0.02 (1 min)	1,160
Number to complete the consent and screener
Adults (ages 26+)	16,500	1	16,500	0.10 (6 min)	1,650
Young adults (ages 18-25)	11,000	1	11,000	0.10 (6 min)	1,100
Total	27,500	1	27,500	0.10 (6 min)	2,750
Number of survey completes
Adults (ages 26+)	3,300	1	3,300	0.33 (20 mins)	1,089
Young adults (ages 18-25)	3,300	1	3,300	0.33 (20 mins)	1,089
Total	6,600	1	6,600	0.33 (20 mins)	2,178
Total Hours	6,088

^aRounded from minutes to hours.

12b. Annualized Cost Burden Estimate

Respondents participate on a purely voluntary basis and are subject to no direct costs other than time to participate. There are also no start-up or maintenance costs. RTI has conducted many smoking-related surveys of similar length among adults. We have examined diagnostic data from each of these prior surveys and estimate that data collection for this study will take approximately 27 minutes per respondent, on average. According to U.S. Department of Labor Bureau of Labor Statistics, the average hourly wage in 2016 was $25.53 for U.S. adults over 18. Thus, assuming an average hourly wage for adults of $25.53, the estimated total opportunity cost to participants will be $155,427. The estimated annual value of respondents’ time for participating in the information collection is summarized in Exhibit 2.

Exhibit 2. Estimated Annual Cost

Type of Respondent	Activity	Annual Burden Hours	Hourly Wage Rate	Total Cost
Adults 18 and older in the United States	Invitation	1,160	$25.53	$29,615
Adults 18 and older in the United States	Consent and Screener	2,750	$25.53	$70,208
Adults 18 and older in the United States	Main Study	2,178	$25.53	$55,604
Total				$155,427

13. Estimates of Other Total Annual Costs to Respondents and Recordkeepers and Capital Costs

There are no capital, start-up, operating, or maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

This information collection is funded through a contract with RTI. The contract was awarded as a result of competition. The total estimated cost to the Federal Government for the collection of data is $353,445, as shown in Exhibit 3. This includes the costs paid to the contractors to program the study, draw the sample, and collect the data. Other activities outside this data collection include coordination with FDA, stimuli development, instrument development, reporting, and RTI IRB. Specific cost information other than the award amount is proprietary to the contractor and is not public information. The cost also includes FDA staff time to design and manage the study, to analyze the resultant data, and to draft a report ($34,915; 640 hours over the life of the study).

Exhibit 3. Itemized Cost to the Federal Government

Government Personnel	Time Commitment	Average Annual Salary	Total
GS-13	15%	$100,203	$15,030
GS-14	15%	$114,590	$17,189
GS-15	2%	$134,789	$2,696
Total Salary Costs			$34,915
Contract Cost			$318,530
Total			$353,445

15. Explanation for Program Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

As described previously, this research seeks to develop measures and methods to support the robustness of consumer perception research conducted to study tobacco product modified risk claims. This research has three goals, including one primary goal and two secondary goals:

1. Primary Goal: To develop and validate survey measures for assessing several specific risk perception constructs that can be used in future research related to studying modified risk tobacco products.

2. First Secondary Goal: To identify populations that may be susceptible to, or disproportionately influenced by, MRTP claims (such as those with high interest in quitting smoking; uncertainty about the risks of a product type; low educational attainment; current tobacco product use; age; whether they have children under age 12 living in the house; race/ethnicity; current U.S. armed forces service; sex) so that they can be overrecruited in future research related to studying modified risk tobacco products.

3. Second Secondary Goal: To assess the performance of two approaches to potentially improve the effectiveness participant debriefing, to increase the likelihood that participants will be fully informed about key aspects of study participation (important because this study will involve exposing participants to tobacco product modified risk claims that have not been authorized by FDA). The most effective approach can be used in future research related to studying modified risk tobacco products.

As discussed in Section 2, conventional statistical techniques, such as factor analysis, descriptive statistics, analysis of variance, and regression models, will be used to analyze the data. The primary goal will be assessed by conducting factor analyses, reliability analyses, t-tests, and correlations (see table of validity analyses in Section 2). The first secondary goal will be assessed by testing whether certain individual traits moderate the effect of the modified risk claims on risk perception and intention outcomes by including interaction terms in regression analyses. The second secondary goal will be assessed by conducting a 2x2 ANOVA to determine which of four debriefing formats (standard and simplified, with and without similar information grouped into bubbles) results in the highest debriefing knowledge scores.

The reporting and dissemination mechanism will consist of a draft and final report. Exhibit 4 shows the schedule for completing these deliverables.

Exhibit 4. Project Time Schedule
Task	Estimated Number of Weeks after OMB Approval
Main study data collected	22 weeks
Final results report completed	34 weeks

17. Reason(s) Display of OMB Expiration is Inappropriate

Not applicable. All data collection instruments will display the expiration date for OMB approval of the information collection.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

Not applicable. There are no exceptions to the certification statement.

15