Supporting Statement_Part B_OMB

Supporting Statement PArt b

0910-NEW

Food and Drug Administration’s Investigation of Consumer Perceptions of Expressed Modified Risk Claims

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The study sample will consist of approximately 2,640 current cigarette smokers, 1,320 current e‑cigarette users, 1,320 current snuff users, and 1,320 current non-users of tobacco. In each of these tobacco user groups, we seek to enroll 50% older adults (aged 26 or older) and 50% young adults (aged 18 to 25). See Exhibit 1 for the sample breakdown.

Exhibit 1. Projected Sample Breakdown by Age and Tobacco Use Status

	Sample Size
Population	Goal
Adults (aged 26 or older)
Current cigarette users	≈1,320
Current e‑cigarette users	≈660
Current snuff users	≈660
Not current tobacco users	≈660
	≈3,300
Young adults (aged 18 to 25)
Current cigarette users	≈1,320
Current e‑cigarette users	≈660
Current snuff users	≈660
Not current tobacco users	≈660
	≈3,300
Study Total	6,600

Respondents to be included in the study will be members of the Lightspeed LLC (Lightspeed) online survey panel. The Lightspeed panel consists of a convenience sample of adults nationwide. Panel inclusion is by invitation only, and Lightspeed invites only pre-validated individuals with known characteristics to participate in their consumer panels. The convenience sample provided by the panel is considered suitable for the purposes of this study. This study is based on predetermined quotas. Individuals who are not members of the Lightspeed panel at the time of survey administration will not be invited to participate in this study. Only those who are invited to the study will have the means to access it, so it is not possible for the non-panel members or uninvited panel members to participate.

Invited panel members who decline to participate in the study during the consenting process will be excluded from the study. Lightspeed respondents who consent will be screened into the study using a series of survey items to determine eligibility based on age and tobacco use. Eligibility assessment will consist of 7 screener items, depending on participants’ responses. These items will determine whether individuals are within the age range of the study; whether they are current cigarette, e-cigarette, or snuff users, or current nontobacco users, and what their educational attainment is. Educational attainment is measured in the screener, although it is not an inclusion criteria, so we can oversample on lower education to determine if this population responds differently to modified risk claims compared to people with higher educational attainment. The screener includes the minimum number of items needed to assess study eligibility and develop the sample needed to achieve study goals. Respondents who choose not to complete a screener item will be ineligible for the study and will be excluded at the point of non-response (rather than after answering all screener items). Furthermore, individuals will be excluded from the study if we have already met the desired sample size for their tobacco use category and age group. The final analytic sample will include only eligible respondents who, after consenting to participate and completing the screener items, complete all critical elements of the survey.

Study Design and Power Calculations

Respondents will be randomized to one of the 24 conditions outlined in Exhibit 2. Each condition will yield data from approximately 275 respondents. Two product types will be studied: snuff and e-cigarettes. Each product type will be represented by three brands and will be depicted in two forms: product packaging and product advertising. Half of the stimuli will include modified risk claims, and the other half will not include claims. In addition, all of the snuff packaging and advertising will include one of four warning labels, which are intended to function as a control variable and are therefore not considered discrete conditions (not shown in Exhibit 2). The product packaging and product advertising stimuli are provided in Attachments 7a and 7b, respectively.

Respondents will be randomized to one condition based on their self-reported tobacco behavior (current cigarette smoker, current e‑cigarette user, current smokeless tobacco user, and not a current tobacco user). For the purpose of the study, we will assign individuals to one tobacco behavior category, even if they are users of multiple product types. To achieve this, we apply the following rules: Individuals will be assigned to a category that is representative of their tobacco use behavior. Users of multiple product types will be assigned to the category with the lowest national prevalence until we have achieved the desired sample for that category. Thus, we will fill the smokeless tobacco user category first, followed by the e‑cigarette user category, the cigarette user category, and the non-tobacco user category.

In addition to being randomized to a modified risk claim condition, all respondents will be randomized to one of four debriefing conditions at the end of the survey (not shown in Exhibit 2), which vary in regard to the product type viewed in the study (e‑cigarettes or snuff) and the presentation format of the debriefing (block of text vs. text bubbles).

Exhibit 2. Conditions and Sample Size for Each Condition

Condition	Snuff			E‑cigarettes
	Brand 1	Brand 2	Brand 3	Brand 1	Brand 2	Brand 3
Pack
Claim	n=275	n=275	n=275	n=275	n=275	n=275
No Claim	n=275	n=275	n=275	n=275	n=275	n=275
Ad
Claim	n=275	n=275	n=275	n=275	n=275	n=275
No Claim	n=275	n=275	n=275	n=275	n=275	n=275

We have investigated the power for the assumed sample sizes for a range of effect sizes calculated by using Cohen’s d. These power analyses suggest that we have sufficient power to detect differences in outcomes for comparisons across the main categories of this study (for example, we can compare outcomes across age and education groups). For these comparisons, we would be able to detect an effect size (d) of 0.2 or more with 80% power. For comparisons by pack/ad or snuff/e cigarette conditions, the power decreases somewhat. However, for pack/ad or snuff/e-cigarette conditions, we would still have adequate power to detect an effect size of 0.25 (or more) with 80% power.

As with any study conducted using opt-in online panels, this study may be subject to several threats to external validity that limit the generalizability of study results. Panelists are recruited into the online panel using convenience sampling methods and thus do not have a known probability of selection into the panel. When inviting panelists to participate in the study, the vendor (Lightspeed) will seek diversity in age, education, and race/ethnicity to ensure a reasonable degree of inclusiveness on key demographic characteristics. FDA will not generate nationally representative results of the targeted sub-population in this study. Note that generating a representative sample for this study would be costly and is not required for the study’s aims. The study will use convenience samples rather than probability samples, and despite the diversity in the sample, the sample in the study is nevertheless still a convenience sample and not representative of national estimates or necessarily of the underlying study panel used. These limitations in generalizability do not affect the internal validity of the study. Such limitations will be noted in the context of describing the results of the study.

2. Procedures for the Collection of Information

Recruitment, Screening and Consent

Sampling, recruitment, and data collection will be conducted by Lightspeed. Lightspeed will recruit study participants through an invitation to its existing online panel. The text of this invitation is shown in Attachment 3. Panel members will take the survey in their own homes or other locations that they choose.

Invitations will be targeted towards panelists who, according to the database, have characteristics that match our study criteria (e.g., tobacco users, nontobacco users). Adult panelists may be contacted in several ways: Survey invitations are distributed via email, but also presented in each panelist’s portal, accessible on computers, tablets, and mobile devices (including via the Lightspeed LifePoints app). Respondents are presented with a blind link to the survey, presented with the estimated length of survey and the proposed point reward. This helps to ensure screening questions are not given away with an initial description. After clicking the online survey link, adult participants will encounter the consent form. If they consent – by clicking a box indicating that they have read the provided material and agree to participate – they will then complete the study screener. Adults who screen into the survey will view one of the experimental images (see Exhibit 2 above) and complete the remaining survey items.

3. Methods to Maximize Response Rates

The survey will use an existing Internet panel to draw a sample. The panel (described in B.1) comprises individuals who share their opinions via the Internet regularly. To help ensure that the participation rate is as high as possible, FDA and the contractor:

• have designed a protocol that minimizes burden (short in length, clearly written, and with appealing graphics); and

• will administer the survey over the Internet, allowing respondents to answer questions at a time and location of their choosing.

Furthermore, Lightspeed will target recruitment to panelists who are likely eligible for the study based on key characteristics including age and tobacco status.

The aim of these methods is to minimize the number of people who spend time on viewing the invitation or completing the screening but are ultimately ineligible to participate in the study based on the inclusion criteria.

4. Test of Procedures or Methods to be Undertaken

RTI will conduct rigorous internal testing of the online survey instrument prior to data collection. The research team will review the online test version of the instrument that we will use to verify that instrument skip patterns are functioning properly, experimental stimuli exposure is working properly, and that all survey questions are worded correctly and are in accordance with the instrument approved by OMB.

5. Individuals Consulted on Statistical Aspects and Individuals Collection and/or Analyzing Data

On behalf of FDA, the contractor, RTI, will work with Lightspeed to collect the data and will then analyze the data as a task order under Contract HHSF223201110005B. Jane Allen, MA., 919-597-5115, is the Project Director for this project. Data analysis will be overseen by the Research Team, including FDA project leads Alexander Persoskie (240-402-6864) and Erin O’Brien (301-796-9335).

4