OMB Supporting Statement Part B

Experimental Study on Consumer Perceptions of Modified Risk Tobacco Products (MRTPs)

B. Statistical Methods

1. Respondent Universe and Sampling Methods

The respondent universe for the experimental studies is as follows:

Study 1: (1) established smokers aged 25 years and older, (2) former smokers aged 25 years and older, (3) established smokers aged 18 to 24 years, (4) experimental smokers aged 18 to 24 years.

Study 2: (1) current smokeless tobacco users aged 25 years and older, (2) adult smokers aged 25 years and older who have never used smokeless tobacco, (3) young adult smokeless tobacco users aged 18 to 24 years, (4) young adult smokers aged 18 to 24 years who have never used smokeless tobacco.

Study 3: (1) current adolescent smokers aged 13 to 17 years, (2) adolescents age 13 to 17 years, who may be susceptible to smoking initiation.

The ten separate quota samples will be selected from the GMI online member panel, a national opt-in email list sample. Sampled panel members will receive an email inviting them to participate in the study. Panel members who choose to participate will complete the questionnaire (see study screener and questionnaire in Appendix A). Completed interviews will be monitored to ensure samples are diverse in terms of age, gender, education, and ethnicity/race. We estimate a total of 3,000 respondents will complete a questionnaire for the three studies combined.

The agency does not intend to generate nationally representative results or precise estimates of population parameters from the experimental study; generating a representative sample of the size necessary for this study, using Random Digital Dialing (RDD) or other similar method, would be cost prohibitive. The study will use convenience samples rather than probability samples. Despite the attempt to match between the study’s sample and the respondent universe in four demographic characteristics, matching is used solely to produce a sample with a reasonable degree of diversity in key demographic characteristics.

2. Procedures for the Collection of Information

For the information collection, GMI will send email invitations to the target audiences using their market research panel. Each invitation will contain the survey title, the length of the survey, incentive amount provided for successful completion of the survey, and instructions for accessing the secure website for the survey. Once a panel member enters the secure web site, a brief introduction will be presented informing the panel member of the confidential and voluntary nature of the study. Individuals who consent to participate in the survey will be able to access the survey by clicking on the link to the survey URL. Respondents who access the questionnaire will be randomly assigned to an experimental condition.

This experimental study will be conducted using an Internet panel and a questionnaire designed to measure responses to the MRTP information and collect smoking status information from the participant. Participants in each study will be randomly assigned to view one of three versions of the MRTP claim stimuli, which will consist of product packaging and an ad that displays one of the two claims (experimental condition) or no claim (control). For study stimuli, see Appendix B. In Study 1, the effect of brand on perceptions of the MRTP claim will be examined by randomly assigning participants to view the stimuli for either (a) their own brand, (b) a brand other than their own, or (c) a novel brand. Participants in Study 2 will view the MRTP claim stimuli for a hypothetical smokeless tobacco product, and Study 3 participants will view stand-alone MRTP claim stimuli (i.e., no product package or ad). RTI, the contract research organization implementing the study, will analyze information collected from the three studies, the results of which will inform the Agency’s efforts to implement section 911 of the Federal Food, Drug, and Cosmetic Act, which gives FDA authority to regulate MRTPs.

Study 1:

In this experiment, we will test several different MRTP claims on cigarettes, including both exposure modification (EM) and risk modification (RM) claims, and one no-claim control condition. We will also examine the effect of brand on participant perceptions of the MRTP claims by randomly assigning participants to view the claims attached to either (a) their own brand, (b) another brand (i.e., a real brand that is not their preferred brand), or (c) a novel brand. In order to experimentally manipulate brand as a factor, we will select participants who currently or formerly preferred one of these three brands: Marlboro, Camel, or Newport. Because the stimuli must be prepared in advance of the study so that we can assure random assignment, we had to select a finite number of brands and select our sample accordingly. We have selected these three brands because they represent the majority of the market share; this will be maximally inclusive and representative of the majority of smokers. In turn, we have created stimuli for these three cigarette brands (plus a hypothetical brand). Thus, participants can be randomly assigned to see their own brand (e.g., a Marlboro smoker views a Marlboro stimulus); another real brand (e.g., a Marlboro smoker views a Camel stimulus); or a hypothetical brand (e.g., a Marlboro smoker views the Durham stimulus). The proposed design implements the experimental study with each of four target groups in study 1: (1) established smokers aged 25 years and older, (2) former smokers aged 25 years and older, (3) established smokers aged 18 to 24 years, (4) experimental smokers aged 18 to 24 years. Each cell will contain 50 respondents for a baseline sample size of 1800.

Study 1 employs a two-by-three-by-three factorial design with 200 participants in each claim-by-brand condition, yielding a total of 1800 participants. The first factor, smoking status, describes whether or not the participant is a current established, experimental or former smoker. The second factor, brand type, describes the product brand on which the claim was displayed. The brand types will be used include: participant’s own brand, a brand other than the participant’s brand, and a novel brand. The third factor, claim type includes MRTP claims about cigarettes, including both EM and RM claims, and a no-claim control condition. Our primary analyses will compare groups of participants exposed to different brand and claim combinations.

Claim Type	Smoking Status	Population	Own Brand	Other Brand	Novel Brand	Total
EM (n=600)	Established Smoker	Young adult	50	50	50	150
		Adult	50	50	50	150
	Experimental Smoker	Young adult	50	50	50	150
	Former Smoker	Adult	50	50	50	150
RM (n=600)	Established Smoker	Young adult	50	50	50	150
		Adult	50	50	50	150
	Experimental Smoker	Young adult	50	50	50	150
	Former Smoker	Adult	50	50	50	150
No Claim (n=600)	Established Smoker	Young adult	50	50	50	150
		Adult	50	50	50	150
	Experimental Smoker	Young adult	50	50	50	150
	Former Smoker	Adult	50	50	50	150
Total (n=1800)			600	600	600	1800

Study 2:

In this study, we will test several different MRTP claims on SLT, including both exposure modification (EM) and risk modification (RM) claims, and a no-claim control condition Participants will view images of packages and ads for a hypothetical brand of SLT. The proposed design implements the experimental study with each of four target groups in study 2: (1) current smokeless tobacco (SLT) users aged 25 years and older, (2) adult smokers aged 25 years and older who have never used smokeless tobacco, (3) young adult smokeless tobacco users aged 18 to 24 years, (4) young adult smokers aged 18 to 24 years who have never used smokeless tobacco. Each cell will contain 50 respondents for a baseline sample size of 600.

Study 2 employs a three-by-two design with 200 participants in each smokeless tobacco user type-by-claim condition, yielding a total of 600 participants. The first factor, describes the participant’s tobacco use status, specifically whether the participant is a current smokeless tobacco user or never used smokeless tobacco. The second factor, claim type, includes MRTP claims about smokeless tobacco, including both EM and RM claims, and a no-claim control condition. Our primary analyses compare groups of participants exposed to different claims by smokeless tobacco user type.

Smokeless Tobacco Use	Population	Claim – EM	Claim –RM	No Claim	Total
Current User (n=300)	Young adult	50	50	50	150
	Adult	50	50	50	150
Never User1 (n=300)	Young adult	50	50	50	150
	Adult	50	50	50	150
Total (n=600)		200	200	200	600

Study 3:

In this experiment, we will test two different MRTP claims on cigarettes, including one exposure modification (EM) and one risk modification (RM) claim, and a no-claim control condition. Adolescent participants will see standalone MRTP claim stimuli, not on packs or ads for tobacco products, or no claim (control). The proposed design implements the experimental study with each of the two target groups in Study 3: (1) current adolescent smokers aged 13 to 17 years, and (2) adolescents age 13 to 17 years who may be susceptible to smoking initiation. Each cell will contain 100 respondents for a baseline sample size of 600.

Study 3 employs a three-by-two design with 200 participants in each tobacco use status-by claim condition, yielding a total of 600 participants. The first factor, describes the participant’s tobacco use status, specifically whether the participant is a current tobacco user or susceptible to tobacco use. The second factor, claim type includes MRTP claims including both EM and RM claims, and a no-claim control condition. Our primary analyses compare groups of participants exposed to different claims by tobacco use status.

Tobacco Use	Claim – EM	Claim –RM	No Claim	Total
Current User (n=300)	100	100	100	300
Susceptible (n=300)	100	100	100	300
Total (n=300)	200	200	200	600

Summary of protocol for all three studies

• Survey screener – confirm eligibility; for Study 1 – identify preferred cigarette brand, to determine condition assignment and fill questionnaire items pertaining to own brand.

• Random assignment to treatment or control.

  • Participants in the experimental groups will view one of two MRTP claims. In Study 1, participants in experimental groups will view the claim on a pack and ad for a brand that is either on (a) their own preferred brand (identified on Screener), (b) another real brand; or (c) a hypothetical brand. In Study 2, all claims will be displayed on a hypothetical brand pack and ad. In Study 3, participants (adolescents) will view the statement alone, and will not be exposed to product packs or ads. Control groups are not exposed to MRTP stimuli claims.

• Subsequent to exposure to claims (or not), participants will answer questions that assess their reactions to the products they viewed and about tobacco products. The questions will assess consumers’ reactions to hypothetical MRTPs, including their understanding of the product, their attitudes and beliefs about it, and their interest in trying and using the product. The control group will be asked all the same questions except those pertaining to the specific MRTP product claim.

Measures

Key Outcomes (measured post exposure):

• Beliefs and Understanding

  • Beliefs about product

  • Product appeal

  • Perceived harm/risk (absolute and comparative)

• Behavioral intentions (intentions to use and purchase product)

• Quit intentions

Covariates and controls:

Age, gender, race, SES (income and education)

In developing the study questionnaire, we have drawn items from existing resources including national surveys (e.g., HINTS, NATS) as well as the published literature. In some cases, items were adapted for the current study. Where necessary, original items were written for this study, such as those specific to the study stimuli (i.e., RM/EM claims). To avoid duplication, some demographic information will be ascertained from the participant database maintained by GMI, rather than included in the study questionnaire.

Analysis plan

Primary analyses

1. Tests of effects of claims on consumer perceptions of product.

• Hypothesis: The presence of a claim (RM/EM) will impact consumer perceptions of the product such that they will perceive it to be less harmful and will report greater intention to try the product.

2. Tests of brand effects (Study 1 only)

• Hypothesis: The presence of a claim on one’s own brand will have a greater impact on a consumer’s (favorable) responses to the product, compared to a claim on another brand or on a novel brand.

Secondary analyses (Exploratory)

1. Examine impact of claim type (RM vs. EM) on primary outcomes.

2. Examine the impact of claims on specific product beliefs and evaluate understanding.

3. Examine potential moderating effects on primary outcomes by: tobacco use status (established vs. experimental vs. former in Study 1; smokeless tobacco user or not, in Study 2) and heaviness of use; age group; and psychological characteristics (openness to new products; concern about health risks).

4. Examine the relationships between product appeal, perceived harm, and intentions to use the product.

The sample design is adequately powered to test the primary research hypotheses:

The presence of a claim (RM/EM) will impact consumer perceptions of the product such that they will perceive it to be less harmful and will report greater intention to try the product.

The sample design is powered to test the primary research hypothesis that consumers who are exposed to claims that a tobacco product reduces risk or exposure to a harmful substance will perceive the tobacco product to be less harmful and will report greater intention to try the product, than consumers who are exposed to the tobacco product without the claim. Sample size calculations for the three studies are the same, and are described below.

For the purpose of sample size calculations, the study outcome is based on an aggregated intention to use index, a continuous measure derived from the set of intention to use questions. Assuming an index with a range of 0 – 100, a mean value of 50, and a standard deviation of 15, tests of the research hypothesis will have 80% statistical power to detect a significant difference of 4.2 points or greater between any two groups of participants viewing different brand-by-claim conditions. In more general terms, this difference equates to minimum detectable effect of 0.28 standard deviation units.

Unusual Problems Requiring Specialized Sampling Procedures

No specialized sampling procedures are involved.

Use of Periodic Data Collection Cycles to Reduce Burden

This is a one-time survey data collection effort.

3. Methods to Maximize Response Rates

Experience with online experimental studies suggests that about 14% of adults and 10% of adolescents who are sent survey invitations will complete a study. FDA will implement several procedures to maximize participation. We have conducted cognitive interviews and will conduct a pretest to help ensure understandability of the questionnaire, to reduce participant burden, and to enhance interview administration. We will keep the study questionnaire at a reasonable length to minimize break-offs. Additionally, the following procedures will be used to maximize cooperation and to achieve the desired response rates:

• A brief introductory paragraph will identify FDA as the sponsor of the study, state the purpose of the study, and encourage participation.

• GMI will provide toll-free telephone numbers to all sampled individuals and invite them to call with any questions or concerns about any aspect of the study. RTI will provide a toll-free telephone number for a RTI project member and a toll-free telephone number for the RTI IRB hotline should participants have any questions about the study or their rights as a study participant.

• GMI data collection staff will work with RTI project staff to address any problems that arise throughout the course of the collection of information.

• Nonrespondents will receive one e-mail reminder from GMI requesting their participation in the survey.

As with any study conducted online, this study may be subject to non-response bias, particularly a self-selection bias. However, the impact of such biases on study conclusions is mitigated by the experimental design of the study, which ensures random assignment of participants to condition. As in a randomized clinical trial, study conclusions are based on comparisons between experimental (and control) conditions.

4. Tests of Procedures or Methods

RTI conducted 9 cognitive interviews for each of the three studies with adult smokers and adult smokeless tobacco users to evaluate and refine the draft questionnaires (see Appendix A). The cognitive interviews identified areas where the instruments were ambiguous, burdensome, or confusing for respondents and the surveys have been revised accordingly.

Additionally, RTI will conduct a pretest with survey panelists from GMI to thoroughly test the programmed questionnaire. At the conclusion of the pretest, all strategies, algorithms, and programs for sampling, survey administration and data compilation will be tested, validated, and readied for launch of the Internet experimental survey. The questionnaire will be revised based on the pretest findings.

5. Individuals Involved in Statistical Consultation and Information Collection

RTI International will manage the information collection on behalf of FDA. Dr. Carol Schmitt is the project director at RTI. RTI will subcontract to GMI to collect the data. Ryan Barry is the project manager at GMI. Analysis and dissemination of the data will be led by Dr. Sarah Johnson at FDA’s Center for Tobacco Products.

Sarah E. Johnson, Ph.D.

Food and Drug Administration
Center for Tobacco Products
Document Control Center
Building 71, Room G335
10903 New Hampshire Avenue
Silver Spring, MD 20993-0002

(301) 796-6892

[email protected]

Carol L. Schmitt, Ph.D.

RTI International

701 13^th Street NW, Suite 750

Washington, DC 20005-3967

(202) 728-2046

[email protected]

Ryan Barry

GMI (Global Market Institute, Inc.)

1100 112^th Ave NE, Suite 200

Bellevue, WA 98004

(508) 503-3653

[email protected]

1 All never users of smokeless tobacco will be current smokers.

9