0874 Supporting Statement Part B 2020

Part B. Statistical Methods

1. Respondent Universe and Sampling Methods

The targeted sample will comprise of adults who have a current asthma diagnosis, who report a lifetime incidence of asthma, or who experience a large number of asthma symptoms. These groups are believed to be the most likely to be targeted by disease awareness and product promotion ads for asthma.

The exclusion criteria for participating will be the following:

1. Reporting fewer than three current asthma symptoms or never diagnosed with asthma;

2. Training or employment as a health care professional;

3. Employment with a pharmaceutical company, an advertising agency, a market research company, or the Department of Health and Human Services (HHS); and

4. Participation in market research within the past three months on the topic of prescription drug advertising.

Given the research objectives, a nonprobability sample will be obtained for the two pretests and the two main studies. FMG will work with ProdegeMR for Study 1 and Study 2 to recruit U.S. adults fitting the inclusion and exclusion criteria discussed above. Both panel providers are responsible for identifying and inviting study participants. FMG will oversee all fielding, hosting, and quality assurance for Study 1 and Study 2 pretest and main studies, in addition to performing the majority of the programming. FMG will also work closely with both panel providers to maximize completion rates for both studies.

2. Procedures for Collection of Information

This research is being conducted to determine how the similarity, temporal positioning, and frequency of exposure to disease awareness communications and prescription drug television promotion impact consumer perception and understanding of the benefits and risks of a prescription drug product. These objectives will be achieved using two experimental studies. The first study will explore the impact on consumer perception and comprehension of different levels of temporal separation between the disease awareness communication and prescription drug promotion within a single period of television programming, as well as the level of similarity versus distinctiveness between these communication types. Temporal separation is defined as the spacing or proximity between the disease awareness communication and prescription drug promotion in the hour-long programming, for example, if they are shown back-to-back or if they are separated by other ads or television programming. Similarity/distinctiveness is defined by variations between the disease awareness communication and prescription drug promotion, including visual and presentation elements such as the setting, actors, and colors. The second study will experimentally examine the impact of disease awareness communication temporal separation and exposure frequency on consumer perception and comprehension. Temporal separation in this second study again refers to the spacing or proximity between the disease awareness communication and prescription drug promotion but is operationally defined as either one day or one week. Exposure frequency is defined as the number of times that participants will view the disease awareness communication, either one, three, or six times. The results of this latter study will examine the practice of “seeding the market,” in which pharmaceutical companies release disease awareness communications before releasing product promotion communications. Similarity versus distinctiveness will also be examined in this study.

We propose the following hypotheses for this research:

Study 1:

H1: Increased perceptual similarity between a disease awareness communication and a prescription drug promotion will result in significantly more conflation of the information presented in both pieces.

H2: Increased temporal proximity between a disease awareness communication and a prescription drug promotion will result in significantly more conflation of the information presented in both pieces.

Study 2:

H1: Increased frequency of exposure to a disease awareness communication before exposure to a prescription drug promotion will result in significantly more conflation of the information presented in both pieces.

H2: Increased temporal proximity between a disease awareness communication and a prescription drug promotion will result in significantly more conflation of the information presented in both pieces.

H3: Increased perceptual similarity between a disease awareness communication and a prescription drug promotion will result in significantly more conflation of the information presented in both pieces.

In each instance, conflation is defined as the extent to which an individual remembers and attributes benefits to a product that is based on information presented in a disease awareness communication and not in the drug promotion.

To address these hypotheses, Study 1 will employ a 3x4 incomplete factorial design in which participants are randomly assigned to one disease awareness communication condition, plus one control condition where participants will not view a disease awareness communication. The extent to which the disease awareness communication is perceptually similar to the product promotion communication will vary, as will the temporal separation of the disease awareness communication and product promotion communication. Table 1 depicts our design visually. Table 2 outlines the timing and sequence of the advertisements for each of the ad proximity experimental conditions.

Table 1.--Study 1 Experimental Design

Disease Awareness Ad	Perceptual Similarity to Product Ad	Disease Awareness and Product Ad Temporal Separation
		Back to back	Within same commercial pod1	In neighboring commercial pods	In non-neighboring commercial pods
Yes	Similar
	Semi-similar
	Distinct
No	N/A

Table 2.--Study 1 Sequence

Condition

Sequence

6min

2min

5min

2min

5min

2min

5min

2min

6min

2min

5min

2min

5min

2min

5min

Back to back

DA, P

DA, P

Same pod

DA, P

DA, P

Neighboring pods

DA

P

DA

P

Non-neighboring pods

DA

P

DA

P

Control

P

P

TV Program

Commercial Pod

DA = Disease Awareness Communication; P = Product Promotion

Study 2 will employ a 2x2x3 factorial design in which participants are randomly assigned to one disease awareness communication condition. The varying factors in Study 2 are the temporal separation between the disease awareness and product promotion communication, the number of exposures to the disease awareness communication, and the perceptual similarity of the disease awareness communication to the product promotion communication. Table 3 visually depicts our design. Of note, to reduce the overall number of experimental conditions for Study 2, no semi-similar experimental condition is used. Table 4 provides details about the timing and sequence of the disease awareness and product ad exposures.

Table 3.--Study 2 Experimental Design

Time Delay Until Product Ad Exposure (Temporal Separation)	Perceptual Similarity of Ads	Exposures to Disease Awareness Ad
		One Exposure	Three Exposures	Six Exposures
One Day	Similar
	Distinct
One Week	Similar
	Distinct

Table 4. Study 2 Sequence

Disease awareness ad exposure phase

Product ad exposure phase

Day

1

2

5

6

9

10

11

12

13

14

15

16

17

Delay

Similarity

Six exposures

1 day

similar

x

x

x

x

x

x

x

distinct

x

x

x

x

x

x

x

1 week

similar

x

x

x

x

x

x

x

distinct

x

x

x

x

x

x

x

Three exposures

1 day

similar

x

x

x

x

distinct

x

x

x

x

1 week

similar

x

x

x

x

distinct

x

x

x

x

One exposure

1 day

similar

x

x

distinct

x

x

1 week

similar

x

x

distinct

x

x

Analysis Plan

For both pretests and main studies, our planned analyses are designed to address the key hypotheses. For both Study 1 and Study 2, we anticipate that the primary analysis will be analysis of variance (ANOVA) to compare the main and interaction effects of the experimental factors.

In Study 1, the first analysis will examine the main effect of disease awareness similarity to product ad on conflation. A one-way ANOVA will first test for significant between-group differences, whereas subsequent planned contrasts (with the control condition as the comparison condition) will examine which disease awareness similarity condition results in increased conflation as compared to the control. Once the main effect of perceptual similarity is established, the second analysis will examine the main effect of disease awareness ad proximity. A planned linear contrast will be able to identify a potential linear relationship between proximity and conflation. This analysis will test the hypothesis that increased proximity between a disease awareness and product ad will increase conflation. Planned comparisons will then indicate which proximity condition results in significantly higher conflation as compared to the control.

In addition to examining main effects, between-subjects ANOVA with perceptual similarity and ad proximity as independent factors will examine any interaction effects between these two factors, such as whether any observed linear relationship between proximity and conflation is stronger for more perceptually similar ads.

For Study 2, we will first test the hypotheses that increased time delay between disease awareness ad exposure and product ad exposure decreases conflation. Then we will test the hypothesis that increased disease awareness ad exposure increases conflation—a planned linear contrast will be able to identify a potential linear relationship between exposure frequency (one, three, or six exposures) and conflation. Any observed main effect of perceptual similarity (similar vs. distinct) will be able to be compared to, and potentially replicate, the main effect of similarity observed in Study 1.

The 2x2x3 between-subjects ANOVA will then be examined for all possible interaction effects:

• Delay x Perceptual similarity;

• Delay x Exposure;

• Exposure x Perceptual similarity

• Delay x Exposure x Perceptual similarity

Interaction effects between exposure and delay, as well as exposure and perceptual similarity, will identify whether delay and perceptual similarity moderate the effect of exposure frequency on conflation. A three-way interaction effect will observe how the interaction between delay and exposure changes for disease awareness ads that are similar to versus distinct from product ads.

Power

We conducted a priori power analyses to ensure we obtained a sufficient sample to detect statistically significant differences in the outcome measures of interest across the different experimental conditions.

Power analyses for Study 1 are based on the incomplete factorial design depicted in Table 1. The power analyses also assume that the analyses conducted on the pretest data would mirror the analyses conducted on the main study data.

Study 1 Pretest: The Study 1 pretest, assuming the need for power of .80, alpha probability of .05, and a medium effect size (f = .25), requires a sample of 270 participants.

Study 1 Main Study: The main study, given the experimental design, and assuming a power of .90, alpha of .05, and a small effect size (f = .10), requires obtaining a sample of 2,105 participants.

Study 2 Pretest: The Study 2 pretest, assuming the need for power of .80, alpha probability of .05, and a medium effect size (f = .25), requires a sample of 158 participants.

Study 2 Main Study: Given the experimental design, and assuming a power of .90, alpha of .05, and small effect size (f = .10), this study requires a sample of 1,269 participants.

3. Methods to Maximize Response Rates and Deal with Non-response

For Study 1, ProdegeMR estimates survey drop-off between 25–30% based on studies with similar length and incentive. ProdegeMR recommends notifying participants of the study time commitment and incentive amount as a strategy to reduce non-response and maximize response rates. ProdegeMR offers a panel of over 17 million members and maintains a database of over 1,000 demographic and behavioral attributers that can be used for sample targeting. Participants will be invited to take part in the study if they meet inclusion criteria and the demographic soft quotas we proposed, increasing the availability of demographic characteristics for the full survey sample, which will increase the options for assessing the risk of non-response bias. We also implemented the attention check items for Study 1 to ensure participant attentiveness throughout the stimuli viewing, with more correct responses resulting in a higher total incentive.

For Study 2, there may be additional retention challenges due to the recurring ad exposures. ProdegeMR estimates a 48% retention rate to view the final ad and complete the survey, and a 70% retention rate to view subsequent ad exposures following initial screening. Similar to Study 1, soft quotas will be used based on cross-classifications of key demographic variables such as gender, age, race/ethnicity, and education. ProdegeMR offers a panel of over 17 million members and maintains a database of over 1,000 demographic and behavioral attributers that can be used for sample targeting. ProdegeMR also invites panel members to participate in surveys, which will increase the availability of demographic characteristics for the full survey sample, thus increasing the options for assessing non-response bias risk. Study 2 follows a graduated payment schedule with participants receiving small amounts following initial screening and for every ad exposure they view, with the largest incentive provided upon completion of the survey.

4. Test of Procedures or Methods to be Undertaken

Two types of pretesting (qualitative and quantitative) are employed as a test of procedures and methods.² The first type of pretesting—already conducted—is qualitative. Cognitive testing with nine individuals was used to refine study stimuli and questions. Additionally, as described in this package, one round of quantitative pretesting for each study will be employed. Pretesting will be used to evaluate the procedures and measures used in the main studies. The pretests will have the same design as the main studies. The primary purpose of the pretests will be to test the questionnaire’s format, the data collection protocol, statistical measures, and any other considerations that may arise. Based on pretest findings, we will refine the survey questions and data collection process, as necessary, to optimize the full-scale study conditions.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing

Data

The contractor, Fors Marsh Group, will collect and analyze data on behalf of FDA as a task order under Contract HHSF223201510003B. Brian Griepentrog, Ph.D., is the Project Director, (571) 858-3757. Review of contractor deliverables and supplemental analyses will be provided by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Kevin R. Betts, Ph.D., (240) 402-5090, and Kathryn Aikin, Ph.D., (301) 796-0569.

1A commercial pod refers to a group of ads into which the test ad is inserted, designed to simulate an advertising break during a television program. As depicted in Table 2, by neighboring commercial pods, we mean commercial pods separated only by television programming and no other commercial pods. By non-neighboring commercial pods, we mean commercial pods separated by both television programming and one or more (one, as studied here) other commercial pods.

2 Pretesting is suggested by OMB as a method to test procedures. See Office of Management and Budget Standards and Guidelines for Statistical Surveys (September, 2006). Available at http://www.whitehouse.gov/sites/default/files/omb/assets/omb/inforeg/statpolicy/standards_stat_surveys.pdf. Last accessed January 12, 2012.

5