Supporting Statement B - Experimental Study of Comparative Direct

Experimental Study of Comparative Direct-to-Consumer (DTC) Advertising

0910-Number

SUPPORTING STATEMENT B

Submitted by

Division of Drug Marketing, Advertising, and Communications

Center for Drug Evaluation and Research

Food and Drug Administration

November, 2011

Revised March, 2012

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

For the entire study, 7,810 (750 for the pretests, 7,060 for the main studies) participants will be recruited to participate in one of the studies. These individuals will include 2,010 participants from a panel of 4,001 prescreened individuals with osteoarthritis (n = 150 for pretest and n = 1,860 for main study) and 5,800 participants from a panel of 9,475 prescreened individuals with either high cholesterol (80% or more of the sample) or high body mass index (BMI; 20% or less of the sample), a proxy for interest in such medications (n = 600 for pretest and n = 5,200 for main test). Study invitations will be sent to individuals with these medical conditions in the existing Knowledge Network panel (see Appendix C for the study invitation and reminder emails).

Knowledge Networks will take the following steps:

1. Identify individuals with the two medical conditions at the rate of no more than one per household;

2. Randomly assign the panel into replicates and then release as many replicates as they think will be necessary under the most optimistic scenario;

3. After a short time in the field (somewhere between a few days and a week), re-evaluate the cooperation rate and then release additional replicates as needed to achieve the required number of completed interviews;

2. Procedures for the Collection of Information

Design Overview

The proposed research will occur in two concurrent phases. The goal of Phase 1 is to (a) explore how consumers understand and interpret print and broadcast ads that explicitly compare the efficacy of two similar drugs and (b) learn whether named comparisons are more likely than unnamed comparisons to promote accurate recall, comprehension, and perceptions. For the purposes of the research described here, named comparisons are ones in which the ad explicitly compares the drug’s efficacy to another named medication (e.g., Drug A was shown to be more effective than Drug B at lowering high cholesterol). Unnamed comparisons are ones in which the ad implicitly compares the drug’s efficacy to other medications (e.g., Compared to other medications, Drug A lowered cholesterol in more patients). These different types of comparisons will be examined in print and television ads and will include appropriate control conditions in a 2 (ad type: print or broadcast) x 3 (comparison type: named, unnamed, or none) design as shown below.

	Named Comparison	Unnamed Comparison	Control Group
Print Ad	Arm #1	Arm #3	Arm #5
Broadcast Ad	Arm #2	Arm #4	Arm #6

The goal of Phase 2 is to (a) determine if consumers infer that one drug is better or more effective than another from ads that include different types of drug label comparisons (i.e., indication, dosing, mechanism of action (MOA)), and (b) if consumers consider switching medications based on these comparisons in advertisements. We will examine three types of drug comparisons that are currently being used in DTC prescription drug ads. An indication-to-indication comparison highlights the approved indications of the advertised drug and the comparator drug (e,g., Drug X is approved to prevent and treat osteoporosis; Drug B is approved to treat osteoporosis). Dosing comparisons are those that compare the dosing schedule or dosing characteristics of two drugs (e.g., You can take Drug A in pill form; Drug B must be injected in a medical office). Finally, mechanism of action comparisons involve differences in the way the two drugs work (e.g., Drug A works by targeting the build up of fat in the arteries; Drug B works by targeting that fat and by disintegrating tangier cells in the esophagus).

We will also explore whether conveying these comparisons with visual images moderates these results. Half of the participants will examine a print ad and the other half will view a television ad. We propose two fully-factorial 2 (comparison type: named or unnamed) x 2 (visual: present or absent) x 3 (drug aspect: indication, dosing, mechanism of action) designs, one for print ads and one for television ads, as shown below. This design also includes two appropriate control groups.

For print ads:

Comparison	Visual	Indication	Dosing	Mechanism of Action	Control Group
Named	Visual	Arm #1	Arm #5	Arm #9	Arm #13
Unnamed	Visual	Arm #2	Arm #6	Arm #10
Named	No Visual	Arm #3	Arm #7	Arm #11
Unnamed	No Visual	Arm #4	Arm #8	Arm #12

For television ads:

Comparison	Visual	Indication	Dosing	Mechanism of Action	Control Group
Named	Visual	Arm #1	Arm #5	Arm #9	Arm #13
Unnamed	Visual	Arm #2	Arm #6	Arm #10
Named	No Visual	Arm #3	Arm #7	Arm #11
Unnamed	No Visual	Arm #4	Arm #8	Arm #12

Procedure

All parts of this study will be administered over the internet. A total of 7,810 interviews will be completed. Participants will be randomly assigned to view one version of a DTC prescription drug print ad or a prescription drug television ad. Following their perusal of this document or video, they will answer questions about their recall and understanding of the benefit and risk information, their perceptions of the benefits and risks of the drug, and their intent to ask a doctor about the medication.

Demographic and health literacy information will be collected. In addition, participants will answer questions about their familiarity with their medical condition and their need for cognition.

Need for cognition (NFC) is one aspect of an individual’s information processing style to take into account when determining the amount of detail to provide patients.¹ NFC refers to the extent to which individuals enjoy and engage in effortful thinking – some individuals enjoy effortful thinking (high NFC) whereas others prefer to avoid it whenever possible (low NFC).   Those individuals high in NFC tend to enjoy processing issue relevant details (e.g., facts and statistics), whereas low NFC individuals tend to rely on message cues (e.g., expert opinion) as their basis for judgment.²  It has been suggested that when providing medical information to individuals with low NFC the information should be simple and include a variety of message cues.³  We will measure NFC using items 38a-c on the questionnaire,⁴ and examine how it relates to benefit and risk perceptions, recall, and comprehension in comparative DTC ads. 

The entire procedure is expected to last approximately 20 minutes. This will be a one-time (rather than annual) information collection.

Participants

Data will be collected using an Internet protocol. Approximately 2,010 consumers who have osteoarthritis will be recruited for phase 1 of the study (n = 150 for the pretest and n = 1,860 for the main test). Approximately 5,800 consumers who have high cholesterol will be recruited for phase 2 of the study (n = 600 for the pretest and n = 5,200 for the main test). Because the task presumes basic reading abilities, all selected participants must speak and read English fluently. Participants must be 18 years or older.

Hypotheses

Phase 1

Research Question 1: To what extent does the presence of an efficacy comparison in DTC advertisements influence consumers’ perceptions, recall, and comprehension of a medication?

Research Question 2: To what extent does the presence of a named versus an unnamed efficacy comparison in DTC advertisements differentially influence consumers’ perceptions, recall, and comprehension of a medication?

Efficacy perceptions

Hypothesis 1: The presence of any efficacy comparison (named or unnamed) will lead consumers to perceive that the advertised medication is more efficacious than the presentation of efficacy information without a comparison (perceived efficacy).

Hypothesis 2: The presence of any efficacy comparison (named or unnamed) will lead consumers to perceive that the advertised medication is more likely to be beneficial than the presentation of efficacy information without a comparison (perceived likelihood of benefit).

Hypothesis 3: The presence of any efficacy comparison (named or unnamed) will lead consumers to believe the advertised medication is more efficacious relative to the comparator medication than the presentation of efficacy information without a comparison (relative efficacy).

We will explore the following research question:

• Does the presence of a named efficacy comparison lead consumers to have differential efficacy perceptions of the advertised medication than the presentation of efficacy information with an unnamed comparison (perceived efficacy, perceived likelihood of benefit, relative efficacy)?

Risk perceptions

We will explore the following research questions:

• Does the presence of any efficacy comparison (named or unnamed) lead consumers to have differential risk perceptions of the advertised medication than the presentation of efficacy information without a comparison (perceived risk, perceived severity of risk, relative risk, perceived likelihood of risk)?

• Does the presence of a named efficacy comparison lead consumers to have differential risk perceptions of the advertised medication than the presentation of efficacy information with an unnamed comparison (perceived risk, perceived severity of risk, relative risk, perceived likelihood of risk)?

Intentions

Hypothesis 4: The presence of any efficacy comparison (named or unnamed) will lead consumers to have greater intentions for behaviors related to the advertised medication than the presentation of efficacy information without a comparison (behavioral intentions).

We will explore the following research question:

• Does the presence of a named efficacy comparison lead consumers to have differential intentions for behaviors related to the advertised medication than efficacy information with an unnamed comparison (behavioral intentions)?

Comparative advantage

Hypothesis 5: The presence of any efficacy comparison (named or unnamed) will lead consumers to believe the advertised medication has a comparative advantage over other medications than the presentation of efficacy information without a comparison (comparative advantage).

Hypothesis 6: The presence of a named efficacy comparison will lead consumers to believe the advertised drug medication has a comparative advantage over the comparator medication than the presentation of efficacy information with an unnamed comparison (comparative advantage).

Recall

We will explore the following research questions:

• Does the presence of any efficacy comparison (named or unnamed) lead consumers to have differential recall of the advertised drug’s benefits than the presentation of efficacy information without a comparison (benefit recall)?

• Does the presence of any efficacy comparison (named or unnamed) lead consumers to have differential recall of the advertised drug’s risks than the presentation of efficacy information without a comparison (risk recall)?

• Does the presence of a named efficacy comparison lead consumers to have differential recall of the advertised drug’s benefits than the presentation of efficacy information with an unnamed comparison (benefit recall)?

• Does the presence of a named efficacy comparison lead consumers to have differential recall of the advertised drug’s risks than the presentation of efficacy information with an unnamed comparison (risk recall)?

Comprehension

We will explore the following research questions:

• Does the presence of any efficacy comparison (named or unnamed) lead consumers to have differential comprehension of the advertised drug’s benefits than the presentation of efficacy information without a comparison (benefit comprehension)?

• Does the presence of any efficacy comparison (named or unnamed) lead consumers to have differential comprehension of the advertised drug’s risks than the presentation of efficacy information without a comparison (risk comprehension)?

• Does the presence of a named efficacy comparison lead consumers to have differential comprehension of the advertised drug’s benefits than the presentation of efficacy information an unnamed comparison (benefit comprehension)?

• Does the presence of a named efficacy comparison lead consumers to have differential comprehension of the advertised drug’s risks than the presentation of efficacy information an unnamed comparison (risk comprehension)?

Phase 2

Research Question 1: To what extent do comparisons in DTC advertisements based on approved drug labels (i.e., indication, dosing, mechanism of action) influence consumers’ perceptions, recall, and comprehension of a medication?

Research Question 2: To what extent do named versus unnamed comparisons in DTC advertisements based on approved drug labels (i.e., indication, dosing, mechanism of action) influence consumers’ perceptions, recall, and comprehension of a medication?

Efficacy perceptions

Hypothesis 1: The presence of any indication or MOA comparison (named or unnamed) will lead consumers to perceive that the advertised medication is more efficacious than the presentation of any indication or MOA information without a comparison (perceived efficacy).

Hypothesis 2: The presence of any indication or MOA comparison (named or unnamed) will lead consumers to perceive that the advertised drug is more likely to be beneficial than the presentation of any indication or MOA information without a comparison (perceived likelihood of benefit).

Hypothesis 3: The presence of any indication or MOA comparison (named or unnamed) will lead consumers to believe the advertised medication is more efficacious relative to the comparator medication than the presentation of any indication or MOA information without a comparison (relative efficacy).

We will explore the following research question:

• Does the presence of any dosing comparison lead consumers to have differential efficacy perceptions of the advertised medication than the presentation of any dosing information without a comparison (perceived efficacy, perceived likelihood of benefit, relative efficacy)?

• Does the presence of a named indication, dosing, or MOA comparison lead consumers to have differential efficacy perceptions of the advertised medication than the presentation of indication, dosing, or MOA information with an unnamed comparison (perceived efficacy, perceived likelihood of benefit, relative efficacy)?

Comparative advantage

Hypothesis 4: The presence of any dosing comparison (named or unnamed) will lead consumers to believe the advertised medication has a comparative advantage over other drugs than the presentation of dosing information without a comparison (comparative advantage).

Hypothesis 5: The presence of a named dosing comparison will lead consumers to believe the advertised medication has a comparative advantage over the comparator medication than the presentation of dosing information with an unnamed comparison (comparative advantage).

Recall

We will explore the following research questions:

• Does the presence of any indication, dosing, or MOA comparison (named or unnamed) lead consumers to have differential recall of the advertised medication’s benefits than the presentation of any indication, dosing, or MOA information without a comparison (benefit recall)?

• Does the presence of a named indication, dosing, or MOA comparison lead consumers to have differential recall of the advertised medication’s benefits than the presentation of indication, dosing, or MOA information with an unnamed comparison (benefit recall)?

Comprehension

We will explore the following research questions:

• Does the presence of any indication, dosing, or MOA comparison (named or unnamed) lead consumers to have differential comprehension of the advertised medication’s benefits than the presentation of indication, dosing, or MOA information without a comparison (benefit comprehension)?

• Does the presence of a named indication, dosing, or MOA comparison (named or unnamed) lead consumers to have differential comprehension of the advertised medication’s benefits than the presentation of indication, dosing, or MOA information with an unnamed comparison (benefit comprehension)?

Research Question 3:

To what extent does a comparison in DTC advertisements based on approved drug labels (i.e., indication, dosing, mechanism of action) influence consumers’ intentions to switch to the advertised medication?

We will explore the following research questions:

• Does the presence of any comparison (named or unnamed) based on approved drug labels (i.e., indication, dosing, mechanism of action) lead consumers to have greater intentions for behaviors related to the advertised medication than the presentation of this information without a comparison (behavioral intentions)?

• Does the presence of a named comparison (named or unnamed) based on approved drug labels (i.e., indication, dosing, mechanism of action) lead consumers to have differential intentions for behaviors related to the advertised medication than the presentation of this information with an unnamed comparison (behavioral intentions)?

Research Question 4: To what extent does the presence of a visual aid in comparative ads based on approved drug labels (i.e., indication, dosing, mechanism of action) perceptions, recall, and comprehension of a medication?

We will explore the following research questions:

Efficacy perceptions

• Does the presence of a visual aid in comparative ads based on approved drug labels (i.e., indication, dosing, mechanism of action) lead consumers to have differential efficacy perceptions of the advertised medication than the presentation of efficacy information without a visual aid (perceived efficacy, perceived likelihood of benefit, relative efficacy)?

Recall

Hypothesis 6: The presence of a visual aid in comparative ads based on approved drug labels (i.e., indication, mechanism of action) will lead consumers to have greater recall of the advertised medication’s benefits than the presentation of indication or MOA information without a visual aid (benefit recall).

Comprehension

Hypothesis 7: The presence of a visual aid in comparative ads based on approved drug labels (i.e., indication, mechanism of action) will lead consumers to have greater comprehension of the advertised medication’s benefits than the presentation of indication or MOA information without a visual aid (benefit comprehension).

Analysis Plan

The following analysis plan pertains to both the drug efficacy comparisons design and the drug label comparative claims design.

For hypotheses regarding drug efficacy comparisons, we will test whether there is a main effect of comparison type (named/unnamed) on our main dependent variables (e.g., perceived efficacy, perceived risk, and behavioral intentions) using one-way ANOVAs (2 comparison type conditions, plus control condition). We will conduct ANOVAs that assesses the main effect of comparison type (named/unnamed), the main effect of ad type (print/broadcast), and the interaction between comparison type and ad type on our main dependent variables. We will conduct ANOVAs both with and without covariates (e.g., demographic characteristics, source credibility) included in the model. In addition, we will test whether effects are moderated by other measured variables (e.g., health literacy, need for cognition). If a main effect is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above).

For hypotheses regarding drug label comparative claims, we will test whether there is a main effect of comparison type (named/unnamed) on our relevant dependent variables using one-way ANOVAs for each of the 3 drug label claims (indication, dosing, mechanism of action). We will conduct ANOVAs that assesses the main effect of comparison type (named/unnamed), the main effect of visual type (visual/no visual), the main effect of ad type (print/broadcast), and the interaction between comparison type, visual type, and ad type on our main dependent variables. We will examine these analyses both with and without covariates (e.g., characteristics, source credibility) included in the model. In addition, we will test whether the main effect is moderated by other measured variables (e.g., health literacy, need for cognition).

Power

The following assumptions were made in deriving the sample size for the study: 1) 0.90 power, 2) 0.05 alpha or 0.0125 alpha (Bonferroni-adjusted for four comparisons) and 3) an effect size between small and medium. The table below shows the sample size required to detect differences with effect sizes ranging from conventionally “small” (f = 0.10) to “medium” (f = 0.25) for the comparison between the named group and unnamed group. Because our strictest analysis in both phases involves one degree of freedom and two groups, the following table applies to both phases.

Table 7. Power Analysis Calculation.

A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, main effects, and interactions) to achieve power of 0.90 (Faul et al., 2007).5
		Effect size f*			Effect size f*
Input
		0.10	0.15	0.25	0.10	0.15	0.25
	α error probability	0.05	0.05	0.05	.0125	.0125	.0125
	Power (1 – β error probability)	0.90	0.90	0.90	0.90	0.90	0.90
	Numerator df	1	1	1	1	1	1
	Number of groups	2	2	2	2	2	2
Output
	Critical F	3.85	3.86	3.89	6.25	6.27	6.34
	Denominator df	1,050	466	168	1,429	635	229
	Sample size per cell	527	235	86	716	319	116

*An effect size of 0.10 is traditionally considered small, whereas an effect size of 0.25 is considered medium (Cohen, 1988).⁶ Here we have shown three different effect sizes centering around small to medium effects.

For Phase I, we will have 310 participants per cell, with a total of 1,860 participants in the 6 cells represented in the table (a 2 x 3 design). With this sample size, we will be able to detect small to medium effects with an unadjusted p-value of .05 and medium effects with a Bonferroni-adjusted p-value of .0125.

The pretest for Phase I will involve 150 participants. This will enable us to have 37 or 38 participants per cell, which will allow us to address our main pretest concerns: (a) that the stimuli function properly; (b) participants perceive the stimuli as realistic; and (c) participants notice the experimental manipulations, especially the comparative claims and the visual aids. Because we plan to only use the pretest as descriptive in planning for the main study, we have not conducted a power analysis.

For Phase II, we will have 200 participants per cell, with a total of 5,200 participants in the 26 cells represented in the table (two 2 x 2 x 3 + 1 designs). With this sample size, although we will not be able to detect effects as small as the previous phase, we will still be able to detect somewhat small to medium effects with an unadjusted p-value of .05 and medium effects with a Bonferroni-adjusted p-value of .0125. Given the sheer number of individuals required in this design, we erred on the side of a smaller but adequate number of individuals per cell for this design.

The pretest for Phase II will involve 600 participants, 300 who will see the print ad and 300 who will see the television ad. To keep the number of participants low while still accomplishing our goals of addressing the stimuli and experimental manipulations, we will test a mix of manipulations and ad formats. Specifically, we will include 50 participants each in the following cells:

For print ads:

Comparison	Visual	Indication	Dosing	Mechanism of Action	Control Group
Named	Visual		n = 50
Unnamed	Visual		n = 50
Named	No Visual	n = 50		n = 50
Unnamed	No Visual	n = 50		n = 50

For television ads:

Comparison	Visual	Indication	Dosing	Mechanism of Action	Control Group
Named	Visual	n = 50		n = 50
Unnamed	Visual	n = 50		n = 50
Named	No Visual		n = 50
Unnamed	No Visual		n = 50

We plan to use the pretest as descriptive in planning the main study, however, we will have the sample size to detect medium size effects with power = .80 and alpha = .10 (see below):

A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, main effects, and interactions) to achieve power of 0.80 (Faul et al., 2007).7
		Effect size f*
Input
		0.10	0.15	0.25
	α error probability	0.10	0.10	0.10
	Power (1 – β error probability)	0.80	0.80	0.80
	Numerator df	1	1	1
	Number of groups	2	2	2
Output
	Critical F	2.71	2.72	2.76
	Denominator df	618	277	99
	Sample size per cell	310	139	50

3. Methods to Maximize Response Rates and to Deal with Issues of Non-Response

This experimental study will use an existing Internet panel to draw a sample.  The panel comprises individuals who share their opinions via the Internet regularly.  The participation rate for two previous studies conducted using the Knowledge Networks panel was 65% (Toll-Free, OMB Control No. 0910-0652; Quantitative, OMB Control No. 0910-0663).. To help ensure that the participation rate is as high as possible, FDA and the contractor will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the experiment over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Email a reminder to the respondents who do not complete the protocol four days after the original invitation to participate is sent;

• Provide a toll-free hotline for respondents who may have questions or technical difficulty as they complete the experiment. 

4. Test Procedures

First, nine participants will complete the procedure to assess blatant glitches in questionnaire wording, programming, and execution of the study. We will also conduct pretests with 750 participants from the same target populations as the main studies before collecting data for the main studies to ensure that stimuli and questionnaire wording is clear. Finally, we will run the main studies as described elsewhere in this document.

5. Individuals Involved in Statistical Consultation and Information Collection

The contractor, RTI International, will collect the information on behalf of FDA as a task order under the Quick-Turn-Around Research Services contract. Pam Williams, Ph.D., is the Project Director for this project, telephone (919) 316-3936. Data analysis will be conducted by RTI and by the Research Team, Division of Drug Marketing, Advertising, and Communications (DDMAC), Office of Medical Policy, CDER, FDA, and coordinated by Amie C. O’Donoghue, Ph.D., 301-796-0574, and Helen W. Sullivan, Ph.D., M.P.H., 301-796-0569.

1 Cacioppo, J.T., & Petty, R.E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131.

2 Cacioppo, J.T., Petty, R.E., Feinstein, J.A., Blair, W., & Jarvis, G. (1996). Dispositional differences in cognitive motivation: the life and times of individuals varying in need for cognition. Psychological Bulletin, 119, 197-253.

3 Williams-Piehota, P., Schneider, T.R., Pizarro, J., Mowad, L., & Salovey, P. (2003). Matching health messages to information-processing styles: Need for cognition and mammography utilization. Health Communication, 15, 375-392.

4 Cacioppo,J.T., Petty, R.E., & Kao, C.F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306-307.

5 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A, (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

6 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

7 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A, (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.