PRA DTC television advertise 12 2009 part B

Experimental Study: Toll-Free Number for Consumer Reporting of Drug Product Side Effects in Direct-to-Consumer Television Advertisements for Prescription Drugs

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The universe for this experimental study is members of the Knowledge Networks Internet panel.  Knowledge Network’s Internet panel consists of 50,000 adult panel members who are recruited by random-digit dialing (RDD) or by using address-based sampling. The sample is nationally representative and statistically accurate. Typical panel members receive 3-4 invitations per month to participate in research projects. 

The 5,250 participants for this study would be drawn from the pool of 50,000 panel members.  Quotas will be used so that the overall sample is in proportion to the U.S. adult population on gender and race/ethnicity.  A range of participants over the age of 30 will be selected to approximate a reasonable sample of individuals for whom high blood pressure may be a concern.  At least 20% of the sample will have achieved a high school education or less. 

FDA does not intend to generate nationally or locally representative results or precise estimates of population parameters from this study.  The sample used is a convenience sample, rather than a probability sample.  Despite the attempt to match between the study’s sample and known population characteristics, matching is used solely to produce samples with a reasonable degree of diversity in key demographic characteristics.  Furthermore, no legitimate weights can be constructed from non-probability samples such as the one used here.  Hence, the Agency does not construe this sample or the results generated from this sample as nationally or locally representative.  Rather, the strength of the experimental study lies in its internal validity, on which meaningful estimates of differences across conditions can be produced and generalized.

Overview

This study will examine the placement of the toll-free statement and the length of time the statement is presented on-screen in a DTC television advertisement for a prescription drug. The primary dependent measure of interest is consumer comprehension of the important risk information in the advertisement. This study will also examine potential differences in comprehension based on the wording of the toll-free statement and the prominence of the statement.

The application of a new piece of information for viewers of DTC ads presents logistical challenges. From a research perspective, the primary issue under investigation is how to impart additional information without increasing “cognitive load,” thus leading to information overload. Cognitive load is an index of the memory demands necessary to process a set of information.¹ As cognitive load increases, more mental resources are necessary to process and understand the information. DTC ads are already quite dense when compared to ads for other products. The risk information in the major statement of the ad should not be compromised by the addition of the toll-free statement. At the same time, it is preferable that the risk information and the toll-free statement information are presented in such a way that both are understandable. We have chosen a set of variables in the current study to investigate issues of cognitive load. They are described briefly below before examining the details of the research design.

1. Placement

The location of the toll-free statement may facilitate or detract from the risk information in the major statement. We have chosen three locations for this information to test which location results in the greatest communication of the risks of the drug and the concept that side effects can be reported. It is possible that locating the toll-free statement before the major statement provides a “prime” for the risk information that follows; that is, the mention of side effects in the toll-free statement will cause consumers to start thinking about side effect-related information, which facilitates comprehension of the risk information that follows. In this case, the two conceptual pieces of information may flow together easily. Conversely, it is possible that the toll-free statement confuses consumers or provides no information for them because they have not yet heard any risk information. Thus, without context, the statement lacks applicability.

Placing the toll-free statement during the major statement likely reduces the comprehension of the risk information for the drug because it divides viewer’s attention between two competing pieces of information. It is possible, however, that the juxtaposition of these two informational concepts are complimentary and therefore do not conflict.

The toll-free statement may serve the best role after the risk information has been presented. In this case, participants have been told about the risks and side effects of the drug before they are told they may report this information. This essentially primes the toll-free statement with the major statement. We do not expect this placement to interfere with the comprehension of risk information, as it is not present during the voicing of risks and has not been introduced to viewers at this point. In addition, the usefulness of the toll-free statement may improve in this condition relative to those discussed above because viewers have been provided with context.

Over time, it is likely that the toll-free statement will become part of the background of the ads as people become accustomed to seeing this statement in all DTC ads. In this respect, people will have the statement as an option if needed but will be able to disregard it to focus on the risk information when desired. Thus, we are testing a condition in which the toll-free statement will be present during the entire ad. This test condition will control for the effect of novelty arising from the fact that consumers have not previously seen this type of statement in TV ads. Presence of the statement during the entire ad may increase noticeability of the toll-free statement initially, but will be unlikely to interfere with risk information in the long run.

2. Statement wording

The second variable, statement type, will have two executions of statement language: the language from the FDAAA versus the language used in the final Toll-Free Number for Reporting Adverse Events on Labeling for Human Drug Products Rule (TFNR) and previously tested by FDA. The wording from these two statements is as follows:

1. “You are encouraged to report negative side effects of prescription drugs to the FDA. Visit www.fda.gov/medwatch, or call 1-800-FDA-1088.” (FDAAA)

2. “Call your doctor for medical advice about side effects. You may report side effects to FDA at 1-800-FDA-1088 or www.fda.gov/medwatch.” (TFNR)

We think it is important to test both the toll-free statement version in FDAAA and the version that we have previously tested with actual consumers. The most obvious reason for this is to make sure that the statement is maximally readable and understandable. It may be valuable, however, to test two statements for another reason.

If the toll-free statement is enacted in broadcast ads, it is possible that because of the boilerplate language, some amount of habituation will occur. That is, after viewers have seen the same language in multiple ads for multiple products, they may “tune out” and not pay attention to the toll-free statement at all. If we test two versions of the statement and find both acceptable, it would be possible to either allow sponsors to choose one statement versus another or to suggest some alternating of the two statements. This is a long-term idea, however, and finding appropriate wording is the primary goal of investigating this variable.

3. Duration

Congress specifically mandates that we investigate the duration of the display of the toll-free statement. As with placement, the length of time the toll-free statement is presented on-screen may influence the cognitive load in the ad. For experimental control, we will look at the duration of the statement while holding placement in the ad (after the major statement of risks) constant. Although this placement should not interfere with the processing of the risk information, it is possible that the duration influences the take away message from the ad. For example, having the statement on for a short amount of time may not give consumers enough time to read and process the message, resulting in lower comprehension of the message but have no impact on the comprehension of the risk information. Alternatively, displaying the toll-free statement for a longer period of time may wipe away memory traces of the risks from the major statement, resulting in lower risk comprehension. Whether this longer duration increases the usefulness of the toll-free statement itself is an empirical question. We will compare these short and long durations to instances where the toll-free statement is present during the entire ad and when there is no toll-free statement at all.

4. Prominence

In addition to superimposing the toll-free statement on the screen during the ad, there are other methods available to increase the prominence of the statement. In particular, having the statement read aloud in the ad voiceover while the statement is on the screen may be considered particularly prominent. Does the additional prominence of the statement compromise the comprehension of the risk information in the major statement? If not, does the additional prominence result in a greater understanding of the toll-free statement itself? It is likely that there is a tradeoff between the gains of emphasizing the toll-free statement and the comprehension of the risk information, given the limited cognitive capacity of viewers. In examining this variable, we are exploring the parameters of this tradeoff.

Design

The design will consist of three parts. Part one will be a between-subjects factorial design examining the placement of the toll-free statement by the type of statement. The first variable, placement, will have four levels: before the major statement of risks, during the major statement of risks, after the major statement of risks, or continuously throughout the whole ad.

In each condition the toll-free statement will appear in the ad as superimposed text at the bottom of the screen. We will also include a control condition in which the statement does not appear.

Part One: Placement by Statement Type

4 x 2 + 1

	Statement Type
Placement	FDAAA	TFNR
Before major statement of risks
During major statement of risks
After major statement of risks
During the whole ad

Plus:

Control (no toll-free statement)

Part two of the study will examine four variations in the duration of the toll-free statement using the language from the FDAAA: short (on screen for approximately 3 seconds after the major statement), long (on screen for approximately 6 seconds after the major statement), on screen during the whole ad, and the control condition of no toll-free statement included. These times were adopted by calculating how long it would take a person reading at an average reading speed to read the statement. As in the first part of this study series, the toll-free statement will appear as superimposed text and a control condition in which the toll-free statement does not appear will be included.

Part Two: Duration*

4 x 1

Short (On screen for approximately 3 seconds after major statement)

Long (On screen for approximately 6 seconds after major statement)

During the Whole Ad

Control (no toll-free statement)

*Using FDAAA statement

Part three of the study will examine two variations in the prominence of the toll-free statement using the language from the FDAAA: Spoken after the major statement with only the website and phone number in superimposed text, and a control condition where the toll-free statement is presented visually after the major statement.

Part Three: Prominence*

2 x 1

Extra Prominent (spoken after major statement of risks, website and phone number on screen)

Control (after major statement of risks)

*Using FDAAA statement

We will investigate these issues in one disease condition, high blood pressure, because high blood pressure has a high incidence rate in the population, is a public health concern, and is likely to occur in both males and females. Further, because there is little broadcast promotion for prescription treatment of high blood pressure at this time, participants should be less familiar with DTC television ads for this type of drug, reducing the potential influence of prior experience.

Our primary dependent variable is comprehension of the risk information mentioned in the major statement. In addition to this variable, we will also examine comprehension of benefit information. We will also examine the noticeability and comprehension of the toll-free statement.

Procedure

Participants will see an advertising pod of four ads: two 15-second non-DTC ads (fillers), an approximately 60-second DTC ad for a fictitious high blood pressure medication, and a 30-second DTC ad for an unrelated medical condition with the same toll-free statement included. We include two DTC ads with the toll-free statement in our protocol because this better approximates what will happen if this statement is enacted. That is, viewers will see the statement in all DTC ads for all products. In this study, we want to avoid the suggestion that there is something particular about the high blood pressure drug class that causes the statement to be mandated. Thus, we will show multiple DTC ads but ask questions regarding only the ad which has been manipulated to test our hypotheses. To maximize response information, the test ad will always be the last ad they see.

After viewing the ads, a structured interview will be conducted. Participants will answer questions about the high blood pressure DTC test ad they have seen. Questions will examine a number of important perceptions about the advertised product, including risk comprehension, risk recall, benefit comprehension, benefit recall, behavioral intention, noticeability of the toll-free statement, and recall of the toll-free statement.

Finally, demographic and health care utilization information will be collected. The entire procedure is expected to last approximately 20 minutes. A total of 5,250 interviews will be completed. This will be a one-time (rather than annual) information collection.

Participants

Data will be collected using an Internet protocol. Two samples of consumers will be recruited: one sample of individuals diagnosed with high blood pressure and another sample of consumers over the age of 21. Both groups will represent a range of education levels. Because the task presumes basic reading abilities, all selected participants must speak English as their primary language.

FDA proposes to conduct two rounds of pretesting with 675 consumers in each round to refine the questionnaire and the stimuli before fielding the main study.

Hypotheses

Overall, we expect effects to be stronger in the high blood pressure sample than in the general population sample, as high blood pressure sufferers will likely have higher involvement with the medical condition.

Risk Comprehension

• Any inclusion of the toll-free statement will reduce the comprehension of risk information.

(Risk comprehension will be highest in control condition for all analyses)

• Placement:

-Conditions in which the statement is presented after the major statement and the statement is present for the whole ad will reduce comprehension least.

(After control condition, risk comprehension will be highest in conditions where statement is present for whole ad or after the major statement; risk comprehension will be lowest when statement is presented during or before the major statement).

• Wording: Type of statement will not influence risk comprehension.

• Placement x Wording: This analysis is exploratory

• Duration:

-Statement will interfere with risk comprehension less when presented in the whole ad than when presented for briefer periods.

-Short duration will result in lower risk comprehension than long duration because it will be displayed for a short time, causing attention to shift twice in quick succession

(Risk comprehension highest in control condition, followed by whole ad condition followed by long duration, and, finally, short duration)

• Prominence: Prominence of statement will not affect risk comprehension.

Benefit Comprehension

• Any inclusion of the toll-free statement will reduce the comprehension of benefit information.

(Benefit comprehension will be highest in control condition for all analyses)

• Placement:

-Conditions in which the statement is presented after the major statement and the statement is present for the whole ad will reduce comprehension least.

(After control condition, benefit comprehension will be highest in conditions where statement is present for whole ad or after the major statement; benefit comprehension will be lowest when statement is presented during or before the major statement).

• Wording: Type of statement will not influence benefit comprehension.

• Placement x Wording: This analysis is exploratory

• Duration:

-Statement will interfere with benefit comprehension most when presented in the whole ad than when presented for briefer periods after the major statement.

-No prediction of differences between short and long duration of statement on benefit comprehension.

(Benefit comprehension highest in control condition, followed short and long duration conditions together, followed by condition where statement is present in whole ad)

• Prominence: Prominence of statement will not affect benefit comprehension.

Toll-Free Statement Recall

• Toll-free statement recall will be higher in any condition where it is included in the ad.

• Placement:

-Recall of statement will be highest in conditions where it is on screen for the whole ad and where it is placed after the major statement.

• Wording: This analysis is exploratory.

• Placement x Wording: This analysis is exploratory

• Duration:

-Recall of the statement will be greatest in the condition where it is present for the whole ad, followed by the condition in which it is located after the major statement.

• Prominence:

-Recall of the statement will be higher in the Extra Prominent condition than in the condition in which it is only in super form after the major statement

Behavioral Intention

• This analysis is exploratory and for completeness.

Power

The following assumptions were made in deriving the sample size: (1) 0.05 alpha and 0.90 power and (2) an effect size between small and medium. The table below presents the per group sample sizes needed to detect differences with effect sizes ranging from conventionally “small” (f=.1) to “medium” (f=.25) across the different designs utilized in this study (see Cohen, 1988, Table 8.4.4, “n to detect f by F test at a = .05”; p. 384).² Thus, based on these assumptions, a per group sample size of 250 should provide adequate power to detect smallish to medium effect sizes across all our designs and a per group sample size of 100 should provide adequate power to detect medium effect sizes in all our designs and smallish effect sizes in some of our designs.

Per group sample size required, by effect size (f) and design, with power = .90, alpha = .05.
	f=.1	f=.15	f=.2	f=.25
5X1 ANOVA	309	138	78	50
4X1 ANOVA	354	158	89	58
2X1 ANOVA	526	234	132	68

Analysis Plan

We will conduct the following analyses separately for the general population sample and the high blood pressure sufferers sample. Once these separate analyses are completed, we will conduct the analyses with the samples combined, using the type of sample as a moderator variable to determine whether any effects differed significantly between the groups.

Part 1: We will test whether there is a main effect of placement on our main dependent variables (i.e., risk comprehension, benefit comprehension, and behavioral intention) using one-way ANOVAs (4 placement conditions, plus control condition). We will conduct ANOVAs that assesses the main effect of placement (4 placement conditions), the main effect of statement type, and the interaction between placement and statement type on our main dependent variables. We will examine logistic regression models predicting toll-free statement recall from placement (4 placement conditions, plus control condition), and from placement, statement type, and the interaction between placement and statement type. We will conduct these analyses both with and without covariates (e.g., demographic and health characteristics) included in the model. In addition, we will test whether any main effects are moderated by other measured variables (e.g., time spent viewing the ad, demographic and health characteristics). If any main effects are significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above).

Part 2: We will test whether there is a main effect of duration on our main dependent variables using one-way ANOVAs and logistic regression models. We will examine these analyses both with and without covariates (e.g., demographic and health characteristics) included in the model. In addition, we will test whether the main effect is moderated by other measured variables (e.g., time spent viewing the ad, demographic and health characteristics). If the main effect is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above).

Part 3: We will test whether there is a main effect of prominence on our main dependent variables using one-way ANOVAs and logistic regression models. We will examine these analyses both with and without covariates (e.g., demographic and health characteristics) included in the model. In addition, we will test whether the main effect is moderated by other measured variables (e.g., time spent viewing the ad, demographic and health characteristics).

Pretesting of Stimuli

Key to our study is the reasonableness and appropriateness of the stimuli we use to approximate television DTC prescription drug ads. Because the particular images are subjective, we will conduct extensive pretesting with consumers similar to our main target audience. This pretesting will involve 675 individuals in two waves. The purpose of the pretesting is to ensure that the stimuli are perceived as realistic. During the pretesting stage, the primary dependent variable will be the success of the particular manipulation. The pretesting will allow us to make changes in the ad stimuli before the actual study commences, thus making participants’ time more valuable.

2. Procedures for the Collection of Information

Respondents will participate in the study via the Internet (see sampling criteria above). They will watch an advertising pod consisting of four advertisements, including one non-test DTC ad, two filler ads for non-pharmaceutical products, and the test ad, always placed last in the series. The non-test ad will have a similar presentation of the toll-free statement to ensure that consumers do not think there is something particular about the stimulus product that would require a special statement. Participants will then answer questions, as shown in Attachment 2. The whole procedure will take no more than 20 minutes.

3. Methods to Maximize Response Rates and to Deal with Issues of Non-Response

This experimental study will use an existing Internet panel to draw a sample. The panel includes people who have expressed interest in sharing their opinions via the Internet and do so regularly. The expected participation rate for the Internet panel is 55 percent when responding to a specific study. To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the experiment over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Administer the experiment to individuals who have expressed interest in participating in Internet studies;

• Email a reminder to the respondents who do not complete the protocol four days after the original invitation to participate is sent;

• Provide contact information on where to get help for respondents who may have questions as they complete the experiment.

4. Test Procedures

The contractor will run nine participants through the procedure to assess questionnaire wording, basic glitches in the programming and execution of the study. This pretest is designed to ensure that questionnaire wording is clear and that procedures for viewing stimuli and proceeding through the experiment are as planned. The stimuli will be tested in two waves of 700 participants to ensure that the stimuli are perceived as realistic.

5. Individuals Involved in Statistical Consultation and Information Collection

The contractor, RTI International, will collect the information on behalf of FDA as a task order under the Quick-Turn-Around Research Services contract. Claudia Squire, Ph.D., is the Project Director for this project, telephone (919) 541-6613. Data analysis will be conducted primarily by the Research Team, Division of Drug Marketing, Advertising, and Communications (DDMAC), Office of Medical Policy, CDER, FDA, and coordinated by Kathryn J. Aikin, Ph.D., 301-796-0569 and Amie C. O’Donoghue, Ph.D., 301-796-0574.

1 Chandler, P. and Sweller, J. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4), 293-332.

2 Cohen, Jacob (1988). Statistical Power Analysis for the Behavioral Sciences, Second Edition. Hillsdale, NJ: Lawrence Erlbaum Associates.

1