OMBSupportingStatement_Quant112709

Experimental Study: Presentation of Quantitative Effectiveness Information to Consumers in Direct-to-Consumer (DTC) Television and Print Advertisements for Prescription Drugs

0910-Number

SUPPORTING STATEMENT

Submitted by

Center for Drug Evaluation and Research

Office of the Commissioner

Food and Drug Administration

December, 2009

A. JUSTIFICATION

1. Circumstances Making the Collection of Information Necessary

The Federal Food, Drug, and Cosmetic Act (the Act) requires that manufacturers, packers, and distributors (sponsors) who advertise prescription human and animal drugs, including biological products for humans, disclose in advertisements certain information about the advertised product's uses and risks.¹ By its nature, the presentation of this information is likely to evoke active trade-offs by consumers, i.e., comparisons with the perceived risks of not taking treatment, and comparisons with the perceived benefits of taking a treatment.² FDA has an interest in fostering safe and proper use of prescription drugs, an activity that engages both risks and benefits. Therefore, an examination of ways to improve consumers’ understanding of this information is central to this regulatory task.

Under the Act, FDA engages in a variety of communication activities to ensure that patients and health care providers have the information they need to make informed decisions about treatment options, including the use of prescription drugs. FDA regulations (21 CFR § 201.57) describe the content of required product labeling, and FDA reviewers ensure that labeling contains accurate and complete information about the known risks and benefits of each drug.

FDA regulations require that prescription drug advertisements that make (promotional) claims about a product also include risk information in a “balanced” manner (21 CFR 202.1(e)(5)(ii)), both in terms of the content and presentation of the information. This balance applies to both the front, display page of an advertisement, as well as including information “in brief summary” about the advertised product’s “side effects, contraindications, and effectiveness”³ usually, but not always, on a separate page. However, beyond the ‘balance’ requirement there is limited guidance and research to direct or encourage sponsors to present benefit claims that are informative, specific, and reflect clinical effectiveness data.

The FDA has recently provided guidance to sponsors about ways to present risk information in prescription drug advertisements.⁴ This guidance notwithstanding, research addressing specifically how to present benefit and efficacy information in prescription drug advertisements is limited. For example, “benefit claims,” broadly defined, appearing in advertisements are often presented in general language that does not inform patients of the likelihood of efficacy and are often simply variants of an “intended use” statement. One content analysis of DTC advertising by Woloshin and Schwartz (2001)⁵ found that information about product benefits and risks is often presented in an unbalanced fashion. The researchers classified the “promotional techniques” used in the advertisements. Emotional appeals were observed in 67% of the ads while vague and qualitative benefit terminology was found in 87% of the ads. Only 9% contained data. However, for risk information, half the advertisements used data to describe side-effects, typically with lists of side-effects that generally occurred infrequently. Similarly, a content analysis by Frosch et al. (2007)⁶ found that only a small proportion of product-claim ads gave specific information about the population prevalence of the medical condition being advertised. The authors criticize DTC for presenting “best-case scenarios that can distort and inflate consumers’ expectations about what prescription drugs can accomplish” (p. 12) without disclosing how many consumers are likely to experience that benefit.

Some research has proposed that providing quantitative information about product efficacy enables consumers to make better choices about potential therapy. One possible format (termed the “drug facts” box by its creators) for this information has recently received attention.⁷ In these studies, the drug facts box format contained information about the product’s efficacy and safety in terms of rate (how many people in the clinical trial experienced a benefit or side effect compared to placebo). As expected, this study showed that consumers who were provided efficacy information used it. Participants receiving efficacy information (without other potentially valuable information about the drug) were more likely to correctly choose the product with the higher efficacy than consumers who saw the brief summary that did not contain this information.

Although these results are intriguing, additional research is necessary to uncover important information about how consumers understand effectiveness information about prescription drug products from direct-to-consumer advertisements. For example, the research to date does not address whether simply adding efficacy rate information and qualitative summations to a consumer-friendly brief summary would enable consumers to find and report the correct answer, or if the presentation of information in a chart format itself increases comprehension.

Further, these data cannot address the best way in which to convey numerical information; percents were used but another format, such as frequencies, may be more effective at communicating quantitative information. Previous research shows that individuals have great difficulty processing numerical concepts (e.g., Beyth-Marom, 1982; Bowman, 2002; Cohen, Ferrell, & Johnson, 2002).⁸ A few studies have attempted to determine what different formats makes these concepts least troublesome (e.g., Fagerlin, Wang, & Ubel, 2005; Lipkus, 2007),⁹ however, most research into the communication of numerical concepts concentrates on risk information. We are not aware of research looking into the integration of quantitative information about effectiveness or benefits into the body of the advertisement itself. The addition of this information may help consumers make better healthcare decisions, provided they can understand it.

It is also not known if ways of communicating product efficacy work equally well across print and television DTC media. To our knowledge, research on presenting quantitative information in risk communication has been conducted exclusively with static modalities. The ideal format for presenting quantitative information may vary as a function of presentation. The amount of mental processing capacity each individual can devote to understanding a message varies depending on how long individuals have to look at the material and whether the material is self-paced or presented at an uncontrollable speed. As a result, some forms of quantitative information may lend themselves to print, rather than broadcast. This particular understanding is crucial to the risk-benefit tradeoff that patients must make with the consultation of a health care professional in order to achieve the best health outcomes.

The proposed study will examine 1) various ways of communicating quantitative efficacy in DTC print ads, and 2) whether the findings translate to DTC television ads.

2. Purpose and Use of the Information Collection

The purpose of this study is to investigate the value of adding quantitative benefit information to DTC advertisements for prescription drugs and to explore a variety of ways to present that information. FDA’s public health mission makes it important to communicate the risks and benefits to consumers and patients as clearly and usefully as possible. The goal of this study is to provide answers to the question of whether adding information of this type helps consumers understand how well the product works, which may lead to more educated discussions with their healthcare providers. This study is designed to be the first in a series of potential studies—sponsored by FDA and others—to investigate this complex issue of quantitative information in an advertising context. Although this study may inform initial policy decisions, we propose this study as the first step in a longer series of studies which will eventually provide stronger evidence for science-based Agency policies.

3. Use of Improved Information Technology and Burden Reduction

Automated information technology will be used in the collection of information for this study. The contracted research firm will collect data through Internet administration. The participant will self-administer the Internet survey via a computer, which will record responses and provide appropriate probes when needed. In addition to its use in data collection, automated technology will be used in data reduction and analysis. Burden will be reduced by recording data on a one-time basis for each respondent, and by keeping surveys to less than 20 minutes.

3. Efforts to Identify Duplication and Use of Similar Information

Although some previous studies have investigated various aspects of print DTC ads,¹⁰ very few studies have varied aspects of television ads in a controlled manner. Published research has typically used content analysis and not rigorous experimental investigation.¹¹ Such research does not permit extrapolation to understanding consumers’ perceptions or intended behavior.

As described in Section A1 (Circumstances Making the Collection of Information Necessary), several research studies exist that provide a basis for our understanding of the communication of risk information in static (print) displays (e.g., Beyth-Marom, 1982; Bowman, 2002; Cohen, Ferrell, & Johnson, 2002; see footnote 8). We are extending this research in two ways: first, we are examining the presentation of quantitative information about the benefit portion of the risk/benefit ratio, and, second, we are also examining this presentation in a dynamic (television) format. To our knowledge, no studies have examined the specific questions we are addressing in this proposed research.

3. Impact on Small Businesses or Other Small Entities

No small businesses would be involved in this data collection.

3. Consequences of Collecting the Information Less Frequently

The proposed data collection is one-time only. There are no plans for successive data collections.

3. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

This collection of information fully complies with 5 CFR 1320.5. There are no special circumstances.

3. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In accordance with 5 CFR 1320.8(d), FDA published a 60 day notice for public comment in the FEDERAL REGISTER of June 22, 2009 (Vol. 74, No. 118). FDA received four comments. In the following section, we outline the observations and suggestions raised in the comments and provide our responses.

Statement 1. All four comments expressed support for the conductance of the research to explore issues of quantitative benefit information. They all described the collection of data as a worthy endeavor which will provide useful information on how best to communicate information in DTC ads.

Statement 2. Two comments suggested enhancing or supplementing the existing behavioral intention questions (questions 13a-d in the questionnaire, Appendix 2). We took this as an opportunity to examine our behavioral intention questions thoroughly. We decided to maintain three of our four behavioral intention questions but remove one of them because of possible redundancy. We also added a new item to this question on the basis of a comment from one of our peer reviewers. Although we took seriously the suggestion to inquire about use of the internet, one of our existing questions already covers this issue. In the interest of brevity, we have decided to streamline this section. Please see Appendix 2 for the revised questionnaire.

Statement 3. One comment suggested including some questions about the risk/benefit tradeoff. We plan to do so and these questions can be seen in questions 23a-d. We labeled this variable “attitude toward drug” because it is easier to analyze and interpret using this term.

Statement 4. Three comments suggested adding different types of participants to our sample, including 1) a general population sample, 2) a sample of participants suffering from a medical condition that they can diagnose themselves, and 3) samples of at least three different medical conditions. We selected high cholesterol because it is prevalent in the population and is commonly advertised DTC. We think adding a medical condition that is symptomatic or can otherwise be self-diagnosed is an excellent suggestion. We hope to explore the research questions in the current study in a variety of other medical conditions in future research.

Statement 5. Two comments suggested comparing the test ad with either the standard of care or with multiple other comparators instead of simply comparing it to placebo. In response, we remind readers that this is the first study to examine issues of quantitative benefit information in print and television DTC ads and that existing literature paints a grim picture of the amount of numerical information viewers may be likely to absorb. Thus, we are using the simplest comparison for this first study. We agree that future studies should examine other types of comparisons; however, we remind readers that only comparisons that are in the approved product labeling can be displayed in promotional pieces.

Statement 6. One comment recommended the use of the Newest Vital Sign health literacy test. We examined this test and considered it for use in our design, but ultimately decided against it for a number of reasons. First, we would have to modify the test so that it could be administered over the Internet rather than in person. It is unclear how some aspects of the test could be altered in such a way. Second, the test takes approximately three minutes when administered in person and may take as long or longer to administer via computer. We believe that numeracy is the key component of health literacy that will influence the results of our study, and we have devoted considerable space in the questionnaire to its measurement (see questions 29a-f, 30a-d and 31a-d). Because of time constraints and the key role of numeracy, we will maintain our current questions to thoroughly examine numeracy and provide basic information on health literacy. We will also include a one-item subjective health literacy item (see question 28). We will continue to examine the Newest Vital Sign measure for future research.

Statement 7. Two comments expressed concern that our study does not address the role of the healthcare provider and overstates the decisions that consumers can make about their prescription drugs. We agree that the healthcare provider is the best person to interpret clinical data and that the consumer or patient does not make the final prescribing decision. Nonetheless, DTC is currently directed at consumers in such a way that they have information about the risk side of the risk/benefit tradeoff but no specific information about the benefit side. This study is designed to assess whether adding specific benefit information will help consumers understand how well the product works, which may ultimately result in better-informed conversations with their healthcare providers.

Statement 8. One comment suggested looking at the results of this study in conjunction with the results of another study we are conducting concerning the role of distraction in television ads in order to inform the development of future research. This is an excellent suggestion that shows a strong understanding of DDMAC’s long-term research goals. We plan to use the results of these two studies, in part, to strengthen the development of our future research.

Statement 9. One comment recommended the inclusion of open-ended recall questions in the questionnaire. We have included some open-ended questions in the revised questionnaire (see questions 4 and 15).

Statement 10. One comment suggested including questions about perceptions of safety and efficacy. A related comment suggested using personal framing rather than asking about “the average person.” We have included questions about safety and efficacy perceptions and these are shown in the revised questionnaire (see questions 15, 16, 17, and 20). We combed through the questionnaire to determine the best framing for each question. Where possible we added personalizing language, but in portions of the questionnaire that measure recall of the words in the ad, we mimicked the language of the ad (see questions 14a-h and 18a-i).

Statement 11. One comment suggested copy testing our mock ad before it is included in the protocol. This is an excellent suggestion that cannot be considered due to limited resources. Nevertheless, we conducted extensive pretesting of the stimuli ad for a previous project and applied the same procedures and concepts to the creation of the current mock ad. Moreover, we conducted limited cognitive testing (of fewer than 9 people) to address such issues and these interviews provided some assurance that our ads were acceptable as were the ads for the other project.

Statement 12. One comment suggested that we show the ads to participants as they would view them at home, i.e., in a clutter reel of ads for the television component and in a group of magazine ads in the magazine component. Although embedding our stimuli within other ads would more closely mimic real viewing, we have several research questions to answer before we reach that point. We are not confident participants will understand any numerical information even when specifically directing them to one ad because this type of information seems to be so difficult for people to understand. We need to establish the basic parameters of statistical and visual information presentation before we can manipulate the realism of the situation and begin to examine other issues such as stopping power and attention.

Statement 13. One comment recommended against using the Internet to administer the study and instead suggested the use of a mall-intercept protocol. Although we recognize that one study cannot address all questions and repeat that the current study is planned to be the first among future studies, we do require several experimental conditions to answer basic presentation and comprehension questions. The resources necessary to conduct this study using a mall-intercept procedure give us less than half of the participants we are currently utilizing. Given that we are using a nationally representative, random digit dialing-based Internet panel to collect our experimental data, we feel that we are obtaining the best value for our funds. We do not feel that the tradeoffs in terms of external validity regarding mall-intercepts are favorable to that method.

Statement 14. One comment recommended including an analysis plan for review, specifically one that addresses what result(s) would support a conclusion that the test ad has achieved a balanced presentation. In response to the first part of this comment, we have included an analysis plan in this current document. In response to the second part of this comment, the primary research question in this study is not whether the information is balanced, but simply how well participants can understand numerical benefit information. Although we will address questions of balance and risk/benefit tradeoff in our questionnaire (see questions 23a-d), our main dependent variables concern the recall and understanding of the benefit information, independent of the other information in the ad. Secondarily, we will examine recall and comprehension of risk information to assess whether it is affected by the inclusion of benefit information and the form the benefit information takes. Finally, we will look at the intersection of benefit and risk information, primarily in risk and benefit perception questions. Our main analyses, however, involve the understanding of benefit information and not in the balance of benefit and risk information. That is an excellent suggestion for future research.

Statement 15. One comment expressed concern that high efficacy may not be the only reason to select one drug over another. We agree. The current research is not designed to examine the multiple factors that a physician or a consumer considers when prescribing or deciding to take a drug. The scope of this project is to investigate the presentation of quantitative benefit information. We have chosen to vary the efficacy of the product (high versus low) as a simple method for determining whether viewers can understand how well the product works when this information is presented in different forms. We maintain that the efficacy of the drug is a major consideration in this decision and therefore represents a reasonable variable to use in this study.

Statement 16. One comment was concerned that data presentation, and in particular the relative frequency presentation, would confuse consumers. This comment reflects the very reason we are conducting the study. Before considering the idea of adding quantitative benefit information to DTC advertising, we want to ensure that we are not causing people to become more confused about their options. We have included the relative frequency condition specifically because we believe consumers do have trouble understanding this format. Sponsors have expressed interest in using this format in their ads and therefore this is a particularly important experimental condition for testing.

Statement 17. One comment suggested that we ask questions about participant age and education. We ask these and other demographic questions in all data collection efforts we conduct and this study is no exception (see questions 39-45).

Statement 18. One comment mentioned that subjective measures of drug efficacy may confuse viewers. We will define high and low efficacy quantitatively based on the range of efficacy currently found in the drug class. We will ask perception questions on Likert scales (e.g., strongly agree to strongly disagree) as well as numerical scales.

Statement 19. One comment suggested that we are basing our entire study on an outdated study from 2001. First, we provided information about the 2001 study to provide background information because it is relevant to the current study but have not based our entire research on it. Second, it is unclear what basic principles of human communication will have changed in the eight years that have passed since the publication of this one study. Finally, although this one study shows that researchers in the field are investigating similar issues, no research currently exists to answer our research questions about the understanding of quantitative information in print and television DTC advertisements.

Statement 20. One comment suggested that 20 minutes is not adequate for participants to complete this study. We have completed similar studies in the past within 20 minutes. We will conduct cognitive testing before the administration of the study to ensure that the protocol can be completed within 20 minutes. Interviews lasting longer than 20 minutes are risky because participants tend not to want to spend that much time on them. Therefore, we will maintain the study at 20 minutes or less.

External Reviewers

In addition to public comment, DDMAC sent materials to three individuals for external peer review. Those individuals are:

• Anthony Cox, Ph.D., Professor of Marketing, Kelley School of Business, IUPUI Indianapolis

• Ellen Peters, Ph.D., Senior Research Scientist, Decision Research Institute, Inc.

• Chung-Tung (Jordan) Lin, Ph.D., Consumer Science Specialist, Consumer Studies Team, CFSAN, FDA

3. Explanation of Any Payment or Gift to Respondents

Internet panel participants are enrolled into a points program that is analogous to a ‘frequent flyer’ card: respondents are credited with points in proportion to their regular participation in surveys. (For the households provided Internet appliances and an Internet connection, their incentive is the hardware and Internet service).  Traditionally, panelists receive 1,000 points for each survey lasting 20 minutes or less. Panelists receive cash-equivalent checks approximately every four to six months in amounts reflecting their level of participation in the panel (1,000 points = $1), which commonly results in distributions in the range of $4 to $6 per month.

3. Assurance of Confidentiality Provided to Respondents

All respondents will be provided with the assurance of confidentiality. The experimental instructions will include information explaining to respondents that their information will be kept confidential.

No personally identifiable information will be sent to FDA. All information that can identify individual respondents will be kept by the independent contractor in a form that is separate from the data provided to FDA. The information will be kept in a secured fashion that will not permit unauthorized access. These methods will all be approved by FDA’s Institutional Review Board (Research Involving Human Subjects Committee, RIHSC) prior to collecting any information.

All electronic data will be maintained in a manner consistent with the Department of Health and Human Services’ ADP Systems Security Policy as described in the DHHS ADP Systems Manual, Part 6, chapters 6-30 and 6-35. All data will also be maintained in consistency with the FDA Privacy Act System of Records #09-10-0009 (Special Studies and Surveys on FDA Regulated Products).

3. Justification for Sensitive Questions

This data collection will not include sensitive questions. The complete list of questions is available in Attachment 2.

3. Estimates of Annualized Burden Hours and Costs

The total annual estimated burden imposed by this collection of information is 1,755 hours for this one-time collection (Table 1).

Table 1. Estimated Annual Reporting Burden^a

Activity	No. of Respondents	Annual Frequency per Response	Total Annual Responses	Hours per Response	Total Hours
Screener	9,000	1	9,000	2/60	270
Questionnaire	4,500	1	4,500	20/60	1,485
Total					1,755

^aThere are no capital costs or operating and maintenance costs associated with this collection of information.

These estimates are based on FDA’s experience with previous consumer studies.

Table 2. Estimated Annual Recordkeeping Burden

Activity	No. of Recordkeepers	Annual Frequency per Recordkeeping	Total Annual Records	Hours per Record	Total Hours
N/A	N/A	N/A	N/A	N/A	N/A

^aThere are no capital costs or operating and maintenance costs associated with this collection of information.

These estimates are based on FDA’s experience with previous consumer studies.

3. Estimates of Other Total Annual Costs to Respondents and Record Keepers

There are no costs to respondents. There are no record keepers.

3. Annualized Cost to the Federal Government

The estimated cost to the Federal Government for the collection of data is $1,026,555.00. This includes the costs paid to the contractors to create stimuli, to program the study, draw the sample, collect the data, and create a database of the results. The cost also includes FDA staff time to design and manage the study, to analyze the resultant data, and to draft a report.

3. Explanation for Programs Changes or Adjustments

This is a new data collection.

3. Plans for Tabulation and Publication and Project Time Schedule

Conventional statistical techniques for experimental data, such as descriptive statistics, analysis of variance, and regression models, will be used to analyze the data. The Agency anticipates disseminating the results of the study after the final analyses of the data are completed, reviewed, and cleared. The exact timing and nature of any such dissemination has not been determined, but may include presentations and articles at trade and academic conferences, publications, and Internet posting.

Table 3. Project Timetable

Task	Estimated Completion Date
External Peer Review	October, 2009
RIHSC Review	November, 2009
30-day FR notice publication	December, 2009
OMB Review of PRA package	January, 2009
Data Collection	February, 2009
Receipt of Data and Methods Report from Contractor	March, 2009
Data Analysis	April, 2009
Draft Report	June, 2010
Internal Review of Draft Report	July, 2010
Revisions	August, 2010
Final Report	September, 2010

3. Reason(s) Display of OMB Expiration Date is Inappropriate

No exemption is requested.

3. Exceptions to Certification for Paperwork Reduction Act Submissions

No exceptions are requested.

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The universe for this experimental study is members of the Knowledge Networks Internet panel.  Knowledge Network’s Internet panel consists of 48,000 adult panel members who are systematically recruited by random-digit dialing (RDD) or by using address-based sampling.  Households without existing Internet service are also eligible, and Knowledge Networks provides these members with laptops or Web TVs to enable their participation. The sample is nationally representative and statistically accurate.  Typically, panel members receive 3-4 invitations per month to participate in research projects. 

The 4,500 participants for this study will be drawn from the pool of 48,000 panel members. All panel members complete prescreening questionnaires on a variety of topics, and we will recruit participants who indicated that they have been medically diagnosed with high cholesterol.  If necessary, we will oversample certain population segments so that the overall sample is in proportion to the U.S. adult population on gender, race/ethnicity, education, and income.  At least 20% of the sample will have achieved a high school education or less. 

2. Procedures for the Collection of Information

Overview of Design

This study will be conducted in two concurrent parts: one examining quantitative information in DTC print advertisements and the other examining such information in DTC television advertisements. Three factors will be examined: drug efficacy, statistical format, and visual format.

We will investigate two levels of drug efficacy (low versus high), defined by a quantifiable, objective metric that can be conveyed in graphical representations of the drug versus the comparator reference drug (in this case, placebo). Specifically, high efficacy will be defined by a large, noticeable difference compared with no treatment, whereas low efficacy will be defined by a minimal difference between the drug and no treatment. We will examine two levels of efficacy to determine whether participants can accurately distinguish between these levels within various formats.

We will investigate five statistical formats, defined as the type of statistical information conveyed: frequency, percent, frequency plus percent, relative frequency, and frequency plus relative frequency. Based on existing literature, we will use the frequency statistical format in all of our visual formats for consistency.

Visual format is defined as various methods through which efficacy can be visually represented. We have chosen to investigate four different formats: pie chart, bar chart, table, and pictograph.

Additionally, we will have a control condition with no specific efficacy information provided. Please see Appendix 3 for the operationalization of each of these conditions. The factors will be combined in a partially crossed factorial design as follows:

		Statistical Format
		Frequency	Percent	Frequency + Percent	Relative Frequency	Frequency + Relative Frequency
Efficacy	Low
	High

AND

		Visual Format
		None	Pie Chart	Bar Chart	Table	Pictograph
Efficacy	Low
	High

+ 1

No Statistical Format/No Efficacy

Procedure

This study will be administered over the internet. A total of 2,250 interviews involving print ads will be completed. Participants in this part of the study will be randomly assigned to view one version of the magazine promotion page and the brief summary page of a prescription drug ad. Following their perusal of this document, they will answer questions about their recall and understanding of the benefit and risk information, their perceptions of the benefits and risks of the drug, and their intent to ask a doctor about the medication.

A total of 2,250 interviews involving television ads will be completed. Participants in this part of the study will be randomly assigned to view one version of a television ad twice and answer the same questions described above.

For both parts, demographic and health care utilization information will be collected. The entire procedure is expected to last approximately 20 minutes. This will be a one-time (rather than annual) information collection.

Participants

Data will be collected using an Internet protocol. Participants will all have reported that a healthcare professional has diagnosed them with high cholesterol and will represent a range of education levels. Because the task presumes basic reading abilities, all selected participants must speak English as their primary language. Participants must be 18 years or older.

Hypotheses

Preface

The proposed research has two main objectives. First, we plan to test several statistical formats to determine whether the presentation of efficacy information in different formats affects perceptions of efficacy. The risk communication literature suggests that presenting numerical risk information as an absolute frequency (e.g., N out of 100) may be the most easily understood format (Fagerlin et al., 2007¹²). Percent, and a combination of absolute frequency and percent, represent increasingly complex statistical formats; however, they may not differ from the baseline of absolute frequency for average consumers. In contrast, the risk communication literature suggests that presenting numerical risk information as a relative frequency (e.g., 10 times higher) is a markedly more complex statistical format that biases perceptions (Fagerlin et al., 2007). Thus, presenting efficacy information as a relative frequency, compared to absolute frequency, may affect perceptions of efficacy. Presenting the combination of absolute frequency and relative frequency may mitigate this effect.

Second, we plan to test several visual formats to determine whether the presentation of a visual format, in conjunction with the presentation of absolute frequency information, affects perceptions of efficacy. The risk communication literature suggests that the addition of visual formats such as bar charts, tables, and pictographs increase people’s understanding of numerical information (Ancker et al., 2006¹³; Lipkus & Hollands, 1999¹⁴). However, not all visual formats are always helpful; for instance, pie charts may only help when people are comparing proportions (Lipkus, 2007¹⁵). Thus, presenting efficacy information with a bar chart, table, and pictograph—but not necessarily with a pie chart—may affect people’s understanding of efficacy information, in comparison to when there is no visual format.

Measuring numeracy will allow us to assess the magnitude of these effects across participants. Similarly, the separate TV and print portions of the study will allow us to assess the magnitude of these effects across these modalities.

Specific Hypotheses

Efficacy effects in print and TV ads

(1) Behavioral intentions, attitude toward drug, and perceived efficacy will be higher in high efficacy conditions than in low efficacy conditions.

(2) We will explore whether there are differences between the no efficacy condition (control) and the low and high efficacy condition on behavioral intentions, attitude toward drug, and perceived efficacy.

(3) Benefit accuracy will be higher in the low and high efficacy conditions than in the no efficacy condition. There will be no difference between the low and high efficacy conditions.

(4) The effects tested in 1 & 3 will be modified by numeracy, such that high numeracy participants will be more likely to show these effects than will low numeracy participants.

(5) Risk recall will not differ by efficacy level (no, low, high).

(6) Perceived risk will be lower in the high efficacy condition compared with the low efficacy condition because, according to the Affect Heuristic (Slovic & Peters, 2006¹⁶), people perceive things that are more beneficial as less risky.

Statistical format effects in print and TV ads

(1) We will test competing hypotheses for behavioral intentions, attitude toward drug, and perceived efficacy.

(1a) Overestimation hypothesis: The first hypothesis rests on the assumption that in the absence of any quantitative information people overestimate the effectiveness of drugs. Accordingly, we would predict that behavioral intentions, attitude toward drug, and perceived efficacy will be higher for participants in the no statistical format condition, compared to all other statistical format conditions. Support for this interpretation will be found if estimates of the benefits are higher in the no statistical format condition than in all other statistical format conditions.

(1b). Peripheral cue hypothesis: The competing hypothesis rests on the assumption that any statistical information will be used as a peripheral cue; that is, participants will not process the quantitative information provided in the various statistical formats but will rather view it as “scientific proof” of the drug’s efficacy. Accordingly, we would predict that behavioral intentions, attitude toward drug, and perceived efficacy will be lower for participants in the no statistical format condition, compared to all other statistical format conditions. Support for this interpretation will be found if, in addition to perceived efficacy effects, estimates on attitude toward the ad “peripheral cue” measures—ratings of how believable, persuasive, informative, etc, the ad is--are lower in the no statistical format condition than in all other statistical format conditions.

(2) Based on the risk communication literature, we predict that the absolute frequency, percent, and absolute frequency and percent conditions may not differ on behavioral intentions, attitude toward drug, or perceived efficacy. However, we predict that behavioral intentions, attitude toward drug, and perceived efficacy will be higher in the relative frequency condition than in the absolute frequency, percent, absolute frequency + percent, and absolute frequency + relative frequency conditions.

(3) The effects tested in hypotheses 1-2 will be modified by numeracy. For instance, we expect that the difference between the relative frequency and the absolute frequency + relative frequency conditions will be greater for high numeracy participants than for low numeracy participants (because high numeracy participants will be more likely to use the additional information provided by the absolute frequency).

(4) Benefit accuracy will be lowest in the no statistical format condition and highest in the absolute frequency condition (Slovic, Monahan, & MacGregor, 2000¹⁷). Tests of other relations between statistical formats will be exploratory. For instance, we might see information overload with some formats (e.g., absolute frequency & relative frequency) which impedes benefit accuracy.

(5) The effects tested in (4) will be modified by numeracy, such that low numeracy participants will show greater differences in benefit accuracy across statistical formats than will high numeracy participants (Peters, Vastfjall, et al., 2006¹⁸).

(6) We expect that risk recall will not differ by statistical format, but we will conduct exploratory analyses to determine whether information overload impedes risk recall.

(7) We expect that perceived risk will be lowest in the relative frequency condition if perceived benefit is indeed highest in this condition (see Slovic & Peters, 2006, footnote 10).

Visual format effects in print and TV ads

(1) We will test competing hypotheses for benefit accuracy, behavioral intentions, attitude toward drug, and perceived efficacy.

(1a) Visual information facilitation hypothesis: The first hypothesis rests on the assumption that participants will, to the extent possible, process and use the information in the visual formats. The risk communication literature suggests that visual representations of risk can increase understanding, and that people have a more difficult time processing this kind of information in pie charts, as compared to other visual formats. Therefore, our first hypothesis is that benefit accuracy will be higher in the bar chart, table, and pictograph conditions—but not necessarily the pie chart condition--than in the no visual format condition. Tests of other relations between visual formats will be exploratory.

(1b) Information overload hypothesis: Alternatively, there may be no differences across visual formats on behavioral intentions, attitude toward drug, perceived efficacy, or benefit accuracy if the visual serves as a distraction or is too much information to process.

(1c) Peripheral cue hypothesis: Behavioral intentions, attitude toward drug, and perceived efficacy—but not benefit accuracy—may be higher in all visual conditions than in the no visual condition if the visual information serves as a peripheral cue.

(2) The effects tested in hypothesis 1 will be modified by numeracy. For instance, we expect that high numeracy participants will be more likely to process the information in the visual formats, and thus more likely to show the pattern of effects outlined in 1a, compared to low numeracy participants.

(3) We expect that perceived risk and risk recall will not differ by visual format but we will conduct exploratory analyses to determine whether information overload impedes risk recall.

Analysis Plan

We will conduct the following analyses separately for the print and television versions of the ad.

Efficacy effects in print and TV ads: We will conduct ANOVAs to test whether the no statistical format/no efficacy condition differs from the low and high efficacy condition on the dependent measures (i.e., benefit accuracy, behavioral intentions, attitude toward drug, perceived efficacy, perceived risk, and risk recall, peripheral cue measures). We will conduct these analyses both with and without covariates (e.g., demographic and health characteristics) included in the model. In addition, we will test whether any main effects are moderated by other measured variables (e.g., numeracy, demographic and health characteristics). If the main effect of efficacy is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above). In addition, the main effect of efficacy (low vs. high) and any interaction it has with statistical format or visual format will be tested in the ANOVAs presented in the following two sections.

Statistical format effects in print and TV ads: We will conduct ANOVAs to test whether the no statistical format/no efficacy condition differs from the other statistical format conditions on the dependent measures. In addition, we will examine the main effect of statistical format in ANOVAs predicting our dependent measures from statistical format, efficacy level, and their interaction. We will conduct these analyses both with and without covariates included in the model. In addition, we will test whether any main effects are moderated by other measured variables. If the main effect of statistical format is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above).

Visual format effects in print and TV ads: To test our hypotheses regarding visual format, we will examine the main effect of visual format in ANOVAs predicting our dependent measures from visual format, efficacy level, and their interaction. We will conduct these analyses both with and without covariates included in the model. In addition, we will test whether any main effects are moderated by other measured variables. If the main effect of visual format is significant, we will conduct pairwise-comparisons to determine which conditions are significantly different from one another. We will also conduct planned comparisons in line with our hypotheses (see above).

Power

The following assumptions were made in deriving the sample size: (1) 0.05 alpha and 0.90 power and (2) an effect size between small and medium. The table below shows the sample size required to detect differences with effect sizes ranging from conventionally “small” (f = .1) to almost “medium” (f = .25) for the largest comparison we plan to analyze (2 x 5). Specifically, visual and statistical format will never be crossed and we plan to conduct the print and broadcast parts of the study as entirely separate analyses.

Table: A priori power analysis to determine sample size needed in F tests (ANCOVA: fixed effects, main effects, and interactions) to achieve power of .90 (Faul et al., 2007).19
		Effect size f*
Input
		.10	.13	.20
	α error probability	.05	.05	.05
	Power ( 1 – β error probability)	.90	.90	.90
	Numerator df	9	9	9
	Number of groups	10	10	10
	Number of covariates	4	4	4
Output
	Noncentrality parameter λ	19.92	19.98	20.20
	Critical F	1.88	1.89	1.90
	Denominator df	1,981	1,171	494
	Total sample size	1,992	1,182	505
	Actual power	.90	.90	.90

*An effect size of .10 is traditionally considered small, whereas an effect size of .25 is considered medium (Cohen, 1988).²⁰ Here we have shown three different effect sizes centering around small to medium effects to show that we will be able to detect fairly small effects.

We will have 118 participants per cell, with a total of 1,180 participants in the 10 cells represented in the table (2 x 5). The table shows that our sample size of 1,180 per portion of the study will be sufficient to detect effects as small as .13.

3. Methods to Maximize Response Rates and to Deal with Issues of Non-Response

This experimental study will use an existing Internet panel to draw a sample.  The panel comprises individuals who share their opinions via the Internet regularly.  The participation rate for similar studies is 65-70% percent without additional efforts to convert non-respondents.  To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the experiment over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Email a reminder to the respondents who do not complete the protocol four days after the original invitation to participate is sent;

• Provide a toll-free hotline for respondents who may have questions or technical difficulty as they complete the experiment. 

4. Test Procedures

The contractor will run nine participants through the procedure to assess questionnaire wording, basic glitches in the programming and execution of the study. This pretest is designed to ensure that questionnaire wording is clear and that procedures for viewing stimuli and proceeding through the experiment are as planned.

5. Individuals Involved in Statistical Consultation and Information Collection

The contractor, RTI International, will collect the information on behalf of FDA as a task order under the Quick-Turn-Around Research Services contract. Doug Rupert, MPH, is the Project Director for this project, telephone (919) 541-6495. Data analysis will be conducted primarily by the Research Team, Division of Drug Marketing, Advertising, and Communications (DDMAC), Office of Medical Policy, CDER, FDA, and coordinated by Amie C. O’Donoghue, Ph.D., 301-796-0574 and Kathryn J. Aikin, Ph.D., 301-796-0569.

APPENDIX 1

FR 60-day notice

APPENDIX 2

Questionnaire, Quantitative Study (for both print and broadcast)

[PROGRAMMER: We need to record time in milliseconds spent on each screen (including questions) throughout protocol.]

Interview Protocol.

(Present Informed Consent Form. Participants will be blind to FDA’s sponsorship.)

Thank you for agreeing to participate in this study today.

Make sure you are comfortable and can read the screen from where you sit. This study is about advertising for a new product. We will show you an ad for a new product and ask you some questions about it. Your answers are anonymous, which means that no one will ever connect your name with your answers. Your help is valuable and we thank you.

[PROGRAMMER: New screen]

Next you will see an ad for a new product.

[Instructions for print version] Even though it is on a computer screen, please read the ad as you would in a magazine if you saw an ad for a product that you might be interested in for yourself. You can take as much time as you want to look over the ad. The ad has two pages. You can flip back and forth between pages using the BACK and FORWARD buttons if that is how you would normally read this ad. Once you are finished reading, please click “next” to move on to the next part of the study. [PROGRAMMER: Record time in milliseconds spent on each page of each print ad.]

[Instructions for TV version] Even though it is on a computer screen, please watch the ad as you would on television if you saw an ad for a product that you might be interested in for yourself. After viewing the ad, the program will instruct you to move on to the next part of the study. [PROGRAMMER: Show ad twice]

Now please answer the following questions based on the ad you saw.

Q1. Do you remember seeing an ad for Votrea?

Yes

No (terminate)

Not sure (terminate)

OMB Control No.

Q2. What type of product is Votrea? [PROGRAMMER: randomize responses]

Over the counter drug

Prescription drug

Herbal supplement

Lens cleaner

Q3. What condition does Votrea treat?

High blood pressure

High cholesterol

Migraine headaches

Seasonal allergies

Q4. What are the benefits of Votrea?

(open ended)

(Gist Comprehension)

Q5. Based on the information in the ad, does Votrea work better than not taking any treatment?

Yes

No

Not sure

(Perceived Benefit)

Q6. Based on the information in the ad, how effective would Votrea be for you?

1 2 3 4 5 6 7

Not at all Moderately Very

effective effective effective

Q7. Based on the information in the ad, how well would Votrea work for you?

1 2 3 4 5 6 7

Not at all Moderately Very

well well well

(Specific Benefit Accuracy)

OMB Control No.

Please answer the following specific questions based on what you learned from the Votrea ad.

Q8. If 100 people take Votrea, how many will lower their bad cholesterol to normal levels?

______ people (fill in the blank. PROGRAMMER: set acceptable range from 0 to 100)

Q9. What percentage (%) of people who take Votrea will lower their bad cholesterol to normal levels? For example,

If no one will lower their bad cholesterol to normal levels, enter 0.

If everyone will lower their bad cholesterol to normal levels, enter 100.

If some but not all will lower their bad cholesterol to normal levels, enter a number between 0 and 100 that reflects the percentage.

______ percent (fill in the blank. PROGRAMMER: set acceptable range from 0 to 100)

Q10. How many more times effective is Votrea than no treatment in lowering bad cholesterol? Enter a number to show how effective Votrea is compared to no treatment. For example,

If taking Votrea is no more effective than no treatment, enter 0.

If taking Votrea is two times more effective than no treatment, enter 2.

If taking Votrea is three times more effective than no treatment, enter 3.

______ times more effective (fill in the blank. PROGRAMMER: set acceptable range from 0 to unlimited upper bound)

Q11. If 100 people take no treatment, how many will lower their bad cholesterol to normal levels?

______ people (fill in the blank. PROGRAMMER: set acceptable range from 0 to 100)

Q12. This ad had a picture or visual showing how well Votrea works.

Yes

No

Not sure

[PROGRAMMER: randomize Q9a-c]

OMB Control No.

Q13. (Behavioral Intention) Please rate how likely or not likely you are to do each of the following behaviors using the scale on this page.

	Not at all likely	Somewhat likely	Very likely	Extremely likely
a. Talk to your doctor about Votrea
b. Look for more information about Votrea
c. Ask your doctor to prescribe Votrea
d. Take Votrea if prescribed

Q14. (Recall of benefits) Please check which of the following were mentioned in the ad as benefits of taking Votrea.

[PROGRAMMER: randomize Q10a-h]

	Yes	No
a. Votrea works with diet and exercise.	x
b. Votrea can lower bad cholesterol to normal levels.	x
c. Votrea works for people with several common risk factors for heart disease.	x
d. Votrea takes the place of diet and exercise.		x
e. Votrea can reduce the risk of diabetes.		x
f. Votrea is the #1 prescribed medication for high cholesterol.		x
g. High cholesterol is a risk factor for heart disease.	x
h. Votrea can raise good cholesterol to normal levels.		x

Q15. What do you remember about the risks of Votrea?

(open ended)

Q16. Based on the information in the ad, how safe would Votrea be for you?

OMB Control No.

1 2 3 4 5 6 7

Not at all Moderately Very

Safe safe safe

Q17. Based on the information in the ad, how risky would Votrea be for you?

1 2 3 4 5 6 7

Not at all Moderately Very

risky risky risky

Q18. (Recall of risks) Please check which of the following were mentioned in the ad as risks of taking Votrea.

[PROGRAMMER: randomize Q14a-i]

	Yes	No
a. TTP is a risk when you take Votrea	x
b. You need blood tests when taking Votrea.	x
c. People with liver problems should not take Votrea.	x
d. Votrea may cause muscle pain or weakness.	x
e. A common side effect of Votrea is blurry vision.		x
f. A common side effect of Votrea is tiredness.	x
g. A common side effect of Votrea is dizziness.		x
h. People with kidney problems should not take Votrea.		x
i. A common side effect of Votrea is joint pain.		x

[PROGRAMMER: randomize order of Q19-Q20]

Q19. Compared to other treatments you could take for high cholesterol, how well do you think Votrea works?

Much better

Somewhat better

About the same

Somewhat worse

OMB Control No.

Much worse

Q20. Compared to other treatments you could take for high cholesterol, how safe or risky do you think Votrea is?

Much safer

Somewhat safer

About the same

Somewhat riskier

Much riskier

(Affect toward drug)

Q21. How good or bad do you feel about this product?

Very bad

Somewhat bad

Neither bad nor good

Somewhat good

Very good

[PROGRAMMER: Randomize order of Q22a-e]

Q22. (Attitude toward the ad) Please rate your agreement or disagreement with each of the following statements.

	Strongly agree	Somewhat agree	Neither agree nor disagree	Somewhat disagree	Strongly disagree
The ad clearly communicated the benefits of Votrea.
The ad clearly communicated the risks of Votrea.
The ad evenly balanced the risks and benefits of Votrea.
The ad communicated the benefits of Votrea in a way that was easy to understand.
T OMB Control No. he ad communicated the risks of Votrea in a way that was easy to understand.

[PROGRAMMER: Randomize order of Q23a-d]

Q23. (Attitude toward drug) Please rate your agreement or disagreement with each of the following statements.

	Strongly agree	Somewhat agree	Neither agree nor disagree	Somewhat disagree	Strongly disagree
The risks and negative effects outweigh the benefits and positive effects of Votrea.
The benefits and positive effects of Votrea outweigh the risks and negative effects
I could deal with the side effects if I reduced my bad cholesterol with Votrea.
Reducing my bad cholesterol to normal levels would not be enough to balance the risks and negative effects from Votrea.

[PROGRAMMER: Randomize order of Q24a-h]

Q24. (Peripheral cue) This ad had information about how well Votrea works. To what extent do you agree or disagree that the information was:

	Strongly agree	Somewhat agree	Neither agree nor disagree	Somewhat disagree	Strongly disagree
Noticeable
Persuasive
Believable
Distracting
Confusing
Scary
Attractive
Informative

[

OMB Control No.

PROGRAMMER: Randomize order of Q25a-h]

Q25. (Peripheral cue) This ad had information about the risks and side effects of Votrea. To what extent do you agree or disagree that the information was:

	Strongly agree	Somewhat agree	Neither agree nor disagree	Somewhat disagree	Strongly disagree
Noticeable
Persuasive
Believable
Distracting
Confusing
Scary
Attractive
Informative

Q26. (Skepticism) How likely is it that the benefit claim presented in this ad is true?

1 2 3 4 5 6 7 8 9

Not at all Extremely

Likely Likely

Q27. (Skepticism) How skeptical are you about the truth of the benefit claim in this ad?

1 2 3 4 5 6 7 8 9

Not at all Extremely

Skeptical Skeptical

Q28. (Subjective health literacy) How often do you need to have someone help you when you read instructions, pamphlets, or other written material from your doctor or pharmacy?

Never

Rarely

Sometimes

Often

Always

Q29. (Objective numeracy) Now here are some questions that require you to use numbers to solve the problem. Some are easy and others are more difficult. No calculators please- we’d like you to answer on your own. Remember, almost everyone will have trouble with these questions, so don’t be upset if some are difficult—just do your best!

[PROGRAMMER: DO NOT randomize Q29a-f]

OMB Control No.

1. What number is the correct answer:

8 + 4 + 11 = ?

1. 14

2. 19

3. 21

4. 23

5. 32

6. Don’t know

2. What is the correct answer:

17 – 8 + 4 = ?

a. 11

1. 13

2. 21

3. 23

4. 29

5. Don’t know

3. What is the correct answer:

100 x 10 x 10 = ?

1. 100

2. 1,000

3. 10,000

4. 100,000

5. 1,000,000

6. Don’t know

4. Imagine that you flip a fair coin 1,000 times. What is your best guess about how many times the coin would come up heads in 1,000 flips?

___ times out of 1,000 [PROGRAMMER: set acceptable range from 0 to 1,000]

5. In the BIG BUCKS LOTTERY, the chance of winning a $10 prize is 1%. What is your best guess about how many people would win a $10 prize if 1,000 people each buy a single ticket to BIG BUCKS LOTTERY?

________ people [PROGRAMMER: set acceptable range from 0 to 1,000]

OMB Control No.

6. In ACME PUBLISHING SWEEPSTAKES, the chance of winning a car is 1 in 1,000. What percent of tickets to ACME PUBLISHING SWEEPSTAKES will win a car?

___ percent

Q30. (Subjective Numeracy 1st part) For each of the following questions, please check the box that best reflects how good you are at doing the following things:

1. How good are you at working with fractions?

2. How good are you at working with percentages?

3. How good are you at calculating a 15% tip?

4. How good are you at figuring out how much a shirt will cost if it is 25% off?

Q31. (Subjective Numeracy 2^nd part) For each of the following questions, please check the box that best reflects your answer:

1. When reading the newspaper, how helpful do you find tables and graphs that are part of a story?

2. When people tell you the chance of something happening, do you prefer that they use words (“it rarely happens”) or numbers (“there is a 1% chance)?

OMB Control No.

3. When you hear a weather forecast, do you prefer predictions using percentages (“there will be a 20% chance of rain”) or predictions using only words (“there is a small chance of rain today”)?

4. How often do you find numerical information to be useful?

Q32. Are you currently taking a prescription medicine for high cholesterol?

Yes

No

Don't know or uncertain

Q33. How severe is your high cholesterol now? Would you describe it as:

Very mild

Mild

Moderate

Serious

Very serious

Q34. In general, how much do you feel you know about high cholesterol? Would you say you know:

A lot

A good bit

Some

Only a slight amount

Nothing at all

OMB Control No.

Q35. In general, how much do you feel you know about treatments for high cholesterol? Would you say you know:

A lot

A good bit

Some

Only a slight amount

Nothing at all

Q36. Roughly, what is your total cholesterol level? Your best guess is OK.

_____

Q37. Have you ever seen any advertising for Votrea before today?

Yes

No

Don’t Remember

Q38. Did you get any help on the questions that had numbers in them from anything or anyone (e.g., calculator, other person, website)? It’s okay if you did—we just want an honest answer.

Yes (write source_____________)

No

Q39. Are you:

• Hispanic or Latino

• Not Hispanic or Latino

Q40. Which of these best represents your ethnic group? You may choose one or more. Would you say that you are:

• American Indian or Alaska Native

• Asian

• Black or African-American

• Native Hawaiian or Other Pacific Islander

• White

• Other

• Don’t know

• Prefer not to answer

Q41. Gender

1 Male 2 Female

OMB Control No.

Q42. How many years of education have you had?

Completed grade school

Completed middle school

Completed high school

Some college

College degree

Some postgraduate work

Postgraduate degree (M.A., Ph.D., M.D., J.D., etc.)

[Include if not available in KN panel database.]

Q43. How often do you exercise?

Q44. Occupation

Q45. What year were you born? _______

[End time: ___________________ ]

You have been very helpful. Thank you very much for your participation!

OMB Control No.

APPENDIX 3

Sample Statistical Formats

NO VISUAL, NO EFFICACY

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

NO VISUAL, HIGH EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65% of people lowered their bad cholesterol to normal levels versus 2% of people with no treatment.

Combination of Absolute Frequency and Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people (65%) lowered their bad cholesterol to normal levels versus 2 out of 100 people (2%) with no treatment.

Relative Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, people were 33 times more likely to lower their bad cholesterol to normal levels, compared to no treatment.

Combination of Relative Frequency and Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65% of people lowered their bad cholesterol to normal levels versus 2% of people with no treatment—that’s 33 times more effective.

BAR GRAPH, HIGH EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

PIE CHART, HIGH EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Votrea No treatment

TABLE, HIGH EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Clinical Trial Results
		Votrea	No Treatment
Outcome: Lowered cholesterol to normal levels	Yes	65	2
	No	35	98
	Total	100 patients	100 patients

PICTOGRAPH, HIGH EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 65 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Votrea

* * * * * * * * * *

* * * * * * * * * *

* * * * * * * * * *

* * * * * * * * * *

* * * * * * * * * *

* * * * * * * * * *

* * * * * ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

No treatment

* * ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

NO VISUAL, LOW EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10% of people lowered their bad cholesterol to normal levels versus 2% of people with no treatment.

Combination of Absolute Frequency and Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people (10%) lowered their bad cholesterol to normal levels versus 2 out of 100 people (2%) with no treatment.

Relative Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, people were 5 times more likely to lower their bad cholesterol to normal levels, compared to no treatment.

Combination of Relative Frequency and Percent:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10% of people lowered their bad cholesterol to normal levels versus 2% of people with no treatment—that’s 5 times more effective.

BAR GRAPH, LOW EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

PIE CHART, LOW EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Votrea No treatment

TABLE, LOW EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Clinical Trial Results
		Votrea	No Treatment
Outcome: Lowered cholesterol to normal levels	Yes	10	2
	No	90	98
	Total	100 patients	100 patients

PICTOGRAPH, LOW EFFICACY

Absolute Frequency:

Votrea reduces bad cholesterol for people with several common risk factors for heart disease. (super: starting from an average bad cholesterol of 160 mg/dL).

With Votrea, 10 out of 100 people lowered their bad cholesterol to normal levels versus 2 out of 100 people with no treatment.

Votrea

* * * * * * * * * *

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

No treatment

* * ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

° ° ° ° ° ° ° ° ° °

1 For prescription drugs and biologics, the Act requires advertisements to contain "information in brief summary relating to side effects, contraindications, and effectiveness" (21 U.S.C. 352(n)).

2 See Swartz, L, Woloshin, S, Black, W, and Welch, HG (1997). The role of numeracy in understanding the benefit of screening mammography. Annals of Internal Medicine, 127(11), 966-72.

3 See Section 502(n) of the Federal Food, Drug and Cosmetic Act.

4 Draft Guidance for Industry: Presenting Risk Information in Prescription Drug and Medical Device Advertising. Available at http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM155480.pdf. Last accessed September 29, 2009.

5 Woloshin, S. and Schwartz, L. (2001). Direct to consumer advertisements for prescription drugs: what are Americans being told. Lancet, 358, 1141-46.

6 Frosch, D.L., Krueger, P.M., Hornik, R.C., Cronholm, P.F., & Barg, F.K. (2007). Creating demand for prescription drugs: A content analysis of television direct-to-consumer advertising. Annals of Family Medicine, 5(1), 6-13.

7 Schwartz, L.M., Woloshin, S., & Welch, H.G. (2007). The drug facts box: Providing consumers with simple tabular data on drug benefit and harm. Medical Decision Making. 27, 655-692; Schwartz, L.M., Woloshin, S., & Welch, H.G. (2009). Communicating drug benefits and harms with a drug facts box: Two randomized trials. Annals of Internal Medicine, 150, 516-527; Woloshin, S., Schwartz, L.M., & Welch, H.G. (2004). The value of benefit data in direct-to-consumer dug ads. Health Affairs, Suppl web exclusives W4-234-245. .

8 Beyth-Marom, R. (1982). How probable is probable? A numerical translation of verbal probability expressions. Journal of Forecasting, 1, 257-269.

Bowman, M.L. (2002). The perfidity of percentiles. Archives of Clinical Neuropsychology, 17, 295-303.

Cohen, D.J., Ferrell, J.M., & Johnson, N. (2002). What very small numbers mean. Journal of Experimental Psychology: General, 131, 424-442.

9 Fagerlin, A., Wang, C., & Ubel, P.A. (2005). Reducing the influence of anecdotal reasoning on people’s health care decisions: Is a picture worth a thousand statistics? Medical Decision Making, 25, 398-405.

Lipkus, I. (2007). Numeric, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Medical Decision Making, 27, 697-713.

10See, for example:

Holmes, E.R. & Desselle, S.P. (2004). Evaluating the balance of persuasive and informative content within product-specific print direct-to-consumer ads. Drug Information Journal, 38, 83-98.

Munce, S.E., Robertson, E.K., Sansom, S.N., & Stewart, D.E. (2004). Who is portrayed in psychotropic drug advertisements? The Journal of Nervous and Mental Disease, 192, 284-288.

11See, for example:

Kaphingst, K.A., DeJong, W., Rudd, R.E., & Daltroy, L. (2004). A content analysis of direct-to-consumer television prescription drug advertisements. Journal of Health Communications, 9, 515-528.

Kaphingst, K.A., Rudd, R.E., DeJong, W., & Daltroy, L. (2005). Comprehension of the information in direct-to-consumer television prescription drug advertisements among adults with limited literacy skills. Journal of Health Communications, 7, 609-619.

Sumpradit, N., Ascione, F.J., & Bagozzi, R.P. (2004). A cross-media content analysis of motivational themes in direct-to-consumer prescription drug advertising. Clinical Therapeutics, 26, 135-154.

12 Fagerlin, A., Ubel, P.A., Smith, D.M., & Zikmund-Fisher, B.J. (2007). Making numbers matter: Present and future research in risk communication. American Journal of Health Behavior, 31, Suppl 1: S47-56.

13 Acker, J.S., Senathirajah, Y., Kukafka, R., & Starren, J.B. (2006). Design features of graphs in health risk communication: A systematic review. Journal of the American Medical Information Association, 13, 608-618.

14 Lipkus, I., & Hollands, J.G. (1999). The visual communication of risk. Journal of the National Cancer Institute Monographs, 25, 149-163.

15 Lipkus, I. (2007). Numerica, verbal, and visual formats of conveying health risks: Suggested best practices and future recommendations. Medical Decision Making, 27, 697-713.

16 Slovic, P., & Peters, E. (2006). Risk perception and affect. Current Directions in Psychological Science, 15, 322-325.

17 Slovic, P., Monahan, J., & MacGregor, DG. (2000). Violence risk assessment and risk communication: the effects of using actual cases, providing instruction, and employing probability versus frequency formats. Law and Human Behavior, 24, 271-96.

18 Peters, E., Vastfjall, D., Slovic, P., Mertz, CK, Massocco, K., & Dickert, S. (2006). Numeracy and decision making. Psychological Science, 17, 407-13.

19 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A, (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavor Research Methods, 39, 175-191.

20 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

18