Supporting Statement Part B Final 3-18-15

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

The eligible study population is U.S., non-institutionalized adults age 18 and older. The selected sample will be drawn from ResearchNow’s opt-in online survey panels according to the specific research objectives for this project. Consumers will be recruited through ResearchNow’s e-Rewards Consumer Panel using a multi-mode approach (e.g., e-mail, online marketing, and by-invitation, with over 300 diverse online and offline affiliate partners and targeted website advertising).This panel is demographically balanced, including racial and ethnic minorities, a wide range of different age groups, and individuals with relatively less educational attainment. This panel provides access to over 336,000 deeply-profiled panelists in the US and Canada who have been diagnosed with either Type 1 diabetes, Type 2 diabetes, high blood pressure or obesity.

Physicians will be recruited through ResearchNow’s Healthcare Panel utilizing a multi-mode approach combining email, fax, and direct mail to recruit physicians to participate in online surveys. This approach provides a greater reach into the U.S. physician market; in total, ResearchNow has access to more than 95% of all U.S. physicians (with an estimated 44,880 physician panelists). In addition to recruitment methods, they also purchase key association and governmental databases that verify a physician’s practicing status. These verification resources include the Drug Enforcement Agency number (DEA#) and the American Medical Association (AMA) Medical Education Number (ME#).

Consumer and physician panel members eligible to participate in the current survey will be contacted through an e-mail invitation from the panel managers which will include a secure, non-identifiable link to the web-based survey. In each survey invite, panelists are informed about the survey topic in a topline, non-leading way before participation. Panelists are rewarded for taking part in surveys with a structured incentive scheme, reflecting the length of survey and nature of the sample. Recruitment will continue until the target sample size for completed surveys is reached.

The survey sample will be drawn from eligible members based on pre-specified criteria. A pre-profiled sample is used to minimize screen-outs and provide a better quality panelist experience. Once a sample has been selected, email invites are automatically randomized so as not to induce bias. ResearchNow places a limit on both the number of invites available to all members (normally two or fewer surveys per month) and on the number of qualified completes to avoid excessive survey participation which would otherwise create survey fatigue and potential bias.

The participants of the study will be volunteers, and will not be randomly or systematically selected by ResearchNow. Therefore, the sample used is a convenience sample, rather than a probability sample. Eligible participants for the consumer sample will be adults who speak English and self-identify as having been diagnosed with diabetes. We will exclude individuals who work in the health care, marketing, advertising, or pharmaceutical industries from the consumer sample. Diabetes sufferers will be recruited from the ResearchNow Consumer Panel. Eligible participants for the physician sample will be adults who speak English and practice patient care at least half time in the areas of Family Practice, General Practice, Internal Medicine, and Endocrinology. Physicians will be selected from ResearchNow’s Healthcare Panel.

We will exclude pretest study participants from the main study.

2. Procedures for the Collection of Information

Design Overview

The design consists of two pretests and a main study. We will conduct two sequential pretest waves prior to main data collection. The purpose of the pretests are to 1) ensure the stimuli are understandable and viewable, 2) identify and address any challenges to embedding the stimuli within the online survey, and 3) ensure the study questions are appropriate and meet the study’s goals. Participants in the pretests will be randomly assigned to one of two versions of an ad. One version will present information about the price of the product relative to a competitor for the same indication (Price Comparison). Another version will present this information with additional contextual information that the two drugs may not be comparable in terms of efficacy and safety and that the acquisition costs do not necessarily reflect actual prices paid (Price Comparison + Additional Context).

Participants in Pretest 1 will be consumers (n=400) who self-identify as having been diagnosed with diabetes. Pretest 2 will be conducted with physicians (n=1,000) who are General Practitioners (e.g., Family Practice, General Practice, Internal Medicine) and Specialists (e.g., Endocrinology). Pretest 2 has a two-fold purpose. In addition to the measurement and stimuli verification issues identified above, we will also conduct an experiment to evaluate the impact of incentive level (level 1 vs. level 2) and study sponsorship (FDA vs. Public Health Agency) disclosure on physician response rates (see Exhibit 1). Endocrinologists will be randomly assigned to $15 (level 1) or $60 (level 2). General Practitioners will be randomly assigned to $10 (level 1) or $40 (level 2). Pretest 2 will therefore provide a comparison of recruitment approaches, identify ways to optimize response rates, and provide a “dry run” of experimental study recruitment procedures.

Exhibit 1: Pretest 2 Design, Incentive Level by Study Sponsorship by Type of Ad

		Type of Ad
		Price Comparison		Price Comparison + Additional Context		TOTAL
	Study Sponsor	FDA	Public Health Agency	FDA	Public Health Agency
Incentive Level	Level 1	125	125	125	125	500
	Level 2	125	125	125	125	500
	TOTAL	250	250	250	250	1,000

In the main study phase, physician (n = 1440) and consumer (n = 1,500) participants will be randomly assigned to view one of three possible versions of a DTC or professional ad, as depicted in Exhibit 2. This sample size will provide us with sufficient power to detect small-to-medium sized effects.

Exhibit 2: Main Study Design

	Type of Price Comparison
Sample	Price information only	Price information + Additional context	No comparison information (Control)
Consumers (DTC ad)	500	500	500
Physicians (Professional ad)	480	480	480

Procedure

Pretests: Each participant will be randomly assigned to view a print ad for a fictitious prescription drug indicated to treat diabetic neuropathy and will be asked to complete an online survey assessing their benefit/risk perceptions, intentions, and attitudes toward the drug. Based on the pretest findings, we will revise and remove survey items prior to full-scale testing.

Main study: Each participant will be randomly assigned to view a print ad for a fictitious prescription drug for diabetic neuropathy and will be asked to complete an online survey assessing their benefit/risk perceptions, intentions, and attitudes toward the drug.

Participants

Eligible consumer participants for the pretest and main study will be adults who speak English and self-identify as having been diagnosed with diabetes. We will exclude individuals who work in the health care, marketing, advertising, or pharmaceutical industries from the consumer sample. Eligible physician participants for the pretest and main study will be adult physicians who speak English who are General Practitioners (e.g., Family Practice, General Practice, Internal Medicine) and Specialists (e.g., Endocrinology). We will also exclude pretest study participants from the main study.

Analysis Plan

We will conduct ANOVAs (for continuous variables) and chi-squares and logistic regressions (for categorical variables) to examine the impact of price information and additional context. Before conducting analyses, we will assess whether the inclusion of covariates is justified. If they are, we will conduct the analyses both with and without covariates (e.g., sex, age, race/ethnicity, education) included in the model. If the one-way ANOVA is significant, we will implement a series of two-way comparisons (e.g., price information only vs. control, price information + additional context vs. control, price information only vs. price information + additional context context) to test for significant differences among the three experimental arms for each population (consumers and physicians).

Power

As shown in Exhibit 2 above, the main study will include 1,500 consumer participants and 1,440 physician participants. We estimated power for analyses of variance (ANOVAs) comparing continuous outcome measures by type of price comparison at p-value of 0.017, using the PASS software program (Hintze, 2011)¹ and testing the following effect sizes for ANOVA specified by Cohen (1988)²:  small (f=0.10), medium (f=0.25), and large (f=0.40).  With the projected sample size, we would have greater than 90% power for small, medium, and large-sized effects. 

3. Methods to Maximize Response Rates and Deal with Non-response

This experimental study will use existing research panels to draw a sample.  The consumer and physician panels comprise of individuals who have signed up to participate in online studies. To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (clearly written and with appealing graphics);

• Administer the survey over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Field the survey for 2 to 4 weeks to allow participants reasonable time to access and complete the survey;

• Provide up to 2 e-mail reminders throughout the course of the field period;

• Provide a Member Services contact person for respondents to contact via email if they have questions or technical difficulty as they complete the survey.

There are several approaches to address the potential for nonresponse bias analysis in this study, such as comparing response rates by subgroups, comparing respondents and nonrespondents on frame variables, and conducting a nonresponse follow-up study.³ For the proposed project, we will compare responders and nonresponders on demographic variables.

4. Test of Procedures or Methods to be Undertaken

The stimuli and draft questionnaire were tested in a previous data collection, “Cognitive Interviews for Comparative Price Information in Direct-to-Consumer and Professional

Prescription Drug Advertisements,” (OMB Control Number 0910-0695). The cognitive testing examined the stimuli and draft measures to refine and improve question wording, narrow the pool of questions, and refine the wording of the context. In addition to this step, we will conduct pretesting to test and further refine the measurement pool to be used in the main study.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, RTI, will collect and analyze the data on behalf of FDA as a task order under Contract HHSF223201210375G. Brian Southwell, Ph.D., 919.541.8037, is the Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, with assistance from the Office of Biostatistics, CDER, and coordinated by Kathryn J. Aikin, Ph.D., 301-795-0569, and Kevin R. Betts, Ph.D., 240-402-5090.

1 Hintze, J. (2011). PASS 11. NCSS, LLC. Kaysville, Utah, USA. www.ncss.com.

2 Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd Ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc.

3 Office of Management and Budget, Standards and Guidelines for Statistical Surveys, September, 2006. www.whitehouse.gov/sites/default/files/omb/inforeg/statpc. Last accessed April 18, 2013.

5