ss Part B 8-2-2017

B. Statistical

1. Respondent Universe and Sampling Methods

The study will be conducted with Research Now, a national market research firm. The sample will come from Research Now’s e-Rewards® Opinion Panel. There are approximately three million panel members in the e-Rewards panel.

Panel members will be invited to participate by receiving an e-mail invitation (Appendix C) and, if interested, can click on a hyperlink within the e-mail and gain access to the screener (see Appendix B). The sample will not be representative of the population, but soft quotas ensure recruitment of a demographically diverse sample with respect to gender, age, and education. Final sample sizes for pretest and main studies are 554 (277 each) and 1,876, respectively.

As detailed in the participant screener, to qualify for this study all participants will meet the following criteria:

• At least 18 years of age

• Experiencing a severe headache or migraine in the past three months or trying to lose weight in the past twelve months

• Not trained or employed as a health care professional

• Not working for a pharmaceutical company, an advertising agency, or a market research company

• Has not participated in marketing research within the last three months

• Uses the Internet once a week or more

The screener will also ask participants to provide other demographic information. After participants are screened, those who are eligible will be randomly assigned to conditions.

2. Procedures for the Collection of Information

Design Overview

Two pretests will be conducted to test the goal instructions, stimuli, questionnaire, and procedure. In Studies 1-4, participants will be randomly assigned to one experimental condition and will view the corresponding study materials (Tables 1-4). Across all studies, we will examine two different character-space-limited formats and two medical conditions. For Pretest 1 and Study 1, the study materials will be a character-space-limited communication about a fictional weight loss drug, embedded in a Google search page about weight loss. The Study 2 materials will be a character-space-limited communication about a fictional drug to treat migraine, embedded in a Google search page about migraine. The Study 3 materials will be a character-space-limited communication about a fictional weight loss drug, embedded in a Twitter search page about weight loss. The Pretest 2 and Study 4 materials will be a character-space-limited communication about a fictional drug to treat migraine, embedded in a Twitter search page about migraine.

Table 1.--Study 1: Google Sponsored Link, Weight Loss
			Motivation
			General search		Learn about treatments
			Risk only landing page	Risk and benefit landing page	Risk only landing page	Risk and benefit landing page
Mobile	Risk Location	In character space-limited communication
		On linked webpage only
Desktop/Laptop	Risk Location	In character space-limited communication
		On linked webpage only

Table 2.--Study 2: Google Sponsored Link, Migraine
			Motivation
			General search		Learn about treatments
			Risk only landing page	Risk and benefit landing page	Risk only landing page	Risk and benefit landing page
Mobile	Risk Location	In character space-limited communication
		On linked webpage only
Desktop/Laptop	Risk Location	In character space-limited communication
		On linked webpage only

Table 3.--Study 3: Twitter, Weight Loss
			Motivation
			General search		Learn about treatments
			Risk only landing page	Risk and benefit landing page	Risk only landing page	Risk and benefit landing page
Mobile	Risk Location	In character space-limited communication
		On linked webpage only
Desktop/Laptop	Risk Location	In character space-limited communication
		On linked webpage only

Table 4.--Study 4: Twitter, Migraine
			Motivation
			General search		Learn about treatments
			Risk only landing page	Risk and benefit landing page	Risk only landing page	Risk and benefit landing page
Mobile	Risk Location	In character space-limited communication
		On linked webpage only
Desktop/Laptop	Risk Location	In character space-limited communication
		On linked webpage only

Procedure

All studies will be 20 minutes long and conducted using an Internet panel. Participants who enter the study on a mobile device will be included in the Mobile conditions, and participants who enter the study on a desktop or laptop will be included in the Desktop/Laptop conditions. We will use quotas to ensure equal sample sizes across device type. Participants will be randomly assigned to see one version of the study materials. All study materials will allow for scrolling and clicking on any links. The study materials will be accessible by participants only. After viewing the study materials, participants will complete a questionnaire that assesses participants’ retention of the risk information and their perceptions of the drug’s risks and benefits (Appendix B). We will also measure covariates such as demographics and health literacy.

Participants

All participants will be 18 years of age or older. We will exclude individuals who work in healthcare or marketing. Half of the studies will have a sample of participants who self-report needing to lose 30 pounds or more; the other half will have a sample of participants who self-report suffering from migraines or severe headaches. We selected these samples to increase the likelihood that participants will be interested in the fictitious study drugs and therefore motivated to pay attention during the study. The studies will be conducted with an Internet panel. Panel members can only participate in one of the studies and participants cannot have participated in the pretests for these studies.

Hypotheses

We hypothesize that participants who see substantive risk information in the character-space-limited communication, compared with link-only participants, will have greater retention of the risk included in the communication and higher perceived risk. We will explore whether including benefit information on the landing page affects retention and perceptions. We will also explore whether including substantive risk information in the character-space-limited communication affects the likelihood that participants notice the communication or click the link to the risk information.

We hypothesize that participants with a search goal, compared with a browse goal, will have greater retention of the benefit and risk information and higher perceived risk because they will be more likely to notice the character-space-limited communication and to click the link to the risk information. We will test these hypotheses in Studies 1-4 to determine whether these effects hold across different medical conditions and different character-space-limited platforms. We will also explore whether differences emerge between mobile and desktop/laptop participants.

Analysis Plan

We will conduct ANOVAs (for continuous variables) and logistic regressions (for dichotomous variables) with interaction terms and planned comparisons to test the hypotheses outline above.

Power

The following tables show the power calculations for the main studies. The assumptions made in deriving the sample size for each pretest study were: 1) 0.80 power, 2) 0.10 alpha and 3) an effect size between small and medium. The assumptions made in deriving the sample size for each main study were: 1) 0.90 power, 2) 0.05 alpha and 3) an effect size between small and medium.

Table 5. – Study 1: A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, main effects, and interactions) to achieve power of 0.90 (Faul et al., 2007).
	Pretest			Main Study
Effect size f*	0.10	0.15	0.20	0.10	0.15	0.20
α error probability	0.10	0.10	0.10	0.05	0.05	0.05
Power (1 – β error probability)	0.80	0.80	0.80	0.90	0.90	0.90
Numerator df	1	1	1	1	1	1
Number of groups	16	16	16	16	16	16
Total Sample Size	620	277	157	1053	469	265

*An effect size of 0.10 is traditionally considered small, whereas an effect size of 0.25 is considered medium (Cohen, 1988).¹ Here we have shown three different effect sizes centering around small to medium effects.

We plan to have 277 participants in each pretest (total = 554) and 469 participants in each main study (total = 1,876). With these sample sizes, we will have sufficient power to detect small-to-medium sized effects.

3. Methods to Maximize Response Rates and Deal with Non-response

This experimental study will use an existing research panel to draw a sample.  The panel comprises individuals who have signed up to participate in online studies.  To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the experiment over the Internet, allowing respondents to answer questions at a time and location of their choosing;

4. Test of Procedures or Methods to be Undertaken

We will conduct nine hour-long qualitative interviews to cognitively test the study stimuli and materials. Two pretests will be conducted to test the goal instructions, stimuli, questionnaire, and procedure. Finally, we will run the main studies as described elsewhere in this document.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Fors Marsh Group, will collect the data on behalf of FDA as a task order under Contract HHSF223201510003B. Caitlin Krulikowski, 571-858-3771, is the contractor’s Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Helen W. Sullivan, Ph.D., MPH, 301-796-4188, and Amie C. O’Donoghue, Ph.D., 301-796-0574.

1 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

2