Supporting Statement Part B Final 4-30-14

B. Statistical Methods (used for collection of information employing statistical methods)

1. Respondent Universe and Sampling Methods

For the pilot and main study, the sample will be drawn from EyeTracking Inc.’s opt-in national panel for eye tracking studies containing about 120,000 members. All testing will take place in professional research facilities in the cities chosen for the study. We expect to fill the samples from EyeTracking Inc.’s panel. However, it is possible that we will have last-minute cancellations requiring us to go to the facility in which we are testing for assistance in recruiting additional participants (i.e., it may be necessary to supplement with off-panel members). These additional participants will experience the same screener questionnaire, incentives, and testing procedure as all other participants.

Eligible participants will be adults who speak English and self-identify as needing to lose more than 30 pounds. We will exclude individuals who work in the health care, marketing, advertising, or pharmaceutical industries and individuals who wear bifocals or hard contact lenses to watch television. We will also exclude pilot study participants from the main study.

The sample will not be representative of the population, but for the main study, the goal of the sample will be to recruit participants with approximately an even number of males and females, and at least 30% having some college (but no degree) or less education.

Participants will be recruited from diverse geographical areas throughout the United States such as Washington, DC; Chicago; San Diego; Denver; Dallas; and New York City. For the pilot study, we plan to recruit 35 individuals in the Washington, DC area in order to have 30 participants. For the main study, we plan to recruit 65 individuals in each of the five targeted areas in order to have 60 participants (for a total of 300 participants in the main study). We will identify key zip code areas for both studies and recruit from Eye Tracking Inc.’s database within a reasonable proximity to those zip codes.

We will initiate the recruiting process by sending an email to a random subset of eye-tracking panel members inviting them to come to Eye Tracking Inc.’s website to fill out a screener for an eye tracking study (See Appendix C for recruitment emails and Appendix D for the screener). Participants can respond to the email blast by following a link to a participant sign up /log in page on the website (www.eyetracking.com/signup) where they log in to their account. They will then be directed to the screener page for this study where they will respond to the screener. After filling out the screener questionnaire online, each qualifying respondent will be shown a list of available times in his or her location and can schedule an appointment time of choice.

2. Procedures for the Collection of Information

Design Overview

This project is designed to use eye tracking technology to determine 1) how superimposed risk information and the MedWatch statement are perceived in DTC ads and 2) the effect of distraction. Eye tracking technology is an effective method to determine the extent to which consumers attend to risk information presented in DTC television ads. This technology allows researchers to unobtrusively detect and measure where a participant looks while viewing a television ad and for how long, and the pattern of their eye movements may indicate attention to and processing of information in the ad. We plan to collect descriptive eye tracking data on participants’ attention to: (1) The superimposed text during the major statement of risk information; and (2) the MedWatch statement. Further, we plan to examine experimentally the effect of distraction, testing two levels of distraction: low and high.

Procedure

We plan to conduct one 60-minute pilot study with 30 participants and then one 30-minute main study with 300 participants. The studies will be conducted in person in at least five different cities across the United States.

The pilot study and main study will have the same design and will follow the same procedure. Participants will be randomly assigned to one of two test conditions (low and high distraction in a DTC television ad). The ad will be for a fictitious weight loss prescription drug. The ads were created and pretested to ensure that consumers perceive different levels of distraction across the ads (OMB Control Number 0910-0695; “Stimuli Development and Pretests for an Attentional Effects Study”). For instance, as the distraction level increases, the number of scene changes and on-screen activity during the major statement increases. Based on pretest findings, we will use two rather than three ads, as we proposed in the 60-day FRN.

When participants start the study, we will explain the study procedure and calibrate the eye tracking device. To collect eye tracking data, we will use an unobtrusive computer-interfaced eye tracker with a minimum speed of 60 Hz. The test images will be shown on a computer monitor with a minimum size of 23 inches and a minimum display resolution of 1,920 x 1,080. To simulate normal television ad viewing, participants will watch an approximately 5 minute video clip followed by a series of three ads. One of the ads will be the study ad. The video clip and non-study ads will be unrelated to health. The order of the non-study ads will be counterbalanced, and only eye tracking data from the study ad will be analyzed. Next, participants will complete a questionnaire that assesses risk perceptions, risk recall, recall of the MedWatch statement, and covariates such as demographics and health literacy. In the pilot study, participants will also answer questions as part of a debriefing interview to assess the study design and questionnaire (Appendix B).

Participants

Eligible participants will be adults who speak English and self-identify as needing to lose more than 30 pounds. We will exclude individuals who work in the health care, marketing, advertising, or pharmaceutical industries and individuals who wear bifocals or hard contact lenses to watch television. We will also exclude pilot study participants from the main study.

Hypotheses

We hypothesize that distracting audio and visuals during the major statement will decrease risk recall, risk perceptions, and attention to super-imposed text during the presentation of risk information and to the Medwatch statement. We expect that attention to super-imposed text during the presentation of risk information will increase risk recall and risk perceptions. We also expect that attention to the Medwatch statement will increase recall of the Medwatch statement. We will also describe how much attention people pay to the super-imposed text during the presentation of risk information and the Medwatch statement and explore whether attention to the Medwatch statement affects risk recall and risk perceptions.

Analysis Plan

For the pilot study, we will conduct descriptive analyses in order to become familiar with the data and to engage in data screening. We will calculate frequency distributions and check the validity of the data (i.e., range checks, frequency of missing responses, response distribution).

For the main study, we will use descriptive data to determine how much attention participants pay to the super-imposed text and Medwatch statement (captured through eye-tracking). We will use one-way between-subjects ANOVAs and chi-squares to test the hypotheses outline above. Graphic visualizations will be created for the statistically significant outcomes in order to illustrate the key findings.

Power

The following tables show the power calculations for the main study (we do not plan to conduct formal hypothesis testing in the pilot study). The assumptions made in deriving the sample size for each study were: 1) 0.90 power, 2) 0.05 alpha and 3) an effect size between small and medium. The tables below shows the sample size required to detect differences with effect sizes seen in the pretests, f = 0.18 and f = 0.23.

Table 3. – Main Study: A priori power analysis to determine sample size needed in F tests (ANOVA: fixed effects, omnibus, one-way) to achieve power of 0.90 (Faul et al., 2007).1
		Main effect of distraction Effect size f*
Input
		0.18	0.23
	α error probability	0.05	0.05
	Power (1 – β error probability)	0.90	0.90
	Numerator df	1	1
	Number of groups	2	2
Output
	Critical F	3.87	3.89
	Denominator df	326	200
	Total Sample Size	328	202

*An effect size of 0.10 is traditionally considered small, whereas an effect size of 0.25 is considered medium (Cohen, 1988).² Here we have shown three different effect sizes centering around small to medium effects.

Based on the power analysis, we will have 150 participants per cell, with a total of 300 participants.

3. Methods to Maximize Response Rates and Deal with Non-response

This experimental study will use an existing research panel to draw a sample.  The panel comprises individuals who have signed up to participate in eye tracking studies.  To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Send a confirmation email automatically to each participant with details about the study, appointment time/length, contact details, and directions to the test location once an appointment has been scheduled

• Call to remind each participant of the appointment and to answer any questions or concerns that he or she may have regarding the study a short time before the actual test date

4. Test of Procedures or Methods to be Undertaken

In a previous data collection (OMB Control Number 0910-0695; “Stimuli Development and Pretests for an Attentional Effects Study”) we created and tested the stimuli (DTC television ads) to be used in this study. As described above, we will run a pilot study to test the methods to be used in this study.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, RTI, will collect and analyze the data on behalf of FDA as a task order under Contract HHSF223200910135G. Pamela Williams, Ph.D., 919-316-3936, is the Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Helen W. Sullivan, Ph.D., M.P.H., 301-796-4188, and Amie C. O’Donoghue, Ph.D., 301-796-0574.

1 Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A, (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39, 175-191.

2 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2^nd Ed). Hillsdale, NJ: Lawrence Erlbaum & Associates, Inc.

5