Supporting Statement Part B - Endorser Status 2020

United States Food and Drug Administration

Endorser Status and Explicitness of Payment in Direct-to-Consumer Promotion

OMB Control No. 0910-NEW

SUPPORTING STATEMENT

Part B. Statistical Methods

1. Respondent Universe and Sampling Methods

Populations selected for the Study 1 pretest and main study will be drawn from Kantar Lightspeed’s proprietary LifePoints Panel, which reaches over 21,000,000 people in the United States using double opt-in methods. Kantar Lightspeed will purposefully pull a sample of respondents from their LifePoints Panel based on Census quotas and benchmarks specified by Westat and FDA. The sample will include respondents who are 18 or older, who are comfortable reading in English, and who are either extremely familiar or very familiar with the celebrity endorser used in the print ad. The sample will comprise 50% male and 50% female participants. Responses will be monitored, and if the desired diversity across race, ethnicity, and education are not being achieved, then quotas will be implemented to supplement the sample.

For the Study 1 pretest and main study, potential respondents will complete an online screener at the beginning of the web questionnaire. Those who are eligible to participate will receive a message on their screen indicating they qualified for the study and will immediately begin viewing the fictitious print ad. The online screener questions are presented in Appendix B.

In order to encourage respondents to complete the Study 1 pretest and main study surveys, up to two reminder emails will be sent to the selected sample within 3 days of the survey launch date.

Study 2 will use different populations for the pretest and the main study. The Study 2 pretest sample will be drawn from Kantar Lightspeed’s proprietary LifePoints Panel. Respondents selected for Study 2 pretest will be adult females age 18 years or older, who are comfortable reading in English, daily or weekly users of Instagram, and extremely or very familiar with either Tonya Michelle, the influencer used in the fictitious Instagram post, or at least one other well-known influencer that targets a similar audience. For the Study 2 pretest, Kantar Lightspeed will also purposefully pull a sample of respondents from their LifePoints Panel based on Census quotas and benchmarks specified by Westat and FDA. As with Study 1, responses will be monitored, and if the desired diversity across race, ethnicity, and education are not being achieved, then quotas will be implemented to supplement the sample.

The Study 2 main study sample will comprise influencer Tonya Michelle’s Instagram followers. Respondents selected for Study 2 main study will be adult females age 18 years or older, who are comfortable reading in English, daily or weekly users of Instagram, and extremely or very familiar with Tonya Michelle. The influencer will post a series of notices that mention her journey with endometriosis and ask her followers to complete a survey for a study being conducted by the FDA about prescription drug education materials. The survey link will be provided to interested respondents in the influencer’s Instagram bio. (See Appendix C for Study 2 recruitment Instagram posts and Instagram stories.)

For the Study 2 pretest and main study, potential respondents will also complete an online screener at the beginning of the web questionnaire. Again, those who are eligible to participate will receive a message on their screen indicating they qualified for the study and will then view the fictitious Instagram post. The online screener questions are presented in Appendix B.

For Study 2, it is important to note that the recruitment criteria will be different between the pretest and the main study. For the Study 2 pretest, potential respondents will be terminated if they select “Not at all familiar” and/or “Somewhat familiar” for all of the influencers listed. In addition, potential respondents who report using Instagram “Less Often,” will also be terminated from the pretest. For the Study 2 main study, potential respondents will be terminated if they are not at all familiar or only somewhat familiar with Tonya Michelle.

In order to encourage respondents to complete the Study 2 pretest survey, up to two reminder emails will be sent to the selected sample, within three days of the survey launch date. In order to do the same for the Study 2 main study, the influencer will post two recruitment Instagram posts and three recruitment Instagram stories over the study field period.

For both Study 1 and Study 2, participants will be excluded from participation if they work for a pharmaceutical, advertising or market research company and/or are employed by the U.S. Department of Health and Human Services.

2. Procedures for the Collection of Information

Part A of the supporting statement described the rationale for conducting the study.

We propose to extend previous research by examining four types of endorsers in two separate studies (celebrity, physician, patient, non-celebrity influencer¹) and examining whether the presence of a disclosure of their payment status influences participant reactions. We propose to also test two different types of disclosure language—one direct and more consumer-friendly, and one less direct.

To complete this research, we propose the following concurrent studies.²

Study 1

Table 1a: Study 1 Design – Pretest (0.80 power, 0.10 alpha, small effect size f=.2)

Payment Disclosure	Endorser			Total
	Celebrity	Physician	Patient
Present	50	50	50	150
Absent	33	33	33	99
Total	83	83	83	249

Table 1b: Study 1 Design – Main Study (0.90 power, 0.05 alpha, small effect size f=.2)

Payment Disclosure	Endorser			Total
	Celebrity	Physician	Patient
Present	81	81	81	243
Absent	54	54	54	162
Total	135	135	135	405

Study 1 will manipulate endorser type (three levels: celebrity, physician, patient) and payment disclosure (two levels: present, absent) within a print DTC ad for a fictitious acne product. For this study, we will recruit 654 general population individuals (249 pretest; 405 main study; Tables 1 and 2) from the Kantar Lightspeed national nonprobability internet panel. All participants must report familiarity with the celebrity to be included in our study. The celebrity will be one who has publicly spoken out about acne. Stock photos will be used to depict a physician and a patient in the other experimental conditions. Participants will be randomly assigned to see one of the endorsers and to see the ad either with or without a payment disclosure. The payment disclosure in Study 1 will be determined in cognitive testing, but will be similar to: “[Endorser] has been paid to appear in this ad for Drug X.”

Study 2

Table 2: Study 2 Design – Pretest (0.80 power, 0.10 alpha, small effect size f=.2)

Payment Disclosure	Endorser		Total
	Influencer	Patient
Present-Direct	50	50	100
Present-Indirect	50	50	100
Absent	33	33	66
Total	133	133	266

Table 2: Study 2 Design – Main Study (0.90 power, 0.05 alpha, small effect size f=.2)

Payment Disclosure	Endorser		Total
	Influencer	Patient
Present-Direct	81	81	162
Present-Indirect	81	81	162
Absent	54	54	108
Total	216	216	432

In Study 2, we will also manipulate endorser type, examining a patient and an internet influencer, one who provides online content to a number of followers. We will also manipulate the explicitness of the payment disclosure, resulting in a 2 (endorser: influencer, patient) by 3 (payment disclosure: present-direct, present-indirect, absent) between-subjects design. The disclosure will be direct (e.g., “Paid ad…”), indirect (e.g., #sp for “sponsored”), or absent. The setting for this study will be an Instagram post for a fictitious endometriosis product. This study partially replicates Study A and extends it by further examining the explicitness of payment as another manipulated variable and using a different set of endorser types and in a different promotional setting.

For Study 2, we will recruit 698 (266 pretest; 432 main study; Tables 2 and 3) followers of an internet influencer who maintains an Instagram page with more than 500,000 followers and has posted about endometriosis. As in the first study, we are not revealing the influencer’s identity in this public forum to maintain the integrity of the study.

In both studies, we are interested in the role of endorsement and disclosure of payment status on participants’ recall, benefit and risk perceptions, and behavioral intentions. Participants will view one promotional piece and answer questions via the Internet. The study is expected to take less than 20 minutes to complete. Dependent variables will include attention to disclosure statement and risk/benefit information; retention of risk/benefit information; recognition of piece as promotion and endorser as paid; perceived benefits and risks, attitudes toward the product, endorser, and ad; and behavioral intentions such as asking a doctor about the drug.

Research Questions

1A. Does the type of endorser influence attitudes toward the endorser, attitudes related to the ad, attitudes to the prescription drug, and/or behavioral intentions?

1B. Does the type of endorser influence viewers’ attention to, retention of, and perceptions about the risks and benefits of the prescription drug advertised?

2A. Do participants notice a payment disclosure?

2B. Does the presence and language of a disclosure, which explicitly reveals that the endorser was paid, influence attitudes related to the ads, and/or behavioral intentions?

2C. Does the presence and language of a disclosure, which explicitly reveals that the endorser was paid, influence viewers’ attention to, retention of, and perceived risks and benefits of the prescription drug advertised?

3. Does the presence and language of a disclosure, which explicitly reveals that the endorser was paid, influence attitudes toward the endorser? Does that then influence attitudes toward the product and risk/benefit perceptions?

4. Do endorser type and disclosure interact to influence attitudes toward the endorser? Does that then influence attitudes toward the product and risk/benefit perceptions?

5. What variables moderate the relationship between endorser and payment disclosure?

Analysis Plan

For ordinal or continuous dependent variables (such as a scale indicating the amount of risk information read by the respondent in the advertisement), 2-way ANOVAs will be conducted to assess both the main effects and the interaction effects of the two independent variables. For dichotomous dependent variables, logistic regression models will be used to include both experimental factors for determining statistically significant differences between groups.

In addition, we will also plan a priori contrasts to make comparisons between specific groups.

Finally, we will conduct mediation and moderation analyses. Mediation analysis will be performed using the PROC CAUSLMED procedure in SAS 9.4.

For moderation analyses, we will fit a regression including the independent variable, the moderating variable, and their interaction. Significance of the interaction term suggests that the moderator variable (e.g. disease state involvement) moderates the relationship between the independent variable and the study outcome variable.

Power

The necessary sample size for the Study 1 and Study 2 pretests was determined by conducting two power analyses using an alpha level of 0.10, a power of 0.80 and a small effect size of f = 0.2. Based on these analyses, we estimate needing a total of 249 completed surveys for the Study 1 pretest and 266 completed surveys for the Study 2 pretest in order to detect a small effect size. For case allocation, the literature suggests that some proportion of consumers may not recall seeing the disclosure statement in the advertisement (see, for example, Boerman, Willemsen, & Van Der Aa, 2017, footnote 2). Therefore, rather than allotting equal numbers of cases to each condition, we will assign more cases to the disclosure present conditions to increase power in these cells for the pretests. Tables 1 and 2 above provide further details of the Study 1 and Study 2 pretest sample allocations by cell.

The needed sample size for the Study 1 and Study 2 main studies was ascertained by conducting two power analyses using an alpha level of 0.05, a power of 0.90 and a small effect size of f = 0.2. Based on these analyses, we estimate needing 405 completed surveys for the Study 1 main study and 432 completed surveys for the Study 2 main study to detect a small effect size. Again, since the literature suggests that some proportion of consumers may not recall seeing the disclosure statement in the advertisement (Boerman et al., 2017), we will assign more cases to the disclosure present conditions in the main studies to increase power in these cells. Tables 2 and 4 above provide further details of the Study 1 and Study 2 main study sample allocations by cell.

3. Methods to Maximize Response Rates and Deal with Non-Response

Both the pretest and main studies will be administered via Internet. To help ensure that the participation rate is as high as possible, FDA and the contractor will:

• Design a protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Use incentive rates that meet industry standards. In addition to offsetting respondent burden, using market-rate incentives tends to increase response rates, reduce sampling bias, and reduce nonresponse bias.

Participants in the pretest and main studies will be convenience samples, rather than probability-based samples of U.S. adults. Rather, the strength of the experimental design used in this study lies in its internal validity, on which meaningful estimates of differences across manipulated conditions can be produced and generalized. This is a counterpoint to observational survey methodologies where estimating population parameters is the primary focus of statistical analysis. The recruitment procedures in this study are not intended to fit the criteria for survey sampling, where each unit in the sampling frame has an equal probability of being selected to participate. In an observational survey study, response rates are often used as a proxy measure for survey quality, with lower response rates indicating poorer quality. Nonresponse bias analysis is also commonly used to determine the potential for nonresponse sampling error in survey estimates. However, concerns about sampling error do not generally apply to experimental designs, where the parameters of interest are under the control of the researcher—rather than being pre-established characteristics of the participants—and each participant has an equal probability of being assigned to any of the experimental conditions.

Generally, there are several approaches to conducting a nonresponse bias analysis, such as comparing response rates by subgroups, comparing respondents and nonrespondents on frame variables, and conducting a nonresponse follow-up study³. For the proposed project, we will examine nonresponse for its descriptive value by comparing our full sample with population estimates for age, race, and gender (Study 1).

4. Test of Procedures or Methods to be Undertaken

Nine cognitive interviews were conducted to assess questionnaire flow and wording.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Westat, will collect and analyze the data on behalf of FDA as a task order under Contract HHSF223201510001B. Jennifer Berktold, Ph.D., 301-294-3964, is the Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Amie C. O’Donoghue, Ph.D., 301-796-0574, and Helen W. Sullivan, Ph.D., M.P.H., 301-796-4188.

1A “non-celebrity influencer” is a person who has gained a following on a blog, a Twitter feed, or other social media outlet.

2For case allocation, the literature suggests that some proportion of consumers may not recall seeing the disclosure statement in the advertisement (see, for example, Boerman, S.C., L.M. Willemsen, and E.P. Van Der Aa (2017). “This post is sponsored” Effects of Sponsorship Disclosure on Persuasion Knowledge and Electronic Word of Mouth in the Context of Facebook.” Journal of Interactive Marketing, 38, pp. 82–92.). Rather than allotting equal numbers of cases to each condition, we will assign more cases to the disclosure present condition to increase power in these cells.

3 Office of Management and Budget, Standards and Guidelines for Statistical Surveys, September, 2006. www.whitehouse.gov/sites/default/files/omb/inforeg/statpc. Last accessed April 18, 2013.

6