Targeted MOA Study SS Part B 10-14-22 final

United States Food and Drug Administration

Targeted Mechanism of Action Presentations in Prescription Drug Promotion

OMB Control No. 0910-NEW

SUPPORTING STATEMENT

Part B. Statistical Methods

1. Respondent Universe and Sampling Methods

The project will recruit individuals from two populations: general population consumers (Study 1) and healthcare providers (HCPs) who specialize in oncology or primary care physicians (PCPs) with oncology experience (Study 2). For each study, we plan to conduct an online pretest and main study administered via an internet panel.

For the pretest and main study, the general population consumer sample will be drawn from a participant panel managed by Toluna. The Toluna community provides access to nearly 6.4 million members in North America, recruited using various methods including web banners, website referrals, pay-per-click, natural search optimization, affiliate marketing, email, and online public relations activities. Toluna’s panel is reflective of the population in the United States and ensures demographic diversity of participants’ genders, ages, race/ethnicity and education levels.

The HCP sample for the pretest and main study will be drawn from Toluna’s Curizon panel, which is an HCP panel used exclusively for healthcare research. Currently, the Toluna panel has access to 5,500 oncologists, 11,950 PCPs, and 2,800 nurse practitioners (NPs) and physician assistants (PAs) in the United States. This panel will be supplemented with partner panels from the Toluna Affiliate Network to reach the total required sample size. Toluna reviews opt-in policies, privacy policies, legal guidelines, and geographic locations of the partner panels and ensures that they follow all industry guidelines and provide adequate responses to ESOMAR’s 28 Questions. Every partner panel is automatically and continuously monitored to gauge performance.

Using a screening questionnaire, we will recruit 720 individuals (n = 360 for the general population consumer study and n = 360 for the HCP study) for two pretests, and 1,080 individuals (n = 540 for the general population consumer study and n = 540 for the HCP study) for two main studies. The eligibility criteria are shown in Table 1.

Table 1. Eligibility Criteria

Study	Inclusion Criteria
Study 1 and Study 2	18 years or older Able to speak and read English Not employed by the Department of Health and Human Services, pharmaceutical companies, or marketing industry Pretest and main study only: Take the survey on desktop, laptop, or tablet Main study only: Did not participate in the study pretest
Study 1 (general population consumers)	Not employed in the healthcare industry
Study 2 (HCPs)	HCPs with oncology experience (to include a mix of physicians, NPs, and PAs with a specialty in oncology and PCPs with oncology experience) Have prescribing authority Prescribe at least one oncology medication in a typical month Provide direct patient care for at least 45% of the time

The HCP sample will include oncologists and PCPs with at least some oncology experience (operationalized as PCPs who wrote at least one prescription for an oncology medication in the last 30 days). The sample will also include nurse practitioners (NPs) and physician assistants (PAs) with a specialty in oncology. As demands on physicians’ time continue to increase, the role of NPs and PAs continues to grow. PAs and NPs hold prescription privileges in all 50 states and Washington, DC, and an analysis of prescriptive authority legislation from 2001 to 2010 revealed that NP and PA state regulations have relaxed, giving both more autonomy in recent years (Herman, 2017). These HCPs play an important role in prescribing medication; thus, it is important for FDA to understand their interpretations of terminology and graphics in prescription drug promotion materials. The pretest and main study samples will include an approximately equal representation of the three groups of HCPs (i.e., 34% oncologists, 33% PCPs with oncology experience, and 33% NPs/PAs with a specialty in oncology).

Participants will be volunteers. We will recruit pretest samples from the same populations as the main study samples, and we will aim for a mix of participants in terms of race/ethnicity, gender, and other characteristics.

Participants will be convenience samples, rather than probability-based samples, of U.S. consumers (Study 1) or HCPs/oncologists (Study 2). In both the pretest and main study, participants will be randomly assigned to one of the 12 experimental conditions within each study. No weighting of the data will be required because the objective of the studies is to estimate the causal effects of experimental manipulations rather than to estimate descriptive statistics for these populations (Solon, Haider, & Wooldridge, 2015). The strength of the experimental design used in this study lies in its internal validity, on which meaningful estimates of differences across conditions can be produced and generalized.

2. Procedures for Collection of Information

Toluna will recruit general population consumers and HCPs from their respective panels by sending email or router invitations to panelists who likely fall within the target population for this project based on their profile information. The invitation will include a secure, nonidentifiable link to the web-based survey. Toluna will use predicted response rates for these target groups to avoid overcontacting panelists.

In the email invitation, panelists will be informed about the survey topic in a topline, nonleading way before participation. Each email invitation will describe the approximate survey length (i.e., approximately 20 minutes), the incentive amount provided for successful completion of the survey, and instructions for accessing the survey through a secure link. To avoid self-selection bias, Toluna will not disclose project details, such as the true purpose of the study, in its email invitations to participants. Individuals who are interested in completing the survey will be able to access it by clicking on a link embedded in the email invitation or in the router invitation. Participants will complete screener items and be presented with a more in-depth description of the survey and the consent form to inform the panelist that the survey is confidential and voluntary. Participants will not be allowed to take the survey on mobile phones.

Panelists are compensated for taking part in surveys using a structured incentive scheme that reflects the length of the survey and the nature of the sample. Recruitment will continue until the target sample size for completed surveys is reached. Participants can complete each survey only once and will be informed that once they leave the survey, they will not be able to re-enter it (therefore, it needs to be completed in one sitting). Toluna estimates that the fielding for the pretest will take 5 to 7 weeks and for the main study the fielding will take 7 to 9 weeks.

Throughout the pretest and main study, we will closely monitor recruitment and data collection to ensure that the screening criteria are met, participants from key demographic groups are adequately represented, and the response rates are acceptable. We will also conduct a soft launch check (when we reach 10% of the target completes) to ensure that participants’ responses are captured accurately.

Randomization Procedures

The pretest with general population consumers will be conducted concurrently with the pretest with HCPs; however, the two pretests will have independent randomization schemes. The same randomization approach will be employed for the main study where the main study with general population consumers will be conducted concurrently with the main study with HCPs but have independent randomization schemes. We will use randomization without replacement where participants will be randomized within groups of size equal to the number of conditions. The block randomization scheme will be set up in advance and programmed into the survey. Participants will be assigned to a block and condition as soon as they click through the consent form. This approach will help equalize sample sizes across conditions, although some variation in sample size will still occur because some participants who are randomized to an experimental condition may drop out or be considered incomplete surveys. No experimental conditions will be closed until each study reaches the target sample size.

Analysis Plan

We will conduct a series of analyses of variance (ANOVAs) tests and logistic regressions to examine the impact of the presence/absence of the MoA claim, the type of MoA graphic and the presence/absence of MoA related disclosure. Before conducting these analyses, we will assess whether the inclusion of covariates in the model is justified. If the ANOVA tests or logistic regressions for the main effect of the three-level factor (i.e., type of MoA graphic) or interaction effects are significant, we will implement a series of planned contrasts to test for significant differences among experimental arms. Within each study, the analyses will be conducted separately for consumers and HCPs.

Power

Pretest. The proposed sample size for each pretest is 360 (n = 30 per experimental condition), which will allow for thorough examination of the survey administration process and an assessment of the survey timing. No power analysis was conducted because the purpose of the pretest will be to test the stimuli, instruments, and procedures.

Main Studies. The proposed sample size for each of the 12 experimental groups (2 x 2 x 3 factorial design) is roughly 45 (N = 540) per study sample (Table 2). Our power analysis suggests that with a sample of 540 participants, the omnibus F tests for analysis of variance (ANOVA) will be able to detect an effect size of F =.15 with power of .90 and an alpha equal to .05. Given FDA’s interest in examining the effects of disclosure on the targeted mechanism of action (MoA) claim and graphics, we assumed a total of 10 pairwise comparisons of interest for the two-way and three-way interactions. As such, using the same power and alpha levels, the main study sample size is also sensitive to detect moderately small differences (F =.18) for up to 10 nonorthogonal planned contrasts, assuming a Bonferroni-adjusted alpha of .005. Finally, with the proposed sample size, we will also be able to detect pairwise differences in proportions between experimental groups on dichotomous outcome variables as low as 24 percentage points for interactions and as low as 14 percentage points between levels of a single factor.

Table 2. Main Study Design and Estimated Sample Size per Cell

Study	Experimental Manipulations
	Disclosure Statement	Targeted MoA Claim	Targeted MoA Graphic
			Graphic 1	Graphic 2	No Graphic
Study 1 (Consumers) N = 540	Present	Present	45	45	45
		Absent	45	45	45
	Absent	Present	45	45	45
		Absent	45	45	45
Study 2 (HCPs) N = 540	Present	Present	45	45	45
		Absent	45	45	45
	Absent	Present	45	45	45
		Absent	45	45	45

3. Methods to Maximize Response Rates and Deal with Non-response

The pretests and main studies will use an existing research panel to draw a sample. The panel comprises individuals who have signed up to participate voluntarily in online studies. To help ensure that the participation rate is as high as possible, FDA will:

• Design an experimental protocol that minimizes burden (short in length, clearly written, and with appealing graphics);

• Administer the pretests and main studies over the Internet, allowing respondents to answer questions at a time and location of their choosing;

• Use incentives to increase response rates, reduce sampling bias, and reduce nonresponse bias; and

• Use both email and router invitations, which have been shown to be an effective strategy to increase response (Boudewyns et al. 2017).

4. Test of Procedures or Methods to be Undertaken

Two types of pretesting (qualitative and quantitative) are employed as a test of procedures and methods. The first type of pretesting—already conducted—is qualitative. Cognitive testing with nine individuals was used to refine study stimuli and questions. Additionally, as described in this package, one round of quantitative pretesting per population will be employed. Pretesting will be used to evaluate the procedures and measures used in the main study. Pretesting will have the same design as the main studies. The primary purpose of pretesting will be to test the questionnaire’s format, the data collection protocol, statistical measures, and any other considerations that may arise. Based on pretest findings, we will refine the survey questions and data collection process, as necessary, to optimize the full-scale study conditions.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing

Data

The contractor, RTI International, will collect and analyze data on behalf of FDA as a task order under Contract HHSF223201510002B. Mihaela Johnson, Ph.D., is the Project Director, (919) 990-8365. Review of contractor deliverables and supplemental analyses will be provided by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Kevin R. Betts, Ph.D., (240) 402-5090, and Helen Sullivan, Ph.D., (301) 796-4188.

References

Boudewyns, V., Tan, S., Betts, K.R., Aikin, K. J., and Squire, C. Contrasting the Effect of Router- versus Email-Based Recruitment on Invitation Response to Online Surveys. American Association for Public Opinion Research 72nd Annual Conference, New Orleans, LA, 2017.

Herman, J. Prescriptive authority update 2017. Contemporary Clinic. (2017, June 3). http://contemporaryclinic.pharmacytimes.com/journals/issue/2017/june2017/prescriptive-authority-update-2017

Solon, G., Haider, S.J., and Wooldridge, J. “What Are We Weighting For?” The Journal of Human Resources, 50(2), 301–316, 2015.

5