SSB Med Tracking FINAL

United States Food and Drug Administration

Perceptions of Prescription Drug Products with Medication Tracking Capabilities

OMB Control No. 0910-NEW

SUPPORTING STATEMENT

Part B: Statistical Methods:

1. Respondent Universe and Sampling Methods

Primary care physicians (PCPs) and consumers will be recruited for participation in this research. The sample for the pretest and main study will be drawn from Kantar’s opt-in panel using a screener. National estimates will not be derived from the recruited groups, so there are no weighting procedures. The randomized experimental design, together with attention to obtaining a sufficiently diverse mix of participants, will ensure the internal validity of the results.

Kantar will recruit 50 consumers and 50 PCPs for the pretest and 350 consumers and 350 PCPs for the main study. Within each of the two groups (consumer and PCPs), individuals will be randomly assigned to one of five experimental conditions for both the pretest and main study. Efforts will be made to include a mix of demographic characteristics (age, gender, race/ethnicity, etc.) to the extent possible. Individuals participating in prior phases of the study will be excluded from the pretest and the main study.

To recruit participants, Kantar will initially send a recruitment email and screener to identify eligible PCPs (Appendix A1) and consumers (Appendix A2). The screener will include questions for all inclusion and exclusion criteria. To qualify for this study, PCPs must be board-certified in family medicine, in internal medicine, or as a general practitioner; spend at least 50 percent of their time on patient care; and treat 1 or more diabetes patients monthly. Eligible consumers must be 18 or older, have been diagnosed with type 2 diabetes by a healthcare provider, be currently receiving treatment for type 2 diabetes, and be comfortable speaking and reading English. Consumers who are employed as healthcare professionals will be ineligible for the consumer sample. PCPs and consumers who work for market research firms, advertising firms, pharmaceutical companies, or the U.S. Department of Health and Human Services are not eligible for the study. PCPs and consumers who have participated in focus groups or interview-based research in the last 3 months are also not eligible.

2. Procedures for the Collection of Information

Part A of the supporting statement described the rationale for conducting the study. We have the following specific questions:

Research questions:

1. When prescription drug promotional communications include claims about a product’s ability to track medication use, do these claims influence perceptions about the product’s risks and/or benefits (including its effect on medication adherence)?

2. If the promotional claims about the product’s ability to track medication use are accompanied by a disclosure that describes what is known about the effect of medication tracking on medication adherence, does this have an influence on perceptions of the product’s risks and/or benefits (including its effect on medication adherence)?

To complete this research, we propose the design in table 1, which varies based on:

• Whether the fictitious prescription drug product includes technology that tracks medication use,

• Whether the prescription drug promotional communication includes a disclosure describing what is known about the tracking technology’s effect on medication adherence, and

• What the disclosure communicates about the tracking technology’s effect on medication adherence (positive effect shown, no effect shown, or unknown effect).

Table 1--Proposed One-Way, Five-Level Design (1 x 5)
Experimental Condition	Claims About Existence of Medication Tracking Technology	Disclosure About Technology’s Effect on Adherence	Content of Disclosure
Drug	No	No	---
Drug + medication tracking technology	Yes	No	---
Drug + medication tracking technology + no adherence data collected	Yes	Yes	No data is available on the technology’s effect on adherence
Drug + medication tracking technology + data show no effect on adherence	Yes	Yes	Data show the technology has no effect on adherence
Drug + medication tracking technology + data show a positive effect on adherence	Yes	Yes	Data show the technology has a positive effect on adherence

Note: Condition 5 is the only condition in which an adherence benefit has been demonstrated for the fictitious product. The evidence required to support a medication adherence claim is not the focus of this study, and the evidence will not be described in the disclosure.

Condition 2 is a control because the drug product does include medication tracking technology, but the promotional communication does not include a disclosure about the technology’s effect on medication adherence. Condition 1 is a true control because the drug product does not include medication tracking technology. Comparisons between conditions 1 and 2 will show us the baseline of this issue, i.e., will indicate whether the fact that the drug product contains a tracking technology will alter perceptions of risks and benefits (including adherence).

Each participant will see one of five versions of a consumer webpage for a fictitious prescription diabetes treatment, as reflected in table 1. They will answer a survey designed to take no more than 20 minutes to complete regarding their perception of the product’s benefits, risks, and effect on adherence.

Hypotheses

FDA has identified four key research questions (RQs) for the study design in table 1. Below we present draft hypotheses for each question.

RQ1: Does the presence of a medication tracking device in a prescription drug product affect perceived adherence and efficacy?

Hypotheses: Research has found that in the theoretical case of medication monitoring for a chronic disease, participants reported that the technology would be an adherence-improving measure in itself (Ref. 1). This suggests people are making judgements about the product’s ability to increase adherence based solely on the type of product rather than any supporting evidence. These types of mental shortcuts or heuristics (Ref. 2) are used by both laypersons and professionals in making judgements, particularly where there is limited information or the information relates to an unfamiliar topic. While heuristics can be accurate and help reduce cognitive burden, they can also produce biased thinking. We expect both consumers and PCPs to use heuristics. However, PCPs will have more familiarity with medication tracking devices and will rely less on heuristics than consumers.

• We hypothesize consumers viewing a prescription drug use-related software (PDURS) product without any supporting information (Condition 2) will use heuristics and perceive greater adherence and efficacy compared to consumers viewing a non-PDURS product (Condition 1). We hypothesize a similar pattern for PCPs, but the mean difference between Condition 1 and Condition 2 among PCPs will be smaller than for consumers.

RQ2: Does a disclosure about the state of knowledge regarding success of the medication tracking device with regard to adherence affect perceived adherence and efficacy?

Hypotheses: Data supporting increased adherence for a medication tracking device should theoretically make a stronger case for adherence. However, if most consumers believe medication tracking devices increase adherence, then providing supporting information about adherence may have little influence above their prior beliefs. In contrast, we expect PCPs to value empirical data and therefore be more influenced by the presence of accompanying supportive evidence for a medication tracking device.

• We hypothesize that perceived adherence and efficacy will be similar for consumers who view supportive data for a medication tracking device (Condition 5) and consumers who view a medication tracking device without any supportive data (Condition 2). We hypothesize that PCPs assigned to Condition 5 will have greater perceived adherence and efficacy compared to PCPs assigned to Condition 2.

RQ3: Does the content of the disclosure (positive data, no effect shown, unknown) affect perceived adherence and efficacy?

Hypotheses: Some clinicians note that a lack of evidence of Abilify MyCite to improve treatment adherence makes it difficult for them to recommend the product to patients (Ref. 3). A disclosure that indicates there is no data to support increased adherence will highlight the lack of evidence and reduce perceived adherence and efficacy.

• For both consumers and PCPs, we hypothesize that the content of the disclosure will have an effect on perceived adherence and efficacy. In particular, a disclosure that indicates there is no data to support adherence for a medication tracking device (Condition 4) will result in lower perceived adherence and efficacy compared to a medication tracking device where the disclosure indicates there is no adherence data available (Condition 3). Additionally, we hypothesize Condition 5 will result in greater perceived adherence and efficacy compared to Condition 3 and Condition 4 between both consumers and PCPs.

RQ4: Do these effects differ by group (PCPs versus consumers)?

Hypotheses: As discussed earlier, we expect consumers will rely more on heuristics in making judgements about medication tracking devices compared to PCPs. Relative to consumers, we expect PCPs to be more familiar with medication tracking devices and value empirical data in informing their decisions. We expect to observe the greatest differences between PCPs and consumers in Condition 5.

• We hypothesize PCPs will have higher means for perceived adherence and efficacy for Condition 5 than consumers in Condition 5. Additionally, the mean difference in perceived adherence and efficacy between Condition 2 and Condition 5 will be greater among PCPs than consumers.

Power

For the pretest, no power analysis was performed. The primary goal of the pretest is to assess and refine the questionnaire, stimuli, and data collection protocols for the main study. Since all of these objectives are qualitative in nature, there is no need to increase public burden and cost by recruiting a large enough sample to make statistically valid comparisons.

For the main study, we performed power analyses for the consumer and PCP studies separately, consistent with the agreed-upon approach of treating these as two parallel studies. We also clarified with FDA that equal power is required for each pairwise comparison of any two experimental conditions, implying equal sample sizes for each experimental condition. The power analysis assumes that we are interested in comparing means between experimental conditions. That is, we are interested in pairwise t-tests of average scores on individual questionnaire items or number of side effects recalled; for example, where each group is an experimental condition. Because statistical power depends on both the difference between groups and the variation in each group, we use effect size in our power calculations. This is most frequently measured via Cohen’s d, which is calculated as:

where and represent the means of the two groups we want to compare, and is the estimate of the population standard deviation. An effect size of around 0.5 is considered a medium effect size, which we would like to detect in this study for each pairwise comparison (Refs. 4 and 5).

Using the software G*Power with an alpha level of 0.05 and power of 0.90, at least 70 cases in each group are required to detect a medium effect size. This implies an overall sample size of 350 (5 experimental conditions multiplied by 70 participants) cases for each of the consumer and PCP studies. Note that because this is an effect size-based calculation, the detectable difference in the means depends on the variability of respondents’ answers to the survey questions. If respondents in a given condition tend to answer a particular item consistently (and therefore there is little variability), we will be able to detect smaller differences in means; however, if participants’ responses vary widely, even a seemingly large difference in means may not be statistically significant.

We also performed a power analysis for an analysis of variance (ANOVA) of conditions 2 to 5. The ANOVA will be able to tell us whether outcomes vary significantly by experimental condition but will not tell us the nature of or direction of any differences—we need the pairwise t-tests for that. However, assuming a sample size of 70 cases per experimental condition and the same alpha level of 0.05 and desired power of 0.90, the ANOVA will be able to detect a smaller effect size (d=0.23, generally considered a medium-to-small effect size).

Analyses

The main analysis will be conducted separately for consumers and PCPs. We will first use t-tests to determine whether presence of a tracking device (Condition 1 versus Condition 2) has an effect on outcomes such as perceived product adherence and efficacy. Next, we will perform ANOVAs on conditions 2 to 5, testing whether manipulation of the other two experimental conditions in the presence of a tracking device has an impact on outcomes. Finally, if the ANOVA shows a statistically significant effect, we will use t-tests to compare outcomes in Condition 2 versus Condition 5.

A sample size of 350 consumers with the indicated medication condition and 350 PCPs will allow us to detect medium effect sizes for these three tests (one ANOVA and two t-tests). The sample will be evenly distributed among the five experimental conditions to support the detection of medium effects in pairwise t-tests. Note that the proposed sample size controls the pairwise error rates rather than the family-wise error rate (doing the latter would require a far larger sample size), so no explicit adjustment for multiple comparisons is planned.

3. Methods to Maximize Response Rates and Deal with Nonresponse

The study will be administered via the internet. To help ensure that the participation rate is as high as possible, FDA and the contractor will:

• Design a protocol that minimizes burden (short in length, clearly written, and with appealing graphics).

• Use incentive rates that meet industry standards. In addition to offsetting respondent burden, using market-rate incentives tends to increase response rates, reduce sampling bias, and reduce nonresponse bias.

Participants will be convenience samples, rather than probability-based samples of U.S. consumers or U.S. PCPs. Rather, the strength of the experimental design used in this study lies in its internal validity, on which meaningful estimates of differences across manipulated conditions can be produced and generalized. This is a counterpoint to observational survey methodologies, where estimating population parameters is the primary focus of statistical analysis. The recruitment procedures in this study are not intended to fit the criteria for survey sampling, where each unit in the sampling frame has an equal probability of being selected to participate. In an observational survey study, response rates are often used as a proxy measure for survey quality, with lower response rates indicating poorer quality. Nonresponse bias analysis is also commonly used to determine the potential for nonresponse sampling error in survey estimates. However, concerns about sampling error do not generally apply to experimental designs, where the parameters of interest are under the control of the researcher—rather than being pre-established characteristics of the participants—and each participant has an equal probability of being assigned to any of the experimental conditions.

Generally, there are several approaches to conducting a nonresponse bias analysis, such as comparing response rates by subgroups, comparing respondents and nonrespondents on frame variables, and conducting a nonresponse follow-up study. We will obtain age, race, and gender demographics of nonrespondents from Kantar and compare descriptively to those of respondents, independently for both the consumer and PCP samples, to assess any potential risk of nonresponse bias.

4. Test of Procedures or Methods to be Undertaken

As part of study development, we conducted a literature review to inform our knowledge of existing relevant research. After development of the survey instrument, we conducted nine cognitive interviews to assess wording, flow, and potential misunderstandings. Upon approval, we will conduct pretesting as described elsewhere in this document.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Westat, will collect and analyze the data on behalf of FDA as a task order under Contract 75F40120A00018. Simani Price, Ph.D., 301-610-5536, is the Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA and coordinated by Amie C. O’Donoghue, Ph.D., 301-796-0574, and Kathryn J. Aikin, Ph.D., 301-796-0569.

References

1. Chai, P.R., S. Carreiro, B.J. Innes, et al., “Oxycodone Ingestion Patterns in Acute Fracture Pain With Digital Pills,” Anesth Analg, vol. 125, Issue 6, pp. 2105–2112, 2017, doi:10.1213/ANE.0000000000002574.

2. Tversky, A. and D. Kahneman, “Judgment Under Uncertainty: Heuristics and Biases,” Science, vol. 185, Issue 4157, pp. 1124–1131, 1974, doi:10.1126/science.185.4157.1124.

3. Papola, D., C. Gastaldon, and G. Ostuzzi, “Can a Digital Medicine System Improve Adherence to Antipsychotic Treatment?,” Epidemiol Psychiatr Sci, vol. 27, Issue 3, pp. 227–229, 2018, doi:10.1017/S2045796018000082.

4. Cohen, J., Statistical Power Analysis for the Behavioral Sciences, 2nd ed., Routledge, 1988, ISBN: 978-1-134-74270-7.

5. Sawilowsky, S., “New Effect Size Rules of Thumb,” Journal of Modern Applied Statistical Methods, vol. 8, Issue 2, pp. 467–474, 2009, doi:10.22237/jmasm/1257035100, available at http://digitalcommons.wayne.edu/jmasm/vol8/iss2/26.

7