Supporting Statement Part B Terms and Phrases 2020

United States Food and Drug Administration

Assessment of Terms and Phrases Commonly Used in Prescription Drug Promotion

OMB Control No. 0910-NEW

SUPPORTING STATEMENT

Part B. Statistical Methods

1. Respondent Universe and Sampling Methods

The Phase 1 interviews will involve 30 members of the general population (consumers) and 30 PCPs. Participants will be recruited by email through itracks (an online and mobile market research service provider) and its partner panels. itracks regularly partners with market research firms such as Plaza Research and Schlesinger Group. Each of these market research firms maintains databases of eligible individuals who can be prescreened for specific characteristics. A decision on which firm(s) to work with will be made in conjunction with RTI. Potential participants will receive an invitation to complete an online screener by e-mail. Once identified as eligible, a selection of potential participants will receive an e-mail inviting them to sign up for an available interview session using itracks’ interview time selection feature. They will also see a copy of the informed consent form at this time and will be asked to provide consent prior to signing up for an appointment. After signing up for a session, participants will receive a confirmation of their appointment time and an e-mail invitation through Microsoft Outlook that will include their appointment time, a link to the itracks login page, and the phone number if they choose to call in themselves (they may also sign up to receive a call through the itracks system that will connect them with the interviewer).

Consumers sampled for the Phase 1 interviews must be aged 18 or older, must not have participated in a focus group or interview during the previous 3 months, and must not work for FDA, the Department of Health and Human Services (DHHS), RTI, in a health-related field (e.g., at a health care practice, a pharmaceutical company, or hospital), or for a market research firm. We will aim to screen for a diverse participant sample with consideration for age, gender, race/ethnicity, education, income, and region.

PCPs sampled for the Phase 1 interviews must specialize in family medicine, internal medicine, or be a general practitioner, spend at least 50% of their time in direct patient care, and practice medicine at least 32 hours per week. Additionally, PCPs sampled must not have participated in a focus group or interview during the previous 3 months, and must not work for FDA, DHHS, or RTI. We will aim to screen for a diverse participant pool with consideration for gender, race/ethnicity, type of practice (solo, small group, large group), years in practice, part of an academic or health care system, and region.

The same inclusion and exclusion criteria as used for the Phase 1 interviews will be applied for the Phase 2 surveys, except for the requirement regarding interview or focus group participation.

The Phase 2 surveys will involve over 1,000 members of each population, general population consumers and PCPs. For the consumer survey, we will use a probability sample selected from an ABS frame, which is derived from commercially available versions of the U.S. Postal Service’s Computerized Delivery Sequence and No-Stat files, and conduct the survey using a web-based platform. For the PCP survey, we will obtain a probability sample from the AMA Masterfile, which is the most comprehensive list of physicians in the United States, and conduct the survey via mail. For each population, we chose the sampling frame and survey mode that has been shown to produce the highest quality results for that population with respect to coverage, response rates, and nonresponse bias. Since we know from past experience that web surveys sent out via e-mail by the AMA are likely to get filtered into junk mail and therefore not be opened, we determined a mail survey is likely to result in higher response rates for the PCP survey.

2. Procedures for Collection of Information

We plan to conduct this research in two phases. First, we will conduct formative semi-structured interviews with members of each population. Second, we will conduct nationally representative, probability-based surveys of members of each population.

Phase 1: Semi-Structured Interviews. In Phase 1 of the research, semi-structured interviews will be conducted by web conferencing using the itracks platform. This approach allows for the participant and interviewer to see each other and includes a whiteboard feature that can be used to show the terms, statements, or passages for participants to read and follow along as the interviewer reads them aloud. This may be helpful in cases where the statements or passages are long, which may make them difficult to understand when read aloud. In addition, the written information may be helpful as a reference as the discussion progresses.

Participation is estimated to take 1 hour. We will start data collection with a soft launch of three interviews per segment (10 percent) to ensure that all processes are working well. Although we do not intend on making major changes to the interview guides as a result of these soft launch interviews, they will provide an opportunity to make minor changes (e.g., adding interviewer notes).

Phase 2: Nationally Representative Surveys. In Phase 2 of the research, primarily closed-ended survey questions will be administered to each population. The closed-ended survey format will allow the team to quantify the frequency or prevalence of certain interpretations or meanings among a nationally representative sample of the general U.S. consumer and PCP populations. Final questions and response options will be informed by key interpretations discovered during the Phase 1 interviews. Participation is estimated at 20 minutes.

We also plan to embed an experiment in the PCP mail survey. Research has shown that including a pen in the survey package can help to increase response rates and time to response, even potentially reducing the number of reminders required.^{1, 2} However, the shipping of pens can be costly and often pens are damaged in the mail (e.g., ink can leak, etc.). To determine whether another token incentive might be as effective at increasing response rates, we will randomize half of the sample to receive a pen and half to receive a packet of sticky notes or other token incentive. We will compare response rates between the two groups to help inform methods for future studies.

Analysis Plan

For the Phase 1 interviews, we will conduct a thematic analysis using a matrix approach to identify themes and mental models common across participants. For each term or phrase, we will explore findings related to what the term or phrase means to participants and what the term or phrase suggests about the drug in question. Data for consumers and PCPs will be analyzed separately. RTI coders will review the data to identify an initial set of codes for each question. Two RTI coders will double code a small set of responses to achieve inter-rater reliability with a goal of achieving a Krippendorff’s alpha of .80 as an indicator of high reliability.³ Once reliability is achieved, the remaining responses will be independently coded.

For the Phase 2 surveys, we will explore the distribution of outcomes through descriptive statistics, such as frequencies and percentages for categorical outcomes and descriptive statistics (e.g., means, standard deviations) for continuous outcomes. We will then produce national-level estimates for both physicians and consumers regarding understanding and interpretation of key terms and phrases as well as evaluations of terms and phrases regarding perceived benefits, risks, effectiveness, importance and others. We will also conduct subgroup analyses to determine whether understanding and interpretations of terms and phrases or other key outcomes of interest differ based on certain individual characteristics. For the subgroup analysis, we anticipate conducting logistic regressions when the outcome is categorical and ordinary least square regressions when the outcome is continuous.

Analyses will be conducted using statistical software appropriate to the application, such as but not limited to Excel for the Phase 1 interviews and SPSS and SAS for the Phase 2 surveys.

Power

For the Phase 2 survey, we set our sample requirements to a 95 percent confidence interval and a 3 percent margin of error assuming an underlying proportion of 0.50 in the population (which is the most conservative estimate and overestimates the sample size relative to alternate proportions). These parameters are commonly used in quantitative survey research^{4, 5, 6, 7} and offer balance between precision and cost. Thus, assuming a total U.S. population of roughly 250 million adults aged 18 or older,⁸ we estimate the number of completed surveys to be 1,067 for the general population survey. Assuming a total population of 209,000 PCPs,⁹ with the same 95 percent confidence interval and ±3 percent margin of error, we estimate the number of completes for the provider survey to be 1,062. These sample sizes would also allow us to detect a mean difference between ±0.15 and 0.30 points.¹⁰

3. Methods to Maximize Response Rates and Deal with Non-response

Strategies to maximize response rates for the Phase 2 consumer survey include:

• Repeated follow-ups

• Small prepaid incentives with larger postpaid incentives

• Tailored contact materials

• Survey length of 20 minutes or less

• Mobile optimization for web surveys

Obtaining an adequate response rate is a well-known challenge in survey research in recent years. Much research has been devoted to finding ways to maximize response rates to ensure high quality data.¹¹ The strategies we propose are based on the most recent literature on enhancing response.

Similarly, obtaining an adequate response from a sample of physicians is a well-documented challenge. Physicians are frequently asked to take part in surveys about medical issues and, given their busy schedules, are often more reluctant than other types of respondents to participate.¹² We will draw from our experience with various strategies to maximize response rates as summarized below:

• Prepaid incentive checks

• Large incentives

• Tailored contact materials

• Mail mode versus e-mail invitation and web mode

• Inclusion of a non-monetary token of appreciation

• FDA sponsorship

• Survey length of 20 minutes or less

4. Test of Procedures or Methods to be Undertaken

Data collection for each phase will begin with a soft launch in order to test and evaluate procedures. The soft launch will utilize the same procedures and methods as the full data collection but utilize only a small sample. In addition, findings from the Phase 1 interviews will be used to refine the questions and response options implemented in the Phase 2 surveys.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing

Data

The contractor, RTI International, will collect and analyze data on behalf of FDA as a task order under Contract HHSF223201510002B. Bridget J. Kelly, Ph.D., MPH, is the Project Director, (202) 728-2098. Review of contractor deliverables and supplemental analyses will be provided by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Kevin R. Betts, Ph.D., (240) 402-5090, and Helen Sullivan, Ph.D., (301) 796-4188.

1Bell, K., L. Clark, C. Fairhurst, et al, “Enclosing A Pen Reduced Time to Response to Questionnaire Mailings.” Journal of Clinical Epidemiology, 74:144-150, 2016.

2Sharp, L., C. Cochran, S.C. Cotton, et al., “Enclosing a Pen with a Postal Questionnaire Can Significantly Increase the Response Rate.” Journal of Clinical Epidemiology, 59:747-754, 2006.

3Krippendorff, K. (2013). Content analysis. An introduction to its methodology (3rd Ed.). Thousand Oaks, CA: Sage Publications, Inc.

4 Bartlett, J.E., J.W. Kotrlik, and C.C. Higgins, “Organizational Research: Determining Appropriate Sample Size in Survey Research.” Information Technology, Learning, and Performance Journal, 19:43-50, 2001.

5 Cochran, W.G. (1997) Sampling Techniques (3rd ed.). New York: John Wiley & Sons.

6 Dillman, D.A., J.D. Smyth, and L.M. Christian. (2014) Internet, Phone, Mail, and Mixed-mode Surveys: The Tailored Design Method (4th Ed.). Hoboken, NJ: John Wiley & Sons, Inc.

7 Krejcie, R.V. and D.W. Morgan, “Determining Sample Size for Research Activities.” Educational and Psychological Measurement, 30: 607-610, 1970.

8 U.S. Census Bureau. (2017) “National Population by Characteristics: 2010-2017.” Retrieved from https://www.census.gov.

9Agency for Healthcare Research and Quality. (2011) “The Number of Practicing Primary Care Physicians in the United States.” Retrieved from http://www.ahrq.gov/research/findings/factsheets/primary/pcwork1/index.html.

10 Krejcie, R.V. and D.W. Morgan, “Determining Sample Size for Research Activities.” Educational and Psychological Measurement, 30: 607-610, 1970.

11Czajka, J.L. and A. Beyler. (2016). Background paper: Declining response rates in federal surveys: Trends and Implications. Mathematica Policy Research. Available at: https://aspe.hhs.gov/system/files/pdf/255531/Decliningresponserates.pdf

12 Kablunde, C., Willis, G. B., McLeod, C. C., Dillman, D.D., Johnson, T. P., Greene, S. M., and M. L. Brown. (2012). Improving the quality of surveys of physicians and medical groups: A research agenda. Evaluation and the Health Professions, 35(4), 477–506.

3