Supporting Statement Part B HCP Survey PPDP 2019

Part B. Statistical Methods

1. Respondent Universe and Sampling Methods

We will use WebMD’s Medscape subscriber network as a frame for sample selection. Healthcare professionals join the Medscape network to have access to medical news and drug updates and can opt-in to participate in market research studies. The Medscape network includes approximately 600,000 American Medical Association (AMA) validated physicians, including both primary care physicians (PCPs) and specialists (SPs) (e.g., cardiologists, rheumatologists, urologists, etc.) and has more than 165,000 nurse practitioners (NPs) and physicians assistants (PAs). The HCP Survey II will include each of these healthcare professional groups (PCPs, SPs, NPs, and PAs) and all will be recruited from Medscape’s network. Specialists will be drawn from ten specialty areas (cardiology, dermatology, endocrinology, neurology, obstetrics/gynecology, oncology, ophthalmology, psychiatry, rheumatology and urology). Overall, the Medscape subscriber network offers a higher coverage of healthcare professionals compared to more traditional internet panels.

The data collection will include a nationally representative sample of HCPs. FDA has selected the following HCP groups for inclusion in the study: physicians (primary care and specialists), nurse practitioners, and physician assistants (Table 1). Eligible physician respondents include individuals with Doctor of Medicine (MD) or Doctor of Osteopathic Medicine (DO) degrees. To qualify for this study, HCPs must practice in an office-based setting, spend at least 50 percent of their time providing patient care, and have prescribing authority. “Office-based setting” is broadly defined and includes federally and non-federally sponsored clinics, health maintenance organizations (HMOs), and large health care systems. Healthcare professionals primarily seeing patients in a hospital setting or research facility are ineligible for the study.

Table 1. Description of healthcare professional groups

Healthcare professional group	Included specialties
Primary Care Physicians (PCPs)	General practitioner, family medicine, internal medicine
Specialists (SPs)	Cardiologists, dermatologists, endocrinologists, neurologists, OB/GYN, oncologists, ophthalmologists, psychiatrists, rheumatologists, urologists
Physician Assistants (PAs)
Nurse Practitioners (NPs)

Sampling. The sample of healthcare professionals for the pretest and the main survey will be drawn from WebMD’s Medscape subscriber network, a group of HCPs who have opted in to receive medical news and drug updates. Using a prequalification screener, sampled HCPs in WebMD’s network will be screened to determine their eligibility for the study. (See Appendix B for screener.) The pretest will be conducted prior to the main study to test data collection process. The pretest sample (n=25) will not overlap with the main study sample (n=2000).

In considering options for the sample, we considered WebMD’s network, GfK’s pre-recruited healthcare research panel, and two lists of providers—AMA’s Masterfile and the National Provider Identifier (NPI) individual-level file. The AMA Masterfile has high coverage of physicians but contains inaccurate and out-of-date contact information. With the NPI file, there is no concerted effort to keep the contact information updated and over-coverage (e.g., of those no longer practicing) is a concern. Additionally, while these files contain physicians, they do not include PAs or NPs, and therefore other sampling frames would be required. The limitations of these files, the challenges of recruiting the target population, and the costs involved with recruiting large samples led us to consider WebMD’s Medscape subscriber network and GfK’s healthcare research panel. Both WebMD’s network and GfK’s healthcare research panel offer the advantage of having current contact information (namely email) that can be used to invite potential respondents to complete the prequalification screener. However, WebMD offers considerably higher coverage of the study population (see Table 2). In addition, WebMD’s network may be less susceptible to selection bias, as recruited research panel members (such as those participating in GfK’s panel) enroll specifically to participate in surveys and other research activities.

Table 2. U.S. healthcare professionals: Vendor counts and estimated population totals

	Vendor counts		Estimated population totals
	WebMD1	GfK1
Primary care providers	197,980	25,000	242,8002
Specialists	465,020	63,000	724,2492
Physician assistants	62,874	1,406	92,0003
Nurse practitioners	102,552	3,509	220,0004

Sources:

¹WebMD and GfK counts were provided by these respective vendors.

²American Medical Association (http://www.mmslists.com/data/countspdf/AMA-SpecialtyByTOPS.pdf)

³Kaiser Family Foundation (http://kff.org/other/state-indicator/total-physician-assistants/?currentTimeframe=0&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D)

⁴American Association of Nurse Practitioners (https://www.aanp.org/all-about-nps/np-fact-sheet)

Weighting. For the main study, weights will be computed for each completed interview. These weights will be used to generalize the completed interviews to a national population of PCPs, SPs, NPs, and PAs. The weights will account for unequal selection probabilities, and will adjust for differential response rates and differential coverage.

The starting point will be the computation of a base weight for each completed interview, which will be the reciprocal of the final probability selection from the frame for the sample unit (accounting fully for utilization of reserve samples). These weights will be trimmed if necessary, and then calibrated to external control totals to adjust for differential nonresponse and differential coverage. The external control totals will be based on the dimensions of gender, age of the professional (divided into five categories¹), year of graduation, and the four U.S. Census Regions. The characteristics age of professional and year of graduation may be combined into a single dimension as they may be highly correlated. These will be developed separately for the four primary strata of PCPs, SPs, NPs, and PAs. We can also consider in consultation with FDA a division for PCPs into Family Practice, General Practice, and Internal Medicine, and/or a division for SPs into specialties.²

For PCPs and SPs, the National Ambulatory Medical Care Survey (NAMCS) will be the source of benchmarks, with supplementation from the AMA Masterfile for specialties not covered by the NAMCS.

We will discuss with FDA a strategy for PCP and SP control totals based on utilizing the AMA Masterfile, the NAMCS Public Use File, and the restricted use NAMCS. For PAs and NPs, we will study and evaluate a number of alternatives for benchmarks. For PAs, for example, the benchmarks could be from the American Academy of Physician Assistants Masterfile. For NPs, the benchmarks could be from the American Academy of Nurse Practitioners Membership Survey.

Extreme weights will be checked and trimming carried out to trim back any extreme weights. A raking-and-trimming software package will be used to rake the weights to the control totals while simultaneously controlling for extreme weights (iterating back and forth between raking steps and trimming steps). Our tentative criterion for trimming are weights more than 4.5 times the median weight for the four provider-type strata. This is done, for example, by the National Assessment of Education Progress (see National Assessment of Educational Progress (2017), and Valliant et al. (2013), Section 14.4). This criterion will be re-evaluated after the base weights are generated.

A set of jackknife (JKn) replicate weights will be generated that will reflect the stratification; these replicate weights will be based on the assumption that the control totals have zero variance. These will be JKn type weights: each weight is generated by systematically deleting a subset within each of the four provider-type stratum and reweighting the remaining sample units with the stratum. These replicate weights will be used in computing standard errors and doing tests. In Table 3, we propose counts of replicate weights by stratum (200 replicate weights all together).

Table 3. Jackknife replicates by provider-type stratum

Primary stratum	Main study sample size	Number of replicates	Deleted sample units per replicate
PCPS	700	70	10
SPs	600	60	10
NPs	350	35	10
PAs	350	35	10
Total	2,000	200	10

Unless otherwise mentioned, all estimates will be weighted estimates utilizing the final calibrated weights. All standard errors will be computed using the weights and the replicate weights. All tests (Chi-square, t-tests) will reflect degrees of freedom from the replicate weights. Most of the analyses will utilize weights and replicate weights in order to appropriately account for the effects of the complex sample design and estimation procedures.

2. Procedures for the Collection of Information

Part A of the supporting statement described the rationale for conducting the study. The general research questions in the survey are as follows:

1. What methods and/or channels are used to disseminate prescription drug

promotional information to health care professionals/prescribers?

2. How knowledgeable and interested are HCPs in clinical trial data and its presence

in prescription drug promotion?

3. How familiar are HCPs with the FDA approval of prescription drugs and how

does this translate into practice?

In addition, given the critical nature of the opioid situation in the United States at this time, we plan to ask several questions about prescription drug promotion of opioid products and understanding of the term “abuse-deterrent formulation.”

For both the pretest and main study, WebMD will initially send a recruitment email (Appendix A) that links to a prequalifying screener (Appendix B) to identify eligible HCPs. Respondents may also learn about the survey through the Medscape website and pop-up ads (Appendix A) that will appear when logged in to their Medscape.com account. The screener will include questions about the amount of time spent seeing patients, demographic questions (age, race/ethnicity, and gender) and a question to confirm the respondent’s specialty. All respondents that meet eligibility requirements will be invited to participate in the survey within 24 hours of completing the screener. Respondents will be paid incentives for completing the screener and survey.

Analysis Plan

We will run frequencies for all survey questions (see Appendix C for a listing of the questionnaire items). Means and standard deviations will be provided for scale items.

For binary questionnaire items, we will produce bar graphs for the four provider-type strata based on weighted percentages for the two binary outcomes. We will carry through a Chi-square test of the null hypothesis that all four provider-type strata have equal binary percentages (a test on three degrees of freedom). This Chi-square test will utilize the replicate weights and be appropriate for the sample design and weighting adjustments. If the Chi-square test rejects the null hypothesis, a multiple comparisons type test will be done to distinguish which strata differ significantly. The multiple-comparisons p-values will be evaluated using the Benjamini-Hochberg criteria, which controls the false discovery rate among independent tests to be no more than 5 percent³ (Benjamini and Hochberg, 1995 and Efron, 2010).

For ordinal questionnaire items with three levels, we will produce bar graphs for the four provider-type strata based on weighted percentages for the three outcomes lined up in a bar. We will carry through a test of the null hypothesis that all four provider-type strata have equal percentages for the three outcomes (treated ordinally).

After the pretests, verbatim responses for open-ended questions will be reviewed and coded. In consultation with FDA, we will develop a coding scheme for each question. After receiving training on the coding scheme, two coders will independently code all responses to the open-ended questions. We will calculate inter-coder reliability, aiming for a Cohen’s Kappa threshold of .75 or higher. Any differences between coders will be discussed and adjudicated by a third reviewer. All coded responses will be transformed into numeric proxy measures for analysis.

Correlation analysis and exploratory factor analysis will be conducted to guide the development of latent variables. Logistic regression models will be conducted to examine relationships between exposure, attitude and knowledge measures and provider decision-making and practices.

Power

Power analysis is not applicable for this data collection as there are no experimental manipulations.

3. Methods to Maximize Response Rates and Deal with Non-Response

Both the pretest and main studies will be administered via Internet. To help ensure that the participation rate is as high as possible, FDA and the contractor will:

• Design a protocol that minimizes burden (reasonable in length, clearly written, and with appealing graphics).

• Use incentive rates that meet industry standards. In addition to offsetting respondent burden, using market-rate incentives tends to increase response rates, reduce sampling bias, and reduce nonresponse bias.

• Use government sponsorship on the survey invite to increase response rate. An experiment conducted by FDA and RTI⁴ found that among endocrinologists, response rates were 6 percentage points higher when FDA was disclosed as the sponsor in the survey invitation than when no sponsor was listed. However, due to concerns raised in the public comments that mentioning FDA could potentially influence subjects’ responses to study questions, we will ensure that all materials reference the U.S. Department of Health and Human Services rather than FDA.

An analysis of item nonresponse will be made in the main survey (and the screener as well if it is an issue there). Item nonresponse rates will be tabulated for the questionnaire items (allowing for skip patterns). An analysis will be made of any questionnaire items that register unusually high item nonresponse rates. Multivariate item nonresponse relationships will be evaluated, including monotonicity patterns such as breakoffs (all items dropped after a particular item), and other types of ‘blocks’ of multivariate item nonresponse. High levels of item nonresponse in particular items will have their correlations with other questionnaire item results in both the screener and main survey analyzed (tabulating how much the item nonresponse is concentrating in a particular subgroup of health providers).

4. Test of Procedures or Methods to be Undertaken

Nine cognitive interviews were conducted per questionnaire to assess questionnaire flow and wording. We plan to conduct two pretests on a larger scale to ensure the main studies will run smoothly. We propose to test 500 individuals in each pretest.

5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing Data

The contractor, Westat, will collect and analyze the data on behalf of FDA as a task order under Contract HHSF223201510001B. Simani Price, Ph.D., 301-610-5536, is the Project Director for this project. Data analysis will be overseen by the Research Team, Office of Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and coordinated by Amie C. O’Donoghue, Ph.D., 301-796-0574, and Kathryn J. Aikin, Ph.D., 301-796-0569.

1 Five categories: 25- to 34-year-olds, 35- to 44-year-olds, 45- to 54-year-olds, 55- to 64-year-olds, and over-65-year-olds.

2 Cardiologists, dermatologists, endocrinologists, neurologists, OB/GYNs, oncologists, ophthalmologists, psychiatrists, rheumatologists, urologists.

3 The false discovery rate is the proportion of rejected null hypotheses for which the null hypothesis is in fact true (a “false discovery”). Benjamini-Hochberg is less conservative than the Bonferroni approach to deciding on significance levels for multiple comparisons (i.e., it allows for more significant results).

4 Aikin, KJ; Betts, K; Boudewyns, V; Stine, A; & Southwell, B. (2016). Physician responsiveness to survey incentives and sponsorship in prescription drug advertising research. Annals of Behavioral Medicine, 50(Suppl.), s251.

5