OMB Approval Memo

FDA DOCUMENTATION FOR THE GENERIC CLEARANCE,

“Testing Communications on Drugs”
(0910-0695)

TITLE OF INFORMATION COLLECTION: Rapid Message Testing with Healthcare Professionals — Search and Rescue Website

DESCRIPTION OF THIS SPECIFIC COLLECTION

1. Statement of need:

The purpose of this project is to conduct timely testing of the revamped “Search and Rescue” campaign website (https://searchandrescueusa.org/). “Search and Rescue” is a prescriber education campaign that gives healthcare providers the resources they need to help prevent the misuse and abuse of medicines in their practices. According to the CDC, in 2018, more than 67,000 people died of drug overdoses in the United States, and more than 46,000 of those overdose deaths involved opioids.

The “Search and Rescue” campaign was developed by the Partnership to End Addiction in collaboration with the FDA and the health marketing agency Razorfish Health (part of the Publicis Health network). The campaign’s goal is to equip prescribers to be proactive in identifying and helping patients at risk for prescription drug abuse. Following an initial year of formative research, the campaign was piloted in Maryland and Rhode Island in 2014, expanded to six states in 2015, and launched nationally in 2016.

More recently, the campaign website has been revamped to provide prescribers with new tools and resources on how to identify and intervene in prescription drug misuse, abuse, and addiction within their practice. One example of a new tool is a guide, created with input from Harvard pediatricians, that prescribers can give to parents, detailing what to ask when their child is prescribed pain relievers. The website has also been recently optimized to enhance its simplicity and intuitiveness.

Communications science tells us that we must test messages with our intended audiences before communicating them. Thus, FDA plans to test this website using cognitive interviews with a small sample of 16 U.S. physicians drawn from a diverse healthcare provider database.

This data collection is the 20^th in a series of FDA rapid message testing projects submitted to OMB under generic clearance. The previous 19 projects have involved testing messages with consumers, and FDA recently expanded its contract to now include testing with healthcare professionals (HCPs). These projects are part of FDA’s effort to make target audience testing part of its routine communication development processes. This project is in keeping with the spirit of the 2015 Executive Order¹ to improve how information is presented to audiences by applying behavioral science insights, and it meets repeated calls from FDA’s Risk Communication and Advisory Committee to conduct message testing with targeted samples of the general public.

2. Intended use of information:

FDA’s contractor Westat will test the website with a small sample of target audience members to ensure the message meets its objectives without causing unintended negative effects. FDA’s Risk Communication and Advisory Committee includes renowned experts and researchers in social sciences, marketing, health literacy, and related fields. From its very first meeting in 2008, the Committee has consistently advised and reaffirmed that testing communications with the target audience is necessary for FDA, and that using small samples is an effective approach for testing and communicating in a timely manner. In fact, research has shown that “saturation,” or the point at which no new information or themes are observed, can occur with as few as 12 interviews, as described in Guest et al (2006).²

FDA will use the collected interview data to refine its messaging by improving the comprehensibility and personal relevance for a higher public health impact. Specifically, FDA is asking Westat to gain insight to the following questions:

• What do participants understand to be the purpose of the website?

• How intuitive do participants find navigation of the website to be?

• Do participants indicate that any of the website’s information is new to them?

• Which resources provided on the website do participants find most useful and least useful?

• What resources do participants recommend adding to the website?

• What improvements do participants suggest for the website?

• Will participants do anything as a result of visiting the website?

The data collected will not be statistically representative of the target audience population. Therefore, the data will not be used for making policy or regulatory decisions.

3. Description of respondents:

We will conduct 16 45-minute interviews with U.S. physicians. Westat has partnered with WebMD Professional/Medscape, a specialist in healthcare professional recruitment, to recruit respondents from its user database. WebMD Professional/Medscape tracks and stores all member activity and assigns a unique ID number which stays with the respondent throughout their entire membership. These tracking records consist of profile information provided during enrollment, profile updates, and past survey involvement. WebMD Professional/Medscape monitors the quality of their data through various quality checks to save time and provide confidence in data accuracy. These quality checks include individual vetting of contact information and enrollment data, as well as review of screener questions and past study response.

We will use a participant screener to recruit a mix of primary care physicians and specialists (neurology, anesthesiology, emergency medicine, surgery, and orthopedics) who prescribe opioids for non-cancer pain to at least 5 patients per month. To the extent possible, the participant pool will be diverse in terms of gender, race/ethnicity, years of practice, geography (i.e., U.S. Census region), and practice setting (i.e., urban, suburban, and rural).

4. Date(s) to be Conducted:

We plan to conduct interviews in September 2020.

5. How the Information is being collected:

We will conduct all interviews remotely using telephone and screen sharing technology with participants on web-enabled devices such as desktop computers, laptops, or tablets. We will ensure that any materials provided to the participants for the test are compatible with these devices.

For each 45-minute interview, a trained interviewer will lead the discussion using a semi-structured interview guide that ensures consistency in major topics but allows flexibility in probing each participant on particular questions.

Note takers will chart their findings into a standardized reporting template so that all notes are organized in a consistent manner. Interviewers will review the notes to ensure accuracy. With the consent of participants, we will audio record each interview.

FDA staff will have the ability to listen to the interview sessions, and this will be made known to participants as part of the informed consent.

6. Confidentiality of Respondents:

We will provide all respondents with informed consent language that ensures they understand the project purpose, that their participation is voluntary, and that their responses will be kept secure to the extent permitted by law. As part of the consent procedure, respondents will be asked whether they allow audio recording of the interview. Recording will not begin before participants have had the opportunity to ask for any clarification and provide consent. Participants will be asked to again confirm their consent when recording begins. Participants who do not allow audio recording may still participate in the interview. In these cases, Westat will take notes that are more detailed than when relying on the audio recording.

No participant’s identifiable information such as name will be included in the interview notes. All interview materials will be stored on a secure network drive, which will only be accessible to individuals granted access to work on the project. Interview notes will be zipped electronically and password-protected for email or secure file transfer delivery. Prior to forwarding any data to FDA, Westat will destroy all names and contact information of participants to protect their personal identity. Additionally, the interview notes and interpretive report delivered to FDA after message testing will omit all information that could be used to identify respondents.

All electronic data storage media that contain confidential, private, or proprietary information will be maintained within secure areas. Data collected in hard copy will be kept in locked cabinets when not in use.

FDA’s Institutional Review Board (IRB) reviewed this study and determined it is exempt from the requirements of 45 CFR §46.101b(2).

7. Amount and justification for any proposed incentive

Recognizing the significant time and other burdens involved with participation in research and to convey the importance of the research to participants, honorariums are intended to help defray these “costs” in order to encourage individuals to participate.³ Numerous empirical studies have established that an honorarium can significantly increase participation rates. ^{4,5,6,7,8,9,10,11,12,13} This is particularly true for HCPs who are more difficult to recruit as study participants than members of the general population.^14,15

There are many reasons why HCPs are difficult to engage in research, most of which are related to their professional demands and time constraints.¹⁶ For example, many HCPs work irregular hours and must respond to clinical emergencies, making them less available to participate in research that must be scheduled in advance. Although researchers try to accommodate HCPs’ demanding schedules (e.g., offer multiple timeslots, conduct sessions early or late in the day, etc.), it is challenging to find times that do not interfere with their patient care and other required activities.¹⁷ The amount of time required for data collection is another factor that limits HCPs’ participation in research. High patient-volume, back-to-back scheduling, and the need to respond to patient emergencies leaves little time in the day to participate in nonessential activities.¹⁸ For example, a 2015 study found that burden was the primary reason for nonresponse in a study involving a web-based survey, with 60% of physicians saying they had insufficient time to complete the survey requests.¹⁹ Building on these earlier findings, a recent survey conducted by The Physicians Foundation²⁰ found that physicians saw an average of 20 patients per day, and about 80% said they were overworked/overextended or at capacity. These time constraints are particularly salient for qualitative data collections like focus groups and interviews because they tend to be more time consuming than surveys.

Studies with HCPs have shown that participation rates vary by specialty, gender, and other factors.²¹ For example, a recent study found that general surgeons (29.6%), pediatricians (29.2%), and psychiatrists (27.1%) were less likely to participate in a web-based survey than neurologists or neurosurgeons (46.6%) and internists (42.9%). Furthermore, Juster and Suzman²² (1995) found that high incentives reduced nonresponse bias for people with high incomes. These findings are particularly relevant for our project because we intend to recruit both primary care physicians as well as those specializing in neurology, anesthesiology, emergency medicine, surgery, and orthopedics.

For this project, WebMD Professional/Medscape will provide $100 to primary care physicians and $150 to specialists at the end of each 45-minute interview in the form of a check. WebMD Professional/Medscape incentivizes respondents for any participation to maintain a quality-filled participant base. Members do not volunteer their time.

In addition, past experience on other projects our contractor has conducted, and their recent consultation with two national research firms (Reckner Healthcare and WebMD Professional/Medscape), show that the amounts offered are close to but less than what physicians such as those we are targeting for this project require to take time out of their already time-constrained clinical practices to participate in these types of research projects. The table below shows that the proposed honorariums are also less than those recently approved by OMB for other FDA research projects involving remote data collections with primary care physicians and specialists.

Project	Generic Clearance #	Remote Data Collection Type (Length)	Incentive	OMB approval date
Studies to Enhance FDA Communications Addressing Opioids and Other Potentially Addictive Pain Medications	0910-0695	Telephone interviews (60 minutes)	Specialists: $250 Primary care: $175	June 20, 2016
Studies to Enhance FDA Communications Addressing Biosimilar Drug Products	0910-0695	Online Interviews (90 minutes)	Specialists: $250	May 2, 2018
Health Care Providers’ Understanding of Opioid Analgesic Abuse-Deterrent Formulations	0910-0847	Online focus groups (90 minutes)	Specialists: $400 maximum if $300 refused Primary care: $300	March 1,2019
End-User Testing Associated with the “Pregnancy and Lactation Labeling Rule” to Improve Health Communications and Prescribing Decisions in Pregnant Women	0910-0497	Online focus groups (90 minutes)	Specialists: $400 maximum if $300 refused Primary care: $300	April 27, 2020

Several studies have explored strategies to improve recruitment of HCPs, and some have examined participation rates by incentive amount and/or type .^{23,24,25,26,27} This research shows that monetary incentives resulted in higher survey response rates compared to nonmonetary incentives and that the value (or perceived value) of the incentive matters, with higher incentives yielding greater participation than lower incentives. Although the incentives in this literature were lower than those proposed here, they involved significantly shorter survey participation and are often from many years ago. However, provision of high-value honorariums is supported by leading qualitative researchers who suggest that focus groups with physicians may require amounts up to or exceeding $500.²⁸

The decision to provide the proposed honorarium amounts is based on the principles set forth in OMB’s guidance on factors that may justify provision of an incentive (Office of Management and Budget, 2006):

• Data quality: One strategy we will implement to improve the quality of the data is to recruit HCPs diverse in terms of gender, race/ethnicity, years of practice, geography, and practice setting. Offering too low an incentive is likely to limit our ability to recruit the diversity we are seeking for this project. Prior studies have shown variations in participation rates by medical specialty, age, and race/ethnicity. Furthermore, the difficulties engaging HCPs for research studies have been well-documented. Numerous studies have shown that honorariums can reduce sample bias which occurs when research participants do not represent the diversity of the intended audience (Griffin et al., 2011; Lesser et al., 2001; Singer & Kulka, 2002).^29,30,31 A biased sample will compromise the usefulness and validity of the findings.

• HCP participant burden. Lack of time and competing demands are significant barriers to research participation among HCPs. Although we will aim to limit burden to the extent possible, the study design requires participation in a 45-minute prescheduled interview session. Participants may need to find coverage for their patients to attend the session and/or work longer days to complete paperwork or follow up with patients. Burden concerns are a common deterrent to participation, particularly among high-volume HCPs or high-demand and/or uncommon specialists, which would lead to sample bias and affect data quality (see above). The honorarium will help offset or reduce concerns about time and other burdens, such as disruptions to patient flow, inconvenience, financial loss, and the need to limit or reschedule patient appointments.

• Past experience: As described previously, the study team has conducted qualitative studies with HCPs, and our experience confirms that they are a very challenging population to recruit. The requested amount is less than the amounts provided for the prior FDA studies that involved qualitative research methods with similar populations (see above).

• Improved coverage of specialized respondents or rare groups: The HCP participants in this study are considered specialized because they have specific knowledge and experience related to the topic area. Although the number of professionally active primary care and specialty physicians in the United States are similar, the latter are distributed across multiple specialty types, increasing specialists’ rarity. For example, anesthesiologists make up just 9% of all specialists, surgeons just 10%, and emergency medicine physicians just 11%.³² These data demonstrate that the pool of potentially-eligible HCPs is limited, and our eligibility criteria impose further eligibility restrictions. Rarity coupled with barriers related to time and other burdens make specialists particularly hard-to-recruit.

8. Questions of a Sensitive Nature

We do not anticipate asking any sensitive questions in the interviews. Instead, the questions will focus on individuals’ reactions to the messages and materials.

Nevertheless, respondents will be told that they may skip any question that they do not want to answer or may stop participating at any time.

9. Description of Statistical Methods

We do not plan to use formal statistical methods in this study but rather qualitative analysis methods. Our analysis approach is based on the Framework method, as described in Spencer et al (2003).³³ Framework is a matrix-based approach to data management, which facilitates both case and theme-based analysis. The Framework method allows for data reduction through summarization and synthesis yet retains links to original data, in this case the interview notes. We will use the qualitative analysis software NVivo, which has included a Framework functionality since 2011. The software will allow us import interview notes, create links between the notes and the Framework matrices, and develop new queries or matrices as needed.

The Framework method will allow us to recognize patterns within the data. Findings will be supported with verbatim participant quotes and grounded in accepted principles of health communications.

BURDEN HOUR COMPUTATION (Number of responses (X) estimated response or participation time in minutes (/60) = annual burden hours):

Type/Category of Respondent	No. of Respondents	Participation Time (minutes)	Burden (hours)
Screener	150	3	7.5
Interviews	16	45	12
		Total	20

REQUESTED APPROVAL DATE: September 24, 2020

NAME OF PRA ANALYST & PROGRAM CONTACT:

Ila S. Mizrachi

Paperwork Reduction Act Staff

[email protected]

(301)796-7726

Brian Lappin

CDER/Office of Communications

[email protected]

(301)796-9126

FDA CENTER: Center for Drug Evaluation and Research (FDA/CDER)

1 https://obamawhitehouse.archives.gov/the-press-office/2015/09/15/executive-order-using-behavioral-science-insights-better-serve-american

2 Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field methods, 18(1), 59-82.

3 Klabunde, C. N., Willis, G. B., McLeod, C. C., Dillman, D. A., Johnson, T. P., Greene, S. M., & Brown, M. L. (2012). Improving the quality of surveys of physicians and medical groups: A research agenda. Evaluation and the Health Professions, 35(4), 477–506.

4 Abreu, D. A., & Winters, F. (1999). Using monetary incentives to reduce attrition in the survey of income and program participation. Proceedings of the Survey Research Methods Section of the American Statistical Association. 533-538.

5 Aikin, K., Betts, K., Boudewyns, V., Stine, A., & Southwell, B. G. (2016). Physician responsiveness to survey incentives and sponsorship in prescription drug advertising research. Annals of Behavioral Medicine, 50(Suppl.), s251.

6 Dykema, J., Stevenson, J., Day, B., Sellers, S., & Bonham, V. (2011). Effects of incentives and prenotification on response rates and costs in a national web survey of physicians. Evaluation and the Health Professions, 34(4): 434–447.

7 Greenbaum, T. L. (2000). Moderating focus groups: A practical guide for group facilitation. Thousand Oaks, CA: Sage Publications, Inc.

8 Martins, Y., Lederman, R.I., Lowenstein, C.L., Joffe, S., Neville, A., Hastings, B.T., & Abel, G.A. (2012). Increasing response rates from physicians in oncology research: a structured literature review and data from a recent physician survey. British Journal of Cancer, 106(6), 1021-1026.

9 Medway, R. L., & Tourangeau, R. (2015). Response quality in telephone surveys: Do prepaid cash incentives make a difference?  Public Opinion Quarterly, 79(2), 524-543.

10 Mercer, A., Caporaso, A., Cantor, D., & Townsend, R. (2015). How much gets you how much? Monetary incentives and response rates in household surveys.  Public Opinion Quarterly, 79(1), 105-129.

11 Shettle, C., & Mooney, G. (1999). Monetary incentives in U.S. government surveys. Journal of Official Statistics, 15(2), 231–250.

12 Thorpe, C., Ryan, B., McLean, S.L., Burt, A., Stewart, M., Brown, J.B. (2009). How to obtain excellent response rates when surveying physicians. Family Practice, 26(1), 65-68.

13 VanGeest, J. B., Johnson, T. P., & Welch, V. L. (2007). Methodologies for improving response rates in surveys of physicians: A systematic review. Evaluation of Health Professionals, 30(4), 303–321.

14 Asch S., Connor S.E., Hamilton E.G., & Fox S.A. (2000). Problems in recruiting community-based physicians for health services research. Journal of General Internal Medicine, 15(8):591-599.

15 Cummings, S.M., Savitz, L.A., & Konrad, T.R. (2001). Reported response rates to mailed physician questionnaires. Health Services Research, 35(6), 1347-1355.

16 Asch S., Connor S.E., Hamilton E.G., & Fox S.A. (2000). Problems in recruiting community-based physicians for health services research. Journal of General Internal Medicine, 15(8):591-599.

17 Bakken S, Lantigua RA, Busacca LV, & Bigger JT. (2019). Barriers, enablers, and incentives for research participation: a report from the Ambulatory Care Research Network (ACRN). J Journal of the American Board of Family Medicine, 22(4):436–45.

18 Capko, J. To improve practice work flow, tackle patient scheduling. Retrieved from http://www.physicianspractice.com/scheduling/improve-practice-work-flow-tackle-patient-scheduling. Accessed January 17, 2019.

19 Cunningham, C.T., Quan, H., Hemmelgarn, B., Noseworthy, T., Beck, C.A., Dixon, E. (2015). Exploring physician specialist response rates to web-based surveys. BMC Medical Research Methodology, 15(32). doi: 10.1186/s13104-014-0969-8.

20 Physcian’s Foundation. (2018). America’s Physicians: Overworked and Burning Out. https://thehospitalleader.org/americas-physicians-overworked-and-burning-out/Accessed March 24, 2020.

21 Cunningham, C.T., Quan, H., Hemmelgarn, B., Noseworthy, T., Beck, C.A., Dixon, E. (2015). Exploring physician specialist response rates to web-based surveys. BMC Medical Research Methodology, 15(32). doi: 10.1186/s13104-014-0969-8.

22 Juster, F. T. & Suzman, R. (1995). An overview of the health and retirement study. Journal of Human Resources, 30, S7-S56.

23 Asch, D.A., Christakis, N.A., & Ubel, P.A. (1998). Conducting physician mail surveys on a limited budget: a randomized trial comparing bill versus $5 bill incentives. Medical Care, 36(1):95–99.

24 Deehan, A., Templeton, L., Taylor, C., Drummond, C., & Strang, J. (1997). The effect of cash and other financial inducements on the response rate of general practitioners in a national postal study. British Journal of General Practice, 47(415):87–90. 

25 Pit, S.W., Vo, T., & Pyakurel, S. (2014). The effectiveness of recruitment strategies on general practitioner's survey response rates - a systematic review. BMC Medical Research Methodology, 6(14). doi: 10.1186/1471-2288-14-76.

26 Thomson, C.E., Paterson-Brown, S., Russell, D., McCaldin, D., & Russell, I.T. (2004). Short report: encouraging GPs to complete postal questionnaires - one big prize or many small prizes? A randomized controlled trial. Family Practice, 21(6):697–698.

27 Young, R.A., Fulda, K.G., Suzuki, S., Hahn, K.A., Espinoza, A.M., Marshall, J.D.. (2011). The influence of research compensation options on Practice-based Research Network (PBRN) physician participation: A North Texas (NorTex) PBRN study. Journal of the American Board of Family Medicine, 24(5), 562-568.

28 Krueger, R.A., & Casey, M.A. (2015). Focus Groups: A practical guide for applied research. Thousand Oaks, CA: Sage Publications, Inc.

29 Griffin, J., Simon, A. B., Hulbert, E., Stevenson, J., Grill, J., Noorbaloochi, S., & Partin, M. (2011). A comparison of small monetary incentives to convert survey non-respondents: a randomized control trial. BMC Medical Research Methodology, 11(81). doi: 10.1186/1471-2288-11-81.

30 Lesser, V.M., Dillman, D.A., Carlson, J., Lorenz, F., Mason, R., & Willits, F. (2001) Quantifying the Influence of Incentives on Mail Survey Response Rates and Nonresponse Bias. Paper presented at the annual meeting of the American Statistical Association, Atlanta, GA.

31 Singer, E. & Kulka, R. A. (2002). Paying Respondents for Survey Participation. In M. Ver Ploeg, R. A. Moffitt, & C. F. Citro (Eds.), Studies of Welfare Populations: Data Collection and Research Issues, Washington, D.C.: National Academy Press.

32 Kaiser Family Foundation. (2018a). Professionally Active Specialist Physicians by Field. Retrieved from https://www.kff.org/other/state-indicator/physicians-by-specialty-area/?dataView=1&currentTimeframe=0&selectedRows=%7B%22wrapups%22:%7B%22united-states%22:%7B%7D%7D%7D&sortModel=%7B%22colId%22:%22All%20Other%20Specialties%22,%22sort%22:%22asc%22%7D

33 Spencer, L., Ritchie, J., & O'Connor, W. (2003). Analysis practices, principles and processes. In Qualitative research practice. London: Sage Publications.

8