Text Analysis PDN Interpretations SS Part A final 10-14-22

United States Food and Drug Administration

Text Analysis of Proprietary Drug Name Interpretations

OMB Control No. 0910-NEW

SUPPORTING STATEMENT

Part A. Justification

1. Circumstances Making the Collection of Information Necessary

Section 1701(a)(4) of the Public Health Service Act (42 U.S.C. 300u(a)(4)) authorizes the FDA to conduct research relating to health information. Section 1003(d)(2)(C) of the Federal Food, Drug, and Cosmetic Act (FD&C Act) (21 U.S.C. 393(d)(2)(C)) authorizes FDA to conduct research relating to drugs and other FDA regulated products in carrying out the provisions of the FD&C Act.

The Office of Prescription Drug Promotion’s (OPDP’s) mission is to protect the public health by helping to ensure that prescription drug promotion is truthful, balanced, and accurately communicated. OPDP’s research program provides scientific evidence to help ensure that our policies related to prescription drug promotion will have the greatest benefit to public health. Toward that end, we have consistently conducted research to evaluate the aspects of prescription drug promotion that are most central to our mission. Our research focuses in particular on three main topic areas: advertising features, including content and format; target populations; and research quality. Through the evaluation of advertising features, we assess how elements such as graphics, format, and disease and product characteristics impact the communication and understanding of prescription drug risks and benefits. Focusing on target populations allows us to evaluate how understanding of prescription drug risks and benefits may vary as a function of audience, and our focus on research quality aims at maximizing the quality of research data through analytical methodology development and investigation of sampling and response issues. This study will inform all three topic areas.

Because we recognize the strength of data and the confidence in the robust nature of the findings are improved through the results of multiple converging studies, we continue to develop evidence to inform our thinking. We evaluate the results from our studies within the broader context of research and findings from other sources, and this larger body of knowledge collectively informs our policies as well as our research program. Our research is documented on our home page, which can be found at https://www.fda.gov/about-fda/center-drug-evaluation-and-research-cder/office-prescription-drug-promotion-opdp-research. The website includes links to the latest Federal Register notices and peer-reviewed publications produced by our office.

As part of the prescription drug regulatory review process, sponsors propose proprietary names for their products. These names undergo a proprietary name review (PNR) that involves the Office of Surveillance and Epidemiology, the relevant medical office, and OPDP. OPDP reviews names to assess for alignment with the FD&C Act, which provides, among other things, that labeling can misbrand a product if false or misleading representations are made (see 21 U.S.C. 321(n) and 352(a)). A proprietary name that appears in labeling could result in such misbranding if it is false or misleading. OPDP reviews, among other things, whether names (1) overstate the efficacy or safety of the drug, (2) suggest drug indications that are not accurate, (3) suggest superiority without substantiation, or (4) are of a fanciful nature that misleadingly implies unique effectiveness or composition. It would be helpful in OPDP’s review of promotional implications of proprietary names for data on consumer and prescriber interpretations of proposed proprietary names to be more readily available for consideration. The proposed research will use text analysis (e.g., topic modeling and sentiment analysis) to learn how consumer and primary care physician (PCP) populations interpret prescription drug names, which will assist OPDP’s consideration of promotional implications.

This proposed research builds upon and extends OPDP’s research entitled "Empirical Study of Promotional Implications of Proprietary Prescription Drug Names" (86 FR 14440). That research involves an experimental design intended to assess names that potentially overstate the efficacy of a product. In contrast, the proposed research involves a survey design that comprises primarily open-ended questions intended to generate text for analysis, an approach that is unrestricted in its ability to assess text with different types of promotional implications (e.g., minimization of risk and unsubstantiated claims of superiority, in addition to overstatement of efficacy). The proposed research will add to the depth and breadth of knowledge we can draw from during the review of proposed proprietary drug names.

2. Purpose and Use of the Information Collection

The key objectives of the proposed research are as follows:

1. To apply new techniques such as topic modeling and sentiment analysis (forms of text analysis) to answer OPDP’s research questions about consumer and PCP interpretations of proprietary prescription drug names.

2. To help develop a methodological approach for assessing consumer and prescriber interpretations of drug names, which can potentially be used in the future as a standard assessment tool.

3. Use of Improved Information Technology and Burden Reduction

Burden will be reduced by recording data on a one-time basis for each respondent, and by keeping study procedures to 20 minutes. Both the consumer and physician samples will self-administer the survey instrument via a computer. In addition to its use in data collection, automated technology will be used in data reduction and analysis.

4. Efforts to Identify Duplication and Use of Similar Information

We conducted a literature search to identify duplication and use of similar information. We conducted a review of the scientific literature by locating relevant articles through keyword searches using popular databases such as PubMed and PsycInfo. We also identified relevant articles from the reference list of articles found through keyword searches. We did not find duplicative work on the present topic.

5. Impact on Small Businesses or Other Small Entities

No small businesses will be involved in this data collection.

6. Consequences of Collecting the Information Less Frequently

The proposed data collection is one-time only. There are no plans for successive data collections.

7. Special Circumstances Relating to the Guidelines of 5 CFR 1320.5

There are no special circumstances for this collection of information.

8. Comments in Response to the Federal Register Notice and Efforts to Consult Outside the Agency

In the Federal Register of November 1, 2021 (86 FR 60254), FDA published a 60-day notice requesting public comment on the proposed collection of information. FDA received two comments that were Paperwork Reduction Act (PRA) related. Within those submissions, FDA received multiple comments that the Agency has addressed. For brevity, some public comments are paraphrased and therefore may not state the exact language used by the commenter. All comments were considered even if they were not fully captured by our paraphrasing in this document. Comments and responses are numbered here for organizational purposes only.

(Comment 1) One comment contended that FDA should revise the questionnaire to capture real-world conditions more closely in which PCPs and consumers form impressions of proprietary names. The comment suggested that while FDA stated that “[t]he experimental names will be names with suspected promotional implications” in the Federal Register notice, the Agency does not approve proprietary names with “suspected promotional implications.” The comment also argued that FDA’s proposed approach would not mimic the real-world conditions in which mention of a drug’s indication triggers a requirement to provide safety information as well. The comment suggested that either FDA could consider providing only the drug name in a way that is similar to the information provided in reminder advertising, or it could provide a balanced presentation as required under the relevant regulations.

(Response 1) As previously described, sponsors propose proprietary names for their products, including those with promotional implications, as part of the prescription drug regulatory review process. One purpose of this study is to investigate methodological options for collecting insights from consumers and providers during the review process that might help FDA make determinations about whether drug names have promotional implications that misbrand a product. As for real-world conditions, our initial focus is on establishing correlation or causation in a more controlled setting--such as a randomized controlled trial or the type of rigorous experimental study we have planned.

(Comment 2) One comment suggested that FDA does not state how the information obtained from the specified study will be useful or how it will be used to inform name reviews. The comment then asserted that the link between this information and the implementation of FDA’s misbranding authorities and proprietary name review, and thus the practical utility of the survey, is unclear.

(Response 2) FDA’s review of proprietary names is conducted to help ensure that proposed proprietary names do not contribute to misbranding a drug or to other violation(s) of the FD&C Act and Agency regulations, particularly when that proprietary name appears in labeling. (See, e.g., 21 U.S.C. 321(n) and 352(a).) We conduct our review of proprietary names in accordance with applicable legal authorities.

The existing study is a first step in exploring the utility of text analysis for collection of data on proprietary prescription drug names. Determining how names are processed and understood by consumers and healthcare providers (HCPs) is important information to be considered in the review of proposed prescription drug names. This program of research is being conducted in order to increase the body of evidence upon which experts can rely when assessing proposed proprietary names.

(Comment 3) One comment stated that FDA should revise question 1. The comment advised that the instructions should make clear that the respondent can write “no impression” if the name does not, in the respondent’s view, communicate any information related to the particular attribute of the drug. In addition, the comment argued that the last question, asking respondents to write a brief narrative, is confusing and unnecessary and that the objective and practical utility of this exercise are unclear.

(Response 3) It was clear in our cognitive interviews that if respondents had no impression based on a drug name, they would be likely to type “nothing” or “no impression” as their response. The purpose of the last exercise is to examine the utility of an implicit measure of attitudes for comparison with the more explicit measures. If this measure proves to be unproductive in pre-testing, we may omit it from the main study. For instance, this implicit measure might be considered unproductive if it does not prompt any additional, unique text relative to what is offered in response to the earlier open-ended items.

(Comment 4) Two comments similarly claimed that questions two through six are leading, potentially confusing, duplicative of another question, or otherwise unnecessary. One comment recommended removing these questions.

(Response 4) These questions have been included as a way of validating the information recorded in question one. Based on other comments, such as one that challenged the use of yes/no questions, we have revised them to a 5-point Likert scale, ranging from strongly disagree to strongly agree. We will assess these questions further as part of pre-testing.

(Comment 5) One comment stated that FDA should limit patient and PCP participation to those who have experience with the fictitious drug indications. It further asserted that FDA should provide detail on how the patients and PCPs will be selected and how FDA will help ensure these participants have relevant experience. The comment suggested that FDA could add an open-ended question requesting that PCPs provide information about their experiences in the disease areas for which the fictitious drugs are intended, patient populations, and settings to understand the real-world value of the responses.

(Response 5) Due to the large number of drug names and indications to be included in this study, the comment’s suggestion is not feasible. However, we will add a measure to the screener to assess PCPs’ and consumers’ experiences with each of the indications. This variable can then be used as a covariate in analyses.

(Comment 6) One comment suggested that to ensure that the survey isolates the impressions given by the proprietary name, FDA should use only fictitious names for the survey.

(Response 6) We have removed all real drug names from the study and replaced them with fictitious names.

(Comment 7) One comment recommended removing all yes/no questions from the survey.

(Response 7) We have done so, changing the yes/no items to Likert-scale items.

(Comment 8) One comment recommended that FDA should acknowledge that proposed names may include “permissible suggestions” and should include such fictitious examples. The comment conjectured that the survey appears to focus only on potential impermissible suggestions that may result from a drug’s proprietary name. The comment submitted that proposed names should also be included that, for example, suggest the dosage form, frequency of delivery, structure of the drug, or general category of the drug’s indications.

(Response 8) A previous study by this research team did include names such as those suggested above (e.g., with the drug’s indication embedded in the name). Those names are not included here to avoid duplication.

(Comment 9) One comment stated that FDA should explain its methodology for the text analysis and allow for stakeholder feedback on the proposed text analysis methodology.

(Response 9) We will examine and present descriptive results for individual names. However, given our goals of understanding promotional implications of prescription drug names across consumers and PCPs, we are also interested in whether there are differences in topic distributions across our treatment and control arms (control versus promotional implications) and between populations (consumers and PCPs). We will use topic modeling and sentiment analysis to answer those questions. We have described the purpose of the study, the design, and the population of interest, and we have provided the questionnaire to numerous individuals upon request.

(Comment 10) One comment expressed concerns about how degrees or levels of misbranding may be established or standardized for evaluating proposed proprietary prescription drug names. It stated that no information has yet been provided by FDA to inform how such standardization will be developed.

(Response 10) This study is not intended either to establish degrees or levels of misbranding or to standardize levels of misbranding for the evaluation of proposed drug names. The key objectives of the proposed research are (1) to apply new techniques such as topic modeling and sentiment analysis to answer OPDP’s research questions about consumer and PCP interpretations of proprietary prescription drug names and (2) to help develop a methodological approach for assessing consumer and prescriber interpretations of drug names.

(Comment 11) A comment objected that FDA has not provided any information on how it will select target names to include in the pre-test and subsequently decide which target names will be used in the main study. The comment expressed concerns that the pre-test will not be able to develop multiple distinct levels of efficacy or indication implication among target names that will be reliably identifiable by HCPs or consumers. The comment asserted that a proprietary name may not be reliably classified and separated into multiple levels of implication.

(Response 11) Our full stimuli are under development during the PRA process. We do not make draft stimuli public during this time because of concerns that this may contaminate our participant pool and compromise our research. In our research proposals, we describe the purpose of the study, the design, the population of interest, and the estimated burden.

Names will be intentionally developed to have promotional implications (e.g., overstatement of efficacy). Many of the names were used in our cognitive interviews. In addition, we will conduct up to two pre-tests, at which point, if any names are not distinguishable from those composed of random syllables, they will be replaced. A similar process was used in another recent study, with reliable results. Participants did distinguish between names created from random syllables and those with promotional implications.

(Comment 12) One comment advised that the pronunciation offered to a respondent would influence a respondent’s impressions and that it would be important for FDA to control for this influence. The comment opined that the pronunciation should result in as neutral a reading as possible, not emphasizing any particular aspect of a name.

(Response 12) All drug names were recorded by the same voiceover specialist in as neutral a manner as possible.

(Comment 13) A comment similarly asserted that the impression formed from a visual cue (drug name written out) would influence and be influenced by an audio cue and vice versa. The comment contended that there would be less bias introduced by listening first to an audio cue. The comment also recommended that an audio cue first be provided, followed by the question about hearing the name, and that the visual image of the name would be presented followed by the question about seeing the name.

(Response 13) We agree that people access both the orthographic and phonological interpretations when they read. However, since our main comparison is within subjects, it is likely that there is some consistency in the order in which any one respondent listens to the pronunciation versus reading the word, and so any variation that may exist should not confound the effects of their own interpretation of the drug names. In addition, the comment’s suggestion would double the number of open-ended questions for every drug name, increasing the survey burden substantially.

(Comment 14) One comment suggested altering the order of the prompts so that after gaining impressions following the audio and visual cues, the brief story or narrative prompt follows.

(Response 14) The currently proposed questionnaire follows this order.

(Comment 15) One comment argued that prompts should not be “double-barreled” and should not lead or prime the respondent to find benefits or other meanings where there may be none. The comment suggested that questions should ask separately about benefits and how well the drug would work and then also ask separately about risks and side effects. The comment suggested rephrasing to “Does the drug name suggest the drug may have a benefit?” or “Does the drug name make you think about how well it might work?”.

(Response 15) We have edited the open-ended section of the study so that these questions are no longer separate items but merely instructions preceding the first question. The phrasing the comment suggested is likely to lead to one-word answers “yes” or “no,” which does not provide the type of text response that is needed to conduct text analysis on the data. We did find in cognitive interviews that participants who did not perceive any meaning from a specific drug name said they would be likely to type “nothing” into the open-ended text box. Thus, we believe the study in its current form does allow for this possibility.

(Comment 16) One comment suggested very general questions should be asked first and then those that are more specific.

(Response 16) We have ordered the prompts from general to specific in line with the suggested comment.

(Comment 17) One comment proposed that researchers may want to consider reducing the number of drugs queried in the survey from 12 to 6 to elicit the richest text data from respondents and that it may be helpful to give a minimum word count for text responses.

(Response 17) Six drugs will not allow for enough power to make comparisons between the groups. However, if we find that we get many breakoffs (participants who begin the survey but do not complete it) in the pre-test (suggesting the survey burden is too high), we will reconsider the study design.

(Comment 18) One comment recommended that an iterative plan for analysis be developed such that there are checks for both internal and external validity at specified intervals. It further proposed that researchers may want to consider a context-specific analysis plan and argued that one common analysis approach or dictionary may not measure risk, side effects, and other constructs accurately across all drugs.

(Response 18) Though the topic modeling approach is designed to be exploratory for this study, we will calculate coherence metrics to assess model fit as well as perform validation exercises to assess if the generated topics can be easily interpreted.

(Comment 19) One comment recommended that an iterative plan for analysis be created based on a set of preliminary data along with the other research materials, such as the questionnaire, sampling plan, etc., so that it can be reviewed before execution of the full research.

(Response 19) We appreciate the comment. The pre-test will provide the valuable insight to create a specific analysis plan for the main study. The pilot data will help us assess assumptions about how respondents will respond to target names.

External Reviewers

In addition to the comments above, the following experts reviewed the study design, methodology, and questionnaires:

1. Margaret Roberts, Ph.D., Associate Professor, Department of Political Science, University of California, San Diego.

2. Richard Netemeyer, Ph.D., Professor of Free Enterprise, McIntire School of Commerce, University of Virginia.

9. Explanation of Any Payment or Gift to Respondents

The respondent incentive will be $50 for PCPs and 1,000 points for consumers (the equivalent of $1.00). The incentive option for consumers allows panelists to redeem their points for cash, gift cards or merchandise. PCPs will receive incentive in the form of a $50 gift card.

Following OMB’s “Guidance on Agency and Statistical Information Collections,” we offer the following justification for our use of these incentives.

Burden on the respondent: As participants often have competing demands for their time and recently even more limited time due to pandemic challenges (e.g., lack of or limited childcare, increase in home or caregiving responsibilities, additional guidelines for sanitization between patients, potentially reduced staff in physician offices) incentives are used to encourage participation in research. When applied in a reasonable manner, incentives are not an unjust inducement and are an approach that recognizes the time burden placed on participants, encourages their cooperation, and conveys appreciation for contributing to this important study. The use of incentives treats participants justly and with respect by recognizing and acknowledging the effort that they expend to participate (Russell, 2000). Incentives must be high enough to equalize the burden placed on respondents with respect to their time and cost of participation, as well as to provide enough motivation for them to participate in the study rather than another activity.

Data quality: OMB’s guidance states that a “justification for requesting use of an incentive is improvement in data quality. For example, agencies may be able to provide evidence that, because of an increase in response rates, an incentive will significantly improve validity and reliability to an extent beyond that possible through other means.”

Several studies have demonstrated that monetary incentives help to increase response rates, particularly among hard-to-reach populations (Shaghaghi, Bhopal, & Sheikh, 2011), convert refusals, and reduce subsequent attrition (Singer, & Kulka, 2022; Pit, Vo, & Pyakurel, 2014;). Empirical studies have established that larger incentives (e.g., $100, $150) perform significantly better than smaller incentives (Church, 1993; Hsu et al., 2016; Shettle & Mooney, 2009; Martinez-Ebers, 1997;). Providing low incentives can result in increased time and cost to recruit the required number of participants for the study as participants may agree to participate and then not show up or drop out early.

Importantly, physicians are a difficult population to recruit for research (Asch et al. 2000; VanGeest, Johnson, & Welch, 2007) and their response rates have been decreasing in recent years. Additionally, there are only a limited number of physicians in online panels. High nonresponse can threaten our ability to achieve the target number of completes for the study. Therefore, it is critical to maximize the number who respond to ensure sufficient power to determine meaningful differences by experimental conditions. An underpowered study increases the chance for Type II error, which may result in erroneously rejecting hypothesized models (Cohen et al., 2003). As such, the honoraria are intended to recognize the time burden placed on participants, encourage their cooperation, and convey appreciation for contributing to this important study. The use of modest incentives is expected to enhance survey response rates and reduce nonresponse bias.

Improved coverage of specialized respondents, rare groups, or minority populations: Incentives are also necessary to ensure a reasonable cross-section of participants, reflecting diversity in age, income, and education. Studies have shown that incentives can reduce nonresponse bias for key subgroups (Lesser et al., 2001; Griffin et al. 2011; Groth, 2010). Leverage-salience theory argues that monetary incentives can help to recruit people who otherwise might not be motivated to respond (e.g., people who do not care about the topic, lack altruistic motives for responding, have competing obligations) or are typically less likely to participate in research (Groves, Presser, & Dipko, 2004; Guyll, Spoth, & Redmond, 2003; Herman, 2017). Using incentives to bring in a cross section of consumers who otherwise may not participate can reduce nonresponse bias if these participants (those less interested in the topic, men, minorities, high income) have different responses and feedback than those who would participate without incentives (Castiglioni & Pforr, 2007).

Below are incentive rates that have also been approved for online surveys with PCPs of similar length.

• $100 for PCPs and specialists for a 20-minute survey web mixed mode (OMB package #0990-0415)

• $55 for PCPs (OMB package #0910-0730)

• $50 for PCPs (OMB package #0910-0860)

We are also providing incentive rates that have also been approved for online surveys with general population consumers of similar length.

• $7.50 for consumers from the general population for a 20-minute online survey (OMB package #0910-0695)

• $7.50 for consumers from the general population for a 30-minute online survey (OMB package #0910-0785)

• $1.50 for consumers from the general population for a 15-minute online survey (OMB package #0910-0885)

• $1.50 for consumers from the general population for a 20-minute online survey (OMB package #0910-0896)

Participants will be compensated only for surveys that they qualify for and complete. Incentive amounts are determined by participant type and the time commitment involved.

RTI has consulted with Ipsos about the $50 incentive. Ipsos has said that $50 is the currently-acceptable amount for PCPs expected to answer a 20-minute survey.

Offering an incentive below these rates may result in increased costs exceeding the amount saved with a lower incentive. Consequences of insufficient incentives include increased time and cost of recruitment, and increased probability of cancelled or postponed interviews.

10. Assurance of Confidentiality Provided to Respondents

In preparing this Supporting Statement, we consulted our Privacy Office to ensure appropriate identification and handling of information collected.

This ICR does not collect personally identifiable information (PII) or information of a personal nature. While PII is collected at the subcontractor website (i.e., name, phone number, email address) it is collected so the individual can be compensated for their participation in various surveys. The individuals who provide their PII are voluntarily signing up to participate in surveys from many companies, not just for the FDA. They are considered an existing pool of participants. Information is not collected on behalf of the FDA. Because the FDA does not collect PII or the information collection is not done on behalf of the FDA, the ICR is not subject to the Privacy Act of 1974 and the requirements of the Privacy Act such as displaying a Privacy Act Statement on a collection form do not apply.

Under the Freedom of Information Act (FOIA) (5 U.S.C. 552), the public has broad access to government documents. However, FOIA provides certain exemptions from mandatory public disclosure of government records (5 U.S.C. 552(b)(1-9)). FDA will make the fullest possible disclosure of records to the public, consistent with the rights of individuals to privacy, the property rights of persons in trade and confidential commercial or financial information.

11. Justification for Sensitive Questions

This data collection will not include sensitive questions.

12. Estimates of Annualized Burden Hours and Costs

12a. Annualized Hour Burden Estimate

FDA estimates the burden of this collection of information as follows:

Table 1.--Estimated Annual Reporting Burden¹

Activity	Number of respondents	Number of responses per respondent	Total annual responses	Average burden per response	Total hours
General Consumer Population
Pretest 1 screener (assumes 80% eligible)	22	1	22	0.08 (5 min.)	1.8
Pretest 1 survey	17	1	17	0.33 (20 min.)	5.6
Pretest 2 screener (assumes 80% eligible)	22	1	22	0.08 (5 min.)	1.8
Pretest 2 survey	17	1	17	0.33 (20 min.)	5.6
Main study screener (assumes 80% eligible)	413	1	413	0.08 (5 min.)	33
Main study survey completes	330	1	330	0.33 (20 min.)	108.9
PCP Population
Pretest 1 screener (assumes 30% eligible)	57	1	57	0.08 (5 min.)	4.6
Pretest 1 survey	17	1	17	0.33 (20 min.)	5.6
Pretest 2 screener (assumes 30% eligible)	57	1	57	0.08 (5 min.)	4.6
Pretest 2 survey	17	1	17	0.33 (20 min.)	5.6
Main study screener (assumes 30% eligible)	1,100	1	1,100	0.08 (5 min.)	88
Main study survey completes	330	1	330	0.33 (20 min.)	108.9
Total					374

¹ As with most online and mail surveys, it is always possible that some participants are in the process of completing the survey when the target number is reached and that those surveys will be completed and received before the survey is closed out. To account for this, we have estimated approximately 10 percent overage for both samples in the study.

² Burden estimates of less than 1 hour are expressed as a fraction of an hour in decimal format.

12b. Annualized Cost Estimate

There are no capital costs or operating and maintenance costs associated with this collection of information. As a voluntary collection being administered at FDA’s expense, we estimate no annualized cost to respondents.

13. Estimates of Other Total Annual Costs to Respondents and/or Recordkeepers/Capital Costs

There are no capital, start-up, operating or maintenance costs associated with this information collection.

14. Annualized Cost to the Federal Government

The total estimated cost to the Federal Government for the research is $858,663.00. This includes the costs paid to the contractor to assist with study design, questionnaire, and stimuli development, recruit a sample, collect and analyze data, write reports of work completed, and present findings. The task order was awarded as a result of competition. Specific cost information other than the award amount is proprietary to the contractor and is not public information.

15. Explanation for Programs Changes or Adjustments

This is a new data collection.

16. Plans for Tabulation and Publication and Project Time Schedule

See part B for detailed information on the design and analysis plan. The Agency anticipates disseminating the results of the study after the final analyses of the data are completed, reviewed, and cleared. The exact timing and nature of any such dissemination has not been determined, but may include presentations at trade and academic conferences, publications, articles, and posting on FDA’s website.

Table 2. Estimated Project Timetable

Task	Estimated Completion Date
IRB review	May, 2022
30-day FRN publication	September, 2022
OMB Review of PRA package	October, 2022
Pretesting	November, 2022
Main Study Data Collection	March, 2023

17. Reason(s) Display of OMB Expiration Date is Inappropriate

FDA will display the OMB expiration date as required by 5 CFR 1320.5.

18. Exceptions to Certification for Paperwork Reduction Act Submissions

There are no exceptions to the certification.

References

Asch, S., Connor, S.E., Hamilton, E.G., and Fox, S.A. “Problems in Recruiting Community-Based Physicians for Health Services Research.” Journal of General Internal Medicine, 15(8):591–599, 2000.

Castiglioni, L., and Pforr, K. The Effect of Incentives in Reducing Non-Response Bias in a Multi-Actor Survey. Presented at the 2nd annual European Survey Research Association Conference, Prague, Czech Republic, 2007.

Church, A.H. “Estimating the Effect of Incentives on Mail Survey Response Rates.” Public Opinion Quarterly, 57(1):62-79, 1993.

Cohen, J., Cohen, P., West, S.G., and Aiken, L.S. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates, 2003.

Griffin, J.M., Simon, A.B., Hulbert, E., Stevenson, J., Grill, J.P., Noorbaloochi, S., and Partin, M.R. “A Comparison of Small Monetary Incentives to Convert Survey Non-Respondents: A Randomized Control Trial.” BMC Medical Research Methodology, 11(1), 81, 2011.

Groth, S.W. “Honorarium or Coercion: Use of Incentives for Participants in Clinical Research.” Journal of the New York State Nurses Association, 41, 11–3, 2010.

Groves, R.M., Presser, S., and Dipko, S. “The Role of Topic Interest in Survey Participation Decisions.” Public Opinion Quarterly, 68(1), 2–31, 2004.

Guyll, M., Spoth, R., and Redmond, C. “The Effects of Incentives and Research Requirements on Participation Rates for a Community-Based Preventive Intervention Research Study.” Journal of Primary Prevention, 24(1), 25–41, 2003.

Herman, J. Prescriptive authority update 2017. Contemporary Clinic. (2017, June 3). http://contemporaryclinic.pharmacytimes.com/journals/issue/2017/june2017/prescriptive-authority-update-2017

Hsu, J.W., Schmeiser, M.D., Haggerty, C., and Nelson, S. “The Effect of Large Monetary Incentives on Survey Completion: Evidence from a Randomized Experiment with the Survey of Consumer Finances.” Public Opinion Quarterly, 81(3), 736–747, 2017.

Lesser, V.M., Dillman, D.A., Carlson, J., Lorenz, F., Mason, R., and Willits, F. Quantifying the Influence of Incentives on Mail Survey Response Rates and Nonresponse Bias. Paper presented at the annual meeting of the American Statistical Association, Atlanta, GA, 2001.

Pit, S.W., Vo, T., and Pyakurel, S. “The Effectiveness of Recruitment Strategies on General Practitioner’s Survey Response Rates—A Systematic Review.” BMC Medical Research Methodology, 14(1), 76, 2014.

Russell, M.L., Moralejo, D.G., and Burgess, E.D. “Participants’ Perspectives.” Journal of Medical Ethics, 26(2), 126–130, 2000.

Shaghagi, A., Bhopal, R. S., and Sheikh, A. “Approaches to Recruiting ‘Hard-to-Reach’ Populations into Research: A Review of the Literature.” Health Promotion Perspectives, 1(2), 86–94, 2011.

Shettle, C., and Mooney, G. “Monetary Incentives in US Government Surveys.” Journal of Official Statistics, 15(2), 231, 1999.

Singer, E. and Kulka, R.A. “Paying Respondents for Survey Participation.” In M. Ver Ploeg, R.A. Moffitt, & C.F. Citro (Eds.), Studies of Welfare Populations: Data Collection and Research Issues, Washington, D.C.: National Academy Press, 2002.

Martinez-Ebers, V. “Using Monetary Incentives with Hard-To-Reach Populations in Panel Surveys.” International Journal of Public Opinion Research. 99(1), 77–86, 1997.

VanGeest, J.B., Johnson, T.P., and Welch, V.L. “Methodologies for Improving Response Rates in Surveys of Physicians: A Systematic Review.” Evaluation and the Health Professions, 30(4), 303–321, 2007.