Supporting Statement - 9-24- 2009

Request for OMB Review
Docket No. 2008-N-0589

Supporting Statement for

Mental Models Study:
Physician Understanding of Drug Product Effectiveness

Submitted by:

Division of Drug Marketing, Advertising, and Communications
Office of Medical Policy
Center for Drug Evaluation and Research
Food and Drug Administration
Department of Health and Human Services

July 2009
(second revision August 2009)

Mental Models Study: Physician Understanding of Drug Product Effectiveness

A. JUSTIFICATION
1. Circumstances Making the Collection of Information Necessary
The Federal Food, Drug, and Cosmetic Act (the Act) requires that manufacturers, packers,
and distributors (sponsors) who advertise prescription human and animal drugs, including biological
products for humans, disclose in advertisements certain information about the advertised product's
uses and risks. 1 By its nature, the presentation of this risk information is likely to evoke active
trade-offs by consumers, i.e., comparisons with the perceived risks of not taking a treatment, and
comparisons with the perceived benefits of taking a treatment. 2 The FDA has an interest in
fostering safe and proper use of prescription drugs, which is an activity that necessitates
understanding of both risks and benefits. Thus, an in-depth understanding of physicians’ processing
of this information, their thinking on relevant topics, and their informational needs are central to this
regulatory task.
Under the Act, FDA engages in a variety of communication activities to ensure that patients
and health care providers have the information they need to make informed decisions about
treatment options, including the use of prescription drugs. FDA regulations (21 CFR § 201.57)
describe the content of required product labeling, and FDA reviewers ensure that labeling contains
accurate and complete information about the known risks and benefits of each drug.
2. Purpose and Use of the Information Collection
As mentioned above, this data collection will provide FDA with insight for evaluating and
improving current communication procedures. It is designed to identify knowledge gaps for FDA

1 For prescription drugs and biologics, the Act requires advertisements to contain "information in brief summary
relating to side effects, contraindications, and effectiveness" (21 U.S.C. 352(n)).
2 See Swartz, L., Woloshin, S., Black, W., & Welch, H.G. (1997). The role of numeracy in understanding the benefit
of screening mammography. Annals of Internal Medicine, 127(11), 966-72.
2

to address, which would ultimately improve practitioner decision making and hence the health
outcomes of the affected patients. This new information collection uses Mental Modeling, a
qualitative research method that compares a model of the decision-making processes of a group or
groups to a model of the same process developed from expert knowledge and experience. In this
study, the decision models of health care providers (HCPs) concerning their understanding of drug
product efficacy and how they communicate their understanding to their patients will be compared
to a model derived from the knowledge and experience of a panel of experts, including rheumatoid
arthritis medical researchers, medical educators and FDA staff who have oversight over drug
reviews, approvals and labeling. FDA will use telephone interviews to determine from the health
care providers the factors that influence their understanding of drug product efficacy and how they
communicate their understanding to their patients. Comparing expert and health care provider
responses will allow for a richer understanding of decisions determining drug product efficacy from
labeling and other sources and how this understanding is communicated to their patients.
FDA regulations require that prescription drug advertisements that make (promotional)
claims about a product also include risk information in a “balanced” manner (21 CFR
202.1(e)(5)(ii)), both in terms of the content and presentation of the information. This balance
applies to both the front display page of an advertisement and the brief summary page. However,
beyond the ‘balance’ requirement there is limited guidance and research to direct or encourage
sponsors to present benefit claims that are informative, specific, and reflect clinical effectiveness
data.
Healthcare professionals have many avenues through which they can learn about the
effectiveness of the products they prescribe, for example, the approved product labeling, published
studies, advertising, and their professional colleagues. They may also gain knowledge about
product effectiveness through their own prescribing experiences with the product.

3

Consumers’ understanding of product efficacy is most likely to come from direct experience
with the product itself. Outside of direct experience, consumers may also learn about products
through informational sources, the product labeling or through advertising. Unlike drug advertising
aimed at healthcare professionals, “benefit claims,” broadly defined, appearing in consumerdirected advertisements are often presented in general language that does not inform patients of the
likelihood of efficacy and are often simply variants of an “intended use” statement. 3 In a study
involving a content analysis of DTC advertising, the researchers classified the “promotional
techniques” used in the advertisements. Emotional appeals were observed in 67% of the ads while
vague and qualitative benefit terminology was found in 87% of the ads. Only 9% contained data.
However, for risk information, half the advertisements used data to describe side-effects, typically
with lists of side-effects that generally occurred infrequently. Lack of specific efficacy information
may cause consumers to under or overestimate a product’s effectiveness relative to how a physician
understands the product’s efficacy.
The Mental Models phase of this research is the first part of a larger research study. In this
first study phase we will explore how physicians conceptualize product efficacy from FDAapproved labeling and other non-promotional sources. The qualitative information in this Mental
Models phase of the research will provide a preliminary framework and help FDA craft subsequent
quantitative studies. Subsequent quantitative phases will examine physician and consumer
responses to various efficacy presentations. The ultimate goal of this multi-phase study is to
explore ways to narrow the gap between consumer comprehension of product efficacy and
physician understanding of product efficacy.

3. How, by Whom, and the Purpose for Collecting this Information

3 Woloshin, S. and Schwartz, L. (2001). Direct to consumer advertisements for prescription drugs: what are Americans
being told. Lancet, 358, 1141-46.
4

The proposed information collection will use “mental modeling,” a qualitative research
method wherein the decision-making processes of a group of physician respondents concerning the
effectiveness of various prescription drug products are modeled and compared to a model based on
expert labeling knowledge and clinical experience in drug effectiveness. The information will be
collected via telephone interviews concerning the factors that influence perceptions and decisions
related to drug effectiveness. This method will help identify physicians’ beliefs, priorities,
informational needs, visions and conceptualizations about how well particular drugs work. A
comparison between expert and physician models based on the collected information may identify
“consequential knowledge gaps” that can be redressed through labeling changes as well as helping
FDA focus future quantitative research on the communication of drug benefit information. Thus,
the information to be collected will be used by FDA to develop and strengthen research materials
and design in future planned quantitative experiments.
FDA/CDER has contracted with Decision Partners, a world leader in risk perception
research, to develop and conduct the Mental Models study, which they will do with the aid and
input of the FDA project officer, an expert in social science research. The first step in the mental
models process is to conduct background research to develop a model based on both experts’
current knowledge and extant literature on drug effectiveness. The resulting “simple expert model”
is a mapping of decision-making factors, relationships and influences, and is used to develop an
interview protocol for a day-long workshop with experts, hereafter referred to as the “expert
elicitation.”
The expert elicitation was conducted November 28, 2007. It included nine experts from a
variety of medical fields, including those versed in drug labeling issues and others with extensive
clinical experience, particularly involving two medical conditions (insomnia, a medical condition
frequently treated by general practitioners, and rheumatoid arthritis, a condition likely treated by

5

specialists). Six experts were internal to FDA, two experts were from NIH, and one expert was
external to the Federal Government, from the Association of Medical Colleges. The expert
elicitation process does not solicit advice, opinions, or recommendations from the group, but instead
tries to determine how each expert perceives the factors related to consumer decision-making, from
their particular expert field. Results from the expert elicitation were used to develop the expert
model, which generally includes adding new concepts and supporting details to the existing simple
expert model. The new draft expert model was validated during a subsequent teleconference with
the research team about a month following the initial elicitation. Following the validation, the
project team finalized the expert model.
The expert model informs the development of the physician interview guide for physician
telephone interviews. Mental models research is typically conducted with cohorts of respondents
who represent categories of people whose mental models are to be compared, both individually with
the expert model and between cohorts, identifying the potential for significant differences among
cohorts. For the research in question, Decision Partners will work under FDA’s direction to
conduct interviews with 40 health care providers to develop a mental model describing how each of
two cohorts learns about drug product efficacy and how their understanding about efficacy is
communicated to their patients. The cohorts are as follows:
1) Primary care providers. This cohort includes office-based practitioners in primary care
(general practice, family practice, and internal medicine) with at least three years of
experience and who engage in patient care at least 50% of the time.
2) Specialists. This cohort includes office-based practitioners in rheumatology with at least
three years of experience and who engage in patient care at least 50% of the time.
Cohorts will be identified and recruited to represent a reasonable range of age, gender, and
ethnicity.

6

Within each cohort, 20 practitioners will be interviewed by trained interviewers in one-onone in-depth telephone interviews. A sample size of 40 (approximately 20 primary care providers
and 20 Rheumatologists) is sufficiently large for the qualitative findings to capture a wide depth and
range of people’s thinking. The interviews will take approximately 45 minutes. Each participant
will receive $100.
Potential physician participants will be randomly identified through a purchased list based
on the American Medical Association’s (AMA) Physician Masterfile. This list tracks all physicians
(MDs and DOs) practicing in the US, not only members of the AMA.
Four pretest interviews were conducted with general practitioners and rheumatologists. The
one-on-one interviews lasted approximately 45 minutes each. These interviews were used to refine
the physician interview protocol.
The health care provider interviews will be used to create a mental model of physician
decision-making factors with respect to drug product effectiveness. Decision Partners will identify
the gaps and inconsistencies between the physician and expert models and provide
recommendations on the areas that will be important for planning the next quantitative research
steps. At the conclusion of the study, the contractor will produce expert and health care provider
decision-making models and prepare a final project report containing recommendations.
FDA intends this collection to be used as formative research. As with our focus group
research (OMB control number 0910-0360)4, the results of this formative research will provide
direction toward potential areas of focus. Further research is necessary, and planned, to test
concepts obtained from these results. This research will be useful in designing survey questions for
the next phases of this research project (which will be submitted for approval at a later date).

4 Based on the focus groups, we learned that one of the medical conditions we had chosen, insomnia, was not the best
choice for general practitioners. As a result, the protocol for general practitioners now asks physicians to discuss the
condition psoriasis.
7

4. Improved Information Technology and Burden Reduction
The study does not use electronic collection of information. Qualitative interview guides are
often unstructured. The questions are generally open-ended, allowing interviewees to respond
without restriction. As opposed to structured questionnaires, the goal of a qualitative inquiry is to
discover the range of meaningful themes and categories, which are often used in follow-up,
quantitative research. Typically, a qualitative interview requires some interaction between the
respondent and the interviewer. While for some qualitative studies it may be appropriate to engage
in an electronic interaction through a computer interface, mental modeling interviews rely on the
subtleties that can only be detected through verbal conversation. The interviews for this research
are conducted over the telephone, which minimizes respondent burden that would be incurred
through time and travel required to conduct the interviews in person. Past experience has shown
that telephone interviews are sufficient to provide the necessary connection and rapport to result in
useful data and similar procedures have been followed in other studies. 5

5. Efforts to Identify Duplication of Similar Information
There is no likelihood of Federal duplication of effort across agencies. We are unaware of
research that addresses issues of communication of drug effectiveness information in any context,
and certainly not in the context of drug labeling and advertising. As FDA has authority in this area,
it is unlikely that other agencies would expend resources to study these issues particular to FDA.
Risk communication research has been conducted within FDA on food terrorism issues and on
physician labeling for pregnancy. These projects are helpful for providing background and
supporting information about the mental modeling process. We note there is an academic group

5 See, for example, Darisi T., Thorne, S., & Iacobelli, C. (2005). Influences on decision-making for undergoing plastic
surgery: a mental models and quantitative assessment. Plastic Surgery, 116, 907-16, and Downs, J. S., Bruine de Bruin,
W., & Fischhoff, B. (2008). Parents' vaccination comprehension and decisions. Vaccine, 26, 1595-1607.
8

that is researching the presentation of efficacy information within the context of a “drug facts” box. 6
The “drug facts” project is helpful for providing background and supporting information about
consumers’ use of one type of efficacy information. The focus of the research outlined in this
document is to explore physicians’ understanding of drug efficacy and how they communicate that
understanding to their patients. Thus, none of the projects described above duplicates the research
proposed here. FDA’s information needs are unique and require a targeted research strategy.

6. Small Businesses or Other Small Entities
This study will have no impact on small businesses or other small entities. The information
collection is completely voluntary and health care providers can complete the interview whenever
they wish.

7. Consequences of Collecting the Information Less Frequently
This is a one time information collection. Without the data collection, FDA would not have
the knowledge or understanding of how best to design future communications to physicians and
future materials for quantitative physician and consumer studies.
8.

Special Circumstances Relating to the Guidelines of 5 CFR 1320.5
This collection fully complies with 5 CFR 1320.5. The only special circumstance associated

with this information collection is the assurance of confidentiality for the information collected
from health care providers. Please see section 10 below for a discussion on confidentiality.

9. Comments in Response to the Federal Register Notice

6 Schwartz, L. M., Woloshin, S., & Welch, H. G. (2007). The drug facts box: Providing consumers with simple tabular
data on drug benefit and harm. Medical Decision Making, 27(5), 655-662; Schwartz, L.M., Woloshin, S., & Welch,
H.G. (2009). Communicating drug benefits and harms with a Drug Facts Box: Two randomized trials. Annals of
Internal Medicine, 150(8), in press.
9

In accordance with 5 CFR 1320.8(d), on November 24, 2008, in Volume 73, pages 7100671007, a 60-day notice was published in the Federal Register requesting public comment on the
information collection provisions. No comments were received from the public.

10. Payment/Gift to Respondent
Decision Partners typically offers an honorarium to interviewees for participation in a
research project. While honoraria for lay participants is generally on the order of between $25-$30,
for the interviewees in this study Decision Partners has suggested that honoraria be $100. Their
experience has been that physicians expect to be compensated more generously, especially given
that they may be contributing up to an hour of their time. FDA has recently completed another
mental models study using this incentive amount (Mental Models Study of Communicating with
Health Care Providers about the Risks and Benefits of Prescription Drug Use for Pregnant and
Nursing Women with Chronic Conditions, OMB control number 0910-0631).

11. Assurance of Confidentiality Provided to Respondents
The contractor collects information for the sample list for the sole purpose of inviting people
to participate in an interview. The information is stored securely and will not be used unless the
person opts to participate in an interview. Under no circumstance is contact information ever
released to a third party.
The information collected from healthcare providers in this research is considered
confidential. This study has been reviewed by FDA’s Institutional Review Board (Research
Involving Human Subjects Committee - RIHSC) Director and has been determined to be exempt
from RIHSC review under 45 CFR 46.101(b)(2). This exemption states that although participants
may be identified, disclosure of information could not place them at risk of civil or criminal liability
or be damaging to their financial standing, employability, or reputation. Despite this exemption, the
10

contract specifies that respondents be given assurances of confidentiality. Confidentiality will be
assured by using an independent contractor to collect the information, by enacting procedures to
prevent unauthorized access to respondent data, and by preventing the public disclosure of the
responses of individual participants. Identifying information will not be included on the data files
delivered to FDA. Confidentiality of the information submitted is protected from disclosure under
the Freedom of Information Act (FOIA) under sections 552(a) and (b) (5 U.S.C. 552(a) and (b), and
by part 20 of the agency’s regulations (21 CFR part 20). In addition, this study has been reviewed
by the FDA Research in Human Subjects Committee Center liaison and has been approved for six
months, subject to reevaluation and continued approval. Part of the criteria for approved status
includes ensuring that identifying information is stripped from the data and that appropriate security
procedures are in place. Respondents are also informed of their rights to privacy and their right to
refuse to participate, to quit at any time, and to skip any questions they want.
12. Justification for Sensitive Questions
No questions of a sensitive nature are asked in this information collection. Further, we are
only asking respondents to speak within the context of their professional capacities.

13. Estimate of Annualized Burden (Total Hours and Wages)
FDA estimates the total annual burden for this one-time collection of information to be 33
hours. FDA estimates that respondents will take 45 minutes (0.75 hours) to complete the interview.
There will be a total of no more than 44 respondents, four for pre-tests and 40 for the data
collection.

11

Estimated Annual Reporting Burden1
Activity

No. of
Respondents

Annual
Frequency per
Response

Total Annual
Responses

Hours per
Response

Total Hours

4

1

4

0.75

3

40

1

40

0.75

30

Pretests/Cognitive
Interview
Study
1

There are no capital costs or operating and maintenance costs associated with this collection of information

Physicians' Wages, 2008
Source: Bureau of Labor Statistics

Occupation
Title (click on
the occupation
Employment
title to view an
occupational
profile)

Median
Hourly

Mean
Hourly

Mean Annual

Mean RSE

106,210

$75.60

$77.64

$161,490

1.00%

46,980

more than
$84.97
$80

$176,740

1.40%

19,750

more than
$92.68
$80

$192,780

1.50%

Pediatricians,
General
29-1066 Psychiatrists

29,170

$70.21

$73.74

$153,370

1.50%

22,140

$154,050

1.60%

29-1067 Surgeons

47,070

$74.13 $74.06
more than
$99.41
$80

$206,770

1.10%

more than
$79.33
$80

$165,000

1.10%

Code

Family and
29-1062 General
Practitioners
29-1063

Internists,
General

Obstetricians
29-1064 and
Gynecologists
29-1065

Physicians and
29-1069 Surgeons, All
262,850
Other

(1) Estimates for detailed occupations do not sum to the totals because the totals include occupations not
shown separately. Estimates do not include self-employed workers.

(2) Annual wages have been calculated by multiplying the hourly mean wage by a "year-round, full-time" h
figure of 2,080 hours; for those occupations where there is not an hourly mean wage published, the annua
has been directly calculated from the reported survey data.
(3) The relative standard error (RSE) is a measure of the reliability of a survey statistic.
The smaller the relative standard error, the more precise the estimate.

12

14. Annualized Capital Costs (Maintenance of Capital Costs)
There are no capital costs or operating and maintenance costs associated with this collection.
15. Cost to Federal Government
The total cost to the Federal government for this data collection is approximately $80,100.00.
This includes fees paid to the contractor to design the study, draw the sample, collect the data,
produce expert and consumer decision-making models, and prepare a final project report containing
recommendations (see table below).
Task

Approximate Cost

Research development and planning

15,000

Data collection

40,000

Client workshops and debriefing meetings

15,000

Participant honorarium

6,600

Sample development

3,500

16. Explanation for Program or Burden Changes
This is a new information collection.

17. Plans for Tabulation and Project Time Schedule
The Agency anticipates disseminating the results of the study after the final quantitative
phases of the research are completed, reviewed, and cleared. Final results of the entire study may
be summarized for publication in a peer-reviewed scientific journal. Currently, there are no plans
to publish summaries or final reports of this particular part of the research in either hard copy or on
the Internet. Subsequent quantitative research informed by this qualitative research will be

13

reported to the public at a future date. This qualitative research may be described as background in
the discussion of the quantitative research.
Revised Project Time Schedule
Task
Pretest protocol
Publish 30-day notice for public comment and OMB ICR for review
OMB review
Conduct mental models research with 40 physicians
Planning meeting to discuss scope, sample and design of Phase II

Time Frame
February, 2009
March, 2009
April-September, 2009
September, 2009
November, 2009

18. Reason(s) Display of OMB Expiration Approval Date is Inappropriate
N/A. The OMB Approval date will be displayed on all materials associated with the study.
19. Exceptions to “Certification for Paperwork Reduction Act Submissions”
No exceptions to the Certification for Paperwork Reduction Act Submissions for this
proposed research are requested.

B. COLLECTION OF INFORMATION EMPLOYING STATISTICAL METHODS
This information collection will not employ statistical methods. It is a qualitative data
collection that provides data that do not depend on either random selection or random assignment to
experimental conditions. Regardless, we include below information about the sample and how it
will be selected, as well as discuss, as appropriate, the methodology in general.
1. Respondent Universe and Sampling Methods
Mental models research is typically conducted with cohorts of respondents who represent
categories of people whose mental models are to be compared, both individually with the expert
model and between cohorts, identifying the potential for significant differences among cohorts. For
the research in question, Decision Partners will work under FDA’s direction to conduct interviews
with 40 health care providers to develop a mental model describing how each of two cohorts makes
14

decisions about drug product efficacy and communicates their understanding to their patients. The
cohorts are as follows:
•

Primary care providers. This cohort includes office-based practitioners in primary care
(general practice, family practice, and internal medicine) with at least three years of
experience and who engage in patient care at least 50% of the time.

•

Specialists. This cohort includes office-based practitioners in rheumatology with at least
three years of experience and who engage in patient care at least 50% of the time.

Potential physician participants will be randomly identified through a purchased list based on the
American Medical Association’s (AMA) Physician Masterfile. This list tracks all physicians (MDs
and DOs) practicing in the US, not only members of the AMA.
Within each cohort, 20 practitioners will be interviewed by trained interviewers in one-onone in-depth telephone interviews. Cohorts will be identified and recruited to represent a
reasonable range of age, gender, and ethnicity.
2. Procedures for the Collection of Information
The mental models interviews of approximately 45 minutes in length will be conducted by
trained researcher interviewers from the contractor. The interviews will take place by telephone and
will be recorded, without any identifying information attributed to the respondents. The interview
protocol is in Attachment A. The remarks from the recorded interviews will be transcribed, coded
and consolidated into a report.
3. Methods to Maximize Response Rates and Deal with Nonresponse
Physicians and other healthcare providers are known to be exceptionally busy professionals
who generally employ office managers, receptionists, secretaries, or nurses as “gatekeepers” to
screen mail and telephone contacts. Because they are considered difficult to reach, the following

15

procedures will be employed to maximize the response rates for the physician/health care provider
respondents.
(1) Pre-notification letters. Literature and practical experience have shown that “cold
calling” physicians for survey participation has a low chance of success. Instead, we will pre-notify
potential respondents through a letter (Attachment B) which FDA will send to the physicians. The
letter describes the purpose of the research and will be signed by the DDMAC Research Team.
Interested persons will be asked to call, email or write back to a designated person at Decision
Partners who will then contact them to schedule an interview.
(2) Callbacks. After the initial contact, additional callbacks will be employed in an attempt
to reach the physician. A negative response from the “gatekeeper” will not be accepted as a
termination. If the respondent is not available, an appointment for a callback will be made with the
gatekeeper, and the respondent will be contacted at the designated appointment time. If it is not
possible to schedule an appointment, the interviewer will leave a telephone number for the
respondent to schedule an appointment to conduct the interview.
(3) Incentives for physicians. Each physician will receive an honorarium of $100 as
incentive for participation. Our experience has shown that physicians expect to be compensated
generously, given that they may be contributing up to an hour of their time.
4. Test of Procedures or Methods to be Undertaken
The contractor, expert in the field of Mental Models research, has reviewed the
questionnaire (interview protocol – Attachment A). This questionnaire was also reviewed by FDA
individuals who are highly experienced with telephone survey design. A total of four (4) pre-test
Mental Models interviews were conducted as an initial test of procedures and respondent
understanding of terminology and questions. Following the pre-test, a number of questionnaire
items were simplified and others were reworded for clarity in response to feedback received by both

16

interviewers and physician respondents. One physician in the pretest commented that the
honorarium offered was low compared to others she receives for participating in research studies.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or
Analyzing Data
Because this is a qualitative study, there are no statistics that will be employed to analyze the
data. The contractor, Decision Partners, will qualitatively code and interpret the data and prepare a
report for FDA that contains the analysis and recommendations. The information collected will be
used to inform two quantitative data collections to be submitted at a later date. Please see section
number A3 above for respondent universe and data collection information.

17

ATTACHMENT A
SCREENER AND INTERVIEW PROTOCOL

18

Decision Partners Task Order 4
Contract # HHSF223200510007I
Physician Understanding of Drug Product Effectiveness
Script for Recruitment Responses

{Note: Have OMB control number available if asked..}
Use SCRIPT A in the following circumstances:
• Potential participant responds to the invitation via phone
• Potential participant responds to the invitation via voicemail, email or fax AND provides a
contact telephone number.
Use SCRIPT B in case where potential participant responds by email or and does not provide a
contact telephone number
SCRIPT A:
Hi XXXXXXX, thank you very much for responding to the invitation to participate in this research we are
conducting on behalf of the U.S. Food and Drug Administration (FDA) into the information needs of
healthcare providers and how they communicate with patients. If you are willing to participate in this
research, your name will be added to a list of potential research participants. If your name is randomly
selected, one of our researchers will call within 1-2 weeks to schedule a 1 hour one-on-one phone interview
at your convenience. The interview, and any other information you provide is confidential to Decision
Partners. As a thank you, we’re offering an honorarium of $100.00.
{Note: If caller requests more information on confidentiality: Decision Partners will not share any
potential identifying information with FDA or any other party. Our report of the interviews will
consolidate the responses of everyone we interview. You will not be identified anywhere in the report
and we will not retain any potentially-identifying information about you after this research.}

The time required to complete this information collection is estimated to average 45 minutes per
response, including the time to review instructions, search existing data sources, gather the data
needed, and complete and review the information collection.
An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for
this information collection is 0910-#### (expires mm/dd/yyyy).
Do you have any questions about this study?
{Note: If caller requests any additional information about the study that cannot be addressed by the
Decision Partners’ associate fielding the initial call, please offer to take down the person’s contact
information and Sara Eggers, Decision Partners’ Research Director, will follow up.}

Would you be interested in participating in this research: Yes/No
{If no, I thank you for your time today.}
{If yes, continue}

19

I have a few questions to ask to make sure we have the most up-to-date information. This should only take a
minute and will be confidential.
1. Can I ask for your first and last name? {Ask them to spell both names.}
2. Are you:
a. A family practice physician or general practitioner? If yes, insert “psoriasis” into Q3. If no, move
to 2.b.
b. A rheumatologist? If yes, insert “rheumatoid arthritis” into Q3
{If neither, thank them for their time and explain that we are interested in interviewing only from one
of these categories.}
3. Have you seen or treated a patient with [rheumatoid arthritis/psoriasis] in the past six months? {If no,
thank them for their time and explain that we are interested in interviewing physicians who have seen a
patient with [psoriasis/RA] in the past six months.}
4. What is the zip code of your primary office or practice?
{If they have more than one, ask for the zip code of the practice where they spend most of their time
with patients}
5. What is the best phone number for us to call to schedule an interview?
a. Is this a home, business or cell number?

6. What is the best time to call (AM, PM, evenings, weekends, anytime)?
7.

Are you Hispanic or Latino/Latina? [Yes/No]

8.

What is your race? Please select one or more:
[American Indian or Alaska Native]
[Asian]
[Native Hawaiian or other Pacific Islander]
[Black or African American]
[White]

Thank you very much for offering to participate in this research. One of Decision Partners Researchers may
be contacting you within 1 – 2 weeks to schedule an interview.

DPRC: Please record the following information:
Date:
Method of Contact: (Phone, email, fax, letter):
Gender:

SCRIPT B
Dear Dr. XXXXX
Thank you very much for responding to the invitation to participate in this research we are conducting on
behalf of the U.S. Food and Drug Administration (FDA) into the information needs of healthcare providers
and how they communicate with patients.

20

If you are willing to participate in this research, your name will be added to a list of potential research
participants. If your name is randomly selected, one of our researchers will call within 1-2 weeks to schedule
a 1 hour one-on-one phone interview at your convenience. The interview, and any other information you
provide is confidential to Decision Partners. As a thank you, we’re offering an honorarium of $100.00.
If you are still interested, please return this email or call our office at 1-877-588-9106 and provide us
with a contact telephone number and best time to reach you in the next day or two. I have a few
questions to ask to make sure we have the most up-to-date information. This should only take a minute and
will be confidential.
Sincerely,
Jan Vonk
Project Coordinator
Decision Partners

21

Decision Partners Task Order 4
Contract # HHSF223200510007I
Physician Understanding of Drug Effectiveness
Mental Models Protocol (Version 14)
Introduction
Hello, this is . I’m a researcher with a company called Decision Partners. We are
conducting research for the U.S. Food and Drug Administration (FDA) focused on healthcare
providers’ thinking about drug benefits and how they communicate that with patients. Thank you for
agreeing to participate. Our conversation will take about 45 minutes.
Our conversation today will be very open-ended. I have a list of questions to help guide our
discussion, but please feel free to raise anything that comes to mind as we go along. In this kind of
an interview, there are no right or wrong answers. All of your comments will add value to our
research.
I also want to assure you that this interview is confidential. We will not identify you as the specific
source of any comments in our report. Instead, our report will consolidate the responses of
everyone we interview.
Having said that, I would like to ask your permission to record our conversation to ensure that my
notes are accurate and complete. Again, I would like to stress that your responses will be kept
confidential and you will not be able to be identified through the recording. May we proceed on that
basis? (Indicate whether consent was given and proceed appropriately. If consent is not given,
terminate the interview.)
Interview Opening
Note: if interviewing a rheumatologist, the focal condition is rheumatoid arthritis (RA); if
interviewing a primary care provider, the focal condition is psoriasis.
Our discussion will take place in three parts. First we’ll talk generally about making choices among
pharmaceutical options, and to focus our conversation we’ll discuss treating patients with
[Rheumatoid Arthritis/Psoriasis]. Next we’ll discuss the information you need to help you make
judgments on drug effectiveness. Finally, we’ll discuss how you communicate about drug
effectiveness to patients.

The time required to complete this information collection is estimated to average 45 minutes per
response, including the time to review instructions, search existing data sources, gather the data
needed, and complete and review the information collection.
An agency may not conduct or sponsor, and a person is not required to respond to, a collection of
information unless it displays a currently valid OMB control number. The OMB control number for
this information collection is 0910-#### (expires mm/dd/yyyy).

22

Interviewer Note: If the Interviewee seems hesitant or uncomfortable with an open-ended
question, remind him or her, as appropriate, that you are interested in understanding their thinking
on the topic.
Part 1: Drug Efficacy, Effectiveness (5 minutes)
1. To start, could you briefly describe your practice to me?
Interviewer, keep brief. Prompt a-c [if not already mentioned]:
a. How many years have you been in practice?

OMB Control No. xxxx-xx

b. Can you describe your experience treating people with [RA/Psoriasis]?
•

For example, about how many of these patients do you see in a week?

c. Would you say that the economic status of the majority of your patients is high, medium,
or low?
d. Can you tell me approximately how many Medicaid patients you see in a typical week?
2. Thank you. Now, thinking broadly, I’d like you to tell me what first comes to mind when you
think of the term ‘drug effectiveness’?
a. How would you describe drug effectiveness to a medical student?
b. How, if at all, would you distinguish between drug efficacy and drug effectiveness?

Interviewer, read: Just to be sure we’re on the same page during our conversation, we are
defining drug effectiveness in this research as the degree to which a drug provides benefit to a
particular patient in the real world setting. By drug efficacy, we mean how the drug has
demonstrated treatment benefits in a controlled clinical study.
c. Do you have any comments on these definitions?

Part 2: Influences on Judgments of Effectiveness (10-15 minutes)
Now I would like to discuss how you assess drug effectiveness of pharmaceutical options for
[RA/psoriasis].
3. To aid in our discussion, I’d like to focus on a particularly interesting or challenging case you
faced in the last month or so when prescribing a pharmaceutical drug to patient with
[RA/Psoriasis]. What made this case particularly interesting or challenging?
a. {If not mentioned} What were your general treatment goals for this patient? Why?
b. {If not mentioned} Briefly, what were your treatment options for this patient? Why?
What were the key things you considered when you assessed whether a particular drug would
likely be more effective or less effective for this patient?
23

Prompt for details around any (a-e) that you haven’t heard mentioned. Ask f & g.
c. What about endpoints, that is, specific patient outcomes, such as such as biophysical
markers, functioning scores, and symptoms, that can be measured to determine
whether the drug is beneficial?
•

How, if at all, did you take into account any endpoints?

•

Which endpoints were most important?

d. What about the drug mechanism, that is the specific biochemical processes through
which a drug produces its pharmacological effect?
•

How, if at all, did you take into account the drug mechanism?

e. What about variability, that is, differences in drug responses among individual patients?
•
f.

How did it affect your assessment of drug effectiveness?

What about tolerability of the drug, that is the degree to which the drug can be taken
without serious side effects or discomfort to patients?
•

How did it affect your assessment of drug effectiveness?

g. What about patient satisfaction, that is, the degree to which the individual regards the
drug as useful, effective and/or beneficial?
•

How did it affect your assessment of drug effectiveness?

h. Were there any other factors?
•
i.

How did <> affect your assessment of drug effectiveness?

Of all of these factors that we have been discussing, which would you say was the most
important factor? Why do you say that?

4. How did you make drug benefit and risk tradeoffs in this case?
Listen for the following concepts. Probe for more details around these concepts if they
arise. For example, ”How did you take XXXXX into account?”
o

More common side effects. (This may be mentioned as tolerability.)

o

More serious, but less common adverse events.

o

The Interviewee’s tolerance or threshold for risk. Probe into the kinds of things that
factor into his or her tolerance for risk.

5. How confident were you at the time of your assessment of the drug’s effectiveness? Please
rate your confidence on a scale from 1 to 5, where 1 is no confidence and 5 is very high
confidence.
a. What factors affected your confidence?
Listen for uncertainty, clinical trials, case reports, previous experience, etc. and probe
for more details into topics that haven’t yet been addressed.
b. What, if anything, would increase your confidence in your assessment of the drug’s
effectiveness? How would this increase your confidence?
c. More generally, how confident are you in understanding the effectiveness of
pharmaceutical options for treating [RA/Psoriasis]. What things affect your confidence?
24

6. Are there any other things you considered when you assessed your patient’s treatment needs
in this case?
Researcher, you may hear about affordability, compliance, etc. If time is not an issue, probe
briefly into these concepts.
OMB Control
xxxx-xx
forNo.
this
7. What, if any, is the role of follow-up in assessing the effectiveness of the drug treatment
patient?

a. What specifically did you or will you look for in the follow-up?
b. More generally, how, if at all, does your thinking on the role of follow-up differ for a
typical patient with [RA/Psoriasis]? Listen for “no follow-up by the patient” as an
indication of effectiveness. If mentioned, probe into their thinking on this.
8. More generally, how, if at all, does your thinking on drug effectiveness in this challenging case
differ for typical patients with [RA/Psoriasis]?
9. How, if at all, does your thinking on drug effectiveness for drugs to treat symptomatic conditions
such as [RA/Psoriasis] differ for drugs to treat more asymptomatic conditions, such as
osteoporosis or high cholesterol?

Part 3: Information seeking (10 minutes)
Now I would like to discuss the information you need to help you make judgments on a drug’s
effectiveness.
10. What types of information do you look for most often when making judgments about the
effectiveness of a drug? Why?
Researchers – if an example is needed, say, Any statistical information, for example?
a. Where do you typically go to get this information?
Researchers –Listen for:
o

Clinical evidence: If clinical data, trials, evidence, studies, etc. are mentioned, go to
Q.12.

o

Prescribing information: If prescribing information, package insert, product label, PDR,
etc. is mentioned, go to Q.13,

11. [If not mentioned]: Do you rely on information from clinical trials when assessing the
effectiveness of a drug?
Researcher: If “yes”, ask a-e. If no, probe in detail for why not and ask d-e.
a. How do you use this information?
b. What things make this information more useful to you?

25

Listen for: types of data (clinical results, statistics, clinical relevance, presentation,
etc.)
c. What things make it less useful to you?
Prompt [if not already mentioned]:
•

Your confidence in the clinical data (and what influences that confidence).
Researcher, they may bring up statistical data. Prompt for what type of
OMB Control No. xxxx-xx
information they are thinking of.

•

The clinical relevance of clinical information. What would improve the clinical
relevance of this information?

d. Generally, how would you rate the usefulness of the information on effectiveness that
comes from clinical trials? Please rate this on a scale from 1 to 5, where 1 is generally
not useful and 5 is generally very useful.
e. How could it be improved?
12. [If not previously mentioned]: Do you rely on the prescribing information when specifically
assessing the effectiveness of a drug? This includes the package insert and it may also be
found in the Physician’s Desk Reference or PDR or summarized in other sources.
Researcher: If “yes”, ask a-g. If no, prompt in detail for why not and ask e-g.
a. Where do you get this prescribing information specifically, from the package insert,
PDR, or some other source?
b. How do you use this information when assessing the effectiveness of a drug?
c. What makes this information more useful to you?
Listen for: types of data (clinical results, statistics), clinical relevance, presentation,
etc.
d. What things make it less useful to you?
e. How would you rate the usefulness of prescribing information on drug effectiveness?
Please rate this on a scale from 1 to 5, where 1 is very low quality and 5 is very high
quality.
f.

How could drug effectiveness in the prescribing information be improved?

g. How, if at all, do you use prescribing information more generally, for information other
than drug effectiveness?
i. Please explain.
13. In general, what information would you like to have more accessible to you when assessing the
effectiveness of a drug?
14. Are there any barriers in your daily practice that limit your ability to seek the information that
you need? Please describe.
Part 4: Communicating drug effectiveness with patients (10 minutes)
Our last set of questions addresses how doctors talk to patients about drug effectiveness.

26

15. Generally speaking, how do you talk to your patients about drug effectiveness? Let’s focus first
on your patients who you are treating for [RA/psoriasis].
Prompt a-g [if not mentioned]
a. How do you talk about treatment benefits?
OMB Control No. xxxx-xx

b. How, if at all, do you discuss the drug mechanism or how the drug works to treat
[RA/Psoriasis]? If no, why don’t you discuss this?
c. How, if at all, do you present clinical study information to your patients?
d. Do you present any quantitative information (e.g. probabilities, likelihoods)?
a. If no, why don’t you discuss clinical information?
e. How, if at all, do you discuss any uncertainty about treatment response?
a. If no, why don’t you discuss uncertainty?
f.

How, if at all, do you discuss variability in treatment response? That is, how the drug
might work differently for different people?
ii. If no, why don’t you discuss variability?

g. How do you generally discuss benefit and risk tradeoffs?
16. How, if at all, do you make comparisons among specific treatment options when talking to
patients?
17. Now thinking more broadly, how, if at all, does your approach to communicating with patients
with [RA/Psoriasis] differ for more asymptomatic conditions, such as osteoporosis or high
cholesterol?
18. What, if any, factors about the patient do you take into account when you think about how best
to communicate to him or her?
a. Under what circumstances, if any, would you not talk about effectiveness? Please
describe
19. Generally-speaking, how would you rate your confidence in your ability to adequately
communicate drug effectiveness to your patients? Please rate this on a scale from 1 to 5,
where 1 is that you generally have a very low degree of confidence and 5 is that you generally
have a very high degree of confidence?
a. Why do you say that?
b. What does your confidence depend on?
c. Is there anything that could increase your confidence?
Part 5: Wrap Up (<5 minutes)
Thank you. We have almost finished the interview. I just have a few final questions to wrap up our
interview.

27

20. Has anything else come to mind during our conversation that you would like to share at this
time?
OMB Control No. xxxx-xx

21. If you could offer one piece of advice for the FDA regarding communicating drug benefits, what
would that be?
That concludes our interview. Your comments will be very useful in this research. If you have any
questions or comments in the days to come, please feel free to contact Sara Eggers, Decision
Partners Research Director, at 1-877-588-9106. Thank you very much for your time today.

28

OMB Control No. xxxx-x

ATTACHMENT B
INVITATION LETTER

29

HHS/FDA Letterhead

(Doctor’s Name and Address)

Dear Dr. ________________,

I am writing to invite you to consider participating in a research study that Decision Partners (DP), an independent
research firm, is conducting on behalf of the U.S. Food and Drug Administration (FDA). This study is part of an
important initiative to improve how drug information is communicated to patients.
Your participation in this study would involve being interviewed over the telephone by a professional researcher at a
pre-arranged time that is convenient for you. If you indicate your willingness to participate, your name will be included
in a pool of candidates from which a sample will be drawn for interviewing. If selected, DP will contact you to schedule
an interview. The interview is expected to take between 45 to 60 minutes. This study is one part of a larger research
project. The final report of the entire research study may be summarized for publication in a peer-reviewed scientific
journal or published on the FDA’s website.
Your participation is strictly voluntary and FDA will not know whether or not you choose to participate. Should you
participate, your responses will be kept completely confidential. No comments in research reports provided to the FDA
will be attributed to you or any other specific individual.
As a token of our appreciation, we will offer you a $100 honorarium for participating in the interview. We understand
that this is a small amount for your time, but hope that you will take this opportunity to help FDA improve how drug
information is communicated to health care providers. We look forward to your reply.
If you wish to participate, please contact Jan Vonk, Decision Partners Project Coordinator, either by email at
[email protected], by phone at 1-(877) 588-9106 or by FAX at 1-(866) 344-5510. Alternatively, if you wish
to respond by letter, please write to:
Jan Vonk, Project Coordinator
Decision Partners, LLC.
313 East Carson St.
Suite 200
Pittsburgh, PA 15219
Thank you for your consideration.
Sincerely,

Kathryn Aikin, Ph.D.
Office of Medical Policy
U.S. Food and Drug
Administration

Amie O’Donoghue, Ph.D.
Office of Medical Policy
U.S. Food and Drug
Administration

30

Helen Sullivan, Ph.D.
Office of Medical Policy
U.S. Food and Drug
Administration

ATTACHMENT C
60-DAY FEDERAL REGISTER NOTICE

31

[Federal Register: November 24, 2008 (Volume 73, Number 227)]
[Notices]
[Page 71006-71007]
From the Federal Register Online via GPO Access [wais.access.gpo.gov]
[DOCID:fr24no08-107]
----------------------------------------------------------------------DEPARTMENT OF HEALTH AND HUMAN SERVICES
Food and Drug Administration
[Docket No. FDA-2008-N-0589]

Agency Information Collection Activities; Proposed Collection;
Comment Request; Mental Models Study of Health Care Providers'
Understanding of Prescription Drug Effectiveness
AGENCY: Food and Drug Administration, HHS.
ACTION: Notice.
----------------------------------------------------------------------SUMMARY: The Food and Drug Administration (FDA) is announcing an
opportunity for public comment on the proposed collection of certain
information by the agency. Under the Paperwork Reduction Act of 1995
(the PRA), Federal agencies are required to publish notice in the
Federal Register concerning each proposed collection of information and
to allow 60 days for public comment in response to the notice. This
notice solicits comments on the information collection provisions of
the Mental Models Study of Health Care Providers' Understanding of
Prescription Drug Effectiveness. Together with other information being
collected, the results from this study will be used to help inform FDA
about how health care providers conceptualize the drug effectiveness
portion of the risk/benefit tradeoff and how that conceptualization
differs from how agency experts think about drug effectiveness. The
information gathered in this study will be used to focus and strengthen
future planned quantitative research. It will also contribute to FDA's
ability to communicate drug effectiveness information to health care
providers in labeling and other communications.
DATES: Submit written or electronic comments on the collection of
information by January 23, 2009.
ADDRESSES: Submit electronic comments on the collection of information
to http://www.regulations.gov. Submit written comments on the
collection of information to the Division of Dockets Management (HFA305), Food and Drug Administration, 5630 Fishers Lane, rm. 1061,
Rockville, MD 20852. All comments should be identified with the docket
number found in brackets in the heading of this document.
FOR FURTHER INFORMATION CONTACT: Elizabeth Berbakos, Office of
Information Management (HFA-710), Food and Drug Administration, 5600
Fishers Lane, Rockville, MD 20857, 301-796-3792.
SUPPLEMENTARY INFORMATION: Under the PRA (44 U.S.C. 3501-3520), Federal
agencies must obtain approval from the Office of Management and Budget
32

(OMB) for each collection of information they conduct or sponsor.
``Collection of information'' is defined in 44 U.S.C. 3502(3) and 5 CFR
1320.3(c) and includes agency requests or requirements that members of
the public submit reports, keep records, or provide information to a
third party. Section 3506(c)(2)(A) of the PRA (44 U.S.C. 3506(c)(2)(A))
requires Federal agencies to provide a 60-day notice in the Federal
Register concerning each proposed collection of information before
submitting the collection to OMB for approval. To comply with this
requirement, FDA is publishing notice of the proposed collection of
information set forth in this document.
With respect to the following collection of information, FDA
invites comments on these topics: (1) Whether the proposed collection
of information is necessary for the proper performance of FDA's
functions, including whether the information will have practical
utility; (2) the accuracy of FDA's estimate of the burden of the
proposed collection of information, including the validity of the
methodology and assumptions used; (3) ways to enhance the quality,
utility, and clarity of the information to be collected; and (4) ways
to minimize the burden of the collection of information on respondents,
including through the use of automated collection techniques, when
appropriate, and other forms of information technology.
Mental Models Study of Health Care Providers' Understanding of
Prescription Drug Effectiveness
Section 1701(a)(4) of the Public Health Service Act (42 U.S.C.
300u(a)(4)) authorizes FDA to conduct research relating to health
information.
[[Page 71007]]
Section 903(b)(2)(c) of the Federal Food, Drug, and Cosmetic Act (the
act) (21 U.S.C. 393(b)(2)(c)) authorizes FDA to conduct research
relating to drugs and other FDA regulated products in carrying out the
provisions of the act.
FDA regulations require that an advertisement that makes claims
about a prescription drug include a ``air balance'' of information
about the benefits and risks of the advertised product, in terms of
both content and presentation (21 CFR 202.1(e)(5)(ii)). In past
research, FDA has focused primarily on the risk component of the risk/
benefit ratio. In the interest of thoroughly exploring the issue of
fair balance, however, the presentation of effectiveness, or benefit,
information is equally important. This component has received less
scrutiny. The proposed information collection described here is the
first step in a three-phase study designed to investigate the role of
effectiveness information in prescription drug print advertising. Along
the way, we plan to investigate how health care providers use labeling
and other materials and experiences to reach conclusions about drug
effectiveness. We will use this information to provide a benchmark with
which to compare the information consumers receive from direct-toconsumer advertisements.
The information collection described here refers only to the
qualitative portion of the study series, Phase I. The purpose of the
proposed information collection is twofold. First, we plan to gather
information in this phase that will help us to determine the proper
concepts about which to inquire and the proper language to use when
asking health care providers in the second phase about the
effectiveness of certain drug products. Second, we will use the
information gathered in this phase to identify gaps in the
33

communication of effectiveness information in FDA sponsored materials,
such as the physician labeling.
The proposed information collection described here (Phase I of a
multi-phase project) will use ``mental modeling,'' a qualitative
research method that compares a model of the decisionmaking processes
of a group or groups to a model of the same decisionmaking processes
developed from expert knowledge and experience. In this study, the
decision models of certain health care providers concerning
effectiveness decisions of various treatment options for individuals
suffering from insomnia or rheumatoid arthritis will be compared to a
decision model concerning drug effectiveness that was derived from the
knowledge and experience of FDA reviewers responsible for product
labeling, National Institutes of Health clinical experts in this field,
and others involved in the training of medical professionals. FDA will
use telephone interviews to determine from the health care providers
the factors that shape their understanding and decisions about the
effectiveness of various drug treatments for their patients. A
comparison between expert and health care provider models based on the
collected information may identify consequential knowledge gaps that
can be redressed through messages designed by FDA and will provide
information for designing the second (quantitative) phase of research
with a national sample of health care providers.
Using a protocol derived from the research that resulted in the
expert model, trained interviewers will conduct one-on-one telephone
discussions with about 20 members of 2 categories of health care
providers, general practitioners and rheumatologists, who provide
direct patient care at least 50 percent of the time.
FDA has selected these two groups of physicians because the first
group is reasonably likely to treat insomnia, whereas the second group
treats rheumatoid arthritis. We selected these two medical conditions
for focus in the next two phases of the research because prescription
drug treatments for both are heavily advertised to consumers, drugs for
these conditions are variable in their risk/benefit profiles, and yet
they are each fairly complex in terms of risk/benefit profiles. Another
function of the current information collection is to determine the
feasibility of using these two medical conditions in the following
quantitative phases.
FDA estimates the burden of this collection of information as
follows:
Table 1.--Estimated Annual Reporting Burden\1\
--------------------------------------------------------------------------------------------------------------Annual Frequency
Hours Per
No. of Respondents
per Response
Total Annual Responses
Response
Total Hours
--------------------------------------------------------------------------------------------------------------40
1
1
0.75
30.0
--------------------------------------------------------------------------------------------------------------Total
................. .......................
.................
30.0
--------------------------------------------------------------------------------------------------------------\1\ There are no capital costs or operating and maintenance costs associated
with this collection of
34

information.
The study will involve about 40 respondents and take approximately
45 minutes each to complete. These estimates are based on the
contractor's extensive experience with mental models research.
Please note that on January 15, 2008, the FDA Division of Dockets
Management Web site transitioned to the Federal Dockets Management
System (FDMS). FDMS is a Government-wide, electronic docket management
system. Electronic comments or submissions will be accepted by FDA only
through FDMS at http://www.regulations.gov.
Dated: November 17, 2008.
Jeffrey Shuren,
Associate Commissioner for Policy and Planning.
[FR Doc. E8-27801 Filed 11-21-08; 8:45 am]
BILLING CODE 4160-01-S

35