CDER DTC Advertising Survey SS Part B

National Direct-to-Consumer Advertising Survey
OMB Control No. 0910-NEW
Supporting Statement Part B: Statistical Methods (used for collection of information
employing statistical methods)
1. Respondent Universe and Sampling Methods
An address-based sample will be drawn from the United States Postal Service (USPS)
Computerized Delivery Sequence File (CDSF). The USPS CDSF has good geographic detail
and is comprehensive, containing close to 100 percent of the households in the United
States. A sample of 5,042 will be drawn in order to obtain at least 1,500 completes, with
1,765 expected completes (see burden table). The goal is to recruit a sample that is
representative of the noninstitutionalized U.S. adult population.
Residential addresses in all 50 States and the District of Columbia will be included in the
sampling frame. To improve coverage, vacant and seasonal addresses will be retained in the
sampling frame. Some addresses will be excluded such as those PO Boxes that also have a
city-style street address (to avoid giving them multiple chances of being sampled) and drop
units (the small percentage of addresses where the mail is to a central address and is
distributed within that address by the management of the address).
Drawing from an ABS sample based on all USPS mailing addresses offers a substantial
improvement in coverage from RDD and web panel approaches. Forty-one percent of
American households are cell phone only (Ref. 1). Combining traditional RDD with cellphone RDD interview can address the coverage error in landline-only samples, but
respondents reached by cell phone are less likely to respond to telephone invitations to
complete surveys (Refs. 2, 3). Panel providers, in contrast, face challenges to coverage that
result from difficulties associated with panel recruitment and retention. The design also
addresses other measurement error concerns such as the ability to reach young adults, who
are much more likely to do surveys on the web than either telephone or paper; and the
completion of paper surveys by households without Internet access.
Prior to the main study, a pilot study will be conducted to test the data collection process. A
sample of 100 addresses will be drawn, with 35 expected completes based on the estimated
response rate (see burden table). The pretest sample will not overlap with the sample drawn
for the main study.
2. Procedures for the Collection of Information
Survey Overview
The 20-minute survey contains questions about respondents' knowledge of FDA's authority
with respect to prescription drug advertising, their exposure to DTC advertising, their beliefs
and attitudes about DTC advertising, and the influence of DTC advertising on further

information search and patient-physician interactions. At the end of the survey, respondents
will be randomly assigned to view one of two ads for fictional prescription drugs intended to
treat high cholesterol. They will be asked questions about FDA's authority regarding
specific claims within the ad. The survey will include a debriefing to inform respondents
that the advertised drug was fictitious. We will also measure other potentially important
characteristics such as demographics, insurance coverage, and prescription drug use.

Procedure
The survey will use a mixed mode approach consisting of web administration, with paper
surveys and additional telephone support for non-respondents. This approach was chosen for
several reasons, including its coverage and cost-efficiency.
The contact approach is based on the guidelines recently recommended by Dillman (Ref. 4)
for improving response quality in mixed-mode web and paper surveys. Contact with
sampled respondents will begin with a notification letter on FDA letterhead introducing the
study and providing the information necessary for completing the survey over the web;
namely the URL and a unique access code. Two $1 bills will be included in the letter. To
reduce error and burden for respondents accessing the web survey, the URL included in
these communications will direct to a brief FDA landing page created for this study
(Appendix B). From there, respondents will click through to the Westat-hosted survey and
enter their access code.
We will use the Hagen-Collier within household sampling scheme, which identifies the
desired household adult (18 and older) in advance of the mailing with four selection rules
(i.e., oldest male, oldest female, youngest male, youngest female). The first and third
household contacts will ask specifically for the sampled member (e.g., youngest male) to
respond, with instructions for selecting an appropriate replacement if the initial sampled
member is not available. This method has been tested in a national health survey for the
National Cancer Institute (Ref. 5). The survey will begin with a reminder of these
instructions. We recognize the sampling procedure will not always be followed correctly by
the household, but even interviewer-administered RDD surveys that utilize the next-birthday
method have selection error rates of about 30 percent (Ref. 5), and cell phones are often
shared between more than one person, making it a challenge to identify one unique
respondent.
In order to encourage more respondents to complete the survey on the web, a reminder/thank
you postcard will be sent 5 days later to the households, and a second letter will be sent to
non-respondents 7 days after the postcard. The reminder/thank you postcard and the second
letter will contain information similar to the initial letter, including the URL and
respondent’s unique access code. As noted in Part B, Section 9, we will conduct an
experiment with the reminder/thank you postcard to test whether a short statement
mentioning the previously paid incentive increases survey response. Half the sample will be
provided language that reminds respondents that they received a cash incentive in the

2

previous letter; the remaining half will be reminded they received a letter but will not be
specifically reminded about the incentive.
For households that do not complete the survey via the website, on the 4th contact attempt a
paper version of the survey will be mailed to non-respondents, 19 days after the original
letter has been sent. This approach, of withholding the paper option and offering alternative
modes sequentially is recommended, as studies have shown that offering a concurrent web
and paper option often results in mode confusion, significantly reducing response rates (Ref.
6). A reminder postcard for the mail survey will be sent 5 days after the initial mail survey is
sent. A tracking number that contains no identifiable data is included on each mailed
questionnaire for processing purposes. The recruitment materials can be seen in Appendices
C-G.
Though telephone administration is not part of the core data collection strategy, an 800number will be provided in all mailed materials for sampled respondents who are unable to
complete the survey on the website or on paper, or need assistance to do so. On mailed
materials, a statement will be included to invite Spanish-language speakers to call the 800number. If Spanish speakers call for assistance, respondents will be offered the option of
conducting the survey over the phone in Spanish. Under this design, a bilingual interviewer
will administer a translated version of the survey and then enter responses into the English
web instrument. The English language survey will be translated into Spanish to support
telephone interviewers with the administration.
As part of the survey, participants will be randomly assigned to view one of two direct-toconsumer prescription drug ads. The ads will be for fictitious drugs that treat high
cholesterol. Following its review, participants will answer questions about the FDA’s role in
the information presented. Because those who call the 800 number are unable to view the ad
on a computer or their phone, these respondents will skip out of questions that require they
view the ad before responding. Moreover, as the Spanish language survey will be telephoneadministered, the ad will not be translated.
Analysis Plan
We will test for any differences between modes (online versus mail survey) and will account
for any mode effects in our analyses. We will weight the data to account for different
probability of selection and nonresponse. We will examine the frequencies for survey items
and the relation between survey items and demographic and health characteristics. We also
plan to compare responses between this survey and FDA's 2002 survey for repeated items.
Conventional statistical techniques for survey data, such as descriptive statistics, t-tests, chisquare tests, and regression models will be used to analyze the data.
Weighting
Weighting helps to compensate for differential probabilities of selection; reduce biases due
to differential non-response; and make the estimates consistent with external population
totals. We will use a classical design-based approach for weighting, with the base weights

3

constructed from the inverse of the probabilities of selection. In the perfect data collection,
this scheme produces unbiased estimates and does not require any model assumptions.
However, these weights must be modified because of imperfections, such as under coverage
and the fact that some people do not respond to the survey. If under coverage and nonresponse are not addressed, the analysis may be biased.
The starting point in the development of the weights is an address-level base weight—the
inverse of the probability of selecting the address from the frame which is a constant since
all addresses are sampled at the same rate. We adjust the base weight for household nonresponse using data available from the sample frame such as the type of area (e.g.,
geography). These variables will be included in the file when the sample is selected. The
next step is to adjust the address-based weights to the adult level. We first account for
sampling on one adult from each household. To do this, the nonresponse-adjusted address
weights are multiplied by the number of adults in the household. Though the variable in the
questionnaire accepts responses from 1-9 adults in a household, for weighting purposes,
households with 4 or more adults will be aggregated into one upper-bound category of 4 or
more adults. The final step is raking the weights to demographic control totals for the U.S.
adult population. The control totals used in the raking will be current totals for persons ages
18 or older as reported in the American Community Survey. The variables planned for the
raking are:
•
•
•
•
•
•

Gender (Male, Female)
Age (18–34, 35-54, 55-64, and 65+)
Race/Hispanic ethnicity (White/Non-Hispanic, Black/Hispanic/Other)
Education (Less than High School, High School, Some College, Bachelors and higher)
Census Region (Northeast, Midwest, South, West)
Metropolitan Area (Yes, No)

Power
The following analysis is based on completed cases where all variables of interest are
assumed to be binary variables. Assuming simple random sampling, for a binary variable
with a probability of success of p, with n completed cases, the variance will be np(1-p).
Using the worst case scenario of a probability of success, p, of 0.5 (50%) this produces a
standard error of
0.5∗(1−0.5)/√ =0.5/√n .
With a sample of 1,500 completed cases, the standard error is 0.013. A 95% confidence
interval for the estimated percentage (50%) would be from 47.5% to 52.5% (twice the
standard error). If we were interested in comparing two subgroup percentages (say p1 and
p2) where each subgroup had about the same sample size (750 completed cases), then the
confidence intervals would be wider due to the reduced size. In this case, the half-width of
the 95% confidence interval for each of the estimates would be about 0.036 rather than the
0.026 for the full sample of size 1,500. For estimating a difference for the subgroup
percentages (d= p1- p2) the standard error is approximately equal to

4

√((p_1 〖(1-p〗_1 )/750+(p_2 〖(1-p〗_2 )/750).
If the estimates for the subgroups are near 50% (the worst case scenario), then the standard
error of the difference is about 3.6 percentage points. This implies that differences in this
situation would have to be somewhat greater than 7 percentage points (twice the standard
error) to be statistically significant.
As noted in the burden chart, we plan to send 5,042 initial survey invitations, which would
result in a sample of 1,765 respondents with an estimated 35% response rate. This should
help ensure that we reach the goal of having at least 1,500 completed cases.
3. Methods to Maximize Response Rates and Deal with Non-response
To help ensure that the participation rate is as high as possible, FDA will:


Send five different contacts to participants.



Send a small cash incentive with the initial request to respond.



Administer the survey in two modes (web and mail), allowing respondents to answer
questions in the mode, time, and location of their choosing;



Provide multiple response modes sequentially.



Design survey that minimizes burden (short in length, clearly written, and with
appealing graphics);

4. Test of Procedures or Methods to be Undertaken
We will conduct a pilot study to test the data collection process. A sample of 100 addresses
will be drawn in order to obtain an estimated 35 completed responses.
5. Individuals Consulted on Statistical Aspects and Individuals Collecting and/or Analyzing
Data
The contractor Westat, will collect the data on behalf of FDA as a task order under Contract
HHSF223201510001B. Jennifer Berktold, Ph.D., 301-294-3964, is Westat’s Project
Director for this project. Data analysis will be overseen by the Research Team, Office of
Prescription Drug Promotion (OPDP), Office of Medical Policy, CDER, FDA, and
coordinated by Helen W. Sullivan, Ph.D., MPH, 301-796-4188, and Kathryn Aikin, Ph.D.,
301-796-0569.
References
1. Blumberg, S.J., and J.V. Luke, Wireless Substitution: Early Release of Estimates from the
National Health Interview Survey, July-December 2013. National Center for Health
Statistics, (2014). (http://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201407.pdf)

5

2. Link, M., M.P. Battaglia, M.R. Frankel, L. Osborn, and A.H. Mokdad, “Reaching the Cell
Phone Generation” Public Opinion Quarterly 71(5), 814–839 (2007).
3. Qayad, M.G., C. Pierannunzi, P.P. Chowdhury, S. Hu, D.M. Town, and L.S. Balluz,
“Landline and Cell Phone Response Measures in Behavioral Risk Factor Surveillance
System” Survey Practice 6(3) (2013).
4. Dillman, D. A., J. D. Smyth, and L. M. Christian, Internet, Phone, Mail, and Mixed-Mode
Surveys: The Tailored Design Method, 4th ed. Hoboken, NJ: John Wiley & Sons, Inc.
(2014).
5. Hicks, W., and D. Cantor, “Evaluating Methods to Select a Respondent for a General
Population Mail Survey,” Paper presented at the AAPOR Conference, Orlando, FL (2012,
May).
6. Medway, R.L. and J. Fulton, “When More Gets You Less: A Meta-Analysis of the Effect
of Concurrent Web Options on Mail Survey Response Rates” Public Opinion Quarterly 76
(4), 733-746 (2012).

6