Download:
pdf |
pdfAttachment G
Question & Answer Brochure
Answers
If you have more questions
about NSDUH, please call
1-800-848-4079
or visit our Web site at
http://nsduhweb.rti.org
For more information on SAMHSA or
RTI International, contact:
NSDUH National Study Director
SAMHSA
1 Choke Cherry Road
Room 2-1113
Rockville, MD 20857
www.samhsa.gov
to your questions
The National Survey on Drug Use and Health (NSDUH) provides up-todate information on alcohol, tobacco, and drug use, mental health and
other health-related issues in the United States. NSDUH is directed by the
Substance Abuse and Mental Health Services Administration (SAMHSA),
part of the U.S. Department of Health and Human Services (DHHS).
The study is being conducted by RTI International, a nonprofit research
organization.
National Survey
on Drug Use
and Health
NSDUH National Field Director
RTI International
3040 Cornwallis Road
Research Triangle Park, NC 27709
www.rti.org
RTI International is a trade name of Research Triangle Institute.
What Is the National Survey on
Drug Use and Health?
Sponsored by the U.S. Department of Health and Human
Services and the Substance Abuse and Mental Health Services
Administration. Conducted by RTI International.
NSDUH began in 1971 and is conducted every year. This year almost
70,000 people from across the United States will be interviewed for this
important study.
Information from NSDUH is used to support prevention and treatment
programs, monitor substance use trends, estimate the need for treatment
facilities and assist with the creation of government policy.
Answers to Your Important Questions about the National Survey on Drug Use and Health
Why Should I Participate?
You are important! Your household was one
of only a few in this area selected for this
study, and no other household or person
can take your place.
Every person who is chosen and completes
the full interview will receive $30 in cash at
the end of the interview in appreciation for
their help.
If chosen for an interview, you will represent
the residents of your community and help us
gather important information that is needed
to make sound policy decisions.
Your participation also provides vital
information to researchers and local, state
and federal agencies to design education,
treatment and prevention programs and
receive funding to support these efforts.
What if I Do Not Smoke, Drink
or Use Drugs?
In order to know the percentage of people
who smoke, drink or use drugs, we also
need to know how many people do not.
The responses of people who do not use
these substances are just as important as the
responses of people who do.
All information collected for this study will be
kept confidential and used only for statistical
purposes, as required by federal law – the
Confidential Information Protection and
Statistical Efficiency Act of 2002 (CIPSEA).
While some questions ask about drug
knowledge and experience, other questions
ask about a number of health-related topics
relevant for all people. You do not need to
know anything about drugs to answer the
questions.
What Is the Substance Abuse
and Mental Health Services
Administration?
How Was I Chosen?
Household addresses, not specific people,
are randomly selected through scientific
methods. Once a household has been
selected, it cannot be replaced for any
reason. This assures that NSDUH accurately
represents the many different types of people
in the United States.
A professional RTI interviewer will visit your
household to ask several general questions
that only take a few minutes to answer.
Afterwards, one or possibly two members of
your household may be asked to complete
the full interview. It is possible that no one
in your household will be chosen for the
interview.
Your household has been chosen at random, but no one
else can take your place. Your participation matters!
What Will Happen During the
Interview?
An interviewer will conduct the interview
with each selected person using a laptop
computer. No prior computer skills are
necessary.
Participants will answer most of the
interview questions in private, entering their
responses directly into the computer. For
other questions, the interviewer will read the
questions aloud and enter the participant’s
responses into the computer.
The interview takes about one hour to
complete. Persons who complete the full
interview will receive $30 at the end of the
interview as a token of our appreciation.
SAMHSA is an agency in the U.S. Department
of Health and Human Services (DHHS).
SAMHSA was created to improve the lives
of people with or at risk for mental and
substance use disorders.
NSDUH is used to help this mission by
gathering data on substance use, problems
related to substance use, and mental health
problems in the United States. The numbers
of people who use various substances, or
have problems related to substance use or
mental health, are important for planning
treatment and prevention services.
SAMHSA selects a qualified survey research
organization to administer NSDUH.
RTI International, a nonprofit research
organization, is under contract with SAMHSA
to conduct NSDUH.
Attachment O
Highlights and Newspaper
Articles
SELECTED HIGHLIGHTS from the
2011 National Survey on Drug Use and Health
Past Month Cigarette Use among Persons Aged 12 or Older,
by Age: 2011
Tobacco Use
•
An estimated 68.2 million Americans
reported current use (during the past
month) of a tobacco product in 2011,
which is 26.5 percent of the population
aged 12 and older. About 56.8 million
(22.1 percent) smoked cigarettes.
•
The graph to the right illustrates past
month cigarette use among persons
age 12 or older.
Alcohol Use
Current Alcohol Use among Persons Aged12-20,
by Age: 2002-2011
•
Slightly more than half of all
Americans age 12 or older, 51.8
percent or an estimated 133.4 million
persons, were current drinkers in the
2011 survey, which is similar to the
131.3 million persons (51.8 percent)
reported in 2010.
•
Although consumption of alcoholic
beverages is illegal for those under
21 years of age, 25.1 percent of this
age group (9.7 million) were current
drinkers in 2011. The graph on the
left displays the current use of
alcohol for 12-20 year olds from
2002 through 2011.
Illicit Drug Use
•
An estimated 22.5 million Americans were current users of illicit drugs in 2011, meaning
they used an illicit drug at least once during the 30 days prior to the interview. This
estimate represents 8.7 percent of the population 12 years old or older.
•
Marijuana is the most commonly used illicit drug, with an estimated 18.1 million current
users, or 7.0 percent of the population 12 years old or older, similar to the 2010 rate of 6.9
percent. Similar to 2010, an estimated 1.4 million persons were current users of cocaine. In
2011, an estimated 6.1 million (2.4 percent) used prescription-type psychotherapeutic drugs
nonmedically in the past month which is lower than the 2010 estimate of 2.7 percent.
Results from the 2011 National Survey on Drug Use and Health: Summary of National Findings,
DHHS/SAMHSA/CBHSQ, September 2012
Mental Health
•
In 2011, an estimated 15.2 million adults, or 6.6 percent of the population aged 18 or older,
had at least one major depressive episode (MDE) in the past 12 months. Among adults, the
percentage having MDE in the past year varied by age and gender, as shown in the graph below.
Major Depressive Episode in the Past Year among Adults Aged 18 or Older,
by Age and Gender: 2011
Substance Dependence or Abuse among Adults Aged 18 or Older,
by Major Depressive Episode in the Past Year: 2011
•
Persons with past year MDE
were more likely than those
without MDE to have used
an illicit drug in the past
year (28.5 vs. 13.4 percent).
•
Similarly, substance
dependence or abuse was
more prevalent among
persons with MDE than
among those without MDE
(20.0 vs. 7.3 percent), as
shown in the graph to the
right.
Results from the 2011 National Survey on Drug Use and Health: Mental Health Findings,
DHHS/SAMHSA/CBHSQ, 2012
Attachment S
Contact Materials Report
2009 NATIONAL SURVEY ON
DRUG USE AND HEALTH
REPORT ON THE CONTACT
MATERIALS FOCUS GROUP
FINDINGS AND
RECOMMENDATIONS
Contract No. 283-2004-00022
RTI Project No. 0209009
Authors:
Project Director: Thomas G. Virag
Doug Currivan
Emilia Peytcheva
Sonia Rodriguez
Stephanie Stolzenberg
Jim Leiman (Morpace, Inc.)
Arturo Obscura (Morpace, Inc.)
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
November 2009
2009 NATIONAL SURVEY
ON DRUG USE AND HEALTH
REPORT ON THE CONTACT
MATERIALS FOCUS GROUP
FINDINGS AND
RECOMMENDATIONS
Contract No. 283-2004-00022
RTI Project No. 0209009
Authors:
Project Director: Thomas G. Virag
Doug Currivan
Emilia Peytcheva
Sonia Rodriguez
Stephanie Stolzenberg
Jim Leiman (Morpace, Inc.)
Arturo Obscura (Morpace, Inc.)
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
November 2009
Table of Contents
Chapter
Page
1.
Introduction ..............................................................................................................................1
2.
Design and Methods ................................................................................................................3
2.1.
2.2.
2.3.
2.4.
3.
Results ......................................................................................................................................9
3.1.
3.1.1.
3.1.2.
3.2.
3.3.
3.4.
3.5.
3.6.
4.
Lead Letter Envelope ...................................................................................................... 9
Considerations in Deciding Whether to Open a Piece of Mail ................................... 9
Reactions to the Two NSDUH Envelopes ................................................................ 10
Lead Letter Text ............................................................................................................ 11
Lead Letter Graphics..................................................................................................... 14
Question and Answer Brochure .................................................................................... 15
Special Concerns of Spanish-speaking Participants ..................................................... 17
Additional Suggestions ................................................................................................. 18
Summary and Recommendations ..........................................................................................22
4.1.
4.2.
4.3.
4.4.
5.
Purpose and Goals of Focus Groups ............................................................................... 3
Focus Group Protocol and Procedures ............................................................................ 4
Recruitment of Participants............................................................................................. 5
Characteristics of Participants......................................................................................... 6
Lead Letter Envelopes .................................................................................................. 22
Lead Letter Text ............................................................................................................ 23
Lead Letter Graphics..................................................................................................... 23
Q&A Brochure .............................................................................................................. 24
References ..............................................................................................................................26
i
List of Appendices
Appendix
Page
A
Focus Group Moderator’s Guide ................................................................................ A-1
B
Participant Informed Consent Form............................................................................ B-1
C
Incentive Payment Receipt ......................................................................................... C-1
D
Sample Focus Group Recruitment Advertisement ..................................................... D-1
E
Focus Group Recruitment and Screening Protocol ......................................................E-1
F
Contact Materials – Lead letter envelope, regular size ................................................ F-1
G
Contact Materials – Lead letter envelope, 9x12 ......................................................... G-1
H
Contact Materials – Lead letter, Version 1a text only ................................................ H-1
I
Contact Materials – Lead letter, Version 2a text only .................................................. I-1
J
Contact Materials – Lead letter, Version 3a text only ................................................. J-1
K
Contact Materials – Lead letters, Version 1b graphics only ....................................... K-1
L
Contact Materials – Lead letters, Version 2b graphics only ........................................L-1
M
Contact Materials – Lead letters, Version 3b graphics only ...................................... M-1
N
Contact Materials – Q & A brochure, Version 1 ........................................................ N-1
O
Contact Materials – Q & A brochure, Version 2 ........................................................ O-1
ii
List of Tables
Table
Page
Table 1.
Characteristics of English Focus Group Participants.................................................. 7
Table 2.
Characteristics of Spanish Focus Group Participants ................................................. 8
Table 3.
Preference Counts for Each Type of Contact Material ............................................. 19
iii
This page intentionally left blank.
iv
1. Introduction
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance
Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S.
civilian non-institutionalized population aged 12 and older. The conduct of NSDUH is
paramount in meeting a critical objective of SAMHSA’s mission to maintain current data on the
prevalence of substance use in the United States.
Like other surveys, the potential for nonresponse bias is an ongoing concern.
Nonresponse bias is a function of the nonresponse rate and differences between respondents and
nonrespondents on key survey variables. On NSDUH, the key survey items are substance use
prevalence rates. Historically, interview response rates have been lower among some
demographic subgroups, such as those 50 and older (50+) (Murphy, Eyerman, and Kennet,
2004), and to a lesser extent those living in urban areas and males (Table B.5 in the 2006
Summary of Findings). Recent NSDUH data indicate sample members who are more likely to be
underrepresented in the interview data:
Men are somewhat less likely than women to participate.
Adults over 50 years old, and especially those 65 and older, are somewhat less likely than
adults under 50 to participate.
Respondents who are categorized in the Other race subgroup and White respondents who
are not Hispanic are somewhat less likely to participate than respondents who are
Hispanic or non-Hispanic Black.
Lower response rates among subgroups may increase bias in overall estimates of prevalence
rates. Increasing response rates across all groups would help minimize bias.
One way to ensure high participation across subgroups in the population would be to
improve any aspects of the survey protocol that could directly influence the decision to
participate. In the spring of 2005, RTI carried out 12 focus groups with potential respondents to
examine the issue of nonresponse among persons 50 and over (Murphy, Schwerin, Hewitt, and
Safir, 2005). This study was designed to explore how persons age 50 and over perceived the
NSDUH interview request. Subjects were shown a video of an NSDUH interviewer contacting a
sample member, enlisting cooperation to complete the household screening, and then enlisting
cooperation for a selected household member to conduct the interview. Participants were also
provided with copies of the materials given to respondents, including the lead letter, question and
answer (Q&A) brochure, and other materials. Focus group members were then asked questions
about the screening and interview process, as well as the contact materials being used during
these processes. One set of recommendations from this study focused on examining how well the
NSDUH contact materials:
establish the legitimacy of the sponsoring and research organizations,
clearly convey the survey objectives and importance of participation, and
describe the selection process and importance of the selected individual’s participation.
Conveying this information to sample members is clearly dependent on the screening
respondent actually seeing the materials. The results of a cursory analysis of whether the
1
screening respondent recalled seeing the lead letter in the 2001 NSDUH 1 indicated that
approximately 48 percent of respondents aged 50+, 48 percent of respondents aged 35 to 49, 45
percent of respondents aged 26 to 34, and 35 percent of respondents aged 18 to 25 recalled
seeing the lead letter.
The investigation described in this report was designed to utilize feedback from focus
groups to identify ways to improve the NSDUH contact materials to maximize cooperation
among sample members. To this end, SAMHSA and RTI undertook two specific tasks. First,
alternative versions of the lead letter envelopes, lead letters, and question and answer (Q&A)
brochure were developed based on survey design principles usually associated with the decision
to participate (Groves, Cialdini, and Couper, 1992).The final versions of the lead letters and
Q&A brochures were also translated into Spanish. 2 Second, focus groups with participants from
different parts of the United States were conducted to evaluate how members of the target
population would react to the different versions of the contact materials. This report describes the
procedures followed and results obtained from 17 focus groups involving both English and
Spanish-speaking adults across the U.S. In addition to describing the key themes emerging from
these focus group discussions, this report presents general recommendations for revising the
contact materials based on the findings.
1
The NSDUH was titled the “National Household Survey on Drug Abuse” (NHSDA) prior to 2002.
For the focus groups, the return address or other text on the lead letter envelopes was not translated into Spanish.
NSDUH lead letters that include the Spanish translation of the letter on the reverse side are currently mailed in the
same envelopes as English-only letters.
2
2
2.Design and Methods
2.1.
Purpose and Goals of Focus Groups
Given that the purpose of the overall investigation was to improve the design of the
NSDUH contact materials in ways likely to maximize cooperation rates, a key objective of this
study was to evaluate alternative versions of the materials through focus group discussions. The
contact materials developed for the focus groups included multiple versions of the lead letter
envelope, the lead letter text without graphics, the lead letter graphics without text, and the Q&A
brochure. The revised contact materials resulting from this study will likely be used for the
upcoming NSDUH redesign, which is currently slated to begin in 2013.
The feedback provided by focus groups will assist in identifying which of the lead letter
envelope, lead letter, and Q&A brochure designs might maximize positive responses to requests
to complete the screener and interview among households selected for NSDUH. As Fowler
(1995) notes, focus groups can quickly broaden researchers’ perspective on how people think
about the issues under study. As such, the focus groups conducted for the NSDUH contact
materials can help in two ways:
1. To examine assumptions about the likely reactions members of the target population
would have toward the materials and participation in the survey and
2. To evaluate assumptions about how members of the target population understand the
key concepts and specific terms used to describe the survey.
The focus group sessions primarily provided participants’ responses to a set of structured
questions and probes, but also allowed participants to add spontaneous comments on the
materials. The qualitative information derived from the focus groups included both overall
reactions to the contact materials as well as specific feedback on individual elements. We also
planned to count participants’ preferences for specific versions of each type of contact material,
to provide some quantitative context for the reactions and feedback.
Another important goal of the focus groups was to involve members of the target
population who represented different regions of the country and different languages spoken. For
this reason, a total of 17 focus groups were conducted across five metropolitan areas – Chicago,
Dallas-Fort Worth, Los Angeles, Raleigh-Durham, and Washington, D.C. To ensure
representation of U.S. residents who speak primarily English and those who speak primarily
Spanish, 11 of the focus groups were conducted in English and six in Spanish.
A final objective in conducting the focus groups was to ensure the groups were conducted
by experienced professionals who did not have a direct stake in the specific study materials or
protocol. This facilitated a more objective treatment of the issues and ensured that detailed
knowledge of the NSDUH did not unduly influence the direction or content of the focus group
discussions. RTI hired professional focus group moderators from Morpace, Inc. to conduct both
the English and Spanish focus groups. In addition to having extensive experience in conducting
focus groups, the Morpace moderators also had experience and training in survey research
methods.
3
2.2.
Focus Group Protocol and Procedures
The focus group protocol guide was developed by RTI and SAMHSA, translated into
Spanish by RTI, and distributed to the Morpace moderators to study. The protocol is included
with this report as Appendix A. The protocol consisted of six major sections:
1. informed consent,
2. introduction,
3. lead letter envelope discussion,
4. lead letter discussion,
5. Q&A brochure discussion, and
6. concluding remarks.
The informed consent part of the protocol took about five minutes and provided an
overview of the study and specifics about the participants’ rights as study volunteers. The
participant informed consent form is included with this report as Appendix B. The introduction
also lasted about five minutes and was intended to set up the discussion rules and acquaint the
participants in each group. The discussion of the lead letter envelopes was designed to take about
15 minutes and involved discussions of (1) how respondents screen their mail and (2) how they
perceive alternative versions of the envelope. About 45 minutes was allotted for the discussion of
the lead letter and intended to cover both alternative versions of the letter text and graphics. The
section on the Q&A brochure was planned for about 30 minutes and was aimed at comparing the
current brochure (version 1) with a redesigned version (version 2). The discussion of the Q&A
brochure involved participants’ preferences in terms of the content, visuals, and design. At the
completion of the discussion of each type of contact material, the moderator encouraged
participants to indicate which version they preferred. The concluding section of the protocol was
intended to give both participants and observers a final opportunity to ask questions or make
comments. At the end of each focus group session, each participant received $75 in cash and
signed a receipt for the payment. The incentive receipt form is included with this report as
Appendix C.
Both the English and Spanish focus groups followed the same protocol. Up to 10
participants comprised each focus group, with most sessions including seven to nine people.
Upon entering the room, participants were presented with two copies of the informed consent
form. The moderator briefly explained the consent form and asked participants to read over it,
sign both copies, return one to the moderator, and keep the other for their records. Next, the
moderator introduced some ground rules related to cell phone use, taking breaks, and side
conversations among participants. He also encouraged participants to share their opinions,
especially when they differed from the views expressed by other participants. The moderator
explained that each session was video and audio recorded and briefly introduced himself. He
then went around the room and asked each participant to briefly introduce himself or herself.
After the introduction, the moderator began following the protocol for the discussion of
each type of contact material. The moderator guided participants through the protocol by asking
questions about the material and following up with probes to address specific points that were
not mentioned by participants. As appropriate, the moderator expanded the discussion to build on
participants’ comments and asked others to share their views. A key strategy used by the
4
moderators was to direct questions to participants who had not yet contributed to the current
discussion. This ensured that most participants were heard throughout the session, so that the
impressions gleaned from the groups were not based solely on a few participants who were most
forthcoming.
Following the discussion of each type of contact material, the moderator asked
participants to mark the version of the material they preferred with the number one. Not all
participants chose to select a preferred version for each type of material. Although some
participants expressed mixed feelings about specific features of different versions, in most
groups a majority of participants did indicate a preference for each type of contact material.
2.3.
Recruitment of Participants
RTI recruiters posted advertisements on craigslist.com for the Raleigh-Durham, North
Carolina, Washington, District of Columbia, Dallas, Texas, Chicago, Illinois, and Los Angeles,
California metropolitan areas. These advertisements explicitly noted that in order to be eligible
for participation, the respondent must be age 18 or older, not currently serving on active duty in
the military, not currently employed by RTI International, and not a family member of an RTI
employee. The text for the recruitment advertisement for English focus groups can be found in
Appendix D.
A key consideration for our recruitment efforts was to reach Spanish-only, or mostly
Spanish, speakers who would not be included in the pool of potential participants in the English
focus groups. Our recent experiences on recruiting this population for focus groups and cognitive
interviews indicated that working with local Hispanic/Latino community organizations was an
effective way to tap into this subpopulation. RTI bilingual recruiters partnered with local
community organizations to legitimize the research in these target communities by assisting with
recruiting and providing a facility at their site for hosting the focus groups. Advertisements were
posted in highly concentrated Hispanic areas, such as Hispanic shopping centers, community
centers, and La Raza, a Spanish-speaking newspaper for the Chicago area. Word of mouth was
the most valuable way of getting calls from prospective participants. The local community
centers assisting with these efforts were El Pueblo in Raleigh, North Carolina, and Casa Central
in Chicago, Illinois.
When a prospective respondent called RTI in response to a recruitment ad, they were
screened using the recruitment scripts found in Appendix E. Specifically, all respondents were
screened for age, race, level of education, total household income, distance from the focus group
site, geographic area (urban/suburban/rural), whether or not they spoke English/Spanish as their
native (primary) language, whether they could read English/Spanish, and how well they could
read English/Spanish (very well, well, or not well). Eligible respondents were informed that the
focus group would be audio and video recorded and then provided with the date and time the
group would be conducted. In order to recruit the desired number of participants and encourage a
sufficient number of them to show up to each focus group, an incentive of $75 was offered.
5
2.4.
Characteristics of Participants
RTI used the screener in Appendix E to recruit participants and place eligible participants
in the appropriate session for their age. Recruiters were asked to strive for the greatest possible
variation in age, gender, and other demographic characteristics, to ensure a heterogeneous set of
viewpoints in each group. The goal was to recruit ten participants for each group to ensure at
least seven or eight people would attend each focus group session.
The first round of recruitment began on September, 23, 2009 and concluded on October
16, 2009. This initial round recruited subjects for focus groups conducted on October 12-13 in
Durham, North Carolina, October 20-21 in Raleigh, North Carolina and October 21 in
Washington, District of Columbia. Round two of recruitment included focus groups held on
November 2-3 in Irvine, California, November 4-5 in Addison, Texas, and November 4-5 in
Chicago, Illinois. This recruitment phase ran from October 19, 2009 through October 30, 2009.
Table 1 and Table 2 presents the final demographic composition of all participants across the
seventeen focus groups, including age, gender, race, education, income, and urbanicity for each
participant.
6
Table 1.
Characteristics of English Focus Group Participants
Gender
Location and
Age Group
Total
Raleigh, NC
18-29
30-49
50+
Washington,
DC
18-29*
50+
Addison, TX
18-29
30-49
50+
Irvine, CA
18-29
30-49
50+
Race
Education
Income
Geographic Area
Number
M
F
White
(NonHisp)
96
33
63
56
24
16
11
84
81
14
35
57
3
8
8
10
3
3
4
5
5
6
5
3
7
3
4
3
1
-
1
1
1
7
7
9
8
6
8
2
2
3
3
3
4
3
7
1
2
-
9*
10
4
5
5*
5
4
6
2*
4
3
-
1
2
7
8
6
8
2
2
3
4
5
6
-
7
10
7
2
2
3
5
8
3
2
6
6
2
4
-
3
1
1
1
3
6
9
4
6
9
7
1
1
-
2
4
3
5
6
4
-
9
10
8
1
4
2
9
6
6
4
6
7
1
1
-
4
3
1
-
9
10
8
7
8
8
2
2
-
1
4
5
8
6
3
-
Black
Other
Race
Edu ≤
HS/GED
Edu >
HS/GED
Inc ≤
$100,000
Inc >
$100,000
Urban
Suburban
Rural
* One no show was replaced by an eligible participant on-site. Since this participant was not completely screened prior to arrival, not all demographic characteristics could
be recorded for inclusion in this table.
7
Table 2.
Characteristics of Spanish Focus Group Participants
Gender
Location and
Age Group
Total
Raleigh, NC
18-29
30-49
50+
Chicago, IL
18-29
30-49
50+
Country of Origin
Puerto Central
South
Rico
America America
Education
Income
Inc ≤
Inc >
$100,000 $100,000
Edu ≤
HS/GED
Edu >
HS/GED
14
28
21
49
1
-
2
4
5
4
5
3
1
3
4
2
-
3
6
5
5
3
5
5
Number
M
F
Mexico
49
20
29
24
8
3
5
8
7
2
3
4
3
5
3
2
3
1
1
1
9
10
10
3
2
6
6
8
4
6
8
4
1
2
3
8
Geographic Area
Urban
Suburban
Rural
0
46
2
1
5
8
7
-
5
8
6
-
1
9
10
10
-
8
10
9
1
1
-
3. Results
3.1. Lead Letter Envelope
3.1.1. Considerations in Deciding Whether to Open a Piece of Mail
Before presenting the lead letter envelopes, the moderators asked focus group participants
to think about the mail they receive each day and identify factors related to their likelihood of
opening a specific piece of mail. This discussion was intended to provide background
information on how people generally perceive the different types of mailings they receive and
how this might affect the likelihood of opening the NSDUH lead letter envelope. The moderators
probed participants by asking the following questions:
What kinds of things do you consider in deciding whether to open a piece of mail?
What kinds of things do you look for on an envelope to determine whether you will
open it?
What kinds of envelopes do you tend to throw out or recycle without opening?
Most participants indicated that their households receive a great amount of mail and that
they spend time sorting and separating the important pieces from what they considered to be
“garbage” or “junk mail.” Some participants indicated that they open all the mail they receive,
often looking for interesting promotions or coupons, but others indicated that they regularly
throw out at least some mail unopened. In some of their households, participants noted that there
is one individual who does most of the mail sorting. In these cases, the rest of the household
members would only receive pieces of mail that the sorter considers to be important mail.
Focus group participants felt the decision to keep a piece of mail and open it is usually
based on what they know about the sender and the characteristics of the envelope. Participants
indicated they usually try to determine whether there is some kind of “connection” between
themselves and the senders of the mail. They typically assess this by examining both the return
address and to whom the mail is addressed. For example, they would be more likely to open a
piece of mail from a company with whom they already have a commercial relationship.
Participants also stated they would be less likely to open mail addressed simply to “Resident,”
and some indicated they routinely throw out mail addressed in this way. Some participants in
Spanish groups who live in apartment buildings mentioned that they never open mail addressed
only to “Resident” because they think this mail was sent to the landlord. This can be a problem
in cases where the mail of more than one family is received in the same mailbox and people do
not want to open their neighbor’s mail by mistake.
For most people, simply adding some version of “Resident of ___________ County” to
the address would not significantly diminish their inclination to ignore mail addressed in this
way. Some participants in Spanish groups and some participants age 50+ in English groups
thought that personalizing the mailing address in this way would make them more likely to open
the mail. Those who preferred adding “Resident of ___________ County” mentioned their
likelihood of opening the envelope would increase because addressing the envelope this way
would indicate to them that the mail contains something important for them and their local
community.
9
In terms of the physical characteristics of the envelopes, participants stated they usually
associate pieces of mail that use bright colors and glossy paper, have messages written on them,
and use bulk mail postage with marketing and promotions. For this reason, they are more likely
to dispose of this mail unopened. Some participants mentioned there are some law firms and
companies that try to make their promotions look more “official” or “important” in order to trick
people into opening them. These participants noted that they identify such efforts with the use of
plain white envelopes with minimal information on the outside about the sender.
Overall, focus group participants indicated the mail they are most likely to open would
have the following characteristics:
mail addressed to a specific person in the household,
mail that includes the name of a company with which the recipient is familiar, and
mail that uses mostly white envelopes with familiar and official-looking logos.
3.1.2. Reactions to the Two NSDUH Envelopes
Following the discussion of what factors influence the decision to open a piece of mail,
the moderators distributed two versions of the lead letter envelope. The first version was a white
standard number 10 size window envelope, printed as shown in Appendix F. The second
envelope was a white 9 x 12 inch catalog envelope, with the return address and address window
arranged in portrait orientation. The image for this larger envelope is exhibited in Appendix G.
Both envelopes included the same agency logo and return address.
Initial Reactions and Likelihood of Opening the Two Versions of the Envelopes
Table 3 on page 18 presents focus group participants’ preferences with respect to each
type of contact material. In the tabulation of participants preferences for the lead letter envelopes,
slightly more than half of the English group participants preferred the larger envelope. Twothirds of the Spanish group participants preferred the larger envelope.
Overall, initial reactions to the envelopes were very positive. Based on their appearance,
most participants indicated they would likely open either the standard size or the larger size
version of the envelope if they received it in their mail. A key factor stated almost universally by
participants in the English language groups was that they would open the envelopes because of
the Department of Health and Human Services (DHHS) return address. People emphasized that
the envelope should look as “business-like” and “official” as possible, and this is generally
accomplished by using the DHHS logo and title. For this reason, most participants stated they
would open the envelope, regardless of the size and the use of the generic addressee of
“Resident.” Because the mailing would be coming from the DHHS, many participants suggested
that there could be important information concerning recent developments in health care. The
Rockville, Maryland return address did not seem to bother participants in any of the cities.
Inclusion of the RTI project number also did not seem to bother anyone, but it was mentioned by
a few that this information did suggest that the mailing might be in reference to a survey.
In the Spanish groups, most participants were not familiar with the DHHS, but did
indicate that they would still open the envelopes because they look “official” and different from
marketing materials. The white envelope and the logo with an eagle in it clearly communicated
to these participants that the mail came from the U.S. government. Similar to the English
language groups, the Rockville, Maryland return address reinforced the idea that the mail is
10
legitimate because people recognized that many government offices are located in that part of the
country.
Specific Reactions to the Different Sizes of the Envelope
Reactions from participants suggested that the larger envelope would certainly garner
more initial attention than the standard size envelope. Across the groups, people suggested that
the larger envelopes are often used to send legal documents, so they look more “official.”
Among the Spanish group participants the larger envelope communicated that the documents
inside are important and cannot be folded. Some of the 18 to 29 year old participants in the
Durham English group preferred the smaller envelope because they felt it looked “more
business-like.” Also most participants in the 50+ Spanish group in Chicago felt there was no
need to use a large envelope, perhaps because it would be a waste of resources.
In some of the English language groups, after discussing the letters the moderator
returned to the question of which of the two envelopes should be used. Once participants realized
that both envelopes would contain only a single sheet of paper, some reversed their earlier
preference for the larger envelope suggesting that it would be a waste of paper and postage.
Some of these people were motivated by a concern for the environment. Others were motivated
by a concern with government spending.
Use of the “Official Business” Endorsement on the Envelopes
The text on the envelopes that reads “OFFICIAL BUSINESS. PENALTY FOR
PRIVATE USE $300” tended to enforce the official nature of the mailing for focus group
participants. Despite this view, the great majority of participants did not understand exactly what
this statement actually means. In the Spanish groups, some people suggested that this statement
means that the envelope must only be opened by the person to whom it is addressed and no one
else. This perception could be problematic, given that the mail is addressed to “Resident” and in
some cases the household receives mail in a depository shared with other households. There
were also a few Spanish group members who felt it was intimidating to have the penalty note on
the envelope.
3.2. Lead Letter Text
Three versions of the lead letter with text only and no graphics were presented to
participants and labeled as versions 1a, 2a, and 3a. These versions varied in how or whether
various aspects of the survey request and protocol were mentioned and the emphasis given to
each element. For example, version 3a provided the study name in the first paragraph, while
versions 1a and 2a did not. Versions 1a and 2a provided the URL for the NSDUH website.
Version 3a of the lead letter text was based on a letter used by the National Survey of Family
Growth and only included a signature from SAMHSA staff (with the RTI signature omitted).
This version thereby attempted to develop a more “personal” approach and appeal to recipients.
The three versions of the letters with text only are presented in numerical order as Appendices H,
I, and J.
Initial Reactions to the Three Alternative Versions of the Letter Text
As Table 3 indicates, focus group participants varied somewhat in their preferences for
each version of the lead letter text. Slightly more than half of the English group participants
preferred version 1a, but Spanish group participant preferences were even more mixed. A
11
plurality of about 43 percent of Spanish group participants preferred version 3a and another 35
percent preferred version 1a. Version 2a was the least popular among both English and Spanish
group participants. Versions 1a and 3a were viewed by participants as being better organized,
shorter, and more direct than version 2a. Participants also felt both of these versions contained
most of the information participants wanted to know. In addition, the way the text was
distributed on the page made versions 1a and 3a easier to read, understand, and communicate to
other members of the household.
One of the main differences between version 1a and version 3a was that the latter
mentions the specific topic of the survey (drug use, alcohol, and tobacco). Many participants felt
that being specific about the topic would increase their interest in the survey and would make
them more likely to participate. In addition, knowing the topic would also prevent surprises at
the time of the interview when the questions start asking about sensitive subjects. Spanish group
participants, who preferred version 3a, suggested that mentioning in the letter (as in the
brochure) that both users and non-users of drugs and alcohol needed to participate would be a
good idea. However, there were strong differences among the Spanish group participants in
terms of reading skills and ability to understand the text of the letters. Only the more educated
participants with a college education were able to talk about differences that were too subtle for
the rest. This may be one reason why the two shorter versions of the letter were preferred.
Version 1a was particularly strong among the younger participants who believed that the
text in the letter was shorter, more professional, and straight to the point. Version 3a was
particularly popular among the older participants in the English language groups, and nearly half
of the Spanish group participants. One of the reasons cited was the more personal tone of the
letter. However, many of the younger and middle aged participants in the English language
groups really disliked this tone particularly mentioning the introduction, “My agency…”
Other Specific Features of the Letter Text
One feature used only in version 2a viewed favorably by participants was how the
“Members of the Household” and “Resident of the [city, county, or state]” made the letter more
seem more personal. These participants were particularly thinking about situations when more
than one person in the household would read the letter.
Participants were asked to compare how the three letters handled the issue of survey
confidentiality. A large number of the participants in the English language groups preferred the
single sentence used in version 3a as it was short and to the point. Some participants pointed out
that the mention of the random selection of the address, not the particular person (version 2a)
further enhanced confidentiality. Furthermore, they liked the fact that the sentence was its own
paragraph that made the topic “stand out.” There was no discussion about the small font text at
the bottom of the letter that further addressed confidentiality.
Participants from the Spanish groups did not notice big differences across letters in terms
of information about confidentiality. To them, the statements in the three versions were sufficient
to make the reader comfortable about participating in the survey. Stating that the confidentiality
of the responses will be protected by federal law was one of the key elements for the Spanish
group participants. Only a few noticed the small font text at the bottom of the page addressing
confidentiality. Once it was pointed out, however, most agreed that it was a positive element and
should be included in the letter.
There were mixed responses to the handling of signatures. Some suggested that two
signatures made the letter appear more official and important; others said they were not very
12
interested in how the government was conducting the survey and a single signature would
suffice. However, if a single signature would be used, it should be the one from DHHS rather
than RTI.
There were mixed reactions to the concept in version 2a of using other organizations to
endorse the survey. The 30 to 49 year old English participants in Durham and Dallas said an
endorsement by the American Medical Association (AMA) might help give some credibility to
the survey but an endorsement by the American Association of Retired Persons (AARP) would
not carry much weight. Adults in the 18 to 29 year old group in Durham did not seem to think
endorsements would add much. The youngest group of adults in D.C. suggested that maybe an
endorsement by the Centers for Disease Control (CDC) might enhance the appeal. In California
there was little support for the concept of using endorsements. There, it was emphasized that the
merits of the study itself should be enough to convince people to participate. In Dallas it was
suggested that the use of endorsements from other organizations carried some risk since some
organizations could be perceived as detractors and this might cause less favorable disposition
towards survey participation. Some of the frequently mentioned organizations were the
American Health Association, the Red Cross, and the March of Dimes.
Organization like AMA and AARP had a low level of awareness among the Spanish
group participants, and their endorsement would not have a strong effect on people’s willingness
to participate in the survey. At the same time, it was suggested that mentioning the endorsement
of a health institution would be more beneficial than endorsements from other types of
institutions. Further, mentioning that the survey was required by the U.S. Congress reinforced
that the survey was official, serious, and relevant, and some of the Spanish group participants
said it would make them feel more obligated to participate. Most of the participants said that
mentioning of the specific government code enabling the survey in version 2a was not needed,
and if anything, might be intimidating.
Nearly everyone in all the English language groups said they would use the internet to
access the RTI website indicated in the first two versions of the letters. Few said they would call
the toll-free number, but only after having visited the website first, and only if they had
additional questions about the survey. The lack of a web site address in version 3a was often
cited by participants as a reason why they did not choose that version of the letter.
Spanish group participants also believed that having a web site address was a very
positive element in the text. Most said that they would immediately go to the web to find more
information and make sure that the survey was legitimate. However, for Spanish group
participants, having a telephone number was also important because there were many
participants who did not have access to the internet, or they did not feel comfortable using
computers.
In general, participants were not familiar with RTI. The three versions of the letter did a
good job explaining the role of RTI – most participants understood that it was the institution that
would conduct the interviews, and they thought it was good to mention that RTI is a non-profit
organization. However, there was little interest in including more details about RTI.
The bolding and highlighting of the incentive sentence caught the attention of the readers.
It did not seem to have a negative connotation and participants admitted that this line would
increase their interest in the letter, and because of it, they would be more likely to pay attention
and read the complete text more carefully. The fact that this information was not bolded in
Version 3 was often cited as a reason for not choosing that version of the letter.
13
After reading the texts, most participants had a good understanding of the ideas
communicated in the letters. The language was appropriate and most of the information people
wanted to know was included. However, some participants believed the letter should include
more specific information about the interviewer’s visit – specifically, when the interviewer will
call on their house, and who in their household they would want to talk to. Spanish group
participants from the two younger groups (18-29 and 30-49 years old) in Chicago expressed
interest in being able to determine whether they would be eligible to participate in the survey by
including eligibility age range in the letter.
3.3. Lead Letter Graphics
In addition to text, the graphics for the lead letters also included three alternative
versions, labeled 1b, 2b, and 3b. These versions varied in the size, format, and content of the
graphics used to “package” the letters. For example, the image on the example identification
badge varied from a dark gray silhouette (versions 1b and 3b) to an actual photo of a person,
printed in color (version 2b). Participants were asked to comment on the graphical layout of the
letter independent of its content (this was accomplished by providing letters with graphics, but
no text). The three versions of the letter graphics are exhibited in numerical order as Appendices
K, L, and M.
Initial Reactions to the Three Alternative Versions of the Letter Graphics
As Table 3 shows, focus group participants indicated mixed preferences for which
version of the lead letter graphics they preferred. None of the versions garnered majority
approval in either the English or Spanish groups, but half of participants in the Spanish groups
preferred version 2b. In contrast, about 45 percent of English group participants preferred
version 1b.
Participants immediately noticed the differences in the headers and sizes of the logos.
They believed the logo was important because it reinforced the official nature of the letter.
Adding Rockville, Maryland and a zip code in the header was also a positive element and helped
to make the letters appear more “business-like” and official. Some participants liked the larger
DHHS logo in version 1b as it allowed them to read it and that was why they preferred that
version over the others.
Other Specific Features of the Letter Graphics
Across the groups there was a difference in opinion regarding the line in the header of
version 1a, “An Important Request from The U.S. Department of Health & Human Services.”
Some participants liked how it emphasized the importance of the survey while others suggested
that it made the letter look “less business-like.” In the Irvine groups it was suggested that the
importance of the survey should be obvious without having to explicitly state it.
Overall, greater support for the use of this tagline in version 1a was evidenced in the 50+
age groups than in the younger-aged groups. Among the 50+ year olds in the Addison groups the
appeal of this tag line was consistent with their preference for the text in letter version 3a, which
begins with, “My agency, …, needs your help.” In Irvine, however, the older adults did not like
the header on version 1b saying that it looked less professional than the other versions. They also
suggested that it was best if the recruitment materials did not make DHHS sound “too
desperate.” Some Addison group participants similarly indicated that the materials should not
appear to be using “hard sell” approaches.
14
Among the Spanish group participants, the tagline did not have any negative effect, but it
was not considered a significant positive element. However, it is important to mention that a
couple of younger participants (who could also speak some English) thought that the word
“solicitud” was usually associated with an application they had to fill out and suggested using the
word “petición” instead.
There was a consistent response to the image of the identification badge using an actual
photograph, as in version 2b. Nearly all participants said their first impression was that this was
the individual who would “show up” at their home. When informed otherwise, nearly everyone
said if the badge had some indication that it was being used for illustrative purposes only, like
having “sample” written across it, they would still prefer to see an actual face on the badge.
Some respondents also thought it was too much to show Ilona Johnson’s signature on the
identification badge and that her signature on the letters (in versions 1b and 2b) was sufficient.
Participant feedback indicated that the way the identification badge is identified must be clear
and somehow visual for people who do not read English. There were a couple of instances where
Spanish group respondents thought that the person in the photo was Ilona Johnson because they
read the name in text on the badge, but they did not understand the words around it.
After the discussion about the badge, most participants agreed that having the hand
written name of the interviewer that would visit their home was a good idea. However, some
Spanish group participants were skeptical about the ability of RTI to actually send the person
who signed the letter.
3.4. Question and Answer Brochure
Focus group participants were shown two versions of the question and answer (Q&A)
brochure. Version 1 was the current Q&A brochure, updated to reflect planned for the NSDUH
redesign. Version 2 was an updated version which used a variety of background colors and
photographs. The brochures included some similar questions and answers, but also had
significant differences in both content and format. The two versions of the Q&A brochures are
displayed in numerical order as Appendices N and O.
Initial Reactions to the Two Alternative Versions of the Q&A Brochure
Overall, version 2 was preferred by participants over version 1. These preferences
differed somewhat between the English and Spanish group participants. Over 80 percent of
participants in the Spanish groups preferred version 2 and 51 percent of English group
participants preferred version 2. Participants in the English groups (24 percent) were also much
more likely to decline to indicate a preference between the two versions compared to those in the
Spanish groups (0 percent).
Most participants indicated that version 2 was more appealing and something that most
people would find more inviting to read. Older participants remarked that a slightly larger type
font made this version easier to read. Participants generally felt that the photos included in
version 2 of the brochure do a good job communicating that different types of people of different
ages, occupations, and walks of life are participating in the survey. This feature led people to
note that version 2 appeared more friendly and personal. Participants also liked the colors used
for this brochure.
The only image that people had some trouble with was the map. Participants weren’t sure
why this image was included and what the different shades of blue and colors signified. Some
15
people believed that the map was communicating the fact that NSDUH is a national survey, but
no one associated this image with the selection of sample units.
Version 1 of the brochure was preferred by a minority of the participants. A negative
comment among 18 to 29 year olds suggested that this version looked like “something from the
80s.” A common complaint among all participants was that version 1 contained too much
information. People suggested this made the brochure somewhat overwhelming and, therefore,
less compelling for them to read. At the same time, a minority of participants in each group
indicated they liked the greater content in version 1. Some of these participants suggested that
this version of the brochure might be more effective for describing the survey to another
household member. These people suggested that dividing the information into more specific
topics in version 1 was a better approach than the way the topics were organized than in version
2.
One section in version 1 of the brochure that was identified as being particularly effective
by some participants was the section with the title “What If I Do Not Smoke, Drink, or Use
Illegal Drugs?” One section that was considered unimportant by most people was “How Does the
Government Conduct the Study?” Words that summarize a number of participants’ reactions that
SAMHSA uses a competitive bid process to select a vendor included “I don’t care.” In each of
the Irvine groups it was pointed out that the way in which version 1 addressed the question of
how the survey data would be used is very good. It was emphasized, particularly in the 50+ age
group, that it is important to tell people how this information will be used.
Overall, participants felt that the information in the brochures is more understandable and
complete, especially compared to the more limited information presented in the letter. The
technical and administrative details of the survey seemed less relevant to participants than
knowing the purpose and utility of the survey. The brochures also do a much better job
explaining that not only drug and alcohol users need to participate in the survey. Some
participants said this will make people more willing to participate, although they also
acknowledged that this might discourage participation among those who have considerable
substance use to report.
Other Specific Features of the Q&A Brochures
When asked if any of the information in either version of the brochure was “confusing,”
participants in the English groups generally indicated they did not have difficulty with most of
the phrases used to describe the survey process, such as “randomly selected” and “chosen at
random through scientific methods.” One concept that some people did have difficulty with was
the juxtaposition of “random” with “scientific” in the phrase “…chosen at random through
scientific methods.” It was suggested by some participants that “random” and “scientific” is a
contradiction. In the Spanish groups, explanations of how participants are selected randomly and
scientifically were not understood by participants with lower education levels. At the same time,
these participants did not express much concern about why they would be asked to participate.
Although much the same information is contained in both versions of the brochures,
participants seemed to have an easier time identifying the listing of relevant web sites in version
1. Since the web site is likely to be used by people to validate information about NSDUH, use of
the format for the web addresses in version 1 of the brochure should be considered.
The mention of computers in the brochure was only mentioned as a concern in the
Durham group of 18 to 29 year olds. This concern was not expressed for them personally, but
instead they suggested some older adults might be concerned about having to use a computer and
16
therefore be less interested in participating. None of the older adults in the English or Spanish
language groups expressed any such concern.
In the Spanish groups some participants mentioned that not using a computer would
make them doubt the legitimacy of the survey because today everybody uses computers for
everything. On the other hand, older participants in the Spanish groups who did not have much
experience working with computers appreciated the information included in the brochures
explaining that knowledge of computers is not necessary. These participants did not quite
understand whether they would actually have to use the computer themselves or whether the
interviewer would enter their responses for them.
Overall, participants felt the brochures do a better job than the letters in explaining the
role of RTI in the study. People noted that the brochures provide more detailed information and
history about both the NSDUH and the institutions involved. Some participants suggested that
this information would cause them to be more favorably disposed towards participating in the
survey.
Participants generally thought it is important to include the logos of SAMHSA and RTI
in the way they are presented in version 2 of the brochure. Even though sample members may or
may not be aware of these institutions, people indicated the logos would make them more
comfortable by reinforcing the importance and legitimacy of the survey.
The 50+ participants in the English groups seemed sincerely motivated to participate in
such a survey if it helped the government’s health planning and related public policy initiatives.
Messages focused in terms of how survey participation would support a worthy endeavor seem
likely to be received favorably by older adults.
3.5. Special Concerns of Spanish-speaking Participants
Overall, Spanish-speaking participants shared many of the same impressions of the
materials as the English group. There are only some special concerns with the Spanish version of
the materials. The most important concern is that the reading level of the letters is only
appropriate for Spanish-speakers with at least a high school education or greater. During the
focus groups strong differences across participants were noticeable in terms of reading skills and
ability to understand the letters text. There was one participant in Chicago who said she did not
understand any of the letters, and others just repeated part of other people's comments. Only the
more educated participants with a college education were able to talk about differences that
seemed too subtle for the rest.
Like English group participants, the text on the envelopes that reads “OFFICIAL
BUSINESS. PENALTY FOR PRIVATE USE $300” was frequently misunderstood by most
Spanish speakers. In addition, many Spanish group participants indicated that they also felt
uncomfortable opening an envelope with this note. Some felt intimidated by it. Participants felt it
meant that the envelope must only be opened by the person that it is addressed to, and no one
else. This was considered to be confusing, particularly if the mail is addressed to “Resident” and
the household receives its mail in a depository with other households. Many participants thought
they would have to pay a fine of $300 if they opened the envelope.
The use of acronyms in Spanish is not as common as in English, and it has always shown
to be problematic for Spanish group participants who are unfamiliar with U.S. government
agencies, associations or U.S. code. Many Spanish-speaking participants pointed out their unease
with acronyms, and recommended avoiding them all together.
17
Explanations of how participants are selected randomly and scientifically were not really
understood by those participants with lower education levels. And the Spanish term ‘al azar’ (at
random) was confusing and many did not understand what that meant.
Spanish group participants said their first impression of the picture on the identification
badge was that it was either the individual who would “show up” at their home or the project
director. When informed otherwise, some participants said they would feel distrustful if the field
interviewer who showed up at their door was not the same person shown in the picture.
Letters addressed to “Resident” made people in the Spanish groups think that it was not
necessarily sent to them. Many thought the letter was addressed to landlords, such as in cases
where people live in rented homes.
3.6. Additional Suggestions
In addition to the discussion of the features of the envelopes, letters, and brochures
already detailed, several further suggestions were identified by focus groups participants:
Participants suggested the use of a regular stamp to make the envelope look more
official and distinguish it from junk mail. This recommendation is in sync with the
Tailored Design Method for survey mailings, proposed by Dillman (Dillman, 2000).
Many participants suggested putting the incentive amount on the envelope or
somehow suggesting that the recipients can get paid for participation in an official
government survey. Such a message would make sample persons more likely to open
the envelope and read the letter.
Generally, participants did not like the fact that the envelope might be addressed to
“Resident”. They preferred “Resident of ____________ County” as this placed them
in a group and suggested the letter contained something that applied to them. Several
suggestions were made to better address this issue. Among the proposed addressees
were “Head of household at [ADDRESS]”, “Randomly selected resident at
[ADDRESS]”, and “Survey to resident at [ADDRESS]”.
In most groups there was some misunderstanding about the statements in letter
versions 2a and 3a regarding the interviewer visit. On first read, many participants
believed the letters stated they would be given a $40 incentive for answering a few
questions (in version 2a) or a five-minute interview (in version 3a). These statements
caused some confusion about the survey protocol. During the discussions, participants
eventually realized the letters were referring to the screening process that would
determine their eligibility to participate in the survey interview.
Several participants suggested mailing the brochure with the initial letter to make the
mailing look more official overall.
18
Table 3.
Preference Counts for Each Type of Contact Material
English Groups
English Totals
Durham,
NC
Washington,
DC
Irvine,
CA
Addison,
TX
Lead letter envelope, regular size
9
9
8
13
Lead letter envelope, 9x12
17
10
14
No preference expressed
0
0
TOTAL
26
Lead letter, Version 1a
Coun
t
Spanish Groups
Spanish Totals
%
Raleigh,
NC
Chicago,
IL
Coun
t
%
39
40.6%
4
6
10
20.4%
10
51
53.1%
15
18
33
67.3%
5
1
6
6.3%
3
3
6
12.2%
19
27
24
96
100.0
%
22
27
49
100.0
%
9
12
18
11
50
52.1%
5
12
17
34.7%
Lead letter, Version 2a
7
3
1
3
14
14.6%
7
4
11
22.4%
Lead letter, Version 3a
8
3
1
8
20
20.8%
8
13
21
42.9%
No preference expressed
2
1
7
2
12
12.5%
0
0
0
0.0%
TOTAL
26
19
27
24
96
100.0
%
20
29
49
100.0
%
Lead letters, Version 1b
12
2
17
13
44
45.8%
10
5*
15
30.0%
Lead letters, Version 2b
8
8
7
6
29
30.2%
9
16
25
50.0%
Lead letters, Version 3b
4
8
2
4
18
18.8%
1
9*
10
20.0%
No preference expressed
2
1
1
1
5
5.2%
0
0
0
0.0%
TOTAL
26
19
27
24
96
100.0
%
20
30
50*
100.0
%
Lead letter envelopes
Lead letter text only
Lead letter graphics only
Q& A brochure
Q & A brochure, Version 1
(current)
Q & A brochure, Version 2 (new)
6
3
5
10
24
25.0%
1
7
8
16.3%
10
16
17
6
49
51.0%
19
22
41
83.7%
No preference expressed
10
0
5
8
23
24.0%
0
0
0
0.0%
TOTAL
26
19
27
24
96
100.0
%
20
29
49
100.0
%
19
* One Spanish group participant selected both versions 1b and 3b
20
This page intentionally left blank.
21
4. Summary and Recommendations
The 17 focus groups conducted in various regions of the U.S. confirmed the importance
of much of the content and formatting elements used in the contact materials, but also
highlighted some key issues that might not be fully addressed in each type of contact materials.
This section synthesizes the focus group reactions into specific recommendations for improving
each type of contact materials. Recommendations accepted by SAMHSA will then be used in
creating a new version of each type of materials for the NSDUH redesign scheduled for 2013.
4.1.
Lead Letter Envelopes
A majority of focus group participants stated a preference for the larger 9x12 envelope.
This majority was greater in the Spanish focus groups. At the same time, participants in the
English groups did express concern about using large envelope for only a letter and no additional
study materials. In addition, the overwhelming majority of participants indicated that they would
open either envelope, primarily because the DHHS logo led them to believe that the mailing was
important. The main benefit of the larger envelope appears to be that it will attract greater
attention, at least in some households. One benefit for NSDUH interviewers is that they would
not have to fold the letters to insert them into the envelopes, which would increase the likelihood
that the address information is visible in the address window.
Based on these results, the recommendation would be to carefully consider the costs and
benefits of using 9x12 envelopes versus the standard number 10 size. Using these larger
envelopes would require additional material costs of about $4,800 and additional postage costs of
approximately $116,000 annually, assuming 200,000 lead letters are mailed each year. Although
focus group participants generally preferred the larger envelope, this preference was tempered by
knowing that only a single letter would be included in the mailing. Given that the standard size
envelope did not generate negative reactions, this size envelope with logo and return address
presented in the focus groups may be equally effective as the larger envelope.
Including the endorsement “OFFICIAL BUSINESS. PENALTY FOR PRIVATE USE
$300” appears to have advantages and drawbacks. Focus group participants felt this statement
made the envelopes look more official, but almost no participants actually understood what this
statement means. The phrase “PENALTY FOR PRIVATE USE $300” was particularly
confusing to many participants. These reactions suggest that it may be advisable to keep the
phrase “OFFICIAL BUSINESS,” but drop the second part of the statement if possible. United
States Postal Service guidelines indicate that this full statement would have to be included on the
envelope. The current envelope used for mailing the lead letters and frequently asked questions
for the National Immunization Survey, a study sponsored by DHHS, includes only the
“OFFICIAL BUSINESS” part of this statement. The potential for using only the first part of this
statement should be investigated further, so that a final decision can be made on whether to
include this statement. Given that most participants indicated they would open the envelope
because of the DHHS logo, including this statement does not seem critical to the effectiveness of
the lead letter envelope.
22
4.2.
Lead Letter Text
A majority of participants in the English groups preferred version 1a of the lead letter
text, but a plurality of Spanish group participants preferred version 3a. In most groups,
participants made compelling arguments for either version. Version 2a was generally viewed as
too lengthy and complicated, and participants therefore felt they could recommend few parts of
the text in this letter as preferable.
Based on these results, the primary recommendation would be to create a hybrid of the
text in version 1a and 3a that would combine the preferred text of each letter and avoid any text
considered to be problematic. For example, many participants indicated version 1a was wellorganized and covered the most important information about the study. Participants also thought
it was useful that version 3a specifically mentions the topic of the survey and indicates (as in the
Q&A brochure) both users and non-users of drugs and other substances are needed to participate.
These reactions can be used to update version 1a to include some of the content and phrasing of
version 3a to produce a stronger letter overall.
One element of the version 2a letter text that could be used in the new letter was
addressing the letter to “Resident of ____________ County.” Some participants did not feel this
would significantly increase their likelihood of opening the letter, but many did feel this would
indicate to them that the mailing is important for them and their local community. For this
reason, it might be worthwhile to investigate the costs and logistics of adding the county, parish,
or district for each addressee.
On the issue of two signatures (included in versions 1a and 2a) versus a single signature
(used in version 3a), participants did not indicate a strong preference. Most felt including both
signatures was the better approach, so recipients would more clearly understand both
SAMHSA’s and RTI’s role in conducting the study. Given that there were really no negative
reactions to including both signatures, it seems like the letter should continue to provide both the
SAMHSA and RTI signatures.
4.3.
Lead Letter Graphics
Focus group participants offered rather mixed preferences for which version of the lead
letter graphics they preferred. None of the versions garnered majority approval in either the
English or Spanish groups, but half of the participants in the Spanish groups preferred version
2b. In contrast, about 45 percent of English group participants preferred version 1b.
Specific elements of the lead letter graphics seemed to heavily influence participant
preferences. The larger DHHS logo on version 1b was often cited as preferable to the smaller
version displayed on versions 2b and 3b. Another key element was the use of a gray silhouette
versus an actual picture in the image of the field interviewer’s identification badge. Overall,
participants preferred the actual picture on the identification badge, even when it was pointed out
to them that the picture could not be tailored to show the actual field interviewer assigned to each
selected household. One qualification on this point is that participants agreed that a watermark or
other graphical feature should be used to indicate the identification badge is only a sample. Many
participants were initially unclear that the picture was just a sample, and therefore they would
have expected the person in the picture to be the actual field interviewer assigned the recipients’
household. One aspect of the graphics in version 1b that was not received favorably by
participants was including the phrase “An Important Request from The U.S. Department of
Health & Human Services” in the header. Most participants viewed this as superfluous.
23
These reactions suggest that the lead letter graphics should incorporate various elements
used across the three versions, including:
the larger DHHS logo used in version 1b
the sample picture on the identification badge used in version 2b, with a watermark
indicating that the badge is just a sample
the line for the interviewers’ name under the identification badge used in all three
versions.
Participants did not express strong preferences for how the return address was presented in the
header. This feature of the header seems unlikely to have a significant impact on recipients’
reaction to the letter.
4.4.
Q&A Brochure
Although a majority of focus group participants preferred version 2 of the Q&A
brochure, preferences did differ significantly between the English and Spanish group
participants. Whereas over 80 percent of participants in the Spanish groups preferred version 2,
51 percent of English group participants preferred version 2. Participants in the English groups
were also much more likely to decline to indicate a preference between the two versions
compared to those in the Spanish groups.
The primary appeals of version 2 of the brochure appeared to be the use of colors and
pictures, as well as the layout and amount of text presented. Participants who perceived version 1
as providing more detail overall than version 2 viewed this alternatively as either a positive or
negative feature. Some felt the additional details were informative and useful, but others thought
these details were overwhelming and would discourage people from reading the brochure.
Feedback on the Q&A brochures indicates development of version 2 should continue, but
useful elements from version 1 should be incorporated into the brochure. For example, some
participants suggested the way the topics were organized in version 1 was a better than in version
2. In addition, the brochure should retain sections viewed as particularly useful by participants
and consider reducing or dropping sections viewed as less important. Participants felt the section
“What If I Do Not Smoke, Drink, or Use Illegal Drugs?” in version 1 of the brochure was
particularly effective. One section that was identified as less important by most people was
“How Does the Government Conduct the Study?” Creating a new brochure should address these
concerns.
Preliminary cost estimates gathered during the design phase for the contact materials
indicated that version 2 of the Q&A brochure should not cost significantly more than version 1 to
print. A final cost estimate can be determined once the brochure design is finalized.
24
This page intentionally left blank.
25
5. References
Dillman, Don. (2000). Mail and Internet Surveys: The Tailored Design Method. NY: John Wiley
and Sons
Fowler, F.J. (1995). Improving Survey Questions: Design and Evaluation. Sage: Thousand Oaks,
CA.
Groves, R. M., Cialdini, R. B., and Couper, M. P. (1992). "Understanding the Decision to
Participate in a Survey." Public Opinion Quarterly 56(4): 475-495.
Murphy, J., Eyerman, J., and Kennet, J. (2004). Nonresponse among persons age 50 and older in
the National Survey on Drug Use and Health. Proceedings of the eighth conference on Health
Survey Research Methods, Atlanta, GA.
Murphy, J., Schwerin, M., Hewitt, D., and Safir, A. (2005) “Nonresponse among Respondents
Aged 50 and Older Potential Respondents Focus Group Report.” Prepared by RTI for Substance
Abuse and Mental Health Services Administration.
26
This page intentionally left blank.
27
Appendices
28
| File Type | application/pdf |
| File Modified | 2013-04-17 |
| File Created | 2013-04-17 |