Download:
pdf |
pdf2015 NSDUH, Supporting Statement
Attachment D – Redesign Pretesting Summary
Report
NATIONAL SURVEY ON DRUG
USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY
REPORT—DRAFT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009 & 0211838
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 17, 2014
This page is intentionally blank.
2015 NATIONAL SURVEY ON
DRUG USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY REPORT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009.486.014 & 0211838.108.006.004
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 2014
Acknowledgments
This publication was developed for the Substance Abuse and Mental Health Services
Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ), by
RTI International (a trade name of Research Triangle Institute), Research Triangle Park, North
Carolina, under Contract Nos. 283-2004-00022 and HHSS283200800004C. Significant
contributors at SAMHSA include Dicy Painter, Joel Kennet, and Peggy Barker. Significant
contributors at RTI include Larry Kroutil and Doug Currivan.
ii
Table of Contents
Chapter
Page
1.
Overview ..............................................................................................................................1
1.1
Background of the Redesign ....................................................................................1
1.2
Goals of the Pretesting .............................................................................................3
1.2.1 Usability Testing ..........................................................................................3
1.2.2 Cognitive Interviewing Phase 1 ...................................................................4
2.
Usability Testing ..................................................................................................................7
2.1
Data Collection and Participant Profiles ..................................................................7
2.1.1 Description of Recruitment Methods ...........................................................7
2.1.2 Respondent Characteristics across Usability Testing ..................................7
2.1.3 Data Collection Procedures..........................................................................8
2.2
Key Findings ............................................................................................................9
2.2.1 Logistical Issues with Answering the "Yes/No" or "Enter All That
Apply" Questions .........................................................................................9
2.2.2 Ability to Use the Drug Images ...................................................................9
2.2.3 Screen Layout Preferences ...........................................................................9
2.2.4 Issues with the Question Text from a Usability Standpoint ........................9
2.3
Changes Implemented ..............................................................................................9
3.
Cognitive Interviewing: Phase 1 ........................................................................................11
3.1
Data Collection and Participant Profiles ................................................................11
3.1.1 Description of Recruitment Methods .........................................................11
3.1.2 Respondent Characteristics across Phase 1................................................11
3.1.3 Data Collection Procedures........................................................................12
3.2
Key Findings ..........................................................................................................13
3.2.1 Identification of Prescription Drugs...........................................................13
3.2.2 Performance of the Misuse Criteria ...........................................................13
3.2.3 Other Issues ................................................................................................14
3.3
Changes Implemented ............................................................................................14
4.
Cognitive Interviewing: Phase 2 ........................................................................................17
4.1
Data Collection and Participant Profiles ................................................................17
4.1.1 Description of Recruitment Methods .........................................................17
4.1.2 Participant Characteristics .........................................................................17
4.1.3 Data Collection Procedures........................................................................18
4.2
Key Findings ..........................................................................................................19
4.2.1 Demographic Items ....................................................................................19
4.2.2 Prescription Drugs and Use/Misuse ...........................................................20
4.2.3 Tobacco, Illicit, and Special Drugs ............................................................21
4.3
Changes Implemented ............................................................................................22
4.3.1 Phase 2, Round 1........................................................................................22
4.3.2 Phase 2, Round 2........................................................................................23
iii
Table of Contents (continued)
Chapter
Page
5.
Discussion ..........................................................................................................................27
5.1
Summary of Major Changes ..................................................................................27
5.2
Usability Testing Advantages ................................................................................27
5.3
Recommendations for Future Pretests ...................................................................28
6.
References ..........................................................................................................................29
iv
List of Attachments
Attachment
Page
A.
Phase 1.1 Summary of Findings ..................................................................................... A-1
B.
Phase 1.2 Summary of Findings ......................................................................................B-1
C.
Phase 1.3 Summary of Findings ......................................................................................C-1
D.
Phase 2.1 Summary of Findings ..................................................................................... D-1
E.
Phase 2.2 Summary of Findings ...................................................................................... E-1
F.
Variable Wording Crosswalk ........................................................................................... F-1
v
This page is intentionally blank.
vi
List of Tables
Table
Page
1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants ........ 8
2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants ....... 12
3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants ....... 18
4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens ......................... 22
5.
List of Screens Presented for the Full and Abbreviated Tutorials .................................... 25
vii
This page is intentionally blank.
viii
1. Overview
1.1
Background of the Redesign
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance
Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S.
civilian, noninstitutionalized population aged 12 and older. The conduct of the NSDUH is
paramount in meeting a critical objective of SAMHSA's mission to maintain current data on the
prevalence of substance use in the United States. In order to continue producing current data,
SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ) must update the
NSDUH periodically to reflect changing substance use and mental health issues. These updates
are necessary because substance use in the United States is a dynamic phenomenon that changes
with time, demographic shifts, economic prosperity, and most importantly, availability of
substances.
CBHSQ plans to redesign the NSDUH for the 2015 survey year to achieve two main
goals: (1) revise the questionnaire to address changing policy and research data needs, and
(2) modify the survey methodology to improve the quality of estimates and the efficiency of data
collection and processing.
The questionnaire will be revised to improve questions that cause known or suspected
problems with data that are collected using the current NSDUH questionnaire. New content that
addresses current data needs will be added. Revisions that are designed to reduce errors
associated with usability problems in the design and layout of the computer-assisted interviewing
(CAI) instrument are planned. The changes include electronic calendars, revised prescription
drug modules, revised front end demographics, a revised smokeless tobacco module, revised
questions for measuring binge alcohol use, a revised hallucinogens module, new questions for
lifetime use of specific inhalants, a new methamphetamine module, a revised special drugs
module, a revised consumption of alcohol module, and a revised back end demographics section.
Specifically, the following are highlights of issues for the modules for prescription drugs and
health care and for interviewer-administered questions that will be addressed by the redesigned
questionnaire.
•
In the current interview procedures, interviewers and respondents mark a printed calendar
with the start of the 30-day and 12-month reference periods. In the redesigned
questionnaire, respondents will be shown electronic versions of these calendars that note
the start of these reference periods, and they will receive a tutorial about these electronic
calendars. The electronic calendars also will appear periodically to remind respondents of
reference periods, and respondents can use a function key to review the calendars at any
time during the self-administered modules.
1
•
The content of the specific prescription drugs in the core prescription drug modules was
last updated in 1999, based on prevalence information from 1998. Since that time, despite
minor updates to the questionnaire, several prescription drugs included in NSDUH have
been discontinued or are otherwise no longer legally available, and recently approved
prescription drugs with abuse potential are not included in the core modules. All
questions measuring misuse of prescription drugs will be moved into the prescription
drug modules. In addition, a review of the drugs that are included in these modules will
result in recommendations for drugs to be added or dropped based on their availability.
•
The current NSDUH questionnaire asks only about misuse of prescription drugs. A
revised structure will be tested that first uses a screener module to ask about any use of
prescription drugs. Respondents who report any use pf prescription drugs will then
receive questions determining whether any of that use constituted misuse.
•
The current NSDUH definition of misuse (i.e., use of drugs that were not prescribed for
you or that you took only for the experience or feeling they caused) combines a behavior
and a motivation. It also does not include overuse of prescribed medication. The revised
definition will focus on behaviors that constitute misuse and will refer to use "in any way
a doctor did not direct you to use it/them." Overuse (i.e., use in greater amounts, more
often, or longer than directed) will be added to the examples of behaviors that constitute
misuse.
•
Except for misuse of OxyContin® in the current pain relievers module and misuse of
methamphetamine in the stimulants module, misuse of specific prescription-type
psychotherapeutic drugs in the current questionnaire is limited to the lifetime period.
Measurement of past year misuse is limited to misuse of any drugs within a
psychotherapeutic category (e.g., pain relievers). However, information about more
recent misuse of specific prescription drugs (e.g., in the past year) is of greater interest to
policymakers and stakeholders who use the NSDUH prescription drug data. Therefore,
questions will measure past year misuse of individual prescription drugs.
•
In the current questionnaire, methamphetamine is included in the module for misuse of
prescription stimulants, and respondents are asked in the stimulants module about misuse
of methamphetamine. However, most methamphetamine that is used in the United States
is manufactured illegally rather than by the pharmaceutical industry. Respondents also
may fail to report methamphetamine use in the context of questions about misuse of
prescription drugs. Therefore, the redesigned questionnaire will include a separate
module for methamphetamine, and the questions in the stimulants module will ask about
misuse of prescription stimulants.
•
For the current questionnaire, respondents are handed printed "pill cards" with pictures of
prescription drugs in a given module to aid respondents in recalling their misuse of
specific drugs. NSDUH incurs the expense of printing pill cards for interviewers who are
new to the survey or reprinting the pill cards when new dosages become available for
some prescription drugs. Also, respondents do not always ask interviewers for the pill
cards when they reach the prescription drug questions. Therefore, electronic drug images
to be displayed on the laptop computer screen will replace the printed pill cards. The
electronic images also will include examples of prescription drugs other than pills (e.g.,
patches, vials).
2
•
The current health care module of the questionnaire is limited in scope. Questions
measuring height, weight, a revised list of health conditions, and details about diagnoses
of those health conditions will be tested and added to the health care module.
•
The current survey includes questions about emergency room visits, hospitalizations,
substance abuse treatment utilization, and mental health service utilization, but does not
include questions about outpatient primary care service utilization or opportunities for
intervention with substance users in primary care settings. Questions related to these
topics will be tested and included in the health care module of the questionnaire.
•
Although audio computer-assisted self-interviewing (ACASI) is used for most of the
sensitive questions, respondents may be likely to consider some questions in intervieweradministered sections to be sensitive (e.g., income). Pretesting efforts will explore the
feasibility of moving health insurance and income questions to an ACASI mode.
Proposed revisions will improve the precision of estimates that exhibit properties
associated with measurement error. Revisions will also be designed to reduce errors associated
with usability problems in the design and layout of the CAI instrument.
1.2
Goals of the Pretesting
The pretesting for the 2015 NSDUH questionnaire redesign had three components:
Usability Testing (Prescription Drugs), Phase 1 (Prescription Drugs) Cognitive Interviewing, and
Phase 2 (Redesigned Modules) Cognitive Interviewing. The purpose of each of these three
components is discussed in the following sections.
1.2.1
Usability Testing
Usability testing was conducted in the first stage of redesign pretesting. Prior to the
usability testing, questions for misuse of pain relievers and stimulants were reworded, producing
two new versions of these questions. One of these versions was a "three-criteria" bulleted list that
asked participants about using a set of prescription pain relievers and stimulants "in any of these
ways" in the past 12 months:
•
without a prescription of your own,
•
in greater amounts, more often, or longer than you were told to take it, or
•
just for the effect it had on you—not for its intended medical use.
A second version ("two criteria") eliminated one criterion from the three-criteria list: use
"in greater amounts, more often, or longer that you were told to take it," but used the same
wording for the remaining two criteria.
In addition, the usability instrument tested the placement of pill images above or below
the question text in order to test the new electronic drug images. Participants were presented with
a total of four versions of the questions. Two versions of the three-criteria questions featured
images above and below the questions. There were also two versions of the two-criteria
questions with images above and below the questions. In addition to soliciting feedback on the
criteria items, usability testing sought to accomplish three primary goals:
3
1. observe participant use of and reaction to different options for proposed laptop
computer screen layouts for the screeners and modules for prescription pain relievers
and prescription stimulants;
2. collect participant feedback and recommendations for improving the design of the
laptop computer screen layouts for the screeners and modules for prescription pain
relievers and prescription stimulants; and
3. collect participant feedback on the electronic reference date calendars and electronic
drug images.
1.2.2
Cognitive Interviewing Phase 1
The second stage of redesign pretesting, cognitive interviewing, occurred in two phases.
The first phase of interviewing focused on redesigned prescription drug modules and a
redesigned health module. The second phase continued to test these modules but expanded the
scope to include the full redesigned NSDUH instrument.
Phase 1 cognitive interviewing took place over three rounds and tested an instrument
with changes to the prescription pain relievers, prescription stimulants, and health modules. The
Phase 1 redesigned instrument included an abbreviated core set of substance use questions that
measured lifetime and 30-day use. The following changes were reflected in the Phase 1
abbreviated core questionnaire:
•
Smokeless tobacco sections (chewing tobacco, snuff, and the new product, snus) were
combined into one section.
•
Hallucinogen items currently included in special drugs for ketamine, tryptamines (DMT,
AMT, and "Foxy"), and Salvia divinorum were moved from the noncore special drugs
module to the core hallucinogens module.
•
New inhalants questions for lifetime use of markers and air duster were added to the core
inhalants module.
•
Prescription pain relievers and prescription stimulants screeners focused on any use of
specific prescription drugs in the past 12 months (i.e., medical use or misuse). Misuse of
prescription pain relievers and stimulants in the corresponding main modules was defined
as use "in any way a doctor did not direct you to use it/them" and was based on the
examples from the three-criteria bulleted list that was implemented during usability
testing. Respondents could be reminded about these criteria by pressing a function key to
see a pop-up screen with the bulleted list. Electronic drug images were placed below the
introductory question text, and some images showed examples other than pills.
•
New health module questions were added.
•
The electronic reference date calendar was featured throughout the instrument.
The goals of Phase 1 cognitive interviewing were to test new and revised question
wordings and response options. This testing included gauging participants' reactions to key terms
and concepts and detecting any issues with question comprehension. A test of the electronic
reference date calendar was also a Phase 1 goal.
4
1.2.3 Cognitive Interviewing Phase 2
The cognitive interviewing in Phase 2 took place over two rounds and was designed to be
a broader test of changes to the instrument. Phase 2, round 1 included all of the modules that
were tested in Phase 1. In addition, the following revisions were made for Phase 2:
•
New items were added to core demographics, including new military veteran questions.
•
Screeners and modules for tranquilizers and sedatives were added to the instrument and
tested.
•
The definition of binge drinking was changed for female respondents to include four or
more drinks on an occasion in the past 30 days.
•
Questions about needle use in the noncore special drugs module were reworded, and
questions about use of methamphetamine and prescription stimulants with a needle were
moved to the corresponding core modules.
•
New questions about medical use of marijuana were added to the noncore blunts module.
•
Education, health insurance, and income were moved to the ACASI portion of the
interview. In addition, the highest level response category for income was revised.
•
A new module introduced proxy respondents to the ACASI.
Phase 2, round 2 included the fully (nonabbreviated) redesigned instrument, with all core
modules, and incorporated audio recordings for the ACASI portion.
The goals of Phase 2 cognitive interviewing included retesting changes that were made to
question wording, routing, and response options following Phase 1. Testing the content that was
new to the cognitive interviewing protocols in Phase 2 was also a goal.
5
This page is intentionally blank.
6
2. Usability Testing
2.1
Data Collection and Participant Profiles
2.1.1
Description of Recruitment Methods
Participants were recruited for usability testing via a recruitment notice that was posted
on Craigslist. Persons who called RTI International in response to the recruitment notice were
administered a series of screening questions. The screening questions included items about the
caller's age and location to establish eligibility for the study; to be eligible, a person needed to be
aged 18 or older and located within 50 miles of Research Triangle Park, NC. The screening
questions also collected information on other demographic characteristics (e.g., gender,
education), computer use, and any use of prescription pain relievers and stimulants in the past 12
months. Questions about use of pain relievers and stimulants were included to gauge participants'
familiarity with the general types of drugs that were included in the usability testing. However,
the usability testing did not focus on participants' actual behavior with respect to prescription
drug use.
2.1.2
Respondent Characteristics across Usability Testing
Table 1 provides information on characteristics of the eight usability participants. The
number of participants was evenly divided between males and females. Participants' ages ranged
from 28 to 60. Among the four participants in the 26 to 34 age category, there were two
participants aged 30 years, one participant who was aged 28, and another aged 32. Among the
three participants aged 50 or older, ages ranged from 53 to 60. There was only one participant in
the 35 to 49 age group. Six participants reported some education beyond high school, while two
participants did not.
As might be expected, because participants were recruited from Craigslist, all reported
using a computer for personal use every day. However, only half reported that most of what they
do at work is done on a computer. Two participants reported using a computer at work less than
daily or never; the screening interviewer was instructed to choose "never" for computer use at
work if participants did not use a computer at work or if they did not have a job.
Five participants reported using prescription pain relievers for any reason in the past 12
months, and one reported using prescription stimulants in that period. None reported use of both
classes of prescription drugs in the past 12 months. Although the screening question included
items about use of prescription pain relievers or stimulants in the past 12 months, use of these
prescription drugs was not an eligibility requirement for the study. Consequently, two
participants reported not using either of these classes of prescription drugs in the past 12 months.
7
Table 1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants
Participant Characteristics
Gender
Male
Female
Total
Age Group
18 to 25
26 to 34
35 to 49
50 or Older
Total
Highest Grade Completed
High School or Less
Beyond High School
Total
Computer Use at Home/For Personal Use
Every Day
Computer Use at Work
Never
Sometimes but Not Every Day
Every Day but Not All Day
Most Work Is on a Computer
Total
2.1.3
Number
4
4
8
0
4
1
3
8
2
6
8
8
1
1
2
4
8
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures that were approved by RTI's Institutional Review Board. Parental permission was not
an issue for usability testing because all participants were adults.
Participants were then presented with a booklet containing a series of tasks. Participants
were shown the task instructions but not the follow-up probes, which were read aloud by the
interviewer. The follow-up questions provided breaks between sets of usability tasks. In the first
two tasks, all participants were shown unlabeled pictures of the pain relievers or stimulants that
were included in the interview. The aim of these tasks was to test participant identification of
prescription drugs solely from the drug images.
In the remaining tasks, participants were presented with hypothetical scenarios that were
read aloud by the interviewer. Participants were instructed to answer the questions on the laptop
computer as though the behavior described in a given scenario was true for them. Some scenarios
tested participants' identification of drugs by name, and others tested participants' ability to
identify drugs based on pictures in the booklet and the corresponding on-screen images. The
interviewer asked participants to read the questions aloud according to how they appeared on the
computer screen and then asked them to indicate how they thought the question should be
answered based on the scenario they were given. To obtain preliminary information about the
cognitive processing of the question, the interviewer also asked why participants thought a
question should be answered a certain way based on the scenario.
8
Although each interview was allowed 90 minutes, interviews generally only required
about 1 hour to complete, including the introduction, informed consent, usability test questions
and tasks, and incentive disbursement. Participants were given a $60 Visa gift card for
completing the interview.
2.2
Key Findings
2.2.1
Logistical Issues with Answering the "Yes/No" or "Enter All That Apply"
Questions
No participants appeared to have difficulty with answering the yes/no questions.
However, some participants had difficulty with the "enter all that apply" questions when the
numbering of response categories continued from the previous question, instead of resuming the
numbering of the first category with 1.
2.2.2
Ability to Use the Drug Images
Some scenarios in the test booklets presented situations in which a drug was shown as a
picture rather than being mentioned by name. Participants were able to correctly answer the
relevant survey question by matching the on-screen drug image to the drug image in the test
booklet. One participant commented specifically about how well laid out the on-screen images
were. These results suggest that participants can successfully use the on-screen images as a recall
tool.
2.2.3
Screen Layout Preferences
Participants as a whole did not have a strong preference for whether the drug images
were placed above or below the question text. Those who expressed a preference were fairly
evenly divided between the two types of layouts.
2.2.4
Issues with the Question Text from a Usability Standpoint
From a usability standpoint, no major issues were identified with respect to whether it
was easier for participants to answer the two-criteria or three-criteria questions. In particular, no
one commented that the three-criteria layout appeared to be too "busy" on the computer screen.
One participant recommended a larger point size for the on-screen font, particularly to aid older
participants in reading the questions.
Some participants talked about the repetitive nature of the questions, but no one explicitly
mentioned that the three-criteria questions were overly repetitive. Rather, comments about the
repetitive nature of the questions appeared to be caused by participants being presented with the
same basic questions four different ways (i.e., with the drug images above or below the questions
and with the two-criteria or three-criteria wording).
2.3
Changes Implemented
Based on the results of the usability testing, either image layout was deemed acceptable
to participants. It was decided that the drug images would be displayed at the top rather than that
the bottom of the screen, and that the introductory question text would appear above the drug
9
images. In addition, response option values for "enter all that apply" questions would be reset to
1 on each relevant questionnaire screen. For example, if there are five drugs displayed per
screen, each will be numbered as 1 through 5, as opposed to being labeled 1 through 5 and then 6
through 10. This will address the issue with entering responses that participants identified with
the "enter all that apply" questions during usability testing.
10
3. Cognitive Interviewing: Phase 1
3.1
Data Collection and Participant Profiles
3.1.1
Description of Recruitment Methods
Phase 1 cognitive interviewing included three rounds of interviews. Interviews were
conducted in Research Triangle Park, NC; Chicago, IL; and Washington, DC. Advertisements
were placed on Craigslist, and flyers were distributed to outpatient substance abuse treatment
programs. In the first round of Phase 1 cognitive interviews, only adults were interviewed. In the
second and third rounds, adolescents were also interviewed. In all three rounds, the recruitment
advertisements targeted past year users of prescription pain relievers and prescription stimulants.
Licit or illicit use was not specified in the advertisements. There were two versions of the
recruitment advertisements for rounds 2 and 3. One version was for adults, and one version was
for adolescents aged 12 to 17.
Persons who responded to the recruitment advertisements were administered a screener
over the telephone to assess their eligibility for the study. The screener took approximately 5
minutes to administer. It included questions about how callers learned about the study (to
identify the source of recruitment), age, gender, place of residence, education, physical
limitations, prior research study participation, and use of prescription pain relievers or stimulants
(for any reason) in the past 12 months. Recruitment was balanced to include people who had
used prescription pain relievers or stimulants in the past 12 months as well as nonusers.
Procedures for obtaining permission from a parent or legal guardian to interview adolescents are
described in Section 3.1.3.
In addition, persons who have had considerable prior experience with survey research
may bias the data and conclusions. Therefore, persons were deemed ineligible if they had
participated in more than one prior research study in the past 12 months (regardless of who
conducted the study or the content) or in any prior RTI research study in this period.
3.1.2
Respondent Characteristics across Phase 1
Table 2 presents an overview of selected demographic characteristics of Phase 1
cognitive interview participants. Out of 40 respondents in Phase 1, 10 were adolescents. As
noted previously, these 10 adolescents were interviewed in rounds 2 and 3 only. Seven of the 10
adolescent respondents were interviewed in round 2. Recruitment was balanced across gender,
with 18 female respondents. In Phase 1, 13 of the respondents were recruited from substance
abuse treatment programs. These 13 substance abuse treatment patients were interviewed in
rounds 1 and 2; recruitment in round 3 was limited to persons from the general population.
Thirty respondents had used prescription pain relievers in the past 12 months and 18 had used
prescription stimulants in the past 12 months. Although not shown in the table, of the 30 adults,
15 had a high school diploma, GED, or lower education. The other 15 had attended at least some
college.
11
Table 2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)1
Female
Enrolled in Treatment Program (Currently or
within Past 12 Months)
12 Month User of Any Prescription Pain
Reliever
12 Month User of Any Prescription Stimulant
Chicago, IL
Washington, DC
Research Triangle Park, NC
1
Round 1
Round 2
Round 3
Total
16
0
7
8
5
7
5
5
9
3
6
0
30
10
18
13
14
7
9
30
7
5
5
6
4
2
3
7
7
3
4
5
18
10
12
18
Mean age not available.
3.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across all three rounds
of Phase 1 data collection. The interview was conducted using the computer-assisted Blaise
interview. The cognitive interviewer first administered introductory questions verifying the
respondent's age and gender. The interviewer introduced the laptop functions to the respondent
and described the feature of the electronic reference date calendar. The respondent was then
asked to conduct the next section of the interview using a self-administered design. In Phase 1,
audio files were not used in the first two rounds, so respondents read questions on the screens
themselves and then answered the questions. In round 3, respondents could listen to audio
recordings of the questions. Interviewers administered think-aloud probes and follow-up probes
to collect data on the cognitive properties of specific questions. The following modules were
tested in all rounds in Phase 1:
•
An abbreviated ensemble screener of drugs
•
Pain relievers screener
•
Stimulants screener
•
Pain relievers main module
•
Stimulants main module
•
Health module
The protocol largely remained the same for rounds 1 and 2. After round 2, slight changes
were made to the cognitive interview protocol. Changes included the following:
•
adding probes for questions that were identified as problematic in the first rounds of
interviewing,
12
•
dropping three categories describing sources for obtaining prescription drugs, and
•
editing selected question logic, question wording, and response options.
Probes were deleted for questions that no longer necessitated testing in the third round.
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
3.2
Key Findings
Summaries of the findings and recommendations from rounds 1, 2, and 3 of Phase 1
cognitive interviewing can be found in Appendices B, C, and D. These summaries also include
recommendations for edits to increase respondent comprehension of key items.
Key findings from Phase 1 interviews can be arranged into three broad categories:
(1) identification of prescription drugs, (2) performance of the misuse criteria, and (3) other
issues not related to prescription drugs. The subsections that follow cover each of these key
findings categories in turn. Changes that were made as a result of these findings are covered in
Section 3.3.
3.2.1
Identification of Prescription Drugs
In most cases, participants recognized the prescription drugs that they took by name.
However, some participants did find the drug images to be helpful, particularly if participants
were unfamiliar with certain prescription drugs.
In the screener, several screens showed brand name prescription drugs (e.g., Vicodin®)
and the generic equivalent (e.g., hydrocodone); "generic" was shown in parentheses next to the
generic drug name. There was some uncertainty about the "generic" term associated with some
drugs. Some participants reported that their doctors would write the brand-name drug on the
prescription, but the pharmacy would substitute the generic. Another issue concerns the number
of generic drug manufacturers and the variety of generic equivalents of specific drugs that may
be dispensed when prescriptions are filled. It would not be realistic to try to include exhaustive
examples of images for generic drugs. Addition of the word "generic" to the drug images in
round 3 (i.e., in addition to its appearance in the response category for the generic) appeared to
be helpful.
Another issue with identifying prescription drugs involved making respondents aware
that the questions applied to any form regardless of its appearance or whether they used the drug
in pill form or another form. Some respondents neglected to report use of liquid forms of the
prescription drug, especially if the images for these particular drugs showed only pills. Other
respondents mentioned using the drug, but in a pill form that was not shown on the screen. An
edit was made to the instrument to address this issue. This edit is described with PR01 in Section
3.3.
3.2.2
Performance of the Misuse Criteria
The wording "in any way a doctor did not direct you to use it" differentiated between
misuse of prescription drugs and use with a prescription and as directed. With few exceptions,
participants who used prescription drugs only with a prescription of their own and as prescribed
13
could determine that they did not use prescription drugs "in any way a doctor did not direct you
to use [them]." Similarly, persons who misused prescription drugs could determine that their use
constituted use "in any way a doctor did not direct you to use it," even if they used prescription
drugs in ways not explicitly listed as examples of misuse.
Later in the module, a question asked respondents which of the specific components of
misuse applied to them. This decomposition question puzzled some respondents. In round 1, the
criterion of overuse was presented as a single response (i.e., use in greater amounts, more often,
or longer than I was told to take it). Some round 1 respondents had trouble determining whether
they should choose this second category or the third one (use in some other way a doctor did not
direct me to use it). Specifically, some respondents thought that they needed to have engaged in
all three of the behaviors that were listed in the second category in order to choose it. Three
scenarios for misuse also arose during the interviews that were not listed as examples in the
instrument. These were (1) using with alcohol, (2) asking a doctor to prescribe a larger dosage of
prescription pills than was needed, and (3) buying prescriptions in other countries where
prescriptions are not needed for purchase.
3.2.3
Other Issues
Respondents had trouble understanding the use of canned air or air duster as an inhalant.
Although no respondents had used canned air as an inhalant, there were difficulties identifying
the product by that term. Problems were identified with respondents entering height and weight
in the new health module, but these were resolved with minor revisions to the screens. For
questions about the number of outpatient doctor visits, participants had difficulty determining
what did or did not constitute a doctor visit. Summaries of the findings and recommendations
from rounds 1, 2, and 3 of Phase 1 cognitive interviewing can be found in Appendices B, C, and
D. These summaries also include recommendations for edits to increase respondent
comprehension of key items.
3.3
Changes Implemented
As a result of all rounds of Phase 1 cognitive interviewing, several changes were made to
the instrument. These are listed below and are accompanied by the variable name.1
•
IN01ii—The word "air duster" was added to this question about canned air in the
inhalants section for increased comprehension.
•
PR01—The text in the introduction to each prescription drug screener was revised to
include "Please note that not all forms, dosages, shapes, or colors of these pain relievers
are shown on these screens, but you should include any form that you have used." This
sentence would convey that respondents should consider forms of the drug other than
pills (e.g., liquid forms), even if the image showed only pill forms.
•
PR07—Reference to the term "China Girl" was removed for fentanyl because none of the
respondents knew it, and fentanyl was the only prescription drug with a slang name.
1
Variables in the health module were renumbered during pretesting. Variable names as they appear in the
questionnaire field test (QFT) instrument are in parentheses.
14
•
PRHOSPYR1—This question about use of prescription drugs only in the hospital was
dropped because many respondents had difficulty answering it.
•
The calendar icon, which reminds respondents how to access the electronic reference date
calendar, was made more visible to encourage respondents to use it.
•
Audio was removed for the answer choices for height and weight because those are
included in the questions.
•
HLTHNEW03 (HLTH19)—This question, which asks the respondent how many times
he or she has visited a doctor in the past year, was revised to streamline the text.
•
HLTHNEW11 (HLTH29) & 12a (HLTH31)—Specs about specific health conditions
were changed to skip questions about whether respondents had specific conditions in the
past 12 months if the age at first diagnosis equals current age.
15
This page is intentionally blank.
16
4. Cognitive Interviewing: Phase 2
4.1
Data Collection and Participant Profiles
4.1.1
Description of Recruitment Methods
Phase 2 of cognitive interviewing included two rounds of interviews. For both rounds of
interviews, recruitment advertisements were placed on Craigslist in Research Triangle Park, NC;
Chicago, IL; and Washington, DC.
The Phase 2 cognitive interview research questions influenced the recruitment goals.
During Phase 2, methodologists sought to test the transition from asking questions about health
insurance and income using an interviewer-administered mode to using ACASI. Because
respondents can nominate a proxy to answer these questions on their behalf, new questions were
developed to provide this proxy with a tutorial on how to use the computer and the interviewing
program. Transition statements were also developed to inform the proxy about their role in the
interview. To test this process and these new screens, RTI recruited adolescents to serve as
respondents and their parents to serve as proxies. In some cases, the parents went on to complete
a full interview after serving as a proxy respondent for their child.
Three versions of the advertisement were posted: (1) a version targeting military
veterans, (2) a version targeting parent-teen pairs, and (3) a final version targeting adolescents
whose parent would participate in a short interview as a proxy for health insurance and income
questions.
Prospective participants who responded to the advertisement were administered a
screening questionnaire. The screener took, on average, approximately 5 minutes to administer.
A variety of demographic questions were asked of prospective participants, including questions
about age, gender, educational attainment, military status, and use of prescription drugs in the
previous 12 months. Answers to screener questions, in addition to the location at which they
would be interviewed and the study population to which they belonged (military veteran, parentteen pair, adolescent with proxy interview, and drug use history), were used to select participants
and reach recruitment targets to the extent that was logistically possible. Because reporting illicit
or prescription drug use was not an eligibility requirement in the first round, few first round
participants were heavy or expert drug users. Therefore, in the second round of interviews, RTI
targeted individuals who had used any of the four psychotherapeutic prescriptions or
methamphetamine and those who had received drug treatment in the past 12 months. For these
reasons, the final sample of participants represents a diverse set of demographic characteristics
and experiences.
4.1.2
Participant Characteristics
Table 3 presents a summary of selected demographic characteristics of Phase 2 cognitive
interview participants. For some demographics, such as age and gender, participants were well
distributed across interviewing rounds. For other characteristics of interest, the distribution
reflects changes in recruiting goals between rounds. Reasons for changes to recruitment goals are
discussed subsequently.
17
Participants ranged in age from 12 to 68. A total of 24 adults participated in Phase 2
interviews, with a relatively even distribution between rounds (round 1 = 10; round 2 = 14). The
mean age of adult participants was 42 years. Among the 16 adolescents, 10 participated in the
first round, and 6 participated in the second round of interviews. The mean age of adolescent
participants was 15 years. Approximately half of the sample was female (i.e., regardless of age).
Among adults who provided information on educational attainment, a majority had at least some
education beyond high school (not shown in Table 3).
Interviews in Phase 2, round 1 included a relatively large number of military veterans,
and it included a much small number of individuals with a history of heavy drug use. As
discussed in Section 4.2, veterans exhibited no problems answering the questions of interest. As
a result, the recruitment goals shifted in the second round of interviews to target individuals with
more drug experience.
Table 3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)2
Female
Veteran
Enrolled in Treatment Program (Currently or Within
Past 12 Months)
High Use Participant*
Chicago, IL
Washington, DC
Raleigh, NC
Round 1
10
10
9
5
2
0
4
3
13
Round 2
14
6
12
2
5
11*
5
5
10
Total
24
16
21
7
7
11
9
8
23
1
Mean age = 42.
Mean age = 15.
* The category "high use" was created for Phase 2, round 2 to refer to individuals who had experience with more
than one or two prescription drugs. This enabled recruitment of individuals who had a history with heavy drug
use.
2
4.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across the two rounds
of Phase 2 data collection. These procedures were also the same as those described in Section
3.1.3. Audio was available to participants in Phase 2. Interviewers administered think-aloud
probes and follow-up probes to collect data on the cognitive properties of specific questions.
In round 1, a selected set of modules was administered to participants. In round 2, the
entire proposed redesigned instrument was administered to participants. In both rounds, probes
were inserted following items of interest.
18
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
4.2
Key Findings
Summaries of the findings and recommendations from rounds 1 and 2 of Phase 2
cognitive interviewing can be found in Appendices D and E. These summaries also include
recommendations for edits to increase respondent comprehension of key items. Key findings
from Phase 2 interviews can be arranged into three broad categories: (1) findings related to the
front- and back-end demographics sections, (2) findings related to prescription drug use and
misuse, and (3) findings related to substances other than prescription drugs. The following
subsections cover each of these key findings categories in turn. Changes that were made as a
result of these findings are discussed in Section 4.3.
4.2.1
Demographic Items
As noted in Section 4.2.1, round 1 interviews included a recruitment focus on military
veterans. In general, veterans were able to understand terms and phrases in the way in which they
were intended. Veterans understood the phrase "full time in the Reserves" (item V2a) despite the
fact that none of them reported ever being in the Reserves. Most participants were also able to
correctly define a combat zone when methodologists probed about the meaning of the term
(QD10c).
For a few participants, the new education question (QD11) was problematic. Interviewer
probing revealed that the order of the response categories was confusing. The question asked
about the highest grade that was completed, and response categories 1 to 11 included language to
that effect. Consequently, a participant originally understood an answer of "12" to mean
finishing the 12th grade. Revisions, as described in Section 4.3.2, were made to address this
item.
A majority of participants understood the questions in the back-end demographics
module. For example, participants correctly understood, despite being on summer break, that
they should report currently attending school (QD17). They also understood, when asked what
grade they were currently attending, to report the grade they are planning to enter after summer
vacation comes to a close. No problems were encountered when defining testing for drugs or
alcohol as part of the job hiring process (QD49), and what the term "random" means in the
context of random drug testing (QD50). One back-end demographic question that RTI noticed
could be moderately confusing to respondents was the item about sources of income
(INTRTINN). In the past this item was interviewer administered. The transition to an ACASI
administration resulted in awkward question wording and thus was revised to account for the
mode.
Most participants were able to answer a question about their height (HLTH04) with no
problems. However, it became clear in one interview that some respondents might not be able to
answer the question in the formats that were available (feet/inches or meters/centimeters). Some
respondents will be able to answer only in inches or only in centimeters because this is often how
height is measured in medical evaluations. Revisions to the available methods of entering height
were therefore implemented.
19
Income branching questions (QI20N, QI21A, and QI21B) were generally understood by
participants. One branch question (QI20N) asks respondents whether they make over or under an
income threshold. Contingent on their answer, respondents are taken to QI21A (for incomes
below $20,000) or QI21B (for incomes over $20,000). The numeric categories carrying over
from QI21A to QI21B was confusing to one participant. That is, there were "20" categories in
the first branch of questions that the participant was not routed to and had not seen. The question
she was routed to (QI21B) began categories at "21." No other question began a list of numeric
categories in such a way, which led to the initial confusion about how to answer the question.
The response category labels were revised to begin at 1.
The instrument tested in Phase 2, round 2 included two transitions from interviewer
administration to self-administration. In both rounds of interviewing, most respondents who were
serving as a proxy believed the proxy tutorial was simple to follow and understand. In a couple
of instances, however, it was apparent that the transition between modes could be improved. In
response, edits were made to these items, as explained in Section 4.3.1.
4.2.2
Prescription Drugs and Use/Misuse
A number of issues related to prescription drugs continued to be explored in the Phase 2
cognitive interviews. Highlighted findings include how participants understood concepts such as
generic, use, misuse, and "not all forms shown"; what motivated participants to misuse
prescriptions; and participants' ability to recall their age at first misuse of prescription drugs.
Relevant findings are discussed in this section, and corresponding changes that were
implemented are discussed in Section 4.3.
In most cases, the concepts that were explored were understood by participants as had
been intended. Most participants were able to identify generic drugs as the less expensive
alternative to a brand name drug. A small proportion of participants showed less clarity about the
chemical equivalency of generic and brand name drugs and that generics also require a
prescription.
In both rounds of interviews, participants shared a similar understanding of the concept of
misuse of prescription drugs. However, questions about misuse are administered to participants
who report the use of prescription drugs. Therefore, this finding must be measured against the
finding that there was less clarity among participants about the concept of use. The intended
understanding of "use" for prescription drugs starting at PR01 was use of any kind. Because the
first question about prescription drug use followed a long list of questions about illicit drug use,
some participants thought that the prescription drug screener questions were asking only about
the misuse of prescription drugs. Conversely, several participants thought the screener questions
were asking only about use of prescription drugs in a way that they were prescribed.
In addition to determining whether participants interpreted the concept of misuse in the
intended manner, the SAMHSA/RTI instrumentation team aimed to learn more about how
participants recalled their first misuse and their motivations for misuse. Questions that measured
motivations for misuse were added for the first time in Phase 2. No problems were apparent in
recalling the age at which misuse first occurred.
20
One cognitive issue was uncovered when the motivation items were tested. This finding
revealed an absence of a specific category for misusing prescription drugs to deal with emotions.
It also became apparent that logic and skip patterns in this series of questions needed editing. A
description of these revisions begins in Section 4.3.1.
Throughout the testing, instructions to respondents that indicated that not all forms of
pills were shown on the screen underwent further modification. Despite this iterative testing,
problems persisted. For instance, only morphine pills were shown on the screen as examples for
this drug. Respondents had difficulty determining whether to report use of morphine in liquid
form that they received by injection or intravenously. In response, a photo of a vial of liquid
morphine was added to this screen, along with the reminder about the forms that drugs can take.
4.2.3
Tobacco, Illicit, and Special Drugs
Nonprescription drug items were also of interest in Phase 2 cognitive interviews. Of
particular interest were new items about new cigarette naming conventions, substances that were
now asked about in the hallucinogen module, and whether participants understood questions
about medical marijuana.
Previous versions of the NSDUH questionnaire asked about the type of cigarettes current
smokers smoked most often (e.g., lights, ultra lights, mediums, or full-flavored cigarettes).
Recent legislation banned cigarette manufacturers from using terms on packaging that referred to
cigarette strength. As a result, many manufacturers migrated to using a color-coded system as an
alternative naming convention. In an attempt to continue to track trends in the types of cigarettes
most smoked, a new question (CGCOLOR) was developed.
In cognitive interviews, the SAMHSA/RTI instrumentation team aimed to determine how
well current smokers could recall the package color of the cigarettes they smoke most often. Half
of those who reported smoking in the past 30 days (current smokers) reported an issue with
answering this question. Participants reported that cigarette packaging includes multiple colors,
indicating that participants failed to understand the purpose of the question about package color.
The multiple colors on packaging left some participants unsure if they should pick the main
color, background color, or color of the text on the packaging. As a result, this question was
dropped from the specifications.
New hallucinogen questions were also of special interest in Phase 2 interviews.
Previously, these questions were housed within the special drugs module. The redesigned
questionnaire included them in the hallucinogens module. A primary objective of testing these
items in Phase 2 was to determine the extent to which participants would be familiar with these
drugs. Table 4 shows the number of participants who had heard of each of these drugs. As can be
seen in the table, participants were most familiar with ketamine.
21
Table 4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens
Number of Participants Who Had
Heard of the Drug
14
8
9
Type of Hallucinogen
Ketamine (LS01i)
DMT, AMT, or Foxy (LS01j)
Salvia Divinorum (LS01k)
All of the participants who were routed to a new item about medical marijuana (MJMM)
demonstrated an understanding of the concept of medical marijuana that was consistent with
what was intended in the question.
4.3
Changes Implemented
4.3.1
Phase 2, Round 1
After Phase 2, round 1 interviews, RTI and SAMHSA met to discuss findings of the
interviews. Decisions were made with regard to what changes were needed and where further
attention should be placed during the next round of interviews. A bulleted list of the changes is
provided, along with a brief review of the reason for each change.
•
QI21B—It was learned in cognitive interviewing that this item had the potential to
confuse respondents since the categories carried over from another income branch
question that respondents who are administered QI21B would not see. Categories were
renumbered starting at 1 to be consistent with other items in the instrument.
•
INTRO1—A sentence about the respondent being able to read the questions in the
ACASI system was included in this item. It was deemed during the meeting between RTI
and SAMHSA that it was unnecessary. It was subsequently removed.
•
HLTH04—It was determined in cognitive interviews that some respondents might have a
difficult time answering a question about their height in the formats provided (feet and
inches or meters and centimeters). Two new options were added to the question. These
options allow respondents to indicate a preference to answer the question about height in
only inches or only centimeters.
•
QD26—During the meeting after Phase 2, round 1, SAMHSA and RTI agreed that the F2
options for more information could be improved by tailoring them to the question on
which they appear. The more information option for QD26 was tailored to indicate that a
respondent could ask for more information about the types of work to include when
answering about working in the past week.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, SVYMOTIV, and
STYMOTIV)—During the meeting after Phase 2, round 1, RTI and SAMHSA reviewed
the questions about motivations for misuse of prescription drugs. Resolutions were
reached on question wording and logic. Several decisions were made on the question
logic. These include the decision to ask relevant motivation questions (PRYMOTIV,
TRYMOTIV, SVYMOTIV, and STYMOTIV) of all respondents who reported
prescription drug misuse in the previous 12 months, the decision to ask about a main
reason when respondents reported multiple motivations for misuse (PRYMOT1,
22
TRYMOT1, SVYMOT1, and STYMOT1), and the decision not to allow respondents to
select the "other" reason as their main reason for misuse.
•
"Not all forms" language—During Phase 2, round 1 interviews, issues continued to be
found with the phrasing of the concept that "not all forms" of drugs are displayed in the
drug images. It was decided during the meeting after Phase 2, round 1 that revised
wording would be tested for Phase 2, round 2. The change can be characterized as one
from negative phrasing ("not all forms…are shown") to a more neutral phrasing ("some
forms…may look different").
•
PRIPROX and PRIPROXR—To reduce burden for respondents who previously
completed an interview or proxy interview, it was decided to recreate filter questions.
Two items were created. One allows the field interviewer (FI) to indicate the respondent
previously completed the tutorial, if they are aware of this fact (PRIPROX). If the FI is
unsure or cannot recall whether a respondent has previously completed the tutorial, a
question is asked of the respondent (PRIPROXR). A response indicating previous
completion of a tutorial routes the respondent to a transition screen where the FI informs
the respondent about the following section (TOPROXY).
4.3.2
Phase 2, Round 2
A number of changes to question wording, question logic, and response options resulted
from the Phase 2, round 2 interviews as well.
•
QD11—RTI learned in cognitive interviews that the order of categories could be
confusing to some respondents. Categories 12 and 13 were reversed so that category 12
now represents a high school diploma and 13 means 12th grade but no diploma.
•
CGCOLOR—About half of cognitive interview participants were unable to understand
the question was asking about the new color code system cigarette manufacturers have
implemented. It was decided that this question should not be included in the
specifications.
•
INTROPR (introduction to the pain relievers screener, which is the first prescription drug
module)—Through interviews it was determined that it was unclear to participants that
prescription drug questions were asking about any kind of use. To emphasize that point,
the word "any" was inserted in the first sentence of introductions to prescription drug
modules (INTROPR, INTROTR, INTROSV, and INTROST) so that it reads "These next
questions are about any use of prescription [drug type]."
•
PR06 (use of pain relievers containing morphine)—Several participants were unclear that
nonpill forms of morphine should be included in their response. The SAMHSA/RTI
instrumentation team added to this screen the reminder to include all forms of the pain
reliever that one has used. It was also decided that an additional image showing a vial of
morphine would be added to the screen. Including this image should prompt respondents
to include liquid forms of morphine they received by injection or intravenously.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, and SVYMOTIV)—A
participant responded that he misused a prescription drug to help him deal with his
emotions. None of the existing response options were appropriate. A response category
"to help me with my feelings or emotions" was added.
23
•
Other reasons for misuse (PRMOTOT, TRMOTOT, SVMOTOT, STMOTOT)—It was
unclear to a participant that the instruction involved typing in the "other reason" for
misusing a prescription drug. The word "other" was inserted into the sentence so that it is
clear that respondents should type in the reason they were thinking of when they
indicated another reason in PRYMOTIV, TRYMOTIV, SVYMOTIV, or STYMOTIV.
•
Other reason is the main reason for misuse (PRYMOT1, TRYMOT1, SVYMOT1, and
STYMOT1)—It was previously decided not to allow respondents to indicate that the
other reason was the main reason for their misuse. However, some respondents chose to
list the other reason as their main reason. It was decided to add a category "the other
reason I reported" to account for this.
•
DRPR—This question asks respondents to think about the pain relievers that they
previously reported misusing. Respondents with heavy levels of prescription drug use
were either not clear which pills were pain relievers or which they reported using versus
misusing. The question was edited to include a list of misused prescription pills for
reference. Wording was also edited to consistently refer to misuse.
•
QI12AN and QI12BN—Some cognitive interview participants had difficulty
understanding the types of welfare or public assistance to which these questions referred.
A bulleted list of the types of assistance respondents should include in their answers was
added to this question.
•
INTRTINN—This question was previously administered by an interviewer. Wording in
that question referred to the sources of income "that we just talked about." This wording
was changed, in case it was awkward to use when the computer is reading the questions.
•
Proxy tutorial—During cognitive interviews, it became clear that it was burdensome to
have respondents answer all of the ACASI tutorial items in their interview if they
previously served as a proxy for another respondent. As a result, an abbreviated tutorial
was created for use in the Questionnaire Field Test. This tutorial will ensure that
respondents are presented with information they did not previously receive when they
served as a proxy. Table 5 shows the screens that are a part of the full and abbreviated
front-end and back-end tutorials.
•
Tutorial transition—During Phase 2, interviewers observed that there was no screen for
transitioning the computer from the interviewer to the respondent. Additional instructions
were needed for interviewers to turn the computer over to the respondent when a proxy
was not used in the interview. Such instructions were added to HINSINT, which was
renamed from NOPROX.
•
Calendar—Where appropriate, the SAMHSA/RTI instrumentation team added reminders
for the respondent to press a function key to access either the 30-day or the 12-month
reference date calendars. These reminders were added to most screens in the instrument
that asked the respondent to report behavior in the past month or year. Where reminders
were not included, the calendars themselves were added.
•
INTROINC—In the final paragraph, this item informs respondents that the upcoming
questions refer to the past calendar year rather than the past 12 months that were referred
to in previous questions. However, this would be confusing to proxy respondents because
they were not administered a battery of questions using the 12-month reference period.
24
Skip logic was added to the item to ensure that proxy respondents are not shown the
paragraph referring to a calendar year.
Table 5.
List of Screens Presented for the Full and Abbreviated Tutorials
Full Front-End Tutorial
IntroAcasi1
IntroAcasi3
IntroAcasi4
IntroAcasi2
HeadPhone
INTRO1
INTRO2
Abbreviated Front-End
Tutorial
Back-End Tutorial
IntrAcasi1b
IntrAcasi3b
IntrAcasi4b
IntroAcasi2
Nohphone
INTRO3
GOTDOG
EYECOLOR
ALLAPPLY
NUMBER
BACKUP
PLAYINFO
rangeerr
calendar
calendr2
calendr3
ANYQUES
EYECOLRB
ALLAPPLY
NUMBERB
BACKUPB
PLAYINFB
Rangeerr
Calendar
calendr2
calendr3
ANYQUES
ANYQUES2
25
This page is intentionally blank.
26
5. Discussion
5.1
Summary of Major Changes
The redesigned instrument was tested across a total of six rounds and variations
throughout the redesign pretesting procedures; these rounds included one round of usability
testing, three rounds of cognitive interviewing to test redesigned prescription drug modules, and
two rounds of instrument cognitive interviewing for the total redesigned instrument. Usability
testing was conducted in October 2009. Cognitive interviewing was conducted from January
2011 through February 2012. Findings from these pretests informed many questionnaire
changes. The major changes included
•
deciding upon the "top of screen" layout, use of the wording "in any way not directed by
a doctor" to describe misuse, and making respondents aware of the three misuse criteria
wording for the prescription drug questions;
•
finalizing new wordings to reference generic drugs and that not all forms of prescription
drugs may be pictured on a given screen;
•
removing questions about hospital use of prescription drugs;
•
improving the display of the calendar icon that reminds respondents that the electronic
calendar is available through a function key;
•
refining questions in the health module about height and weight for ease of selfadministration; and
•
adding a transition to the proxy tutorial.
A crosswalk of question text that is included in the main NSDUH questionnaire and the
pretesting questionnaires and ultimately is recommended for inclusion in the field test
questionnaire is included as Attachment F. This document serves as a reference to track question
development across the pretesting phases.
5.2
Usability Testing Advantages
Conducting a series of usability tests followed by increasingly in-depth cognitive
interviews allows for a detailed, iterative examination of potential questionnaire problems. The
usability testing identified early on that respondents did not have problems with the screen layout
and that people understood the purpose and function of the onscreen pictures. One advantage of
conducting a small round of usability testing was that it allowed the study team to eliminate
some errors that might have burdened cognitive interviewing and clouded the findings of those
interviews. A significant recurring theme of comprehending "generic" medications was first
observed in the usability testing. Cognitive interviewers, as a result, knew to probe the questions
about generic drugs for additional issues in subsequent rounds.
27
5.3
Recommendations for Future Pretests
Given the advantages of the usability testing, RTI recommends including it in future
rounds of pretesting. Usability testing is especially adept at blending testing of new equipment
and screen layouts and features. Now that the reference date calendars and drug images are part
of the CAI instrument, there may be usability implications of transitioning to new hardware.
With the addition of a new tablet device that has the potential to display show cards and other FI
materials, usability testing is recommended for any respondent and FI interaction with the tablet.
The recurring themes of the redesigned cognitive and usability testing suggest the need to
investigate the following issues for the field test:
•
The prescription drugs sections are longer. However, use of prescription drugs in the past
12 months was a criterion for recruiting cognitive interview participants. Consequently,
the administration time for the prescription drug questions and the overall administration
time may be shorter with a broader cross-section of respondents in the field test.
Nevertheless, it will be important to track the average interview times during the field test
to identify sections that may be still too long.
•
It will be important for field test interviewers to report notes and experiences on the
proxy transition into the second ACASI portion of the interview. This is a situation that is
likely to be dependent on many variables that are present within the field (rather than lab)
setting.
•
It will be important for the instrumentation team to track reports of usability concerns
with the electronic calendars.
•
Some questions that were administered smoothly in the cognitive testing lab may be
awkward with the wider variety of respondents in the field. Field interviewers will be
encouraged to note awkward wording and choppy or missing transitions.
Overall, iterative rounds of pretesting, combinations of usability testing and cognitive
interviewing, and revisions to instruments and protocols in response to findings have resulted in
an instrument that is well positioned to be tested in a field setting.
28
6. References
Office of Applied Studies. (2008, February 14). The NSDUH Report: Use of specific
hallucinogens: 2006. Rockville, MD: Substance Abuse and Mental Health Services
Administration.
29
Appendix B
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 1 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 1 Results
I.
Highlights
Respondent Characteristics
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain relievers.
Some reported noticing behaviors that concerned them, like needing to take more
medication because the drugs were having less effect for pain relief. They
stopped taking the medicine, lived with the pain for a while, and then would
resume taking the medicine as prescribed. Others in this group reported not liking
the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Identification of Prescription Drugs
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same question.
Some participants in particular reported that their doctors would write the brandname drug on the prescription but the pharmacy would substitute the generic.
Therefore, one issue for remaining rounds would be assisting persons in reporting
February 14, 2011
2
what they actually took, regardless of the name of the drug that was written on a
prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular color.
Additional options also may need to be considered for clarifying for respondents
that pictures may not show all possible examples of a particular pill, or forms
other than pills.
Performance of the Nonmedical Use Criteria
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for stimulants.
An issue for persons who obtained drugs outside of the United States without a
prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
February 14, 2011
3
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
MJ01 – Most participants were able to correctly identify what a blunt was.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
February 14, 2011
4
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
thinking that Vidodan and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
February 14, 2011
5
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
PR03 – One person used Darvocet in the past 12 months.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
February 14, 2011
6
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
PR10 – No one used any of these pain relievers.
February 14, 2011
7
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
ST02 – None of the participants had used any of these drugs.
ST03 - None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
February 14, 2011
8
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
February 14, 2011
9
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
February 14, 2011
10
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
February 14, 2011
11
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
February 14, 2011
12
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
February 14, 2011
13
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
February 14, 2011
14
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
February 14, 2011
15
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
February 14, 2011
16
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
February 14, 2011
17
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
February 14, 2011
18
Appendix C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 2 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 2 Results
I.
Highlights
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
May 9, 2011
2
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
May 9, 2011
3
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
May 9, 2011
4
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
Round 2: No probe.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
MJ01 – Most participants were able to correctly identify what a blunt was.
Round 2: No probe.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
May 9, 2011
5
Round 2: No probe.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens
will become a problem here, we should ask them to specify which “other” hallucinogens they
have used. However, use of drugs other than hallucinogens would not be determined until the
May 9, 2011
6
data are processed. The NSDUH data editing procedures since 1999 continue to treat
respondents as hallucinogen users, even if they specify use of drugs other than hallucinogens.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
May 9, 2011
7
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it
is not clear whether people who have inhaled these products will be able to identify the
substance by the examples given, and that people who have inhaled other aerosol products
(but not these) will be able to answer correctly that they have not used these inhalants. Google
results show that this type of product also is referred to as gas duster and the brand name Dust
Off. We should not continue to edit this question without feedback from users of aerosol
products about whether these are appropriate examples. Feedback from users of aerosol
products also would help to evaluate whether these users can correctly distinguish between use
of these and other aerosol products.
IN01l: Based on observations, SAMHSA requested that the question be revised so
that it no longer references substances that were “listed.” (This revision also
applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
May 9, 2011
8
thinking that Vicodin and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being
shown on the screen.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
May 9, 2011
9
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
PR03 – One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
May 9, 2011
10
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
May 9, 2011
11
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the
module. Perhaps it should read: “Remember, not all forms of these pain relievers may be
shown on the screens.
PR10 – No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
May 9, 2011
12
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
PRHOSPYR1 – Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in
where you used this (these) drug(s). This might draw more attention to the fact that the
question is not just confirming earlier reports of the drug. Another alternative would be to
ask respondents whether they used any of these drugs when they were in the hospital
(rather than “only” when they were in the hospital). If use of specific prescription drugs in
the hospital is reported, ask which drugs the respondents took when they were not in the
hospital. Questions to capture information about use of prescription drugs only in the
hospital may be dropped if respondents continue to have difficulty answering them.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
May 9, 2011
13
ST02 – None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
ST03 - None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
Round 2: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
May 9, 2011
14
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
May 9, 2011
15
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
May 9, 2011
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
16
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
May 9, 2011
17
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
May 9, 2011
18
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
May 9, 2011
19
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
May 9, 2011
20
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
May 9, 2011
21
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the
current versions of these questions for SAMHSA to use in evaluating categories for
combination or deletion.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
May 9, 2011
22
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
Phase 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
May 9, 2011
23
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
Round 2: See HLTHNEW01 results for recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
May 9, 2011
24
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
May 9, 2011
25
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order of
the health conditions in this question.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes:
Round 2: SAMHSA observers gave a reminder for RTI cognitive interviewers to be
sure to administer spontaneous probes in the cognitive interviewing in situations
where interviewers think it is necessary.
Informed Consent Statement:
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
May 9, 2011
26
likely to be small, the IRB may still require prospective respondents to be informed of
this.
May 9, 2011
27
Attachment C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 3 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 3 Results
I.
Highlights
General Themes (All Rounds)
Listed below is a summary of overall themes that emerged from all three rounds
of cognitive interviewing; these themes are not necessarily in the order of
question administration. Specific findings are discussed in further detail in the
remainder of the report.
Participants generally were able to recognize the prescription drugs by name,
by using the pictures, or by using both.
Although more questions were required, asking separately about any use of
specific prescription drugs in the past 12 months and nonmedical use of the
drugs that were used in that period simplified the cognitive task for
participants.
There was some uncertainty about the “generic” term associated with some
drugs. Addition of the word “generic” to the pill images in Round 3 appeared
to be helpful.
The wording “in any way a doctor did not direct you to use it” differentiated
between medical and nonmedical users of prescription drugs. With few
exceptions, participants who used prescription drugs only for medical reasons
could determine that they did not use prescription drugs “in any way a doctor
did not direct you to use (them).” Similarly, nonmedical users could determine
that their use constituted use “in any way a doctor did not direct you to use it,”
even if they used prescription drugs in ways not explicitly listed as examples
of nonmedical use.
Further testing is needed for questions about use of canned air as an inhalant
and injection of stimulants.
Revision of the questions about height and weight (HLTHNEW01 and
HLTHNEW02 series) following Round 1 helped to improve participant
understanding and the task of answering these questions.
For questions about the number of outpatient doctor visits (HLTHNEW03 and
HLTHNEWDK), participants had some difficulty in determining what did or
did not constitute a doctor visit, and in determining their number of visits.
August 17, 2011
2
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
Respondent Characteristics (Round 3)
Overall, 12 respondents participated in Round 3, including 3 adolescents and 9
adults. Each of these participants was recruited from the general public. A few of
the participants were heavy drug users. Two reported use of multiple pain
relievers in the past 12 months, although one of these participants reported
August 17, 2011
3
misusing only one pain reliever. Another participant was a recovering heroin
addict who was intimately familiar with a wide variety of prescription drugs.
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Identification of Prescription Drugs (Round 3)
Respondents were able to identify most of the drugs either by name or picture.
Consistent with findings in earlier rounds, participants often recognized the drugs
by name, although some did find the pill images to be helpful. The Round 3
instrument added the word ‘generic’ to the pill image in order to help respondents
August 17, 2011
4
understand this term. Many respondents understood that generic forms of pills are
not the name brand, and are often cheaper. Some respondents thought that these
generics were “weaker.” Some respondents reported being confused about what
the specific generic pill was a generic of, but seemed to understand that pills had
generic forms and that pharmacies would occasionally substitute generics for
brand name drugs.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
August 17, 2011
5
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
Performance of the Nonmedical Use Criteria (Round 3)
Most respondents were able to differentiate between medical and nonmedical use.
Two respondents misreported their use as medical when it was not. One
respondent took a pill for longer than he had been told to take it, but did not
initially think of that as nonmedical use. Another participant did not think that
she had misused a pill because it was prescribed, because so many of the pills that
she takes are not prescribed. However, she took the pills more often than was
prescribed, and therefore reported misuse. One respondent reported using
“Tylenol 3” without a prescription, but he had really used over-the-counter (OTC)
Tylenol.
While decomposing her use, one participant reported that she had misused the
pills by taking the pills in greater amounts than prescribed and more often than
prescribed. Upon probing, it was discovered that she was describing the same
behavior.
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
August 17, 2011
6
Other Issues (Round 3)
Asking additional probes about familiarity with different names for air duster did
not reveal a term that was more familiar to participants.
The audio pauses in the middle of the questions to allow respondents to look at
the pictures of the pain relievers and stimulants. In some cases, the pause was too
long, while in other cases, the pause was too short.
Respondents did not have any problems with the F2 key, which is used to pull up
additional information about ways of misuse. On occasion, respondents used this
button to aid in their response process.
II.
Ensemble Screener Findings
CG17REV – Round 1: Of the 16 participants who responded to this question, 4
have heard of snus and 3 correctly identified what it was. The fourth person
thought that it goes up your nose.
Round 2: No probe.
Round 3: No probe.
AL01 – Round 1: Most, but not all, participants had heard of the alcopops that
were listed in the protocol. Descriptions included being fruity, girly, ready-made
and flavored. Most participants compared them to wine coolers, and 2
participants mentioned that 4 Loko is similar. These same two participants
identified Sparks as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
Round 3: No probe.
MJ01 – Round 1: Most participants were able to correctly identify what a blunt
was.
Round 2: No probe.
Round 3: No probe.
CC01 – Round 1: Descriptions of crack included: smoked, more addictive, costs
less, crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
August 17, 2011
7
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Round 1: Five participants reported use of cocaine or crack. There were
no inconsistencies in their narrative of use.
Round 2: No probe.
Round 3: No probe.
HE01 – Round 1: Only 1 participant had heard of cheese, while most had heard
of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
Round 3: No probe.
SD17a – Round 1: Almost all participants had heard of methamphetamine. They
had no other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
Round 3: No probe.
HALINTRO – Round 1: Most participants were able to correctly indentify
examples of hallucinogens. Incorrect examples included: marijuana, cocaine,
crack, heroin, alcohol, crystal meth, and Ambien. Participants included a number
of substances that can make one hallucinate, regardless of whether that substance
was a hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 3: Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included marijuana, crack, paint,
markers, hairspray, depression medication, Percocet, and weed laced with
embalming fluid.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
August 17, 2011
8
Action Item: Look at 2010 data for LS01a-LS01f and refusal follow-ups to determine the
numbers of respondents who got different LSFILL patterns.
Peggy also noted that inclusion of the three new hallucinogens will affect the Prior Substance
Use module – i.e., to add these three new substances and follow the same pattern as for LSD,
PCP, and Ecstasy. It is fine to include this in Round 2.2.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
Round 3: No probe.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens will
become a problem here, we should ask them to specify which “other” hallucinogens they have
used. However, use of drugs other than hallucinogens would not be determined until the data
are processed. The NSDUH data editing procedures since 1999 continue to treat respondents as
hallucinogen users, even if they specify use of drugs other than hallucinogens.
Round 3: No probe.
INHINTRO: Round 1: Most participants were able to correctly indentify
examples of inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
Round 3: Most participants were able to correctly identify examples of
inhalants. One reported computer spray duster.
IN01a: Round 1: Responses about a definition of ‘for kicks or to get high’
included: to escape, for a euphoric feeling, excess use, stupor, for fun or a desired
effect, recreational, another frame of mind, to feel good, get outside yourself,
heightened senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
August 17, 2011
9
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
Round 3: Definitions included for fun, trying to get a buzz, doing it
recreationally, just to do it.
Recommendation: Respondents understand this wording and it should continue to be used in
the question.
IN01e & h: Round 1: Two participants reported using an inhalant, but correctly
decided against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Round 1: Most participants had not heard about inhaling markers and
pens to get high. Many were incredulous that this actually happens, and none
reported doing it. However, 2 people had heard of this. (We may observe greater
awareness of this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
Round 3: No respondents reported inhaling markers or felt tip pens.
IN01ii: Round 1: There was a good deal of confusion about canned air. Many
participants referred to this as ‘air duster’. A Google search of ‘air duster’
revealed a number of products by that name. One participant thought that canned
air had to do with filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics air
duster” to make the meaning clearer.
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it is
not clear whether people who have inhaled these products will be able to identify the substance
August 17, 2011
10
by the examples given, and that people who have inhaled other aerosol products (but not these)
will be able to answer correctly that they have not used these inhalants. Google results show
that this type of product also is referred to as gas duster and the brand name Dust Off. We
should not continue to edit this question without feedback from users of aerosol products about
whether these are appropriate examples. Feedback from users of aerosol products also would
help to evaluate whether these users can correctly distinguish between use of these and other
aerosol products.
Round 3: Four respondents had never heard of any alternate terms for
canned air, including computer duster, canned computer duster, canned air
duster, and Dust Off. Two respondents had heard of all of these names.
Three respondents had heard of computer duster. No respondents reported
inhaling this.
IN01l: Round 1: Based on observations, SAMHSA requested that the question be
revised so that it no longer references substances that were “listed.” (This revision
also applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Round 1: Participants seemed to understand the difference between
over-the-counter drugs and prescription drugs. Some descriptions of over-thecounter drugs included: lower dose, pick it off the shelves, don’t alter your state
of mind, not as strong, can’t get high off of them, bought without doctor's
permission, and off the shelf. Prescriptions were described as: containing
narcotics, higher dose, controlled substance, bought from the pharmacist,
stronger, gives you a high, more powerful, more dangerous, easier to abuse, more
expensive, harder to get, need a prescription from a doctor with a signature on it,
narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
NOTE: Screen shots were sent to SAMHSA on May 31, 2011 that showed section headings for
pain relievers. SAMHSA made the decision on June 14 not to add headings to the CAI.
August 17, 2011
11
Round 3: No probe.
PR01: Round 1: Participants reported using a multiple drugs on this list. Two
participants reported thinking that Lorcet and Percocet were the same thing.
Another reported thinking that Vicodin and hydrocodone were the same, along
with Norco. He reported taking Norco as opposed to hydrocodone, but reported
the hydrocodone. One participant was given the pain reliever in the hospital.
Participants were able to distinguish medical and nonmedical use in their
narratives of use. For instance, one participant reported taking the pain relievers
as prescribed on the first three days, but then took nine pills in one day. He
acknowledged that he did not take them as prescribed. Participants discussed
using the pain relievers for recreation, following the instructions for use, or
stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs and
their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to determine
whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being shown
on the screen.
August 17, 2011
12
Round 3: Respondents were able to correctly identify which of these pills
they had taken in the past 12 months. They reported that the pictures and
drug names helped them with this identification.
Similar to the findings of Round 2, respondents reported two different
understandings of the sentence about not including all forms of the drug on
the screen. Three respondents correctly reported thinking that there could
be different forms, such as a liquid or a capsule or additional dosages, which
were not listed on the screen. Other respondents understood that not all pain
relievers were listed, including those that have different names.
Duplicating the labeling of generic pills on the screen appeared to increase
visibility of the term. Most respondents noticed this and most had a general
understanding of its meaning. However, respondents did not necessarily
understand that the generic on the screen was a generic for the other name
brand pills on the screen.
Respondents correctly understood the 12 month reference period.
Recommendation: Consider expanding the sentence about not showing all forms of the drugs
on the screen to provide examples of other forms, such as liquids or patches.
Add "such as different shapes, colors, or dosages." There is less worry about liquids because
respondents are likely to have gotten these only in the hospital.
PR02: Round 1: 11 participants reported using these drugs. One participant said
that he thought that Percocet and Lorcet were the same thing, but the names on
the bottles were different, so he must be wrong. All participants were able to
answer the question based just on the names, although 3 reported that the pictures
were helpful. One reported that his prescription is for Percocet, but his pharmacy
gives him generic oxycodone for insurance reasons, so he sometimes gets
confused about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
August 17, 2011
13
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
Round 3: Respondents did not have any problem identifying which pills they
had used. Five respondents had used Percocet.
PR03 – Round 1: One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
Round 3: One person used Darvocet in the past 12 months.
PR04 – Round 1: One person had used Ultram and tramadol. Another used
tramadol when he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
Round 3: One respondent had used Ultram, but in a liquid form in an IV.
Two respondents used Tramadol.
PR05 – Round 1: Seven people had used Tylenol with codeine. One participant
called this a Tylenol 3. Another reported that he also used Tylenol 4, which is the
pill with the 4 on it. The FDA's Center for Drug Evaluation and Research lists
these as "Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as Tylenol
with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
Round 3: Respondents continued to recognize the names of Tylenol 3 and 4
and to refer to the pill by these names. Six respondents reported using this.
However, one adolescent respondent was actually referring to the OTC
Tylenol and mistakenly believed that this was what was being asked.
Another adolescent respondent reported taking a red and white gel tab OTC
Tylenol. Based on the pictures that were shown on the screen, he decided not
to report this use.
PR06 – Round 1: 2 participants reported using morphine in the past 12 months.
Others reported use, but added that this was not in the past 12 months.
August 17, 2011
14
Participants distinguished between the liquid and pill forms of morphine,
reporting they had it in an IV.
Recommendation: Consider whether an image of a liquid morphine ampule should be shown in
the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
Round 3: One participant reported using both morphine and MS Contin.
The morphine was used in IV form in the hospital.
PR07 – Round 1: Participants were familiar with a number of these drugs. Some
reported lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
Round 3: One respondent used the Fentanyl patch. Three respondents
reported hearing the name China White as another name for Fentanyl. One
of these respondents reported the alternate name as White China. No
respondents had heard of any of the other names that we probed about, nor
had they heard of Sublimaze.
Two participants, when asked, said that the lettering on the package of the
Duragesic was hard to read. Others reported that the pictures were of
limited use because they knew that had not used these drugs.
August 17, 2011
15
Recommendation: Get rid of "China Girl." This is the only prescription drug
with a slang name.
PR08 – Round 1: 2 participants reported use of Suboxone. One was using it to
stop going through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
Round 3: One respondent used Suboxone and Subutex. He used one in
rehab for heroin addiction and another on his own to avoid a relapse into
heroin use.
PR09 – Round 1: 2 participants reported use of Dilaudid. One said that he used
this only in the hospital and asked if that counted. Two participants had used
methadone, at least one of which was at a methadone clinic under supervision.
One used Demerol and said that the picture helped him identify it. Finally, one
person reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12 months
(i.e., not just in pill form) should be reported in PR09. 2. Consider whether instructions need to
include reports of any medications that people took while in the hospital. However, receiving
dosages of pills in a hospital is a different experience than receiving a prescription that will be
filled for use at home. Participants may be unable to identify which pills they took in a hospital,
or to recall that they took pills at all. Do we anticipate that use of pain relievers (or specific
pain relievers) will be significantly underestimated if we do not instruct respondents to report
pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the module.
Perhaps it should read: “Remember, not all forms of these pain relievers may be shown on the
screens.
Round 3: One respondent reported being very addicted to Dilaudid.
Another respondent said that the pictures of the drugs were an excellent
confirmation to his thinking that he took this drug.
August 17, 2011
16
PR10 – Round 1: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
Round 3: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the
combination of the drug names and the pictures of the drugs aided them in
their reporting.
PR11 – Round 1: Other pain relievers that participants reported that they used in
the past 12 months included non-steroidal anti-inflammatory drugs (NSAIDS)
such as naprosyn or diclofenac (brand name Cataflam). Other drugs that were
reported included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
Round 3: Respondents reported that people might report a number of other
pills at this question, including Valium (tranquilizer), Lexapro
(antidepressant), Narco (pain reliever), Paxil (antidepressant), Celexa
(antidepressant), and neurontin (anticonvulsant). [Note: neurontin and
Gabapentin are equivalent.]
PRHOSPYR1 – Round 2: Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
August 17, 2011
17
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in where
you used this (these) drug(s). This might draw more attention to the fact that the question is
not just confirming earlier reports of the drug. Another alternative would be to ask
respondents whether they used any of these drugs when they were in the hospital (rather than
“only” when they were in the hospital). If use of specific prescription drugs in the hospital is
reported, ask which drugs the respondents took when they were not in the hospital. Questions
to capture information about use of prescription drugs only in the hospital may be dropped if
respondents continue to have difficulty answering them.
NOTE: SAMHSA made the decision to drop the hospital questions for Round 3.
IV.
Stimulant Screener
INTROST – Round 1: Most participants had heard of No Doz and Dexatrim. 3
people mentioned 5 Hour Energy as an example of a stimulant. One person
mentioned Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
Round 3: Most respondents were familiar with at least one of the stimulants
that were listed. Many of the respondents had heard of all four.
Recommendation: Keep this list of over the counter stimulants.
ST01 – Round 1: One participant replied that he was not sure which one to
report, because Dexedrine is the brand name, used by the company while
Dextroamphetamine is the chemical name. He was also familiar with the mix of
amphetamine and dextroamphetamine. One participant reported Vyvanse here as
a dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
August 17, 2011
18
Round 3: One respondent used Adderall while another used Adderall XR.
These respondents had no problem answering the question.
ST02 – Round 1: None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
Round 3: One respondent used Concerta in the past 12 months. He stated
that the coating on it made it difficult to dissolve under the tongue.
ST03 - Round 1: None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST04 – Round 1: Two of the participants used Phentermine. One bought it in
Mexico, while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST05 – Round 1: One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
Round 3: One participant used Vyvanse for ADHD.
Some respondents reported that drug names were helpful while answering
these questions. Others reported that the pictures were helpful because the
names were really long. There were no pictures that were not helpful.
August 17, 2011
19
ST06 – Round 1: One participant reported getting Meridia in Mexico. (NOTE:
This is no longer available in the US).
Round 2: No others were reported.
Round 3: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Round 1: Participants did not report using any stimulants in their
lifetimes. One participant later reported that her son was prescribed medicine for
ADD. She took one of his pills so that she could see how it affected him.
However, she neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
Round 3: In response to the question about lifetime use of stimulants, one
respondents noted that we did not include Adipex (which is a form of
phentermine).
A couple of respondents noted that it would be helpful for the numbers on
the pill images to be bolder and more distinct.
V.
Pain Reliever Module Findings
PRL01- Round 1: In describing examples of using a prescription pain reliever in
a way not directed by a doctor, two participants mentioned using a prescription
pain reliever with other drugs or with alcohol. One specifically reported taking
pain relievers with a glass of wine because “they just work better” that way.
Otherwise, participants described ways of use not directed by a doctor that are
included in the definition, such as:
August 17, 2011
20
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining pain
relievers with other drugs or alcohol as a way of using pain relievers that is not directed by a
doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
Round 3: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
PRY01 – Round 1: All participants routed to this question about Vicodin
understood that we were asking this question to get at abuse of prescription pain
relievers or getting them off the street. Many participants used the word “abuse”
when asked what this question was asking. Examples of such behavior include
not using it as prescribed, taking it without a prescription, ‘over doing it,’ getting
high, or selling them. When asked to provide a narrative of the last time
participants used Vicodin, two participants described taking a prescribed medicine
more than the doctor said to because it was needed to make the pain better. A
third participant described having a prescription for dental pain, but using it more
than prescribed because of liking the feeling it caused. Another had a prescription
but was abusing the Vicodin and Norco highly at the time. One participant
mentioned taking their prescription with alcohol. Two participants mentioned
using it without a prescription. One of these used it only as a last resort because
nothing else was available to feed an addiction.
August 17, 2011
21
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them to
use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
Round 3: In response to a probe about what the question was trying to measure,
three respondents said it was to measure abuse. Other mentions include, “using it
without doctor’s orders,” “not taking it any way to cause me to become dependent,”
and “not taking it for other things.” Other responses include:
“If you didn’t follow your prescription”
“If someone gives you their Percocet, if you take it more often or for fun”
“Are you using a prescription drug that wasn’t prescribed by your doctor or
any doctor?”
Respondents who took Vicodin in the past 12 months reported a combination of use
and misuse. Some respondents took pills in greater amounts than prescribed
because of the pain.
PRY01a – Round 1: Participants reported numerous mechanisms by which they
remembered their age at first use, including:
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
August 17, 2011
22
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
Round 3: One respondent reported just remembering the age of first misuse.
PRY02b – Round 1: One participant chose the year because it was not in 2011,
the past couple of weeks. Another just knew the year was 2009.
PRY01d – Round 1: One participant reported that the calendar helped with
choosing the month.
PRY02 – Round 1: The one participant who was asked what this question about
Lortab was getting at reported, “the addictiveness of prescription drugs,” and
“how easy it is to abuse them.” Two participants were asked to describe the last
time they used Lortab. One reported using someone else’s prescription and the
other reported using it when heroin was unavailable.
PRY02a – Round 1: The two participants who received this question distinctly
remembered events associated with the first time they used Lortab (finding
grandmother’s pills and hanging out with friends). One reported that it was
someone else’s prescription and the other reported snorting it.
PRY02c – Round 1: One participant realized he had the year wrong when
answering this question. He mixed up how old he was last summer.
PRY02d – Round 1: The participant remembered the month was July or August
because of how hot it was outside.
PRY03 – Round 1: The one participant who was asked what this question about
Lorcet was getting at described snorting it as using it a way it was not prescribed,
so as to get an immediate high. Two participants answered this question as “no”
because they used this as prescribed, following directions as needed for pain.
PRY04 – Round 1: Two participants were asked to describe what this question
about hydrocodone was getting at. One reported using it recreationally, not
following the doctor’s instructions, taking more than necessary, or snorting it.
Another described a new type of misuse: asking for a larger dose at time of
prescription with the knowledge that a large dose was not really necessary. This
participant still endorsed use of hydrocodone “in a way a doctor did not direct you
August 17, 2011
23
to use it.” When asked to describe the last time the pill was used, one participant
reported getting it for a sore throat in the ER and another reported getting the pills
from an acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as examples
of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
Round 3: Two respondents had used hydrocodone in the past 12 months.
One respondent received a pill from his mother. Another was abusing the
drug because of addiction.
PRY04a – Round 1: Participants did not have trouble reporting their age,
although one wanted to see a calendar for month of last use. When asked how
they were misusing the pills, one reported taking 9 in one day and another
reported taking it with alcohol. A third participant in answering this question
expressed some confusion about Vicodin and hydrocodone. This person reported
thinking that they were the same thing and appeared to be confused by the fact
that we asked about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth date.
It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
Round 3: Respondents did not have any problems remembering their age at
first misuse.
PRY05 - Round 1: One participant described using OxyContin with a
prescription. Her mother kept it locked away and gave it to her at allotted times;
otherwise, she would have abused it. Another participant reported using it once
when a friend provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
Round 3: One respondent took this as directed.
August 17, 2011
24
PRY05a – Round 1: One participant remembered the age at first use easily
because it was in the summer, “after my birthday.” Another had a hard time
determining whether it was at age 19 or age 20. It was after high school but
before college. When using OxyContin at that time, one participant reported
using someone else’s prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
Round 3: Another participant associated use of this drug with a relationship
that he had. When he started using this, he was injecting it.
PRY06 – Round 1: Participants who were asked about what this question about
Percocet is getting at understood it to mean abuse. When asked about how they
used it in a way not directed by a doctor, one reported popping and snorting them,
but with a prescription. Another reported using another’s prescription. One
participant used it but did not report using it in any way not directed by a doctor.
The Percocet made this person sleepy and nauseated, and they did not care to
finish the whole prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
Round 3: Five respondents reported using this prescription. One
respondent did not report misuse when he took one pill for pain that had
been prescribed to him for an earlier procedure. He did not report this
because the bottle said the pills were for pain, and they had been prescribed
to him.
PRY06a – Round 1: Participants remembered their ages by associating it with
events in their lives: sophomore year in college, a friend visiting from Florida, the
diagnosis of migraines. Another had no idea of the age at first use. Participants
reported first using Percocet in a way a doctor did not direct them to use it by
using it without a prescription, mixing it, popping it, and taking it “more than I
should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
August 17, 2011
25
Round 3: One participant had some trouble remembering her age at first
abuse. She estimated the age.
PRY09 - Round 1: When asked what this question about oxycodone is getting at,
one participant who did not answer affirmatively said she was scared of it and
asked, “Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
Round 3: One respondent reported using this as directed. Two others took
this without a prescription.
One respondent asked for a calendar. She did not notice the calendar icon
on the top of the screen.
Recommendation: Make the icon for the calendar bolder or more visible.
PRY09a – Round 1: Participants did not have trouble remembering their ages,
but one asked, “Do you really think that people remember the months and the
years?”
Round 3: One participant estimated her age at first abuse.
PRY10: Round 3: One participant reported taking Darvocet as directed.
She stated that she did not abuse this.
PRY13: Round 3: One participant reported taking Ultram as directed. She
stated that she did not abuse this.
PRY17: Round 3: Three participants had used Tramadol in the past 12
months. One received a prescription for a broken wrist, one received a
prescription for pain, and the third did not have a prescription. Neither
respondent with prescriptions abused the drug.
PRY18 – Round 1: When describing use of Tylenol with codeine “in any way a
doctor did not direct you to use it,” examples included: using someone else’s
prescription because their hydrocodone ran out and a friend gave them something
to tide them over (stave off withdrawal) and taking more than prescribed.
August 17, 2011
26
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
Round 3: Six respondents had used Tylenol with codeine. One respondent
noted that he did not think that he had misused it because he had a
prescription, but after looking at the ways of misuse he was taking more than
were prescribed.
PRY18a – Round 1: Participants did not have trouble remembering their age of
first use.
Round 3: One participant said that she did not know her age at first misuse.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY22 – Round 3: One participant got this prescription from a doctor and
took the pills as directed.
PRY24 – Round 1: One participant described using an IV drip of morphine in the
hospital, and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
Round 3: One respondent received a couple of morphine pills from a friend
and took them all at once.
PRY24a – Round 3: This respondent did not have any trouble remembering
age at first abuse.
PRY28 – Round 3: One respondent received the fentanyl patch at the
hospital.
PRY28a – Round 1: One participant in particular had trouble remembering when
he turned specific ages for first use of fentanyl “in any way a doctor did not direct
you to use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – Round 1: A participant described their use of Suboxone “in any way a
doctor did not direct you to use it” as a backup for when this person could not get
August 17, 2011
27
into a methadone clinic. This made it relatively easy for this participant to recall
the age and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
Round 3: One participant used this in the methadone clinic and then
subsequently bought it from a drug dealer to help with heroin addiction.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – Round 1: A participant who answered this question affirmatively
recalled using Demerol when he had run out of codeine and/or Vicodin. A friend
with chronic pain shared the Demerol with him to extend his prescription. He
was able to recall his age by the month and year and by his birth date.
Round 3: One participant used the Demerol as prescribed.
PRY33 – Round 1: A participant remembered using Dilaudid in the hospital as
an injection for back pain. He reported using this according to doctor
instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
Round 3: This participant goes to the hospital to get Dilaudid, because she is
addicted to it. If she cannot get Dilaudid, she will seek out other drugs.
PRY33a – Round 3: The first time she used it, she had a prescription but she
used more than that.
PRY34 – Round 1: A participant routed to this question answered it as “no.” The
person had not used methadone “in any way a doctor did not direct you to use it”
because the person attends a methadone treatment program every day and
receives the methadone there as directed.
PRL02 – Round 1: One participant reported using a pain reliever with alcohol as
a type of use not directed by a doctor in response to this question.
PRM02DKRE – Round 1: The participant who received this question about their
“best guess” thought their answer was accurate.
August 17, 2011
28
PRM03 – Round 1: Two participants reported how they were able to remember
whether or not they were drinking alcohol. One said because it was yesterday,
and another knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
Round 3: No respondents had taken the pills and drank at the same time.
One respondent was not sure how to report this, because she drank within a
week of taking the pills and was not sure how long alcohol stays in your
system. She ultimately decided to answer ‘no’.
PRY41 – Round 1: One participant understood this question but volunteered that
the wording “at least once” threw her off. She thought that the answers would say
the same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
August 17, 2011
29
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
Round 3: One participant reported “some other way.” She included this
because she injected the drug instead of taking it by mouth. She also picked
the other four response options. After probing, she stated that she was
counting the same behavior of taking more drugs than were prescribed as
both number 2 and 3.
PRY42B – Round 1: Participants did not have trouble remembering how they got
pain relievers the last time. One additional way of getting a prescription pain
reliever reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the current
versions of these questions for SAMHSA to use in evaluating categories for combination or
deletion.
Round 3: One respondent said that she got it from more than one doctor
because she would get her prescriptions at the ER, and there are multiple
doctors that prescribe them.
VI.
Stimulants Module Findings
STL01 – Round 1: A participant reported not having a prescription for
phentermine, but in Mexico, they did not need one. This person pointed out that
you can buy prescription drugs in Mexican drug stores and airports. Technically
in Mexico they were not prescription drugs.
August 17, 2011
30
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in any
way a doctor did not direct you to use it,” and how to instruct respondents to count or not count
this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – Round 1: When describing last use of Adderall, a participant reported
that a friend provided it in 2009. There was no prescription and they were mixing
it with other things.
Round 3: The participant answered “no.” The participant had a
prescription for ADHD and last took Adderall this morning, according to the
prescription.
STY01a – Round 1: A participant reported remembering the exact night in 2009
hanging out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY02 – Round 3: The respondent took Adderall XR for ADHD. He had a
prescription.
STY03a – Round 1: A participant remembered their age at first use of Dexedrine
very well because it was associated with a recent move and new
classmates/friends.
STY08 – Round 3: This respondent did not abuse Concerta.
STY20 – Round 1: The same participant who got phentermine in another country
described it here. Another participant reported using phentermine strictly to lose
weight, as directed, and answered STY20 as “no.”
STY25 – Round 1: The same user of phentermine mentioned getting it outside
the U.S. Another participant reported snorting as a third way of use.
Round 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY25a and STY25b: No respondents in any of the three rounds reported
using stimulants with a needle in the past 12 months.
August 17, 2011
31
STY26b – Round 1: One participant had been prescribed stimulants for ADD
while in rehab. When the participant emerged from rehab, this person told the
doctor about having a prescription for stimulants and was able to get another
prescription easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
VII.
Health Module
HLTHNEW01 – Round 1: 5 participants had difficulty entering their height into
the computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom of
the task bar. Participants also did not read the screen about entering inches. They were still
looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
Round 3: No respondents had any difficulty entering their height or weight.
Recommendation: Questions are functioning properly. Keep as is.
HLTHNEW01a – Round 1: Participants were not accustomed to seeing the
question mark that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – Round 1: No problems entering weight. See HLTHNEW01
results for recommendations.
Round 2: See HLTHNEW01 results for recommendations.
Action Item: Do not play the audio for the answer choices for height and weight because those
are included in the questions.
HLTH05 – Round 1: One participant reported 8. Upon probing, he said that he
was in the hospital for 8 days and 7 nights and had missed the part of the question
that specified nights.
August 17, 2011
32
HLTHNEW03 – Round 1: A few participants included dental appointments in
their estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
HLTHNEW03 – Round 1: Most participants had a reasonable understanding
about what a health care professional was. One participant would not count a
chiropractor, or a person at the eye care center because they have not been to
medical school. One participant said that there were none at the drug treatment
center, while another reported that visits to the methadone clinic would count.
Other participants expressed doubt about including specialists and physical
therapists. One participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a number
of constructs, such as a reference period, thinking about your own health versus someone else’s,
the definition of a health care professional, and exclusions to the rule. The respondent burden
has the potential to be high, as does the measurement error.
Round 3: Depending upon the extent of visiting doctor offices, respondents
struggled with accurately counting the number of visits. One respondent
initially included dental visits despite the instruction not to.
Action Item: Delete the explanatory text from HLTHNEW03. Consider moving the
instructions to appear before the question, and include a probe.
HLTHNEW05 – Round 1: There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows, but
there should be a linkage between the series of questions.
HLTHNEW06 – Round 1: One participant reported that his dentist asked him
about his alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
August 17, 2011
33
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04 because
the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06 focuses
only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
Round 3: Respondents did not have a problem with this question. Many
reported that doctors asked if they drink and then how often and how much.
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order
of the health conditions in this question.
HLTHNEW11 & 12a – Round 3: The respondent was first diagnosed at their
current age, but the follow up question asked whether they had cancer or
heart disease in the past 12 months.
Recommendation: Change the specs to skip questions about whether respondents had specific
conditions in the past 12 months if the age at first diagnosis equals their current age.
Action Item: Implement this change.
HLTHNEW20 – Round 1: One participant asked if we were referring to the past
12 months or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – Round 1: All three participants who reported high blood
pressure were confused by this question. One participant said that he has been
told that it was a little high, but was not sure that this was a chronic state. One
participant said that after the first time, the doctor knew that he had it and they
August 17, 2011
34
talked about it, but he didn’t tell him that he had it again. He said that he was on
medicine that was controlling it. The third participant said that they discuss his
high blood pressure, but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - Round 1: There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an awkward
stopping point, and does not seem to flow after the last few questions. Recommend adding
another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – Round 1: After the stimulant main module, one participant
reported that many of the questions are redundant. However, no other participants
made any comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes: Round 2: SAMHSA observers gave a reminder for RTI cognitive
interviewers to be sure to administer spontaneous probes in the cognitive interviewing
in situations where interviewers think it is necessary.
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
likely to be small, the IRB may still require prospective respondents to be informed of
this.
Round 3: No additional issues.
August 17, 2011
35
Attachment D
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 1 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.1 Results
December 7, 2011
Respondent Characteristics
During the first round of Phase 2 interviewing, survey methodologists interviewed twenty
respondents. Of these twenty, ten were adolescents and ten were adults. Almost all respondents
in the first round of Phase 2 interviewing were recruited from the general population. A couple
of respondents were recruited from drug treatment centers in the Raleigh, North Carolina area.
In order to be recruited in the cognitive interview sample, respondents did not necessarily need to
report use of illicit or prescription drugs in the past 12 months during the telephone screener.
Therefore, most respondents were casual users or often non users of the drugs that were tested in
this round. Few were expert or heavy users.
Round 2.1 tested the transition from the questionnaire asking health insurance and income items
using interviewer administration to using a self-administered mode. Particularly, transitioning to
a proxy respondent who completes the health insurance and income items on behalf of the
respondent using self-administration warranted thorough testing. Interviewers and parents
provided valuable feedback about this process.
In general, proxy respondents were able to successfully answer questions about adolescent health
insurance and family income. The ACASI tutorial worked well, and proxy respondents reported
understanding the questions as well as why they were being asked to answer them. Individual
findings are listed below.
In addition, new questions measured details of military service and status. In order to thoroughly
test these new items, round 2.1 interviewed five veterans of the military. These veterans
provided feedback on the new items in the front end CAPI section of the interview.
Specific feedback on individual items is detailed below.
Veteran Questions
V2a – Respondents understood the use of the phrase “full time in the Reserves.” None of these
respondents reported ever being in the Reserves.
QD10c – Respondents were able to correctly define a combat zone. Among the definitions were
“being in combat,” “being in the fighting,” and Iraq and Kuwait. One respondent said that she
did not serve in a combat zone as women were not allowed to do so when she was in the
military. Another respondent incorrectly reported that he served in a combat zone. He had a
dangerous assignment of guarding prisons in the United States, so he reported yes. After
repeating the question, the respondent acknowledged that he should respond ‘no.’ Respondents
identified with the mention of hazard pay in this question.
2
Hallucinogens
LS01i – Eight respondents had heard of Ketamine. Most reported hearing about it on TV.
LS01j – Three respondents had heard of DMT and one had heard of Foxy. No respondents
reported use.
LS01k – One respondent thought that salvia divinorum was a substitute for marijuana. Another
thought that it was synthetic marijuana. A third person reported hearing of the substance.
Inhalants Screener
IN01ii – No respondents reported use of computer keyboard cleaner just to get high. Fewer than
half of the respondents knew about the spray. A few respondents had heard the term “spray
computer cleaner” or “compressed gas duster” before. One respondent suggested identifying the
spray by its brand names. Others did not have many recommendations for other words used to
describe the spray.
Methamphetamine Module
ME01 – Three respondents in the sample reported lifetime use of methamphetamine. One
respondent reported use of it twenty years ago and one respondent reported using it for two years
three years ago. No respondents reported 12-month use of methamphetamines. Those that
reported lifetime use did not have any problems with the module.
Prescription Drugs
PR01 – One respondent thought that we were only interested in misuse of prescription pain
relievers at this question. A couple of respondents were unaware that the next eight screens were
going to ask about other pain relievers and therefore wanted to report their use here.
Recommendation: Continue to measure whether asking about illicit use of substances before
pain reliever use is priming the respondent to only think about misuse.
Recommendation: Discuss whether we should convey to respondents that more than one screen
asks about pain relievers.
Generic Pills
Although most respondents understood that generic drugs are a cheaper version of name brand
drugs, they were not always clear on what the name brand equivalents of the generic drugs
pictured were. Two respondents reported that generic drugs are over the counter.
3
Not All Forms
The statement that not all forms of a particular drug may be shown on a screen continues to
cause problems for respondents. Many respondents reported thinking that it was referring to
pain relievers with a different name than the one shown on the screen. Others correctly thought
that we were acknowledging that certain pills might take various forms, not all of which were
pictured. A couple of respondents said that they did not understand the statement. Despite the
statement about not all forms, one respondent did not report intravenous use of morphine
because only the pills were shown on the screen.
Recommendation
Evaluate whether new probes should be used to test these questions in Round 2.2 or if the
wording should be further revised.
Pill images
Respondents that had used the drugs report that the pill images aid in their recall. Those that
report not using the pill images often report not using any pills.
Tranquilizers and Sedatives
No new issues were detected with the inclusion of the tranquilizer and sedatives module.
TR06 – When asked whether there were any other tranquilizers that respondents would report, a
few were mentioned. Paxil, Valium, promethazine (phenergen), and trazedone were reported by
respondents.
INTROSV – Respondents had for the most part heard of Benadryl, while fewer respondents had
heard of the other sedatives. Suggestions for other OTC sedatives included Adderall, ibuprofen,
motion sickness pills and melatonin.
Definitions of misuse
Respondents were able to distinguish between use and misuse of prescription drugs and reported
their use correctly.
Special Drugs
SD01 – Most respondents who named examples mentioned “Nyquil.” Other examples were
Sudafed, Tylenol, aspirin, Motrin, Benadryl, and Claritin. Many mentioned “cough syrup.” One
R he’d heard of ‘purple stuff.’ He wasn’t specific and probing lead to him discussing some drink
that a professional football player had been caught with that caused a suspension or fine. He said
he thought it was codeine or Dimetapp or Sudafed like substance. (Note: he is probably referring
to prescription strength cough syrup here.) Only 2 respondents had used OTC medicines. One
4
had taken Benadryl and another had taken Nyquil while he was in bootcamp because he was not
able to drink alcohol at the time.
SD21 – One respondent suggested that “the hospital” be added to this item.
Health
HLTH04 – One respondent only knew he was 54 inches tall, and he did not know how to convert
it to feet and inches.
HLTH19 – Four respondents expressed not knowing whether the dentist should count. None
counted the visits initially, but considered including them after a probe. Of the four, three
decided to count them and one did not.
Recommendation: Wording about not including dentist visits was recently deleted from the
specifications for this item. RTI and SAMHSA should discuss whether this is still appropriate.
HLTH22 – Four respondents reported doctors discussing their drinking with them.
Back End Demographics
QD17 – Fifteen respondents said that they would report being in school even if they were on
summer break. One respondent said he did not know and another said he would say “no” even if
he was just on summer vacation.
QD18 – Respondents were asked a hypothetical question about this item. Most were asked if
they would report being in 7th or 8th grade, if the question was administered during the summer
after they had finished 7th grade. Only one of 7 people probed said “7th grade”. Two participants
correctly reported about their current situation, saying that they would report the grade that they
were “going into.”
QD26 – One adolescent was confused as to whether this question asked about the different
places he had worked or the number of bosses he’d had. Another adolescent was not sure
whether she should count babysitting. She decided that she would NOT count it.
QD49 - One respondent had trouble with ‘DK’ for items QD47 & QD48. She had forgotten that
she needed to use the function keys to indicate this as her answer. She answered ‘DK’ to both
QD47/48. Another participant said that was a difficult question to answer because it depended
on the job one was applying to. Only certain job classifications are subject to testing, which at
her employer (a local government) were ones classified as ‘drug sensitive.’ These jobs are ones
like police officers, welfare workers, or anyone who drives a government vehicle. They do not
test for other job classifications, like the one she is in.
QD50 – The 4 Rs who were probed all mentioned that this question was asking about being
tested “at random.”
5
QD54 – One adolescent R mentioned she spent half the week in one home and half the week in
the other, then asked if she should count “the one I spend the most time in?” It was not hard for
her to choose the one she spent the most time in.
Proxy Tutorial
Two respondents had trouble with entering a “1” to answer the first question in the tutorial.
Recommendation: Change the first question in the tutorial to a “Press Enter to continue” screen.
Two respondents thought it was odd to be asked about the teenager’s income, since they did not
work.
In the income branching questions, one R tried to enter “1” for the first option, but the
appropriate number was “21.” R recognized her mistake and entered “21.”
Recommendation: Change the response options for this question to begin with 1, as opposed to
21.
One respondent complained that the tutorial “went into too much detail.” R thought it was too
slow and instead of letting his intuition take over it bogged him down. He mentioned reading the
script that was for the interviewer (on the demo screens where the interviewer points to the keys,
etc.) and said that “The script was for you but I was thinking of what it said rather than what it
was trying to teach me.”
Additional Issues
In order to reduce respondent burden, a filter question could be added to identify proxies who
have already been introduced to the interview program, whether during their own interview or
while serving as a proxy for another respondents.
Additional instructions and introductions are needed in the back end demographics section. In
cases where a proxy is not nominated, the screens do not clearly indicate which questions are
interviewer administered and which are self-administered.
6
NATIONAL SURVEY ON DRUG
USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY
REPORT—DRAFT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009 & 0211838
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 17, 2014
This page is intentionally blank.
2015 NATIONAL SURVEY ON
DRUG USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY REPORT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009.486.014 & 0211838.108.006.004
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 2014
Acknowledgments
This publication was developed for the Substance Abuse and Mental Health Services
Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ), by
RTI International (a trade name of Research Triangle Institute), Research Triangle Park, North
Carolina, under Contract Nos. 283-2004-00022 and HHSS283200800004C. Significant
contributors at SAMHSA include Dicy Painter, Joel Kennet, and Peggy Barker. Significant
contributors at RTI include Larry Kroutil and Doug Currivan.
ii
Table of Contents
Chapter
Page
1.
Overview ..............................................................................................................................1
1.1
Background of the Redesign ....................................................................................1
1.2
Goals of the Pretesting .............................................................................................3
1.2.1 Usability Testing ..........................................................................................3
1.2.2 Cognitive Interviewing Phase 1 ...................................................................4
2.
Usability Testing ..................................................................................................................7
2.1
Data Collection and Participant Profiles ..................................................................7
2.1.1 Description of Recruitment Methods ...........................................................7
2.1.2 Respondent Characteristics across Usability Testing ..................................7
2.1.3 Data Collection Procedures..........................................................................8
2.2
Key Findings ............................................................................................................9
2.2.1 Logistical Issues with Answering the "Yes/No" or "Enter All That
Apply" Questions .........................................................................................9
2.2.2 Ability to Use the Drug Images ...................................................................9
2.2.3 Screen Layout Preferences ...........................................................................9
2.2.4 Issues with the Question Text from a Usability Standpoint ........................9
2.3
Changes Implemented ..............................................................................................9
3.
Cognitive Interviewing: Phase 1 ........................................................................................11
3.1
Data Collection and Participant Profiles ................................................................11
3.1.1 Description of Recruitment Methods .........................................................11
3.1.2 Respondent Characteristics across Phase 1................................................11
3.1.3 Data Collection Procedures........................................................................12
3.2
Key Findings ..........................................................................................................13
3.2.1 Identification of Prescription Drugs...........................................................13
3.2.2 Performance of the Misuse Criteria ...........................................................13
3.2.3 Other Issues ................................................................................................14
3.3
Changes Implemented ............................................................................................14
4.
Cognitive Interviewing: Phase 2 ........................................................................................17
4.1
Data Collection and Participant Profiles ................................................................17
4.1.1 Description of Recruitment Methods .........................................................17
4.1.2 Participant Characteristics .........................................................................17
4.1.3 Data Collection Procedures........................................................................18
4.2
Key Findings ..........................................................................................................19
4.2.1 Demographic Items ....................................................................................19
4.2.2 Prescription Drugs and Use/Misuse ...........................................................20
4.2.3 Tobacco, Illicit, and Special Drugs ............................................................21
4.3
Changes Implemented ............................................................................................22
4.3.1 Phase 2, Round 1........................................................................................22
4.3.2 Phase 2, Round 2........................................................................................23
iii
Table of Contents (continued)
Chapter
Page
5.
Discussion ..........................................................................................................................27
5.1
Summary of Major Changes ..................................................................................27
5.2
Usability Testing Advantages ................................................................................27
5.3
Recommendations for Future Pretests ...................................................................28
6.
References ..........................................................................................................................29
iv
List of Attachments
Attachment
Page
A.
Phase 1.1 Summary of Findings ..................................................................................... A-1
B.
Phase 1.2 Summary of Findings ......................................................................................B-1
C.
Phase 1.3 Summary of Findings ......................................................................................C-1
D.
Phase 2.1 Summary of Findings ..................................................................................... D-1
E.
Phase 2.2 Summary of Findings ...................................................................................... E-1
F.
Variable Wording Crosswalk ........................................................................................... F-1
v
This page is intentionally blank.
vi
List of Tables
Table
Page
1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants ........ 8
2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants ....... 12
3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants ....... 18
4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens ......................... 22
5.
List of Screens Presented for the Full and Abbreviated Tutorials .................................... 25
vii
This page is intentionally blank.
viii
1. Overview
1.1
Background of the Redesign
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance
Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S.
civilian, noninstitutionalized population aged 12 and older. The conduct of the NSDUH is
paramount in meeting a critical objective of SAMHSA's mission to maintain current data on the
prevalence of substance use in the United States. In order to continue producing current data,
SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ) must update the
NSDUH periodically to reflect changing substance use and mental health issues. These updates
are necessary because substance use in the United States is a dynamic phenomenon that changes
with time, demographic shifts, economic prosperity, and most importantly, availability of
substances.
CBHSQ plans to redesign the NSDUH for the 2015 survey year to achieve two main
goals: (1) revise the questionnaire to address changing policy and research data needs, and
(2) modify the survey methodology to improve the quality of estimates and the efficiency of data
collection and processing.
The questionnaire will be revised to improve questions that cause known or suspected
problems with data that are collected using the current NSDUH questionnaire. New content that
addresses current data needs will be added. Revisions that are designed to reduce errors
associated with usability problems in the design and layout of the computer-assisted interviewing
(CAI) instrument are planned. The changes include electronic calendars, revised prescription
drug modules, revised front end demographics, a revised smokeless tobacco module, revised
questions for measuring binge alcohol use, a revised hallucinogens module, new questions for
lifetime use of specific inhalants, a new methamphetamine module, a revised special drugs
module, a revised consumption of alcohol module, and a revised back end demographics section.
Specifically, the following are highlights of issues for the modules for prescription drugs and
health care and for interviewer-administered questions that will be addressed by the redesigned
questionnaire.
•
In the current interview procedures, interviewers and respondents mark a printed calendar
with the start of the 30-day and 12-month reference periods. In the redesigned
questionnaire, respondents will be shown electronic versions of these calendars that note
the start of these reference periods, and they will receive a tutorial about these electronic
calendars. The electronic calendars also will appear periodically to remind respondents of
reference periods, and respondents can use a function key to review the calendars at any
time during the self-administered modules.
1
•
The content of the specific prescription drugs in the core prescription drug modules was
last updated in 1999, based on prevalence information from 1998. Since that time, despite
minor updates to the questionnaire, several prescription drugs included in NSDUH have
been discontinued or are otherwise no longer legally available, and recently approved
prescription drugs with abuse potential are not included in the core modules. All
questions measuring misuse of prescription drugs will be moved into the prescription
drug modules. In addition, a review of the drugs that are included in these modules will
result in recommendations for drugs to be added or dropped based on their availability.
•
The current NSDUH questionnaire asks only about misuse of prescription drugs. A
revised structure will be tested that first uses a screener module to ask about any use of
prescription drugs. Respondents who report any use pf prescription drugs will then
receive questions determining whether any of that use constituted misuse.
•
The current NSDUH definition of misuse (i.e., use of drugs that were not prescribed for
you or that you took only for the experience or feeling they caused) combines a behavior
and a motivation. It also does not include overuse of prescribed medication. The revised
definition will focus on behaviors that constitute misuse and will refer to use "in any way
a doctor did not direct you to use it/them." Overuse (i.e., use in greater amounts, more
often, or longer than directed) will be added to the examples of behaviors that constitute
misuse.
•
Except for misuse of OxyContin® in the current pain relievers module and misuse of
methamphetamine in the stimulants module, misuse of specific prescription-type
psychotherapeutic drugs in the current questionnaire is limited to the lifetime period.
Measurement of past year misuse is limited to misuse of any drugs within a
psychotherapeutic category (e.g., pain relievers). However, information about more
recent misuse of specific prescription drugs (e.g., in the past year) is of greater interest to
policymakers and stakeholders who use the NSDUH prescription drug data. Therefore,
questions will measure past year misuse of individual prescription drugs.
•
In the current questionnaire, methamphetamine is included in the module for misuse of
prescription stimulants, and respondents are asked in the stimulants module about misuse
of methamphetamine. However, most methamphetamine that is used in the United States
is manufactured illegally rather than by the pharmaceutical industry. Respondents also
may fail to report methamphetamine use in the context of questions about misuse of
prescription drugs. Therefore, the redesigned questionnaire will include a separate
module for methamphetamine, and the questions in the stimulants module will ask about
misuse of prescription stimulants.
•
For the current questionnaire, respondents are handed printed "pill cards" with pictures of
prescription drugs in a given module to aid respondents in recalling their misuse of
specific drugs. NSDUH incurs the expense of printing pill cards for interviewers who are
new to the survey or reprinting the pill cards when new dosages become available for
some prescription drugs. Also, respondents do not always ask interviewers for the pill
cards when they reach the prescription drug questions. Therefore, electronic drug images
to be displayed on the laptop computer screen will replace the printed pill cards. The
electronic images also will include examples of prescription drugs other than pills (e.g.,
patches, vials).
2
•
The current health care module of the questionnaire is limited in scope. Questions
measuring height, weight, a revised list of health conditions, and details about diagnoses
of those health conditions will be tested and added to the health care module.
•
The current survey includes questions about emergency room visits, hospitalizations,
substance abuse treatment utilization, and mental health service utilization, but does not
include questions about outpatient primary care service utilization or opportunities for
intervention with substance users in primary care settings. Questions related to these
topics will be tested and included in the health care module of the questionnaire.
•
Although audio computer-assisted self-interviewing (ACASI) is used for most of the
sensitive questions, respondents may be likely to consider some questions in intervieweradministered sections to be sensitive (e.g., income). Pretesting efforts will explore the
feasibility of moving health insurance and income questions to an ACASI mode.
Proposed revisions will improve the precision of estimates that exhibit properties
associated with measurement error. Revisions will also be designed to reduce errors associated
with usability problems in the design and layout of the CAI instrument.
1.2
Goals of the Pretesting
The pretesting for the 2015 NSDUH questionnaire redesign had three components:
Usability Testing (Prescription Drugs), Phase 1 (Prescription Drugs) Cognitive Interviewing, and
Phase 2 (Redesigned Modules) Cognitive Interviewing. The purpose of each of these three
components is discussed in the following sections.
1.2.1
Usability Testing
Usability testing was conducted in the first stage of redesign pretesting. Prior to the
usability testing, questions for misuse of pain relievers and stimulants were reworded, producing
two new versions of these questions. One of these versions was a "three-criteria" bulleted list that
asked participants about using a set of prescription pain relievers and stimulants "in any of these
ways" in the past 12 months:
•
without a prescription of your own,
•
in greater amounts, more often, or longer than you were told to take it, or
•
just for the effect it had on you—not for its intended medical use.
A second version ("two criteria") eliminated one criterion from the three-criteria list: use
"in greater amounts, more often, or longer that you were told to take it," but used the same
wording for the remaining two criteria.
In addition, the usability instrument tested the placement of pill images above or below
the question text in order to test the new electronic drug images. Participants were presented with
a total of four versions of the questions. Two versions of the three-criteria questions featured
images above and below the questions. There were also two versions of the two-criteria
questions with images above and below the questions. In addition to soliciting feedback on the
criteria items, usability testing sought to accomplish three primary goals:
3
1. observe participant use of and reaction to different options for proposed laptop
computer screen layouts for the screeners and modules for prescription pain relievers
and prescription stimulants;
2. collect participant feedback and recommendations for improving the design of the
laptop computer screen layouts for the screeners and modules for prescription pain
relievers and prescription stimulants; and
3. collect participant feedback on the electronic reference date calendars and electronic
drug images.
1.2.2
Cognitive Interviewing Phase 1
The second stage of redesign pretesting, cognitive interviewing, occurred in two phases.
The first phase of interviewing focused on redesigned prescription drug modules and a
redesigned health module. The second phase continued to test these modules but expanded the
scope to include the full redesigned NSDUH instrument.
Phase 1 cognitive interviewing took place over three rounds and tested an instrument
with changes to the prescription pain relievers, prescription stimulants, and health modules. The
Phase 1 redesigned instrument included an abbreviated core set of substance use questions that
measured lifetime and 30-day use. The following changes were reflected in the Phase 1
abbreviated core questionnaire:
•
Smokeless tobacco sections (chewing tobacco, snuff, and the new product, snus) were
combined into one section.
•
Hallucinogen items currently included in special drugs for ketamine, tryptamines (DMT,
AMT, and "Foxy"), and Salvia divinorum were moved from the noncore special drugs
module to the core hallucinogens module.
•
New inhalants questions for lifetime use of markers and air duster were added to the core
inhalants module.
•
Prescription pain relievers and prescription stimulants screeners focused on any use of
specific prescription drugs in the past 12 months (i.e., medical use or misuse). Misuse of
prescription pain relievers and stimulants in the corresponding main modules was defined
as use "in any way a doctor did not direct you to use it/them" and was based on the
examples from the three-criteria bulleted list that was implemented during usability
testing. Respondents could be reminded about these criteria by pressing a function key to
see a pop-up screen with the bulleted list. Electronic drug images were placed below the
introductory question text, and some images showed examples other than pills.
•
New health module questions were added.
•
The electronic reference date calendar was featured throughout the instrument.
The goals of Phase 1 cognitive interviewing were to test new and revised question
wordings and response options. This testing included gauging participants' reactions to key terms
and concepts and detecting any issues with question comprehension. A test of the electronic
reference date calendar was also a Phase 1 goal.
4
1.2.3 Cognitive Interviewing Phase 2
The cognitive interviewing in Phase 2 took place over two rounds and was designed to be
a broader test of changes to the instrument. Phase 2, round 1 included all of the modules that
were tested in Phase 1. In addition, the following revisions were made for Phase 2:
•
New items were added to core demographics, including new military veteran questions.
•
Screeners and modules for tranquilizers and sedatives were added to the instrument and
tested.
•
The definition of binge drinking was changed for female respondents to include four or
more drinks on an occasion in the past 30 days.
•
Questions about needle use in the noncore special drugs module were reworded, and
questions about use of methamphetamine and prescription stimulants with a needle were
moved to the corresponding core modules.
•
New questions about medical use of marijuana were added to the noncore blunts module.
•
Education, health insurance, and income were moved to the ACASI portion of the
interview. In addition, the highest level response category for income was revised.
•
A new module introduced proxy respondents to the ACASI.
Phase 2, round 2 included the fully (nonabbreviated) redesigned instrument, with all core
modules, and incorporated audio recordings for the ACASI portion.
The goals of Phase 2 cognitive interviewing included retesting changes that were made to
question wording, routing, and response options following Phase 1. Testing the content that was
new to the cognitive interviewing protocols in Phase 2 was also a goal.
5
This page is intentionally blank.
6
2. Usability Testing
2.1
Data Collection and Participant Profiles
2.1.1
Description of Recruitment Methods
Participants were recruited for usability testing via a recruitment notice that was posted
on Craigslist. Persons who called RTI International in response to the recruitment notice were
administered a series of screening questions. The screening questions included items about the
caller's age and location to establish eligibility for the study; to be eligible, a person needed to be
aged 18 or older and located within 50 miles of Research Triangle Park, NC. The screening
questions also collected information on other demographic characteristics (e.g., gender,
education), computer use, and any use of prescription pain relievers and stimulants in the past 12
months. Questions about use of pain relievers and stimulants were included to gauge participants'
familiarity with the general types of drugs that were included in the usability testing. However,
the usability testing did not focus on participants' actual behavior with respect to prescription
drug use.
2.1.2
Respondent Characteristics across Usability Testing
Table 1 provides information on characteristics of the eight usability participants. The
number of participants was evenly divided between males and females. Participants' ages ranged
from 28 to 60. Among the four participants in the 26 to 34 age category, there were two
participants aged 30 years, one participant who was aged 28, and another aged 32. Among the
three participants aged 50 or older, ages ranged from 53 to 60. There was only one participant in
the 35 to 49 age group. Six participants reported some education beyond high school, while two
participants did not.
As might be expected, because participants were recruited from Craigslist, all reported
using a computer for personal use every day. However, only half reported that most of what they
do at work is done on a computer. Two participants reported using a computer at work less than
daily or never; the screening interviewer was instructed to choose "never" for computer use at
work if participants did not use a computer at work or if they did not have a job.
Five participants reported using prescription pain relievers for any reason in the past 12
months, and one reported using prescription stimulants in that period. None reported use of both
classes of prescription drugs in the past 12 months. Although the screening question included
items about use of prescription pain relievers or stimulants in the past 12 months, use of these
prescription drugs was not an eligibility requirement for the study. Consequently, two
participants reported not using either of these classes of prescription drugs in the past 12 months.
7
Table 1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants
Participant Characteristics
Gender
Male
Female
Total
Age Group
18 to 25
26 to 34
35 to 49
50 or Older
Total
Highest Grade Completed
High School or Less
Beyond High School
Total
Computer Use at Home/For Personal Use
Every Day
Computer Use at Work
Never
Sometimes but Not Every Day
Every Day but Not All Day
Most Work Is on a Computer
Total
2.1.3
Number
4
4
8
0
4
1
3
8
2
6
8
8
1
1
2
4
8
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures that were approved by RTI's Institutional Review Board. Parental permission was not
an issue for usability testing because all participants were adults.
Participants were then presented with a booklet containing a series of tasks. Participants
were shown the task instructions but not the follow-up probes, which were read aloud by the
interviewer. The follow-up questions provided breaks between sets of usability tasks. In the first
two tasks, all participants were shown unlabeled pictures of the pain relievers or stimulants that
were included in the interview. The aim of these tasks was to test participant identification of
prescription drugs solely from the drug images.
In the remaining tasks, participants were presented with hypothetical scenarios that were
read aloud by the interviewer. Participants were instructed to answer the questions on the laptop
computer as though the behavior described in a given scenario was true for them. Some scenarios
tested participants' identification of drugs by name, and others tested participants' ability to
identify drugs based on pictures in the booklet and the corresponding on-screen images. The
interviewer asked participants to read the questions aloud according to how they appeared on the
computer screen and then asked them to indicate how they thought the question should be
answered based on the scenario they were given. To obtain preliminary information about the
cognitive processing of the question, the interviewer also asked why participants thought a
question should be answered a certain way based on the scenario.
8
Although each interview was allowed 90 minutes, interviews generally only required
about 1 hour to complete, including the introduction, informed consent, usability test questions
and tasks, and incentive disbursement. Participants were given a $60 Visa gift card for
completing the interview.
2.2
Key Findings
2.2.1
Logistical Issues with Answering the "Yes/No" or "Enter All That Apply"
Questions
No participants appeared to have difficulty with answering the yes/no questions.
However, some participants had difficulty with the "enter all that apply" questions when the
numbering of response categories continued from the previous question, instead of resuming the
numbering of the first category with 1.
2.2.2
Ability to Use the Drug Images
Some scenarios in the test booklets presented situations in which a drug was shown as a
picture rather than being mentioned by name. Participants were able to correctly answer the
relevant survey question by matching the on-screen drug image to the drug image in the test
booklet. One participant commented specifically about how well laid out the on-screen images
were. These results suggest that participants can successfully use the on-screen images as a recall
tool.
2.2.3
Screen Layout Preferences
Participants as a whole did not have a strong preference for whether the drug images
were placed above or below the question text. Those who expressed a preference were fairly
evenly divided between the two types of layouts.
2.2.4
Issues with the Question Text from a Usability Standpoint
From a usability standpoint, no major issues were identified with respect to whether it
was easier for participants to answer the two-criteria or three-criteria questions. In particular, no
one commented that the three-criteria layout appeared to be too "busy" on the computer screen.
One participant recommended a larger point size for the on-screen font, particularly to aid older
participants in reading the questions.
Some participants talked about the repetitive nature of the questions, but no one explicitly
mentioned that the three-criteria questions were overly repetitive. Rather, comments about the
repetitive nature of the questions appeared to be caused by participants being presented with the
same basic questions four different ways (i.e., with the drug images above or below the questions
and with the two-criteria or three-criteria wording).
2.3
Changes Implemented
Based on the results of the usability testing, either image layout was deemed acceptable
to participants. It was decided that the drug images would be displayed at the top rather than that
the bottom of the screen, and that the introductory question text would appear above the drug
9
images. In addition, response option values for "enter all that apply" questions would be reset to
1 on each relevant questionnaire screen. For example, if there are five drugs displayed per
screen, each will be numbered as 1 through 5, as opposed to being labeled 1 through 5 and then 6
through 10. This will address the issue with entering responses that participants identified with
the "enter all that apply" questions during usability testing.
10
3. Cognitive Interviewing: Phase 1
3.1
Data Collection and Participant Profiles
3.1.1
Description of Recruitment Methods
Phase 1 cognitive interviewing included three rounds of interviews. Interviews were
conducted in Research Triangle Park, NC; Chicago, IL; and Washington, DC. Advertisements
were placed on Craigslist, and flyers were distributed to outpatient substance abuse treatment
programs. In the first round of Phase 1 cognitive interviews, only adults were interviewed. In the
second and third rounds, adolescents were also interviewed. In all three rounds, the recruitment
advertisements targeted past year users of prescription pain relievers and prescription stimulants.
Licit or illicit use was not specified in the advertisements. There were two versions of the
recruitment advertisements for rounds 2 and 3. One version was for adults, and one version was
for adolescents aged 12 to 17.
Persons who responded to the recruitment advertisements were administered a screener
over the telephone to assess their eligibility for the study. The screener took approximately 5
minutes to administer. It included questions about how callers learned about the study (to
identify the source of recruitment), age, gender, place of residence, education, physical
limitations, prior research study participation, and use of prescription pain relievers or stimulants
(for any reason) in the past 12 months. Recruitment was balanced to include people who had
used prescription pain relievers or stimulants in the past 12 months as well as nonusers.
Procedures for obtaining permission from a parent or legal guardian to interview adolescents are
described in Section 3.1.3.
In addition, persons who have had considerable prior experience with survey research
may bias the data and conclusions. Therefore, persons were deemed ineligible if they had
participated in more than one prior research study in the past 12 months (regardless of who
conducted the study or the content) or in any prior RTI research study in this period.
3.1.2
Respondent Characteristics across Phase 1
Table 2 presents an overview of selected demographic characteristics of Phase 1
cognitive interview participants. Out of 40 respondents in Phase 1, 10 were adolescents. As
noted previously, these 10 adolescents were interviewed in rounds 2 and 3 only. Seven of the 10
adolescent respondents were interviewed in round 2. Recruitment was balanced across gender,
with 18 female respondents. In Phase 1, 13 of the respondents were recruited from substance
abuse treatment programs. These 13 substance abuse treatment patients were interviewed in
rounds 1 and 2; recruitment in round 3 was limited to persons from the general population.
Thirty respondents had used prescription pain relievers in the past 12 months and 18 had used
prescription stimulants in the past 12 months. Although not shown in the table, of the 30 adults,
15 had a high school diploma, GED, or lower education. The other 15 had attended at least some
college.
11
Table 2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)1
Female
Enrolled in Treatment Program (Currently or
within Past 12 Months)
12 Month User of Any Prescription Pain
Reliever
12 Month User of Any Prescription Stimulant
Chicago, IL
Washington, DC
Research Triangle Park, NC
1
Round 1
Round 2
Round 3
Total
16
0
7
8
5
7
5
5
9
3
6
0
30
10
18
13
14
7
9
30
7
5
5
6
4
2
3
7
7
3
4
5
18
10
12
18
Mean age not available.
3.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across all three rounds
of Phase 1 data collection. The interview was conducted using the computer-assisted Blaise
interview. The cognitive interviewer first administered introductory questions verifying the
respondent's age and gender. The interviewer introduced the laptop functions to the respondent
and described the feature of the electronic reference date calendar. The respondent was then
asked to conduct the next section of the interview using a self-administered design. In Phase 1,
audio files were not used in the first two rounds, so respondents read questions on the screens
themselves and then answered the questions. In round 3, respondents could listen to audio
recordings of the questions. Interviewers administered think-aloud probes and follow-up probes
to collect data on the cognitive properties of specific questions. The following modules were
tested in all rounds in Phase 1:
•
An abbreviated ensemble screener of drugs
•
Pain relievers screener
•
Stimulants screener
•
Pain relievers main module
•
Stimulants main module
•
Health module
The protocol largely remained the same for rounds 1 and 2. After round 2, slight changes
were made to the cognitive interview protocol. Changes included the following:
•
adding probes for questions that were identified as problematic in the first rounds of
interviewing,
12
•
dropping three categories describing sources for obtaining prescription drugs, and
•
editing selected question logic, question wording, and response options.
Probes were deleted for questions that no longer necessitated testing in the third round.
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
3.2
Key Findings
Summaries of the findings and recommendations from rounds 1, 2, and 3 of Phase 1
cognitive interviewing can be found in Appendices B, C, and D. These summaries also include
recommendations for edits to increase respondent comprehension of key items.
Key findings from Phase 1 interviews can be arranged into three broad categories:
(1) identification of prescription drugs, (2) performance of the misuse criteria, and (3) other
issues not related to prescription drugs. The subsections that follow cover each of these key
findings categories in turn. Changes that were made as a result of these findings are covered in
Section 3.3.
3.2.1
Identification of Prescription Drugs
In most cases, participants recognized the prescription drugs that they took by name.
However, some participants did find the drug images to be helpful, particularly if participants
were unfamiliar with certain prescription drugs.
In the screener, several screens showed brand name prescription drugs (e.g., Vicodin®)
and the generic equivalent (e.g., hydrocodone); "generic" was shown in parentheses next to the
generic drug name. There was some uncertainty about the "generic" term associated with some
drugs. Some participants reported that their doctors would write the brand-name drug on the
prescription, but the pharmacy would substitute the generic. Another issue concerns the number
of generic drug manufacturers and the variety of generic equivalents of specific drugs that may
be dispensed when prescriptions are filled. It would not be realistic to try to include exhaustive
examples of images for generic drugs. Addition of the word "generic" to the drug images in
round 3 (i.e., in addition to its appearance in the response category for the generic) appeared to
be helpful.
Another issue with identifying prescription drugs involved making respondents aware
that the questions applied to any form regardless of its appearance or whether they used the drug
in pill form or another form. Some respondents neglected to report use of liquid forms of the
prescription drug, especially if the images for these particular drugs showed only pills. Other
respondents mentioned using the drug, but in a pill form that was not shown on the screen. An
edit was made to the instrument to address this issue. This edit is described with PR01 in Section
3.3.
3.2.2
Performance of the Misuse Criteria
The wording "in any way a doctor did not direct you to use it" differentiated between
misuse of prescription drugs and use with a prescription and as directed. With few exceptions,
participants who used prescription drugs only with a prescription of their own and as prescribed
13
could determine that they did not use prescription drugs "in any way a doctor did not direct you
to use [them]." Similarly, persons who misused prescription drugs could determine that their use
constituted use "in any way a doctor did not direct you to use it," even if they used prescription
drugs in ways not explicitly listed as examples of misuse.
Later in the module, a question asked respondents which of the specific components of
misuse applied to them. This decomposition question puzzled some respondents. In round 1, the
criterion of overuse was presented as a single response (i.e., use in greater amounts, more often,
or longer than I was told to take it). Some round 1 respondents had trouble determining whether
they should choose this second category or the third one (use in some other way a doctor did not
direct me to use it). Specifically, some respondents thought that they needed to have engaged in
all three of the behaviors that were listed in the second category in order to choose it. Three
scenarios for misuse also arose during the interviews that were not listed as examples in the
instrument. These were (1) using with alcohol, (2) asking a doctor to prescribe a larger dosage of
prescription pills than was needed, and (3) buying prescriptions in other countries where
prescriptions are not needed for purchase.
3.2.3
Other Issues
Respondents had trouble understanding the use of canned air or air duster as an inhalant.
Although no respondents had used canned air as an inhalant, there were difficulties identifying
the product by that term. Problems were identified with respondents entering height and weight
in the new health module, but these were resolved with minor revisions to the screens. For
questions about the number of outpatient doctor visits, participants had difficulty determining
what did or did not constitute a doctor visit. Summaries of the findings and recommendations
from rounds 1, 2, and 3 of Phase 1 cognitive interviewing can be found in Appendices B, C, and
D. These summaries also include recommendations for edits to increase respondent
comprehension of key items.
3.3
Changes Implemented
As a result of all rounds of Phase 1 cognitive interviewing, several changes were made to
the instrument. These are listed below and are accompanied by the variable name.1
•
IN01ii—The word "air duster" was added to this question about canned air in the
inhalants section for increased comprehension.
•
PR01—The text in the introduction to each prescription drug screener was revised to
include "Please note that not all forms, dosages, shapes, or colors of these pain relievers
are shown on these screens, but you should include any form that you have used." This
sentence would convey that respondents should consider forms of the drug other than
pills (e.g., liquid forms), even if the image showed only pill forms.
•
PR07—Reference to the term "China Girl" was removed for fentanyl because none of the
respondents knew it, and fentanyl was the only prescription drug with a slang name.
1
Variables in the health module were renumbered during pretesting. Variable names as they appear in the
questionnaire field test (QFT) instrument are in parentheses.
14
•
PRHOSPYR1—This question about use of prescription drugs only in the hospital was
dropped because many respondents had difficulty answering it.
•
The calendar icon, which reminds respondents how to access the electronic reference date
calendar, was made more visible to encourage respondents to use it.
•
Audio was removed for the answer choices for height and weight because those are
included in the questions.
•
HLTHNEW03 (HLTH19)—This question, which asks the respondent how many times
he or she has visited a doctor in the past year, was revised to streamline the text.
•
HLTHNEW11 (HLTH29) & 12a (HLTH31)—Specs about specific health conditions
were changed to skip questions about whether respondents had specific conditions in the
past 12 months if the age at first diagnosis equals current age.
15
This page is intentionally blank.
16
4. Cognitive Interviewing: Phase 2
4.1
Data Collection and Participant Profiles
4.1.1
Description of Recruitment Methods
Phase 2 of cognitive interviewing included two rounds of interviews. For both rounds of
interviews, recruitment advertisements were placed on Craigslist in Research Triangle Park, NC;
Chicago, IL; and Washington, DC.
The Phase 2 cognitive interview research questions influenced the recruitment goals.
During Phase 2, methodologists sought to test the transition from asking questions about health
insurance and income using an interviewer-administered mode to using ACASI. Because
respondents can nominate a proxy to answer these questions on their behalf, new questions were
developed to provide this proxy with a tutorial on how to use the computer and the interviewing
program. Transition statements were also developed to inform the proxy about their role in the
interview. To test this process and these new screens, RTI recruited adolescents to serve as
respondents and their parents to serve as proxies. In some cases, the parents went on to complete
a full interview after serving as a proxy respondent for their child.
Three versions of the advertisement were posted: (1) a version targeting military
veterans, (2) a version targeting parent-teen pairs, and (3) a final version targeting adolescents
whose parent would participate in a short interview as a proxy for health insurance and income
questions.
Prospective participants who responded to the advertisement were administered a
screening questionnaire. The screener took, on average, approximately 5 minutes to administer.
A variety of demographic questions were asked of prospective participants, including questions
about age, gender, educational attainment, military status, and use of prescription drugs in the
previous 12 months. Answers to screener questions, in addition to the location at which they
would be interviewed and the study population to which they belonged (military veteran, parentteen pair, adolescent with proxy interview, and drug use history), were used to select participants
and reach recruitment targets to the extent that was logistically possible. Because reporting illicit
or prescription drug use was not an eligibility requirement in the first round, few first round
participants were heavy or expert drug users. Therefore, in the second round of interviews, RTI
targeted individuals who had used any of the four psychotherapeutic prescriptions or
methamphetamine and those who had received drug treatment in the past 12 months. For these
reasons, the final sample of participants represents a diverse set of demographic characteristics
and experiences.
4.1.2
Participant Characteristics
Table 3 presents a summary of selected demographic characteristics of Phase 2 cognitive
interview participants. For some demographics, such as age and gender, participants were well
distributed across interviewing rounds. For other characteristics of interest, the distribution
reflects changes in recruiting goals between rounds. Reasons for changes to recruitment goals are
discussed subsequently.
17
Participants ranged in age from 12 to 68. A total of 24 adults participated in Phase 2
interviews, with a relatively even distribution between rounds (round 1 = 10; round 2 = 14). The
mean age of adult participants was 42 years. Among the 16 adolescents, 10 participated in the
first round, and 6 participated in the second round of interviews. The mean age of adolescent
participants was 15 years. Approximately half of the sample was female (i.e., regardless of age).
Among adults who provided information on educational attainment, a majority had at least some
education beyond high school (not shown in Table 3).
Interviews in Phase 2, round 1 included a relatively large number of military veterans,
and it included a much small number of individuals with a history of heavy drug use. As
discussed in Section 4.2, veterans exhibited no problems answering the questions of interest. As
a result, the recruitment goals shifted in the second round of interviews to target individuals with
more drug experience.
Table 3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)2
Female
Veteran
Enrolled in Treatment Program (Currently or Within
Past 12 Months)
High Use Participant*
Chicago, IL
Washington, DC
Raleigh, NC
Round 1
10
10
9
5
2
0
4
3
13
Round 2
14
6
12
2
5
11*
5
5
10
Total
24
16
21
7
7
11
9
8
23
1
Mean age = 42.
Mean age = 15.
* The category "high use" was created for Phase 2, round 2 to refer to individuals who had experience with more
than one or two prescription drugs. This enabled recruitment of individuals who had a history with heavy drug
use.
2
4.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across the two rounds
of Phase 2 data collection. These procedures were also the same as those described in Section
3.1.3. Audio was available to participants in Phase 2. Interviewers administered think-aloud
probes and follow-up probes to collect data on the cognitive properties of specific questions.
In round 1, a selected set of modules was administered to participants. In round 2, the
entire proposed redesigned instrument was administered to participants. In both rounds, probes
were inserted following items of interest.
18
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
4.2
Key Findings
Summaries of the findings and recommendations from rounds 1 and 2 of Phase 2
cognitive interviewing can be found in Appendices D and E. These summaries also include
recommendations for edits to increase respondent comprehension of key items. Key findings
from Phase 2 interviews can be arranged into three broad categories: (1) findings related to the
front- and back-end demographics sections, (2) findings related to prescription drug use and
misuse, and (3) findings related to substances other than prescription drugs. The following
subsections cover each of these key findings categories in turn. Changes that were made as a
result of these findings are discussed in Section 4.3.
4.2.1
Demographic Items
As noted in Section 4.2.1, round 1 interviews included a recruitment focus on military
veterans. In general, veterans were able to understand terms and phrases in the way in which they
were intended. Veterans understood the phrase "full time in the Reserves" (item V2a) despite the
fact that none of them reported ever being in the Reserves. Most participants were also able to
correctly define a combat zone when methodologists probed about the meaning of the term
(QD10c).
For a few participants, the new education question (QD11) was problematic. Interviewer
probing revealed that the order of the response categories was confusing. The question asked
about the highest grade that was completed, and response categories 1 to 11 included language to
that effect. Consequently, a participant originally understood an answer of "12" to mean
finishing the 12th grade. Revisions, as described in Section 4.3.2, were made to address this
item.
A majority of participants understood the questions in the back-end demographics
module. For example, participants correctly understood, despite being on summer break, that
they should report currently attending school (QD17). They also understood, when asked what
grade they were currently attending, to report the grade they are planning to enter after summer
vacation comes to a close. No problems were encountered when defining testing for drugs or
alcohol as part of the job hiring process (QD49), and what the term "random" means in the
context of random drug testing (QD50). One back-end demographic question that RTI noticed
could be moderately confusing to respondents was the item about sources of income
(INTRTINN). In the past this item was interviewer administered. The transition to an ACASI
administration resulted in awkward question wording and thus was revised to account for the
mode.
Most participants were able to answer a question about their height (HLTH04) with no
problems. However, it became clear in one interview that some respondents might not be able to
answer the question in the formats that were available (feet/inches or meters/centimeters). Some
respondents will be able to answer only in inches or only in centimeters because this is often how
height is measured in medical evaluations. Revisions to the available methods of entering height
were therefore implemented.
19
Income branching questions (QI20N, QI21A, and QI21B) were generally understood by
participants. One branch question (QI20N) asks respondents whether they make over or under an
income threshold. Contingent on their answer, respondents are taken to QI21A (for incomes
below $20,000) or QI21B (for incomes over $20,000). The numeric categories carrying over
from QI21A to QI21B was confusing to one participant. That is, there were "20" categories in
the first branch of questions that the participant was not routed to and had not seen. The question
she was routed to (QI21B) began categories at "21." No other question began a list of numeric
categories in such a way, which led to the initial confusion about how to answer the question.
The response category labels were revised to begin at 1.
The instrument tested in Phase 2, round 2 included two transitions from interviewer
administration to self-administration. In both rounds of interviewing, most respondents who were
serving as a proxy believed the proxy tutorial was simple to follow and understand. In a couple
of instances, however, it was apparent that the transition between modes could be improved. In
response, edits were made to these items, as explained in Section 4.3.1.
4.2.2
Prescription Drugs and Use/Misuse
A number of issues related to prescription drugs continued to be explored in the Phase 2
cognitive interviews. Highlighted findings include how participants understood concepts such as
generic, use, misuse, and "not all forms shown"; what motivated participants to misuse
prescriptions; and participants' ability to recall their age at first misuse of prescription drugs.
Relevant findings are discussed in this section, and corresponding changes that were
implemented are discussed in Section 4.3.
In most cases, the concepts that were explored were understood by participants as had
been intended. Most participants were able to identify generic drugs as the less expensive
alternative to a brand name drug. A small proportion of participants showed less clarity about the
chemical equivalency of generic and brand name drugs and that generics also require a
prescription.
In both rounds of interviews, participants shared a similar understanding of the concept of
misuse of prescription drugs. However, questions about misuse are administered to participants
who report the use of prescription drugs. Therefore, this finding must be measured against the
finding that there was less clarity among participants about the concept of use. The intended
understanding of "use" for prescription drugs starting at PR01 was use of any kind. Because the
first question about prescription drug use followed a long list of questions about illicit drug use,
some participants thought that the prescription drug screener questions were asking only about
the misuse of prescription drugs. Conversely, several participants thought the screener questions
were asking only about use of prescription drugs in a way that they were prescribed.
In addition to determining whether participants interpreted the concept of misuse in the
intended manner, the SAMHSA/RTI instrumentation team aimed to learn more about how
participants recalled their first misuse and their motivations for misuse. Questions that measured
motivations for misuse were added for the first time in Phase 2. No problems were apparent in
recalling the age at which misuse first occurred.
20
One cognitive issue was uncovered when the motivation items were tested. This finding
revealed an absence of a specific category for misusing prescription drugs to deal with emotions.
It also became apparent that logic and skip patterns in this series of questions needed editing. A
description of these revisions begins in Section 4.3.1.
Throughout the testing, instructions to respondents that indicated that not all forms of
pills were shown on the screen underwent further modification. Despite this iterative testing,
problems persisted. For instance, only morphine pills were shown on the screen as examples for
this drug. Respondents had difficulty determining whether to report use of morphine in liquid
form that they received by injection or intravenously. In response, a photo of a vial of liquid
morphine was added to this screen, along with the reminder about the forms that drugs can take.
4.2.3
Tobacco, Illicit, and Special Drugs
Nonprescription drug items were also of interest in Phase 2 cognitive interviews. Of
particular interest were new items about new cigarette naming conventions, substances that were
now asked about in the hallucinogen module, and whether participants understood questions
about medical marijuana.
Previous versions of the NSDUH questionnaire asked about the type of cigarettes current
smokers smoked most often (e.g., lights, ultra lights, mediums, or full-flavored cigarettes).
Recent legislation banned cigarette manufacturers from using terms on packaging that referred to
cigarette strength. As a result, many manufacturers migrated to using a color-coded system as an
alternative naming convention. In an attempt to continue to track trends in the types of cigarettes
most smoked, a new question (CGCOLOR) was developed.
In cognitive interviews, the SAMHSA/RTI instrumentation team aimed to determine how
well current smokers could recall the package color of the cigarettes they smoke most often. Half
of those who reported smoking in the past 30 days (current smokers) reported an issue with
answering this question. Participants reported that cigarette packaging includes multiple colors,
indicating that participants failed to understand the purpose of the question about package color.
The multiple colors on packaging left some participants unsure if they should pick the main
color, background color, or color of the text on the packaging. As a result, this question was
dropped from the specifications.
New hallucinogen questions were also of special interest in Phase 2 interviews.
Previously, these questions were housed within the special drugs module. The redesigned
questionnaire included them in the hallucinogens module. A primary objective of testing these
items in Phase 2 was to determine the extent to which participants would be familiar with these
drugs. Table 4 shows the number of participants who had heard of each of these drugs. As can be
seen in the table, participants were most familiar with ketamine.
21
Table 4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens
Number of Participants Who Had
Heard of the Drug
14
8
9
Type of Hallucinogen
Ketamine (LS01i)
DMT, AMT, or Foxy (LS01j)
Salvia Divinorum (LS01k)
All of the participants who were routed to a new item about medical marijuana (MJMM)
demonstrated an understanding of the concept of medical marijuana that was consistent with
what was intended in the question.
4.3
Changes Implemented
4.3.1
Phase 2, Round 1
After Phase 2, round 1 interviews, RTI and SAMHSA met to discuss findings of the
interviews. Decisions were made with regard to what changes were needed and where further
attention should be placed during the next round of interviews. A bulleted list of the changes is
provided, along with a brief review of the reason for each change.
•
QI21B—It was learned in cognitive interviewing that this item had the potential to
confuse respondents since the categories carried over from another income branch
question that respondents who are administered QI21B would not see. Categories were
renumbered starting at 1 to be consistent with other items in the instrument.
•
INTRO1—A sentence about the respondent being able to read the questions in the
ACASI system was included in this item. It was deemed during the meeting between RTI
and SAMHSA that it was unnecessary. It was subsequently removed.
•
HLTH04—It was determined in cognitive interviews that some respondents might have a
difficult time answering a question about their height in the formats provided (feet and
inches or meters and centimeters). Two new options were added to the question. These
options allow respondents to indicate a preference to answer the question about height in
only inches or only centimeters.
•
QD26—During the meeting after Phase 2, round 1, SAMHSA and RTI agreed that the F2
options for more information could be improved by tailoring them to the question on
which they appear. The more information option for QD26 was tailored to indicate that a
respondent could ask for more information about the types of work to include when
answering about working in the past week.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, SVYMOTIV, and
STYMOTIV)—During the meeting after Phase 2, round 1, RTI and SAMHSA reviewed
the questions about motivations for misuse of prescription drugs. Resolutions were
reached on question wording and logic. Several decisions were made on the question
logic. These include the decision to ask relevant motivation questions (PRYMOTIV,
TRYMOTIV, SVYMOTIV, and STYMOTIV) of all respondents who reported
prescription drug misuse in the previous 12 months, the decision to ask about a main
reason when respondents reported multiple motivations for misuse (PRYMOT1,
22
TRYMOT1, SVYMOT1, and STYMOT1), and the decision not to allow respondents to
select the "other" reason as their main reason for misuse.
•
"Not all forms" language—During Phase 2, round 1 interviews, issues continued to be
found with the phrasing of the concept that "not all forms" of drugs are displayed in the
drug images. It was decided during the meeting after Phase 2, round 1 that revised
wording would be tested for Phase 2, round 2. The change can be characterized as one
from negative phrasing ("not all forms…are shown") to a more neutral phrasing ("some
forms…may look different").
•
PRIPROX and PRIPROXR—To reduce burden for respondents who previously
completed an interview or proxy interview, it was decided to recreate filter questions.
Two items were created. One allows the field interviewer (FI) to indicate the respondent
previously completed the tutorial, if they are aware of this fact (PRIPROX). If the FI is
unsure or cannot recall whether a respondent has previously completed the tutorial, a
question is asked of the respondent (PRIPROXR). A response indicating previous
completion of a tutorial routes the respondent to a transition screen where the FI informs
the respondent about the following section (TOPROXY).
4.3.2
Phase 2, Round 2
A number of changes to question wording, question logic, and response options resulted
from the Phase 2, round 2 interviews as well.
•
QD11—RTI learned in cognitive interviews that the order of categories could be
confusing to some respondents. Categories 12 and 13 were reversed so that category 12
now represents a high school diploma and 13 means 12th grade but no diploma.
•
CGCOLOR—About half of cognitive interview participants were unable to understand
the question was asking about the new color code system cigarette manufacturers have
implemented. It was decided that this question should not be included in the
specifications.
•
INTROPR (introduction to the pain relievers screener, which is the first prescription drug
module)—Through interviews it was determined that it was unclear to participants that
prescription drug questions were asking about any kind of use. To emphasize that point,
the word "any" was inserted in the first sentence of introductions to prescription drug
modules (INTROPR, INTROTR, INTROSV, and INTROST) so that it reads "These next
questions are about any use of prescription [drug type]."
•
PR06 (use of pain relievers containing morphine)—Several participants were unclear that
nonpill forms of morphine should be included in their response. The SAMHSA/RTI
instrumentation team added to this screen the reminder to include all forms of the pain
reliever that one has used. It was also decided that an additional image showing a vial of
morphine would be added to the screen. Including this image should prompt respondents
to include liquid forms of morphine they received by injection or intravenously.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, and SVYMOTIV)—A
participant responded that he misused a prescription drug to help him deal with his
emotions. None of the existing response options were appropriate. A response category
"to help me with my feelings or emotions" was added.
23
•
Other reasons for misuse (PRMOTOT, TRMOTOT, SVMOTOT, STMOTOT)—It was
unclear to a participant that the instruction involved typing in the "other reason" for
misusing a prescription drug. The word "other" was inserted into the sentence so that it is
clear that respondents should type in the reason they were thinking of when they
indicated another reason in PRYMOTIV, TRYMOTIV, SVYMOTIV, or STYMOTIV.
•
Other reason is the main reason for misuse (PRYMOT1, TRYMOT1, SVYMOT1, and
STYMOT1)—It was previously decided not to allow respondents to indicate that the
other reason was the main reason for their misuse. However, some respondents chose to
list the other reason as their main reason. It was decided to add a category "the other
reason I reported" to account for this.
•
DRPR—This question asks respondents to think about the pain relievers that they
previously reported misusing. Respondents with heavy levels of prescription drug use
were either not clear which pills were pain relievers or which they reported using versus
misusing. The question was edited to include a list of misused prescription pills for
reference. Wording was also edited to consistently refer to misuse.
•
QI12AN and QI12BN—Some cognitive interview participants had difficulty
understanding the types of welfare or public assistance to which these questions referred.
A bulleted list of the types of assistance respondents should include in their answers was
added to this question.
•
INTRTINN—This question was previously administered by an interviewer. Wording in
that question referred to the sources of income "that we just talked about." This wording
was changed, in case it was awkward to use when the computer is reading the questions.
•
Proxy tutorial—During cognitive interviews, it became clear that it was burdensome to
have respondents answer all of the ACASI tutorial items in their interview if they
previously served as a proxy for another respondent. As a result, an abbreviated tutorial
was created for use in the Questionnaire Field Test. This tutorial will ensure that
respondents are presented with information they did not previously receive when they
served as a proxy. Table 5 shows the screens that are a part of the full and abbreviated
front-end and back-end tutorials.
•
Tutorial transition—During Phase 2, interviewers observed that there was no screen for
transitioning the computer from the interviewer to the respondent. Additional instructions
were needed for interviewers to turn the computer over to the respondent when a proxy
was not used in the interview. Such instructions were added to HINSINT, which was
renamed from NOPROX.
•
Calendar—Where appropriate, the SAMHSA/RTI instrumentation team added reminders
for the respondent to press a function key to access either the 30-day or the 12-month
reference date calendars. These reminders were added to most screens in the instrument
that asked the respondent to report behavior in the past month or year. Where reminders
were not included, the calendars themselves were added.
•
INTROINC—In the final paragraph, this item informs respondents that the upcoming
questions refer to the past calendar year rather than the past 12 months that were referred
to in previous questions. However, this would be confusing to proxy respondents because
they were not administered a battery of questions using the 12-month reference period.
24
Skip logic was added to the item to ensure that proxy respondents are not shown the
paragraph referring to a calendar year.
Table 5.
List of Screens Presented for the Full and Abbreviated Tutorials
Full Front-End Tutorial
IntroAcasi1
IntroAcasi3
IntroAcasi4
IntroAcasi2
HeadPhone
INTRO1
INTRO2
Abbreviated Front-End
Tutorial
Back-End Tutorial
IntrAcasi1b
IntrAcasi3b
IntrAcasi4b
IntroAcasi2
Nohphone
INTRO3
GOTDOG
EYECOLOR
ALLAPPLY
NUMBER
BACKUP
PLAYINFO
rangeerr
calendar
calendr2
calendr3
ANYQUES
EYECOLRB
ALLAPPLY
NUMBERB
BACKUPB
PLAYINFB
Rangeerr
Calendar
calendr2
calendr3
ANYQUES
ANYQUES2
25
This page is intentionally blank.
26
5. Discussion
5.1
Summary of Major Changes
The redesigned instrument was tested across a total of six rounds and variations
throughout the redesign pretesting procedures; these rounds included one round of usability
testing, three rounds of cognitive interviewing to test redesigned prescription drug modules, and
two rounds of instrument cognitive interviewing for the total redesigned instrument. Usability
testing was conducted in October 2009. Cognitive interviewing was conducted from January
2011 through February 2012. Findings from these pretests informed many questionnaire
changes. The major changes included
•
deciding upon the "top of screen" layout, use of the wording "in any way not directed by
a doctor" to describe misuse, and making respondents aware of the three misuse criteria
wording for the prescription drug questions;
•
finalizing new wordings to reference generic drugs and that not all forms of prescription
drugs may be pictured on a given screen;
•
removing questions about hospital use of prescription drugs;
•
improving the display of the calendar icon that reminds respondents that the electronic
calendar is available through a function key;
•
refining questions in the health module about height and weight for ease of selfadministration; and
•
adding a transition to the proxy tutorial.
A crosswalk of question text that is included in the main NSDUH questionnaire and the
pretesting questionnaires and ultimately is recommended for inclusion in the field test
questionnaire is included as Attachment F. This document serves as a reference to track question
development across the pretesting phases.
5.2
Usability Testing Advantages
Conducting a series of usability tests followed by increasingly in-depth cognitive
interviews allows for a detailed, iterative examination of potential questionnaire problems. The
usability testing identified early on that respondents did not have problems with the screen layout
and that people understood the purpose and function of the onscreen pictures. One advantage of
conducting a small round of usability testing was that it allowed the study team to eliminate
some errors that might have burdened cognitive interviewing and clouded the findings of those
interviews. A significant recurring theme of comprehending "generic" medications was first
observed in the usability testing. Cognitive interviewers, as a result, knew to probe the questions
about generic drugs for additional issues in subsequent rounds.
27
5.3
Recommendations for Future Pretests
Given the advantages of the usability testing, RTI recommends including it in future
rounds of pretesting. Usability testing is especially adept at blending testing of new equipment
and screen layouts and features. Now that the reference date calendars and drug images are part
of the CAI instrument, there may be usability implications of transitioning to new hardware.
With the addition of a new tablet device that has the potential to display show cards and other FI
materials, usability testing is recommended for any respondent and FI interaction with the tablet.
The recurring themes of the redesigned cognitive and usability testing suggest the need to
investigate the following issues for the field test:
•
The prescription drugs sections are longer. However, use of prescription drugs in the past
12 months was a criterion for recruiting cognitive interview participants. Consequently,
the administration time for the prescription drug questions and the overall administration
time may be shorter with a broader cross-section of respondents in the field test.
Nevertheless, it will be important to track the average interview times during the field test
to identify sections that may be still too long.
•
It will be important for field test interviewers to report notes and experiences on the
proxy transition into the second ACASI portion of the interview. This is a situation that is
likely to be dependent on many variables that are present within the field (rather than lab)
setting.
•
It will be important for the instrumentation team to track reports of usability concerns
with the electronic calendars.
•
Some questions that were administered smoothly in the cognitive testing lab may be
awkward with the wider variety of respondents in the field. Field interviewers will be
encouraged to note awkward wording and choppy or missing transitions.
Overall, iterative rounds of pretesting, combinations of usability testing and cognitive
interviewing, and revisions to instruments and protocols in response to findings have resulted in
an instrument that is well positioned to be tested in a field setting.
28
6. References
Office of Applied Studies. (2008, February 14). The NSDUH Report: Use of specific
hallucinogens: 2006. Rockville, MD: Substance Abuse and Mental Health Services
Administration.
29
Appendix B
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 1 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 1 Results
I.
Highlights
Respondent Characteristics
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain relievers.
Some reported noticing behaviors that concerned them, like needing to take more
medication because the drugs were having less effect for pain relief. They
stopped taking the medicine, lived with the pain for a while, and then would
resume taking the medicine as prescribed. Others in this group reported not liking
the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Identification of Prescription Drugs
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same question.
Some participants in particular reported that their doctors would write the brandname drug on the prescription but the pharmacy would substitute the generic.
Therefore, one issue for remaining rounds would be assisting persons in reporting
February 14, 2011
2
what they actually took, regardless of the name of the drug that was written on a
prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular color.
Additional options also may need to be considered for clarifying for respondents
that pictures may not show all possible examples of a particular pill, or forms
other than pills.
Performance of the Nonmedical Use Criteria
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for stimulants.
An issue for persons who obtained drugs outside of the United States without a
prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
February 14, 2011
3
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
MJ01 – Most participants were able to correctly identify what a blunt was.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
February 14, 2011
4
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
thinking that Vidodan and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
February 14, 2011
5
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
PR03 – One person used Darvocet in the past 12 months.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
February 14, 2011
6
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
PR10 – No one used any of these pain relievers.
February 14, 2011
7
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
ST02 – None of the participants had used any of these drugs.
ST03 - None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
February 14, 2011
8
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
February 14, 2011
9
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
February 14, 2011
10
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
February 14, 2011
11
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
February 14, 2011
12
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
February 14, 2011
13
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
February 14, 2011
14
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
February 14, 2011
15
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
February 14, 2011
16
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
February 14, 2011
17
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
February 14, 2011
18
Appendix C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 2 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 2 Results
I.
Highlights
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
May 9, 2011
2
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
May 9, 2011
3
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
May 9, 2011
4
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
Round 2: No probe.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
MJ01 – Most participants were able to correctly identify what a blunt was.
Round 2: No probe.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
May 9, 2011
5
Round 2: No probe.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens
will become a problem here, we should ask them to specify which “other” hallucinogens they
have used. However, use of drugs other than hallucinogens would not be determined until the
May 9, 2011
6
data are processed. The NSDUH data editing procedures since 1999 continue to treat
respondents as hallucinogen users, even if they specify use of drugs other than hallucinogens.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
May 9, 2011
7
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it
is not clear whether people who have inhaled these products will be able to identify the
substance by the examples given, and that people who have inhaled other aerosol products
(but not these) will be able to answer correctly that they have not used these inhalants. Google
results show that this type of product also is referred to as gas duster and the brand name Dust
Off. We should not continue to edit this question without feedback from users of aerosol
products about whether these are appropriate examples. Feedback from users of aerosol
products also would help to evaluate whether these users can correctly distinguish between use
of these and other aerosol products.
IN01l: Based on observations, SAMHSA requested that the question be revised so
that it no longer references substances that were “listed.” (This revision also
applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
May 9, 2011
8
thinking that Vicodin and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being
shown on the screen.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
May 9, 2011
9
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
PR03 – One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
May 9, 2011
10
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
May 9, 2011
11
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the
module. Perhaps it should read: “Remember, not all forms of these pain relievers may be
shown on the screens.
PR10 – No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
May 9, 2011
12
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
PRHOSPYR1 – Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in
where you used this (these) drug(s). This might draw more attention to the fact that the
question is not just confirming earlier reports of the drug. Another alternative would be to
ask respondents whether they used any of these drugs when they were in the hospital
(rather than “only” when they were in the hospital). If use of specific prescription drugs in
the hospital is reported, ask which drugs the respondents took when they were not in the
hospital. Questions to capture information about use of prescription drugs only in the
hospital may be dropped if respondents continue to have difficulty answering them.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
May 9, 2011
13
ST02 – None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
ST03 - None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
Round 2: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
May 9, 2011
14
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
May 9, 2011
15
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
May 9, 2011
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
16
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
May 9, 2011
17
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
May 9, 2011
18
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
May 9, 2011
19
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
May 9, 2011
20
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
May 9, 2011
21
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the
current versions of these questions for SAMHSA to use in evaluating categories for
combination or deletion.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
May 9, 2011
22
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
Phase 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
May 9, 2011
23
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
Round 2: See HLTHNEW01 results for recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
May 9, 2011
24
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
May 9, 2011
25
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order of
the health conditions in this question.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes:
Round 2: SAMHSA observers gave a reminder for RTI cognitive interviewers to be
sure to administer spontaneous probes in the cognitive interviewing in situations
where interviewers think it is necessary.
Informed Consent Statement:
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
May 9, 2011
26
likely to be small, the IRB may still require prospective respondents to be informed of
this.
May 9, 2011
27
Attachment C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 3 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 3 Results
I.
Highlights
General Themes (All Rounds)
Listed below is a summary of overall themes that emerged from all three rounds
of cognitive interviewing; these themes are not necessarily in the order of
question administration. Specific findings are discussed in further detail in the
remainder of the report.
Participants generally were able to recognize the prescription drugs by name,
by using the pictures, or by using both.
Although more questions were required, asking separately about any use of
specific prescription drugs in the past 12 months and nonmedical use of the
drugs that were used in that period simplified the cognitive task for
participants.
There was some uncertainty about the “generic” term associated with some
drugs. Addition of the word “generic” to the pill images in Round 3 appeared
to be helpful.
The wording “in any way a doctor did not direct you to use it” differentiated
between medical and nonmedical users of prescription drugs. With few
exceptions, participants who used prescription drugs only for medical reasons
could determine that they did not use prescription drugs “in any way a doctor
did not direct you to use (them).” Similarly, nonmedical users could determine
that their use constituted use “in any way a doctor did not direct you to use it,”
even if they used prescription drugs in ways not explicitly listed as examples
of nonmedical use.
Further testing is needed for questions about use of canned air as an inhalant
and injection of stimulants.
Revision of the questions about height and weight (HLTHNEW01 and
HLTHNEW02 series) following Round 1 helped to improve participant
understanding and the task of answering these questions.
For questions about the number of outpatient doctor visits (HLTHNEW03 and
HLTHNEWDK), participants had some difficulty in determining what did or
did not constitute a doctor visit, and in determining their number of visits.
August 17, 2011
2
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
Respondent Characteristics (Round 3)
Overall, 12 respondents participated in Round 3, including 3 adolescents and 9
adults. Each of these participants was recruited from the general public. A few of
the participants were heavy drug users. Two reported use of multiple pain
relievers in the past 12 months, although one of these participants reported
August 17, 2011
3
misusing only one pain reliever. Another participant was a recovering heroin
addict who was intimately familiar with a wide variety of prescription drugs.
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Identification of Prescription Drugs (Round 3)
Respondents were able to identify most of the drugs either by name or picture.
Consistent with findings in earlier rounds, participants often recognized the drugs
by name, although some did find the pill images to be helpful. The Round 3
instrument added the word ‘generic’ to the pill image in order to help respondents
August 17, 2011
4
understand this term. Many respondents understood that generic forms of pills are
not the name brand, and are often cheaper. Some respondents thought that these
generics were “weaker.” Some respondents reported being confused about what
the specific generic pill was a generic of, but seemed to understand that pills had
generic forms and that pharmacies would occasionally substitute generics for
brand name drugs.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
August 17, 2011
5
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
Performance of the Nonmedical Use Criteria (Round 3)
Most respondents were able to differentiate between medical and nonmedical use.
Two respondents misreported their use as medical when it was not. One
respondent took a pill for longer than he had been told to take it, but did not
initially think of that as nonmedical use. Another participant did not think that
she had misused a pill because it was prescribed, because so many of the pills that
she takes are not prescribed. However, she took the pills more often than was
prescribed, and therefore reported misuse. One respondent reported using
“Tylenol 3” without a prescription, but he had really used over-the-counter (OTC)
Tylenol.
While decomposing her use, one participant reported that she had misused the
pills by taking the pills in greater amounts than prescribed and more often than
prescribed. Upon probing, it was discovered that she was describing the same
behavior.
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
August 17, 2011
6
Other Issues (Round 3)
Asking additional probes about familiarity with different names for air duster did
not reveal a term that was more familiar to participants.
The audio pauses in the middle of the questions to allow respondents to look at
the pictures of the pain relievers and stimulants. In some cases, the pause was too
long, while in other cases, the pause was too short.
Respondents did not have any problems with the F2 key, which is used to pull up
additional information about ways of misuse. On occasion, respondents used this
button to aid in their response process.
II.
Ensemble Screener Findings
CG17REV – Round 1: Of the 16 participants who responded to this question, 4
have heard of snus and 3 correctly identified what it was. The fourth person
thought that it goes up your nose.
Round 2: No probe.
Round 3: No probe.
AL01 – Round 1: Most, but not all, participants had heard of the alcopops that
were listed in the protocol. Descriptions included being fruity, girly, ready-made
and flavored. Most participants compared them to wine coolers, and 2
participants mentioned that 4 Loko is similar. These same two participants
identified Sparks as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
Round 3: No probe.
MJ01 – Round 1: Most participants were able to correctly identify what a blunt
was.
Round 2: No probe.
Round 3: No probe.
CC01 – Round 1: Descriptions of crack included: smoked, more addictive, costs
less, crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
August 17, 2011
7
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Round 1: Five participants reported use of cocaine or crack. There were
no inconsistencies in their narrative of use.
Round 2: No probe.
Round 3: No probe.
HE01 – Round 1: Only 1 participant had heard of cheese, while most had heard
of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
Round 3: No probe.
SD17a – Round 1: Almost all participants had heard of methamphetamine. They
had no other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
Round 3: No probe.
HALINTRO – Round 1: Most participants were able to correctly indentify
examples of hallucinogens. Incorrect examples included: marijuana, cocaine,
crack, heroin, alcohol, crystal meth, and Ambien. Participants included a number
of substances that can make one hallucinate, regardless of whether that substance
was a hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 3: Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included marijuana, crack, paint,
markers, hairspray, depression medication, Percocet, and weed laced with
embalming fluid.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
August 17, 2011
8
Action Item: Look at 2010 data for LS01a-LS01f and refusal follow-ups to determine the
numbers of respondents who got different LSFILL patterns.
Peggy also noted that inclusion of the three new hallucinogens will affect the Prior Substance
Use module – i.e., to add these three new substances and follow the same pattern as for LSD,
PCP, and Ecstasy. It is fine to include this in Round 2.2.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
Round 3: No probe.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens will
become a problem here, we should ask them to specify which “other” hallucinogens they have
used. However, use of drugs other than hallucinogens would not be determined until the data
are processed. The NSDUH data editing procedures since 1999 continue to treat respondents as
hallucinogen users, even if they specify use of drugs other than hallucinogens.
Round 3: No probe.
INHINTRO: Round 1: Most participants were able to correctly indentify
examples of inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
Round 3: Most participants were able to correctly identify examples of
inhalants. One reported computer spray duster.
IN01a: Round 1: Responses about a definition of ‘for kicks or to get high’
included: to escape, for a euphoric feeling, excess use, stupor, for fun or a desired
effect, recreational, another frame of mind, to feel good, get outside yourself,
heightened senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
August 17, 2011
9
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
Round 3: Definitions included for fun, trying to get a buzz, doing it
recreationally, just to do it.
Recommendation: Respondents understand this wording and it should continue to be used in
the question.
IN01e & h: Round 1: Two participants reported using an inhalant, but correctly
decided against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Round 1: Most participants had not heard about inhaling markers and
pens to get high. Many were incredulous that this actually happens, and none
reported doing it. However, 2 people had heard of this. (We may observe greater
awareness of this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
Round 3: No respondents reported inhaling markers or felt tip pens.
IN01ii: Round 1: There was a good deal of confusion about canned air. Many
participants referred to this as ‘air duster’. A Google search of ‘air duster’
revealed a number of products by that name. One participant thought that canned
air had to do with filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics air
duster” to make the meaning clearer.
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it is
not clear whether people who have inhaled these products will be able to identify the substance
August 17, 2011
10
by the examples given, and that people who have inhaled other aerosol products (but not these)
will be able to answer correctly that they have not used these inhalants. Google results show
that this type of product also is referred to as gas duster and the brand name Dust Off. We
should not continue to edit this question without feedback from users of aerosol products about
whether these are appropriate examples. Feedback from users of aerosol products also would
help to evaluate whether these users can correctly distinguish between use of these and other
aerosol products.
Round 3: Four respondents had never heard of any alternate terms for
canned air, including computer duster, canned computer duster, canned air
duster, and Dust Off. Two respondents had heard of all of these names.
Three respondents had heard of computer duster. No respondents reported
inhaling this.
IN01l: Round 1: Based on observations, SAMHSA requested that the question be
revised so that it no longer references substances that were “listed.” (This revision
also applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Round 1: Participants seemed to understand the difference between
over-the-counter drugs and prescription drugs. Some descriptions of over-thecounter drugs included: lower dose, pick it off the shelves, don’t alter your state
of mind, not as strong, can’t get high off of them, bought without doctor's
permission, and off the shelf. Prescriptions were described as: containing
narcotics, higher dose, controlled substance, bought from the pharmacist,
stronger, gives you a high, more powerful, more dangerous, easier to abuse, more
expensive, harder to get, need a prescription from a doctor with a signature on it,
narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
NOTE: Screen shots were sent to SAMHSA on May 31, 2011 that showed section headings for
pain relievers. SAMHSA made the decision on June 14 not to add headings to the CAI.
August 17, 2011
11
Round 3: No probe.
PR01: Round 1: Participants reported using a multiple drugs on this list. Two
participants reported thinking that Lorcet and Percocet were the same thing.
Another reported thinking that Vicodin and hydrocodone were the same, along
with Norco. He reported taking Norco as opposed to hydrocodone, but reported
the hydrocodone. One participant was given the pain reliever in the hospital.
Participants were able to distinguish medical and nonmedical use in their
narratives of use. For instance, one participant reported taking the pain relievers
as prescribed on the first three days, but then took nine pills in one day. He
acknowledged that he did not take them as prescribed. Participants discussed
using the pain relievers for recreation, following the instructions for use, or
stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs and
their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to determine
whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being shown
on the screen.
August 17, 2011
12
Round 3: Respondents were able to correctly identify which of these pills
they had taken in the past 12 months. They reported that the pictures and
drug names helped them with this identification.
Similar to the findings of Round 2, respondents reported two different
understandings of the sentence about not including all forms of the drug on
the screen. Three respondents correctly reported thinking that there could
be different forms, such as a liquid or a capsule or additional dosages, which
were not listed on the screen. Other respondents understood that not all pain
relievers were listed, including those that have different names.
Duplicating the labeling of generic pills on the screen appeared to increase
visibility of the term. Most respondents noticed this and most had a general
understanding of its meaning. However, respondents did not necessarily
understand that the generic on the screen was a generic for the other name
brand pills on the screen.
Respondents correctly understood the 12 month reference period.
Recommendation: Consider expanding the sentence about not showing all forms of the drugs
on the screen to provide examples of other forms, such as liquids or patches.
Add "such as different shapes, colors, or dosages." There is less worry about liquids because
respondents are likely to have gotten these only in the hospital.
PR02: Round 1: 11 participants reported using these drugs. One participant said
that he thought that Percocet and Lorcet were the same thing, but the names on
the bottles were different, so he must be wrong. All participants were able to
answer the question based just on the names, although 3 reported that the pictures
were helpful. One reported that his prescription is for Percocet, but his pharmacy
gives him generic oxycodone for insurance reasons, so he sometimes gets
confused about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
August 17, 2011
13
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
Round 3: Respondents did not have any problem identifying which pills they
had used. Five respondents had used Percocet.
PR03 – Round 1: One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
Round 3: One person used Darvocet in the past 12 months.
PR04 – Round 1: One person had used Ultram and tramadol. Another used
tramadol when he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
Round 3: One respondent had used Ultram, but in a liquid form in an IV.
Two respondents used Tramadol.
PR05 – Round 1: Seven people had used Tylenol with codeine. One participant
called this a Tylenol 3. Another reported that he also used Tylenol 4, which is the
pill with the 4 on it. The FDA's Center for Drug Evaluation and Research lists
these as "Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as Tylenol
with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
Round 3: Respondents continued to recognize the names of Tylenol 3 and 4
and to refer to the pill by these names. Six respondents reported using this.
However, one adolescent respondent was actually referring to the OTC
Tylenol and mistakenly believed that this was what was being asked.
Another adolescent respondent reported taking a red and white gel tab OTC
Tylenol. Based on the pictures that were shown on the screen, he decided not
to report this use.
PR06 – Round 1: 2 participants reported using morphine in the past 12 months.
Others reported use, but added that this was not in the past 12 months.
August 17, 2011
14
Participants distinguished between the liquid and pill forms of morphine,
reporting they had it in an IV.
Recommendation: Consider whether an image of a liquid morphine ampule should be shown in
the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
Round 3: One participant reported using both morphine and MS Contin.
The morphine was used in IV form in the hospital.
PR07 – Round 1: Participants were familiar with a number of these drugs. Some
reported lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
Round 3: One respondent used the Fentanyl patch. Three respondents
reported hearing the name China White as another name for Fentanyl. One
of these respondents reported the alternate name as White China. No
respondents had heard of any of the other names that we probed about, nor
had they heard of Sublimaze.
Two participants, when asked, said that the lettering on the package of the
Duragesic was hard to read. Others reported that the pictures were of
limited use because they knew that had not used these drugs.
August 17, 2011
15
Recommendation: Get rid of "China Girl." This is the only prescription drug
with a slang name.
PR08 – Round 1: 2 participants reported use of Suboxone. One was using it to
stop going through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
Round 3: One respondent used Suboxone and Subutex. He used one in
rehab for heroin addiction and another on his own to avoid a relapse into
heroin use.
PR09 – Round 1: 2 participants reported use of Dilaudid. One said that he used
this only in the hospital and asked if that counted. Two participants had used
methadone, at least one of which was at a methadone clinic under supervision.
One used Demerol and said that the picture helped him identify it. Finally, one
person reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12 months
(i.e., not just in pill form) should be reported in PR09. 2. Consider whether instructions need to
include reports of any medications that people took while in the hospital. However, receiving
dosages of pills in a hospital is a different experience than receiving a prescription that will be
filled for use at home. Participants may be unable to identify which pills they took in a hospital,
or to recall that they took pills at all. Do we anticipate that use of pain relievers (or specific
pain relievers) will be significantly underestimated if we do not instruct respondents to report
pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the module.
Perhaps it should read: “Remember, not all forms of these pain relievers may be shown on the
screens.
Round 3: One respondent reported being very addicted to Dilaudid.
Another respondent said that the pictures of the drugs were an excellent
confirmation to his thinking that he took this drug.
August 17, 2011
16
PR10 – Round 1: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
Round 3: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the
combination of the drug names and the pictures of the drugs aided them in
their reporting.
PR11 – Round 1: Other pain relievers that participants reported that they used in
the past 12 months included non-steroidal anti-inflammatory drugs (NSAIDS)
such as naprosyn or diclofenac (brand name Cataflam). Other drugs that were
reported included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
Round 3: Respondents reported that people might report a number of other
pills at this question, including Valium (tranquilizer), Lexapro
(antidepressant), Narco (pain reliever), Paxil (antidepressant), Celexa
(antidepressant), and neurontin (anticonvulsant). [Note: neurontin and
Gabapentin are equivalent.]
PRHOSPYR1 – Round 2: Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
August 17, 2011
17
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in where
you used this (these) drug(s). This might draw more attention to the fact that the question is
not just confirming earlier reports of the drug. Another alternative would be to ask
respondents whether they used any of these drugs when they were in the hospital (rather than
“only” when they were in the hospital). If use of specific prescription drugs in the hospital is
reported, ask which drugs the respondents took when they were not in the hospital. Questions
to capture information about use of prescription drugs only in the hospital may be dropped if
respondents continue to have difficulty answering them.
NOTE: SAMHSA made the decision to drop the hospital questions for Round 3.
IV.
Stimulant Screener
INTROST – Round 1: Most participants had heard of No Doz and Dexatrim. 3
people mentioned 5 Hour Energy as an example of a stimulant. One person
mentioned Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
Round 3: Most respondents were familiar with at least one of the stimulants
that were listed. Many of the respondents had heard of all four.
Recommendation: Keep this list of over the counter stimulants.
ST01 – Round 1: One participant replied that he was not sure which one to
report, because Dexedrine is the brand name, used by the company while
Dextroamphetamine is the chemical name. He was also familiar with the mix of
amphetamine and dextroamphetamine. One participant reported Vyvanse here as
a dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
August 17, 2011
18
Round 3: One respondent used Adderall while another used Adderall XR.
These respondents had no problem answering the question.
ST02 – Round 1: None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
Round 3: One respondent used Concerta in the past 12 months. He stated
that the coating on it made it difficult to dissolve under the tongue.
ST03 - Round 1: None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST04 – Round 1: Two of the participants used Phentermine. One bought it in
Mexico, while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST05 – Round 1: One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
Round 3: One participant used Vyvanse for ADHD.
Some respondents reported that drug names were helpful while answering
these questions. Others reported that the pictures were helpful because the
names were really long. There were no pictures that were not helpful.
August 17, 2011
19
ST06 – Round 1: One participant reported getting Meridia in Mexico. (NOTE:
This is no longer available in the US).
Round 2: No others were reported.
Round 3: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Round 1: Participants did not report using any stimulants in their
lifetimes. One participant later reported that her son was prescribed medicine for
ADD. She took one of his pills so that she could see how it affected him.
However, she neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
Round 3: In response to the question about lifetime use of stimulants, one
respondents noted that we did not include Adipex (which is a form of
phentermine).
A couple of respondents noted that it would be helpful for the numbers on
the pill images to be bolder and more distinct.
V.
Pain Reliever Module Findings
PRL01- Round 1: In describing examples of using a prescription pain reliever in
a way not directed by a doctor, two participants mentioned using a prescription
pain reliever with other drugs or with alcohol. One specifically reported taking
pain relievers with a glass of wine because “they just work better” that way.
Otherwise, participants described ways of use not directed by a doctor that are
included in the definition, such as:
August 17, 2011
20
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining pain
relievers with other drugs or alcohol as a way of using pain relievers that is not directed by a
doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
Round 3: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
PRY01 – Round 1: All participants routed to this question about Vicodin
understood that we were asking this question to get at abuse of prescription pain
relievers or getting them off the street. Many participants used the word “abuse”
when asked what this question was asking. Examples of such behavior include
not using it as prescribed, taking it without a prescription, ‘over doing it,’ getting
high, or selling them. When asked to provide a narrative of the last time
participants used Vicodin, two participants described taking a prescribed medicine
more than the doctor said to because it was needed to make the pain better. A
third participant described having a prescription for dental pain, but using it more
than prescribed because of liking the feeling it caused. Another had a prescription
but was abusing the Vicodin and Norco highly at the time. One participant
mentioned taking their prescription with alcohol. Two participants mentioned
using it without a prescription. One of these used it only as a last resort because
nothing else was available to feed an addiction.
August 17, 2011
21
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them to
use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
Round 3: In response to a probe about what the question was trying to measure,
three respondents said it was to measure abuse. Other mentions include, “using it
without doctor’s orders,” “not taking it any way to cause me to become dependent,”
and “not taking it for other things.” Other responses include:
“If you didn’t follow your prescription”
“If someone gives you their Percocet, if you take it more often or for fun”
“Are you using a prescription drug that wasn’t prescribed by your doctor or
any doctor?”
Respondents who took Vicodin in the past 12 months reported a combination of use
and misuse. Some respondents took pills in greater amounts than prescribed
because of the pain.
PRY01a – Round 1: Participants reported numerous mechanisms by which they
remembered their age at first use, including:
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
August 17, 2011
22
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
Round 3: One respondent reported just remembering the age of first misuse.
PRY02b – Round 1: One participant chose the year because it was not in 2011,
the past couple of weeks. Another just knew the year was 2009.
PRY01d – Round 1: One participant reported that the calendar helped with
choosing the month.
PRY02 – Round 1: The one participant who was asked what this question about
Lortab was getting at reported, “the addictiveness of prescription drugs,” and
“how easy it is to abuse them.” Two participants were asked to describe the last
time they used Lortab. One reported using someone else’s prescription and the
other reported using it when heroin was unavailable.
PRY02a – Round 1: The two participants who received this question distinctly
remembered events associated with the first time they used Lortab (finding
grandmother’s pills and hanging out with friends). One reported that it was
someone else’s prescription and the other reported snorting it.
PRY02c – Round 1: One participant realized he had the year wrong when
answering this question. He mixed up how old he was last summer.
PRY02d – Round 1: The participant remembered the month was July or August
because of how hot it was outside.
PRY03 – Round 1: The one participant who was asked what this question about
Lorcet was getting at described snorting it as using it a way it was not prescribed,
so as to get an immediate high. Two participants answered this question as “no”
because they used this as prescribed, following directions as needed for pain.
PRY04 – Round 1: Two participants were asked to describe what this question
about hydrocodone was getting at. One reported using it recreationally, not
following the doctor’s instructions, taking more than necessary, or snorting it.
Another described a new type of misuse: asking for a larger dose at time of
prescription with the knowledge that a large dose was not really necessary. This
participant still endorsed use of hydrocodone “in a way a doctor did not direct you
August 17, 2011
23
to use it.” When asked to describe the last time the pill was used, one participant
reported getting it for a sore throat in the ER and another reported getting the pills
from an acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as examples
of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
Round 3: Two respondents had used hydrocodone in the past 12 months.
One respondent received a pill from his mother. Another was abusing the
drug because of addiction.
PRY04a – Round 1: Participants did not have trouble reporting their age,
although one wanted to see a calendar for month of last use. When asked how
they were misusing the pills, one reported taking 9 in one day and another
reported taking it with alcohol. A third participant in answering this question
expressed some confusion about Vicodin and hydrocodone. This person reported
thinking that they were the same thing and appeared to be confused by the fact
that we asked about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth date.
It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
Round 3: Respondents did not have any problems remembering their age at
first misuse.
PRY05 - Round 1: One participant described using OxyContin with a
prescription. Her mother kept it locked away and gave it to her at allotted times;
otherwise, she would have abused it. Another participant reported using it once
when a friend provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
Round 3: One respondent took this as directed.
August 17, 2011
24
PRY05a – Round 1: One participant remembered the age at first use easily
because it was in the summer, “after my birthday.” Another had a hard time
determining whether it was at age 19 or age 20. It was after high school but
before college. When using OxyContin at that time, one participant reported
using someone else’s prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
Round 3: Another participant associated use of this drug with a relationship
that he had. When he started using this, he was injecting it.
PRY06 – Round 1: Participants who were asked about what this question about
Percocet is getting at understood it to mean abuse. When asked about how they
used it in a way not directed by a doctor, one reported popping and snorting them,
but with a prescription. Another reported using another’s prescription. One
participant used it but did not report using it in any way not directed by a doctor.
The Percocet made this person sleepy and nauseated, and they did not care to
finish the whole prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
Round 3: Five respondents reported using this prescription. One
respondent did not report misuse when he took one pill for pain that had
been prescribed to him for an earlier procedure. He did not report this
because the bottle said the pills were for pain, and they had been prescribed
to him.
PRY06a – Round 1: Participants remembered their ages by associating it with
events in their lives: sophomore year in college, a friend visiting from Florida, the
diagnosis of migraines. Another had no idea of the age at first use. Participants
reported first using Percocet in a way a doctor did not direct them to use it by
using it without a prescription, mixing it, popping it, and taking it “more than I
should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
August 17, 2011
25
Round 3: One participant had some trouble remembering her age at first
abuse. She estimated the age.
PRY09 - Round 1: When asked what this question about oxycodone is getting at,
one participant who did not answer affirmatively said she was scared of it and
asked, “Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
Round 3: One respondent reported using this as directed. Two others took
this without a prescription.
One respondent asked for a calendar. She did not notice the calendar icon
on the top of the screen.
Recommendation: Make the icon for the calendar bolder or more visible.
PRY09a – Round 1: Participants did not have trouble remembering their ages,
but one asked, “Do you really think that people remember the months and the
years?”
Round 3: One participant estimated her age at first abuse.
PRY10: Round 3: One participant reported taking Darvocet as directed.
She stated that she did not abuse this.
PRY13: Round 3: One participant reported taking Ultram as directed. She
stated that she did not abuse this.
PRY17: Round 3: Three participants had used Tramadol in the past 12
months. One received a prescription for a broken wrist, one received a
prescription for pain, and the third did not have a prescription. Neither
respondent with prescriptions abused the drug.
PRY18 – Round 1: When describing use of Tylenol with codeine “in any way a
doctor did not direct you to use it,” examples included: using someone else’s
prescription because their hydrocodone ran out and a friend gave them something
to tide them over (stave off withdrawal) and taking more than prescribed.
August 17, 2011
26
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
Round 3: Six respondents had used Tylenol with codeine. One respondent
noted that he did not think that he had misused it because he had a
prescription, but after looking at the ways of misuse he was taking more than
were prescribed.
PRY18a – Round 1: Participants did not have trouble remembering their age of
first use.
Round 3: One participant said that she did not know her age at first misuse.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY22 – Round 3: One participant got this prescription from a doctor and
took the pills as directed.
PRY24 – Round 1: One participant described using an IV drip of morphine in the
hospital, and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
Round 3: One respondent received a couple of morphine pills from a friend
and took them all at once.
PRY24a – Round 3: This respondent did not have any trouble remembering
age at first abuse.
PRY28 – Round 3: One respondent received the fentanyl patch at the
hospital.
PRY28a – Round 1: One participant in particular had trouble remembering when
he turned specific ages for first use of fentanyl “in any way a doctor did not direct
you to use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – Round 1: A participant described their use of Suboxone “in any way a
doctor did not direct you to use it” as a backup for when this person could not get
August 17, 2011
27
into a methadone clinic. This made it relatively easy for this participant to recall
the age and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
Round 3: One participant used this in the methadone clinic and then
subsequently bought it from a drug dealer to help with heroin addiction.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – Round 1: A participant who answered this question affirmatively
recalled using Demerol when he had run out of codeine and/or Vicodin. A friend
with chronic pain shared the Demerol with him to extend his prescription. He
was able to recall his age by the month and year and by his birth date.
Round 3: One participant used the Demerol as prescribed.
PRY33 – Round 1: A participant remembered using Dilaudid in the hospital as
an injection for back pain. He reported using this according to doctor
instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
Round 3: This participant goes to the hospital to get Dilaudid, because she is
addicted to it. If she cannot get Dilaudid, she will seek out other drugs.
PRY33a – Round 3: The first time she used it, she had a prescription but she
used more than that.
PRY34 – Round 1: A participant routed to this question answered it as “no.” The
person had not used methadone “in any way a doctor did not direct you to use it”
because the person attends a methadone treatment program every day and
receives the methadone there as directed.
PRL02 – Round 1: One participant reported using a pain reliever with alcohol as
a type of use not directed by a doctor in response to this question.
PRM02DKRE – Round 1: The participant who received this question about their
“best guess” thought their answer was accurate.
August 17, 2011
28
PRM03 – Round 1: Two participants reported how they were able to remember
whether or not they were drinking alcohol. One said because it was yesterday,
and another knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
Round 3: No respondents had taken the pills and drank at the same time.
One respondent was not sure how to report this, because she drank within a
week of taking the pills and was not sure how long alcohol stays in your
system. She ultimately decided to answer ‘no’.
PRY41 – Round 1: One participant understood this question but volunteered that
the wording “at least once” threw her off. She thought that the answers would say
the same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
August 17, 2011
29
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
Round 3: One participant reported “some other way.” She included this
because she injected the drug instead of taking it by mouth. She also picked
the other four response options. After probing, she stated that she was
counting the same behavior of taking more drugs than were prescribed as
both number 2 and 3.
PRY42B – Round 1: Participants did not have trouble remembering how they got
pain relievers the last time. One additional way of getting a prescription pain
reliever reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the current
versions of these questions for SAMHSA to use in evaluating categories for combination or
deletion.
Round 3: One respondent said that she got it from more than one doctor
because she would get her prescriptions at the ER, and there are multiple
doctors that prescribe them.
VI.
Stimulants Module Findings
STL01 – Round 1: A participant reported not having a prescription for
phentermine, but in Mexico, they did not need one. This person pointed out that
you can buy prescription drugs in Mexican drug stores and airports. Technically
in Mexico they were not prescription drugs.
August 17, 2011
30
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in any
way a doctor did not direct you to use it,” and how to instruct respondents to count or not count
this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – Round 1: When describing last use of Adderall, a participant reported
that a friend provided it in 2009. There was no prescription and they were mixing
it with other things.
Round 3: The participant answered “no.” The participant had a
prescription for ADHD and last took Adderall this morning, according to the
prescription.
STY01a – Round 1: A participant reported remembering the exact night in 2009
hanging out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY02 – Round 3: The respondent took Adderall XR for ADHD. He had a
prescription.
STY03a – Round 1: A participant remembered their age at first use of Dexedrine
very well because it was associated with a recent move and new
classmates/friends.
STY08 – Round 3: This respondent did not abuse Concerta.
STY20 – Round 1: The same participant who got phentermine in another country
described it here. Another participant reported using phentermine strictly to lose
weight, as directed, and answered STY20 as “no.”
STY25 – Round 1: The same user of phentermine mentioned getting it outside
the U.S. Another participant reported snorting as a third way of use.
Round 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY25a and STY25b: No respondents in any of the three rounds reported
using stimulants with a needle in the past 12 months.
August 17, 2011
31
STY26b – Round 1: One participant had been prescribed stimulants for ADD
while in rehab. When the participant emerged from rehab, this person told the
doctor about having a prescription for stimulants and was able to get another
prescription easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
VII.
Health Module
HLTHNEW01 – Round 1: 5 participants had difficulty entering their height into
the computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom of
the task bar. Participants also did not read the screen about entering inches. They were still
looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
Round 3: No respondents had any difficulty entering their height or weight.
Recommendation: Questions are functioning properly. Keep as is.
HLTHNEW01a – Round 1: Participants were not accustomed to seeing the
question mark that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – Round 1: No problems entering weight. See HLTHNEW01
results for recommendations.
Round 2: See HLTHNEW01 results for recommendations.
Action Item: Do not play the audio for the answer choices for height and weight because those
are included in the questions.
HLTH05 – Round 1: One participant reported 8. Upon probing, he said that he
was in the hospital for 8 days and 7 nights and had missed the part of the question
that specified nights.
August 17, 2011
32
HLTHNEW03 – Round 1: A few participants included dental appointments in
their estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
HLTHNEW03 – Round 1: Most participants had a reasonable understanding
about what a health care professional was. One participant would not count a
chiropractor, or a person at the eye care center because they have not been to
medical school. One participant said that there were none at the drug treatment
center, while another reported that visits to the methadone clinic would count.
Other participants expressed doubt about including specialists and physical
therapists. One participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a number
of constructs, such as a reference period, thinking about your own health versus someone else’s,
the definition of a health care professional, and exclusions to the rule. The respondent burden
has the potential to be high, as does the measurement error.
Round 3: Depending upon the extent of visiting doctor offices, respondents
struggled with accurately counting the number of visits. One respondent
initially included dental visits despite the instruction not to.
Action Item: Delete the explanatory text from HLTHNEW03. Consider moving the
instructions to appear before the question, and include a probe.
HLTHNEW05 – Round 1: There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows, but
there should be a linkage between the series of questions.
HLTHNEW06 – Round 1: One participant reported that his dentist asked him
about his alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
August 17, 2011
33
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04 because
the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06 focuses
only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
Round 3: Respondents did not have a problem with this question. Many
reported that doctors asked if they drink and then how often and how much.
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order
of the health conditions in this question.
HLTHNEW11 & 12a – Round 3: The respondent was first diagnosed at their
current age, but the follow up question asked whether they had cancer or
heart disease in the past 12 months.
Recommendation: Change the specs to skip questions about whether respondents had specific
conditions in the past 12 months if the age at first diagnosis equals their current age.
Action Item: Implement this change.
HLTHNEW20 – Round 1: One participant asked if we were referring to the past
12 months or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – Round 1: All three participants who reported high blood
pressure were confused by this question. One participant said that he has been
told that it was a little high, but was not sure that this was a chronic state. One
participant said that after the first time, the doctor knew that he had it and they
August 17, 2011
34
talked about it, but he didn’t tell him that he had it again. He said that he was on
medicine that was controlling it. The third participant said that they discuss his
high blood pressure, but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - Round 1: There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an awkward
stopping point, and does not seem to flow after the last few questions. Recommend adding
another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – Round 1: After the stimulant main module, one participant
reported that many of the questions are redundant. However, no other participants
made any comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes: Round 2: SAMHSA observers gave a reminder for RTI cognitive
interviewers to be sure to administer spontaneous probes in the cognitive interviewing
in situations where interviewers think it is necessary.
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
likely to be small, the IRB may still require prospective respondents to be informed of
this.
Round 3: No additional issues.
August 17, 2011
35
Attachment D
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 1 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.1 Results
December 7, 2011
Respondent Characteristics
During the first round of Phase 2 interviewing, survey methodologists interviewed twenty
respondents. Of these twenty, ten were adolescents and ten were adults. Almost all respondents
in the first round of Phase 2 interviewing were recruited from the general population. A couple
of respondents were recruited from drug treatment centers in the Raleigh, North Carolina area.
In order to be recruited in the cognitive interview sample, respondents did not necessarily need to
report use of illicit or prescription drugs in the past 12 months during the telephone screener.
Therefore, most respondents were casual users or often non users of the drugs that were tested in
this round. Few were expert or heavy users.
Round 2.1 tested the transition from the questionnaire asking health insurance and income items
using interviewer administration to using a self-administered mode. Particularly, transitioning to
a proxy respondent who completes the health insurance and income items on behalf of the
respondent using self-administration warranted thorough testing. Interviewers and parents
provided valuable feedback about this process.
In general, proxy respondents were able to successfully answer questions about adolescent health
insurance and family income. The ACASI tutorial worked well, and proxy respondents reported
understanding the questions as well as why they were being asked to answer them. Individual
findings are listed below.
In addition, new questions measured details of military service and status. In order to thoroughly
test these new items, round 2.1 interviewed five veterans of the military. These veterans
provided feedback on the new items in the front end CAPI section of the interview.
Specific feedback on individual items is detailed below.
Veteran Questions
V2a – Respondents understood the use of the phrase “full time in the Reserves.” None of these
respondents reported ever being in the Reserves.
QD10c – Respondents were able to correctly define a combat zone. Among the definitions were
“being in combat,” “being in the fighting,” and Iraq and Kuwait. One respondent said that she
did not serve in a combat zone as women were not allowed to do so when she was in the
military. Another respondent incorrectly reported that he served in a combat zone. He had a
dangerous assignment of guarding prisons in the United States, so he reported yes. After
repeating the question, the respondent acknowledged that he should respond ‘no.’ Respondents
identified with the mention of hazard pay in this question.
2
Hallucinogens
LS01i – Eight respondents had heard of Ketamine. Most reported hearing about it on TV.
LS01j – Three respondents had heard of DMT and one had heard of Foxy. No respondents
reported use.
LS01k – One respondent thought that salvia divinorum was a substitute for marijuana. Another
thought that it was synthetic marijuana. A third person reported hearing of the substance.
Inhalants Screener
IN01ii – No respondents reported use of computer keyboard cleaner just to get high. Fewer than
half of the respondents knew about the spray. A few respondents had heard the term “spray
computer cleaner” or “compressed gas duster” before. One respondent suggested identifying the
spray by its brand names. Others did not have many recommendations for other words used to
describe the spray.
Methamphetamine Module
ME01 – Three respondents in the sample reported lifetime use of methamphetamine. One
respondent reported use of it twenty years ago and one respondent reported using it for two years
three years ago. No respondents reported 12-month use of methamphetamines. Those that
reported lifetime use did not have any problems with the module.
Prescription Drugs
PR01 – One respondent thought that we were only interested in misuse of prescription pain
relievers at this question. A couple of respondents were unaware that the next eight screens were
going to ask about other pain relievers and therefore wanted to report their use here.
Recommendation: Continue to measure whether asking about illicit use of substances before
pain reliever use is priming the respondent to only think about misuse.
Recommendation: Discuss whether we should convey to respondents that more than one screen
asks about pain relievers.
Generic Pills
Although most respondents understood that generic drugs are a cheaper version of name brand
drugs, they were not always clear on what the name brand equivalents of the generic drugs
pictured were. Two respondents reported that generic drugs are over the counter.
3
Not All Forms
The statement that not all forms of a particular drug may be shown on a screen continues to
cause problems for respondents. Many respondents reported thinking that it was referring to
pain relievers with a different name than the one shown on the screen. Others correctly thought
that we were acknowledging that certain pills might take various forms, not all of which were
pictured. A couple of respondents said that they did not understand the statement. Despite the
statement about not all forms, one respondent did not report intravenous use of morphine
because only the pills were shown on the screen.
Recommendation
Evaluate whether new probes should be used to test these questions in Round 2.2 or if the
wording should be further revised.
Pill images
Respondents that had used the drugs report that the pill images aid in their recall. Those that
report not using the pill images often report not using any pills.
Tranquilizers and Sedatives
No new issues were detected with the inclusion of the tranquilizer and sedatives module.
TR06 – When asked whether there were any other tranquilizers that respondents would report, a
few were mentioned. Paxil, Valium, promethazine (phenergen), and trazedone were reported by
respondents.
INTROSV – Respondents had for the most part heard of Benadryl, while fewer respondents had
heard of the other sedatives. Suggestions for other OTC sedatives included Adderall, ibuprofen,
motion sickness pills and melatonin.
Definitions of misuse
Respondents were able to distinguish between use and misuse of prescription drugs and reported
their use correctly.
Special Drugs
SD01 – Most respondents who named examples mentioned “Nyquil.” Other examples were
Sudafed, Tylenol, aspirin, Motrin, Benadryl, and Claritin. Many mentioned “cough syrup.” One
R he’d heard of ‘purple stuff.’ He wasn’t specific and probing lead to him discussing some drink
that a professional football player had been caught with that caused a suspension or fine. He said
he thought it was codeine or Dimetapp or Sudafed like substance. (Note: he is probably referring
to prescription strength cough syrup here.) Only 2 respondents had used OTC medicines. One
4
had taken Benadryl and another had taken Nyquil while he was in bootcamp because he was not
able to drink alcohol at the time.
SD21 – One respondent suggested that “the hospital” be added to this item.
Health
HLTH04 – One respondent only knew he was 54 inches tall, and he did not know how to convert
it to feet and inches.
HLTH19 – Four respondents expressed not knowing whether the dentist should count. None
counted the visits initially, but considered including them after a probe. Of the four, three
decided to count them and one did not.
Recommendation: Wording about not including dentist visits was recently deleted from the
specifications for this item. RTI and SAMHSA should discuss whether this is still appropriate.
HLTH22 – Four respondents reported doctors discussing their drinking with them.
Back End Demographics
QD17 – Fifteen respondents said that they would report being in school even if they were on
summer break. One respondent said he did not know and another said he would say “no” even if
he was just on summer vacation.
QD18 – Respondents were asked a hypothetical question about this item. Most were asked if
they would report being in 7th or 8th grade, if the question was administered during the summer
after they had finished 7th grade. Only one of 7 people probed said “7th grade”. Two participants
correctly reported about their current situation, saying that they would report the grade that they
were “going into.”
QD26 – One adolescent was confused as to whether this question asked about the different
places he had worked or the number of bosses he’d had. Another adolescent was not sure
whether she should count babysitting. She decided that she would NOT count it.
QD49 - One respondent had trouble with ‘DK’ for items QD47 & QD48. She had forgotten that
she needed to use the function keys to indicate this as her answer. She answered ‘DK’ to both
QD47/48. Another participant said that was a difficult question to answer because it depended
on the job one was applying to. Only certain job classifications are subject to testing, which at
her employer (a local government) were ones classified as ‘drug sensitive.’ These jobs are ones
like police officers, welfare workers, or anyone who drives a government vehicle. They do not
test for other job classifications, like the one she is in.
QD50 – The 4 Rs who were probed all mentioned that this question was asking about being
tested “at random.”
5
QD54 – One adolescent R mentioned she spent half the week in one home and half the week in
the other, then asked if she should count “the one I spend the most time in?” It was not hard for
her to choose the one she spent the most time in.
Proxy Tutorial
Two respondents had trouble with entering a “1” to answer the first question in the tutorial.
Recommendation: Change the first question in the tutorial to a “Press Enter to continue” screen.
Two respondents thought it was odd to be asked about the teenager’s income, since they did not
work.
In the income branching questions, one R tried to enter “1” for the first option, but the
appropriate number was “21.” R recognized her mistake and entered “21.”
Recommendation: Change the response options for this question to begin with 1, as opposed to
21.
One respondent complained that the tutorial “went into too much detail.” R thought it was too
slow and instead of letting his intuition take over it bogged him down. He mentioned reading the
script that was for the interviewer (on the demo screens where the interviewer points to the keys,
etc.) and said that “The script was for you but I was thinking of what it said rather than what it
was trying to teach me.”
Additional Issues
In order to reduce respondent burden, a filter question could be added to identify proxies who
have already been introduced to the interview program, whether during their own interview or
while serving as a proxy for another respondents.
Additional instructions and introductions are needed in the back end demographics section. In
cases where a proxy is not nominated, the screens do not clearly indicate which questions are
interviewer administered and which are self-administered.
6
Appendix E
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 2 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.2 Results
February 29, 2011
Respondent Characteristics
Twenty participants were interviewed during the second round of Phase 2 Questionnaire
Redesign Cognitive Interviewing. Summary information about the participants is presented in the
table below. This table shows that the sample was comprised largely of adults, females, and
those who had significant experience with drugs.
NSDUH Cognitive Interviewing Round 2.2
Participant Information Summary (n = 20)
n
Adults
14
Adult mean age = 39
Adolescents
6
Adolescent mean age = 14
Females
12
Enrolled in treatment program
5
(currently or within past 12 months)
High use participants
11
All participants were recruited from the general population using online ads that were placed in
Chicago, IL; Washington, D.C., and Research Triangle Park, NC. However, since most users
interviewed in the first round were casual users, the second round targeted heavy drug users and
individuals who had recently received treatment. Of the 14 adult interviews, 5 were currently
enrolled or had been enrolled in a treatment program in the past 12 months and 11 were
categorized as ‘high use’ participants. High use was defined as having used multiple prescription
drugs.
As in the previous round, Round 2.2 tested the second transition from interviewer to ACASI
administration for proxy responses to health insurance and income items. Feedback from parents
and the interviewers about the transitions was provided. All proxy respondents were able to
successfully transition and answer proxy questions. Findings about the process are provided
below.
Some questions were tested for the first time in Round 2.2. These questions asked respondents
about the color of the pack of cigarettes they smoked and about their motivations for misusing
prescription drugs. By recruiting ‘high use’ participants we were able to thoroughly test these
new items.
Specific feedback on individual items is provided below.
2
Veteran Questions
QD10c – Recruitment in Round 2.2 did not focus on Veterans as heavily compared to Round 2.1.
Therefore, only one participant was routed to this question. She correctly defined a combat zone.
She initially indicated that a combat zone could be a “zone even in Peacetime” but clarified that
it meant “places where you are taking on fire.”
Front End Demographics
QD11 – Participants were generally able to answer this question with no trouble. Two
participants initially failed to understand that the question asked about the highest grade finished,
but ultimately they realized the meaning of the question and changed their answers to the correct
one. One participant had trouble with the numeric categories. She answered “12” but then
changed her answer to “15.” When the interviewer probed to learn more about the change from
not finishing high school to taking some college credits the participant realized she should have
selected “13” to indicate earning a high school diploma. A scripted probe lead to one final
noteworthy comment about this question. One participant had taken credit courses during her
service in the military which she wanted to report. Since no option was available for having
taken military training/courses she realized her best option was to answer “13-regular high
school diploma.”
Recommendation: This question asks about the highest grade completed. Response categories
are generally the same as the number of the grade. However, the form of category 12 is markedly
different. If a respondent answers ‘12’ they are indicating that they did not receive a high school
diploma. Respondents who do not thoroughly review the text accompanying the response options
may be inclined to answer ‘12’ to indicate they have a high school diploma. Therefore, we
should consider changing the order of the response options so that this inclination will not invite
measurement error. An alternative would be to add a question about receiving a high school
diploma if the respondent answers 12.
Tobacco
CGCOLOR – Eight of the participants reported smoking cigarettes in the past 30 days. Of these,
four considered the question to be “self-explanatory” and found it easy to recall the color of the
pack. The other half of participants, however, noted complicating issues with the question. Two
found the question somewhat confusing because packs of cigarettes have more than one color on
the packaging. One of these participants noted that he wasn’t sure whether he should or could
select more than one color or if he should “just pick the main color.” A similar comment was
made by another participant who said this question made her think of the two different colors on
the packaging—the background color (black) and the color of the text (blue). One participant,
who previously indicated the question was self-explanatory, said that his brand of cigarettes
3
(Newport) only comes in green packaging. Upon probing he elaborated that the shade of the pack
indicates the strength of the cigarette—lighter colors indicate lighter and darker colors indicate
fuller flavors. Finally, one other participant was unable to answer the question because he had
been given a single cigarette from someone else and wasn’t sure what color the pack was.
Respondents did not necessarily understand that this question was asking about the strength of
the cigarette and its association with color.
Recommendation: Consider whether language could be added to clarify what we mean by
“color” in the question. We should continue to attempt to reach the tobacco consultant about
these questions.
CGMENTH1 – One participant volunteered that this question was difficult for her to answer.
She smokes Camel Crush cigarettes, which allow the smoker to choose between regular and
menthol flavors. The cigarettes are by default regular flavor. By squeezing a point on the
cigarette a smoker can activate a menthol flavoring. The participant sometimes smokes them
only as non-menthol, other times she smokes them only as menthol, and occasionally she
switches from non-menthol to menthol at some midpoint.
CG34 – One participant indicated this question was difficult for him to answer. He had only used
cigars to make blunts, but had never smoked tobacco cigars. He noted this prior to answering the
question. The participant indicated he would answer “yes” to this question because it does not
specify what the cigar had in it.
MJ01 – One participant indicated this question was difficult to answer because she was not sure
what was meant by “used.” She pointed out that the alcohol questions were very specific about
what counted as drinking (a sip or two does not count), but we were less specific about
marijuana. She had taken one toke of a marijuana cigarette in her lifetime and was not sure that
counted as “used.”
Hallucinogens
LS01i – Six participants had heard of Ketamine. There was variety in the places people had
heard about Ketamine, what participants thought the drug was intended for, and the effects the
drug has on a user. Several participants indicated they thought it was a prescription drug for
animals. One thought it was the “date rape drug.” One participant indicated she had heard about
it from kids in her neighborhood and another had heard about it on television. Only one
participant had tried ketamine. He used at a rave “back when he partied with some white boys.”
He said it is “like meth but it’s for people who don’t want to do meth” and that it “makes you
feel like [expletive] in the morning.”
LS01j – One participant had heard of all three, one had heard of AMT and Foxy, one had heard
of DMT only, and one had heard of Foxy only. In addition, one participant reported using DMT
with the same group with which he had used Ketamine. He said it had a similar effect as
4
marijuana. He decided to not use DMT anymore and to instead “just use weed” because he got
the same effect from it and marijuana was from a plant.
LS01k –Six participants had heard of salvia. One had heard of it on television, one from a movie,
and four did not specify where they had heard of it. Two of the participants said saliva is similar
to marijuana, while one said that when you smoke it you “lose your mind for 20 minutes.” Two
indicated it was legal and one said he thought you could buy it at smoke shops.
Inhalants Screener
IN01ii – Findings on this item were similar to those in Phase 2.1 interviews. None of the
participants reported use of computer keyboard cleaner just to get high. Fewer than half of the
participants knew about the spray and of these only two made comments which reveal they
understood it could be used to get high. Seven participants had heard the term “spray computer
cleaner” or “compressed gas duster” before. One participant said he had heard it called by the
brand name “Dust Off.” And, one participant thought that computer keyboard cleaner might be
called “alcohol” perhaps suggesting the participant thinks that is a main ingredient.
Methamphetamine Module
ME01 – Just one participant in this round of interviews reported lifetime use of
methamphetamine. It was during a time when a close relative of the participant was diagnosed
with cancer. Someone the participant knew had experience with methamphetamine and
suggested it could help with relaxation. The participant reported no issues with the module.
Generic Pills
Each of the participants characterized generic drugs as a less expensive version of a brand name
drug. Three participants made comments about the quality of generics compared to name brand
drugs, two of which questioned the effectiveness of generics compared to name brand drugs.
Not All Forms
There continue to be problems associated with the statement that not all forms of a drug may be
shown on a screen. Participants in this round clearly thought this statement referred to how pills
look, but did not seem to understand that forms also included other modes of drug
administration—i.e., injectable drugs. For example, many of the participants indicated that the
statement meant that generic alternatives might not be presented when the brand name drug was
pictured, that not all shapes and/or colors would be shown, and that the text imprinted on the pill
might look different than what is shown. None of the participants mentioned the possibility that
drugs could be something other than a pill, and as mentioned below two participants noted
trouble with answering PR06 because morphine is often administered through an IV.
5
Recommendation: Discuss modifying the language to clarify that “not all forms” means both
the color/shape and mode of administration. This language has been tested and modified over
many rounds of interviewing, so we should review previous revisions and findings when
exploring whether to revise it further. See also the recommendation following PR06.
Pain Relievers
PR01 – It was evident in this round of interviewing that there is confusion about what kind of use
these questions are about. Four participants thought we were interested in any kind of use (use or
misuse), one thought we were only interested in misuse, and seven thought the question was
asking only about prescribed use. Upon probing one participant paged back to INTROPR and
pointed out that it only talks about “use of prescription pain relievers.” To him this suggested he
should only think about use that is prescribed.
Recommendation: Consider revising the following sentence: “These next questions are about
the use of prescription pain relievers” to say “These next questions are about any use of
prescription pain relievers.” Bolding ‘any’ may reinforce that we are measuring both use and
misuse. This probe was added to the instrument to address concerns that respondents would think
that we were only measuring illicit use of prescriptions, because most of the previous substances
that were asked about are illicit.
PR06 – Two participants had difficulty understanding that non-pill forms counted in this
question, which asks about morphine.. One asked whether “injections counted” and the other had
had morphine through an IV drip and initially failed to report it.
Recommendation: Consider adding a reminder on this screen that not all forms of the drugs
may be shown on the screen, because morphine is very likely to be administered in liquid form.
PRY42B – One participant volunteered that she had a problem with only being allowed to select
one way for the last time she used a prescription pain reliever. She said that she got the drug in
multiple ways. By forcing her select one way the respondent said we were forcing her to provide
a socially desirable answer, because she would select the way that did not make her look like a
drug addict.
Recalling age of first misuse
Participants used landmark events to recall dates of first use. For example, one participant
recalled first use of Xanax by recalling the group of people he was “hanging around with” during
that period of his life and also recalled his approximate age when spending time with this group.
Three other participants recalled first use because it was at a time of stress associated with life
changes, or medical events they or their family members experienced.
6
Definitions of misuse
Participants had no problems making the distinction between use and misuse of prescription
drugs. Misuse was correctly reported. One respondent reported that she sold her prescription
drugs, which should be considered misuse. Upon probing she indicated that a doctor did not tell
her to sell her pills.
Ways in which participants misused prescription drugs
Only one participant mentioned misusing a prescription in a way different than defined in the
section introductions. She indicated she used less of the drug than prescribed. She believed this
way fit in the “in any other way a doctor did not direct” category. Most other participants
indicated they had used the drug without a prescription. However, several made a point to say at
different junctures that they “knew” how a doctor would prescribe it and their use was consistent
with that “knowledge.”
Motivations for misuse
Participants were generally able to answer questions about the reasons for use the last time they
had used a drug in a way a doctor had not prescribed, and in a similar manner most had no
problem identifying the main reason for this misuse. A few issues were noted, however.
PRYMOTIV – Of the participants routed to this item, three mentioned either an issue with or
problem in answering questions about the reasons for misuse. One participant had previously
indicated she had used a prescription in a way a doctor had not directed her to by using less than
prescribed. She noted that there was no response alternative that fit her situation. One respondent
noted that he took the pills to escape his emotions, and inidicated that a response option about
dealing with emotions should be included.
PRMOTOT – When one participant selected “some other reason” for PRYMOTIV and was
presented PRMOTOT he was unclear the question was asking him to specify the “other reason.”
Recommendation: Revise the wording to say, Please type in the other readon you used [RX
DRUG] that time. This revision inserts “other” into the sentence.
PRMOT1 – Participants did not display or report any problems answering this item. However,
one interviewer noted that currently we do not allow “some other reason” from PRMOTOT to be
selected as the main reason in PRMOT1.
Recommendation: Discuss including “some other reason.” This will require developing an
alternative audio fill for the “some other reason” such as “The other reason you reported.”
Alternatively, we could acknowledge to respondents that they can only pick from reasons 1-7
7
from PRMOTOT as the main reason, but we would need to consider the limitations of this
approach.
TRYMOTIV – Of the participants routed to this item, one again noted that the response
categories did not fit her situation because she answered “yes” to misuse, but had actually used
less than a doctor had directed her to. Another participant mentioned difficulty answering this
item because he did not think his depression fit in any of the categories.
SVYMOTIV – One participant again noted that he thought we should have a response alternative
that captured dealing with emotions. The other participant who was routed to this item was able
to successfully match his answer to a response alternative.
Special Drugs
SD01 – Consistent with findings from Phase 2.1 interviews, popular examples of OTC drugs for
use just to get high were “Nyquil,” “Benadryl,” and the generic terms such as “cough syrup,” or
“cough medicine.” Others mentioned were “mouthwash,” “paragesic,” “triple-C,” “No-doze,”
“Sudafed,” “Vicks,” Robitussin,” “Formula 44,” “sleep aids,” “Tylenol,” and “Tylenol PM.” One
participant mentioned “purple drink” which seemed to be similar to a substance a participant in a
Phase 2.1 interview mentioned. However, we believe a prescription drug may be involved with
this substance. One other participant mentioned “red juice,” describing an OTC syrup.
Blunts
MJMM – It was clear to all five of the participants who were routed to this item that the question
was asking about marijuana prescribed by a doctor. In response to the probe, all five mentioned
prescribed use. Two participants differentiated this use from use with marijuana bought from
“someone else” or “someone on the street.”
Substance Dependence and Abuse
DRPR -- One participant reported heavy use of prescription pain relievers. Some of this was
misuse, but much of it was prescribed use. It was evident she was not clear this question was
asking about misuse.
Recommendation: Consider listing drugs the respondent has misused to clarify which drugs
respondents should be thinking of while answering this question. This will also provide a
reminder about which drugs are pain relievers, as opposed to other classes of drugs.
Recommendation: Confirm that these questions should use the clause “or that you used only for
the experience or feeling they caused” as there seemed to be some disagreement or confusion
about this.
8
Drug Treatment
TX04c – One participant was unsure whether AA should be counted for this item.
Recommendation: Discuss moving the AA item earlier in the sequence of treatment questions
so that it comes first.
Health
HLTH04 – None of the participants reported or displayed any problems answering about their
height. One participant was unsure how to continue after typing in his answer for “feet.” He
asked, “Do I press enter?”
HLTH18 – One adolescent participant was unclear whether the question included going with
someone else to the hospital for their inpatient stay.
HLTH19 – Generally speaking, participants listed and counted valid medical professionals to
report an answer. One reported a visit to a dentist.
HLTH22 – Three participants reported doctors discussing their drinking with them. These
reports included only being asked on a form, being asked anytime one saw a new doctor, and a
general discussion about “overindulgence” that was unrelated to actual use.
HLTH25 – One participant listed a number of health conditions, but upon probing indicated that
he was reporting times his doctor had asked or discussed these with him. It seems he may have
confused language in HLTH22 with that in HLTH25—that is language about a doctor discussing
these issues with him.
QI12AN – Two participants had difficulty understanding this question. Each had trouble
understanding what types of welfare or public assistance should be reported.
Recommendation: Add a bulleted list of the types of welfare respondents should include when
reporting.
Back End Demographics
QD17 – Nine participants said that they would correctly report being in school even if they were
on summer break. One said he was answer “no” even if he was on spring or summer break.
QD18 – A hypothetical question was asked of participants about being between 7th and 8th
grades. One participant incorrectly said she would say the 7th grade.
QD07 – One participant was unhappy with the omission of a response category for cohabiting.
He said he has been in a committed relationship for approximately four years and thinks the
category “have never been married” does not accurately reflect his situation.
9
QD26 – None of the participants had trouble answering this question about whether they had
worked at a job or business in the past week.
QD49 - Participants did not have any problems understanding this item about whether their
workplace had drug or alcohol testing as part of the hiring process.
QD50 – One participant was probed on this item and he correctly understood what was meant by
drug testing “random basis.” He said it means “out of the blue…like names out of hat and you
pick one.”
PERAGEYR – One participant was unable to answer this question about the age of each
household member because he did not know his parents’ ages.
Income
INTRTINN – This item about sources of income may be confusing to some respondents since
the item is now self-administered whereas it was previously interviewer administered.
Recommendation: Consider modifying language for this item since it is now self-administered
where as it was previously interviewer administered. For example, use of the term “we” in the
clause “those we just talked about” might be awkward since the respondent is interacting with a
computer.
Proxy Tutorial
In general, proxy participants for this round found the tutorial “easy” or “straightforward.” One
proxy participant asked for clarification about whether she could proceed before reviewing all
instructions. In another interview, an interviewer noted that for proxy respondents who have
previously completed an interview or proxy interview there was no transition screen making for
an awkward transition to the ACASI questions.
Recommendation: Add a screen at the beginning of the front-end proxy tutorial that allows
respondents who have previously completed a NSDUH interview or proxy interview to complete
an abbreviated rather than full tutorial.
Recommendation: We should add one more screen to the proxy tutorial transition that indicates
that the interviewer should turn the computer over to a proxy who has already completed the
tutorial elsewhere.
Recommendation: INTROINC says, These questions refer to the calendar year 2011 rather
than to the past 12 months that were referred to in some earlier questions. We recommend not
using this paragraph when there is a proxy, because they have not answered a large battery of
questions with a reference period of the past 12 months. QHI14 is the only question in the health
insurance and income items that asks about the past 12 months. This question asks about the
number of months in the past 12 that the sample person was without health insurance.
10
Additional notes
The Substance Dependence and Abuse section, starting at screen INTRODR, only requires
having smoked ‘all or part of a cigarette’ one time in the past 30 days for a respondent to receive
questions about abuse. One respondent had only smoked one cigarette and many questions did
not apply to him.
Recommendation: Consider whether there should be a higher barrier of entry to this and similar
sections of the Dependence module.
Currently the calendar and calendar reminders are only provided in the core items.
Recommendation: Add calendars and calendar reminders throughout the instrument.
11
Appendix F
Variable Wording Crosswalk
Attachment F. Variable Wording Crosswalk
QFT Variable
(Testing
Outcome)
V2b
(new
question,
tested, no
problems)
QD05 (not
tested,
implemented
in QFT)
QD10c
(new
Phase
Introduced
QFT Instrument Wording
Phase 2
[IF QD10 = 2] Are you currently
serving full‐time in a Reserve
component? Full‐time service does
not include annual training for the
Reserves or National Guard.
NA
Phase 2
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
GUAMANIAN OR
CHAMORRO
6
SAMOAN
7
OTHER PACIFIC ISLANDER
8
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
9
OTHER (SPECIFY)
[IF QD10a=1] Did you ever serve on
active duty in the United States
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
to attempt to better
identify respondents in
the Reserves.
2012 Question Wording
Not included in 2012
QD05 HAND R SHOWCARD 2 Which
of these groups describes you? Just
give me the number or numbers from
the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE (AMERICAN
INDIAN INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
After testing, added two
new race categories to
this question in response
to feedback.
No changes between initial testing and This question was added
final QFT wording.
to better determine if
Not included in 2012
2
QFT Variable
(Testing
Outcome)
question,
tested, no
problems)
QD11
(modified
question;
tested;
problems
found;
revised for
final QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
Armed Forces or Reserve
components in a military combat
zone or an area where you drew
imminent danger pay or hostile fire
pay?
HAND R SHOWCARD 5. What is the
highest grade or year of school you
have completed? Just give me the
number from the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING
COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 REGULAR HIGH SCHOOL
DIPLOMA
13 12TH GRADE, NO DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT,
BUT NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
Initial Test Wording
Notes/Testing Results
respondents who
indicate that they are on
active duty are actually in
a Reserve component.
2012 Question Wording
What is the highest grade or year of
school you have completed?
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE, NO DIPLOMA
13 REGULAR HIGH SCHOOL
DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT, BUT
NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
The original question
used in 2012 had no
category for completing
the 12th grade without
receiving a diploma. This
category was inserted as
response #12, but during
testing it was found that
the order of categories
could be confusing to
some respondents. As a
result of testing,
categories 12 and 13
were reversed so that
category 12 now
represents a high school
diploma and 13 means
12th grade, but no
diploma.
HAND R SHOWCARD 4. What is
the highest grade or year of school
you have completed?
Please tell me the number from
the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NEVER ATTENDED SCHOOL
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE COMPLETED
13 COLLEGE OR UNIVERSITY /
1ST YEAR COMPLETED
14 COLLEGE OR UNIVERSITY /
2ND YEAR COMPLETED
15 COLLEGE OR UNIVERSITY /
3RD YEAR COMPLETED
16 COLLEGE OR UNIVERSITY /
4TH YEAR COMPLETED
3
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCOLOR)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
QFT Instrument Wording
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE,
MD, DDS, DVM, LLB, JD)
DK/REF
Item dropped from questionnaire.
Initial Test Wording
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE, MD,
DDS, DVM, LLB, JD)
DK/REF
CGCOLOR [IF (CG11 = 1‐2 OR CG11 =
4 OR CG11 = 7 OR CG11 = 10 OR CG11
= 12 OR CG11 = 14 OR CG11 = 17‐21
OR CG11 = 24‐25 OR CG11a = 31 AND
CG13 = 1) OR IF (RCG11 = 1‐2 OR
RCG11 = 4 OR RCG11 = 7 OR RCG11 =
10 OR RCG11 = 12 OR RCG11 = 14 OR
RCG11 = 17‐21 OR RCG11 = 24‐25 OR
RCG11a = 31 AND RCG13 = 1) OR IF
(RRCG11 = 1‐2 OR RRCG11 = 4 OR
RRCG11 = 7 OR RRCG11 = 10 OR
RRCG11 = 12 OR RRCG11 = 14 OR
RRCG11 = 17‐21 OR RRCG11 = 24‐25
OR RRCG11a = 31 AND RRCG13 = 1]
What color is the pack of
[CG11/CG11a/RCG11/RCG11a/RRCG1
1/RRCG11a FILL] cigarettes you
smoked most often during the past 30
days?
1
Blue
2
Green
3
Gold
4
Red
5
Silver
6
White
Notes/Testing Results
2012 Question Wording
17 COLLEGE OR UNIVERSITY /
5TH OR HIGHER YEAR
COMPLETED
DK/REF
Not included in 2012
Recent legislation
banned cigarette
manufacturers from
using terms on packaging
that referred to cigarette
strength. In an attempt
to continue to track
trends in the types of
cigarettes most smoked,
a new question was
developed to capture
data based on the new
cigarette color code
system. Most
respondents had a
problem with the
question and reported
that the cigarette
packaging included
multiple colors,
indicating that
participants failed to
understand the purpose
of the question.
4
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCLR2)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
CG25
Phase 1
(CG17REV)
(question
revised in
order to
administer
abbreviated
core for
cognitive
interviewing;
new wording
tested and
implemented
)
ALREF
Phase 1
QFT Instrument Wording
Item dropped from questionnaire.
Initial Test Wording
7
Some other color
DK/REF
CGCLR2 [IF CGCOLOR = 7] Please
type in the color of the pack of
cigarettes you smoked most often
during the past 30 days.
The next questions are about your
use of “smokeless” tobacco such as
snuff, dip, chewing tobacco, or
“snus.”
Have you ever used “smokeless”
tobacco, even once?
Have you ever, even once, tried any
“smokeless” tobacco such as snuff,
dip, chewing tobacco, or “snus”?
NOTE: Example text from ALREF is
ALREF [IF AL01 = REF] The answers
Notes/Testing Results
This follow‐up question
to an answer of “Some
other color” to CGCOLOR
was removed along with
the initial question due
to participant
misunderstanding of the
question.
An abbreviated core
module was used for
cognitive interviewing.
The version of the
smokeless tobacco
question that was used
eliminated the
introductory wording,
“The next questions are
about your use of . . .”
since only the lifetime
use of smokeless tobacco
was assessed. The
revised wording that was
tested and maintained
combined snuff and
chewing tobacco, since
incidence use rates of
these are quite low. In
addition, a new product
called “snus” has
recently emerged on the
market and it is also
included in the QFT
wording. No changes to
the question were made
as a result of the testing.
A streamlined version of
2012 Question Wording
Not included in 2012
These next questions are about your
use of snuff, sometimes called dip.
Snuff is a finely ground form of
tobacco that usually comes in a
container called a tin. You can use
snuff by placing a pinch or dip in
your mouth between your lip and
gum or between your cheek and
gum. Snuff can also be inhaled
through the nose. Snuff is sold in
both loose form and in ready‐to‐use
packets.
Have you ever used snuff, even
once?
[IF AL01 = REF] The answers that
5
QFT Variable
(Testing
Outcome)
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
AL08
(new
question,
tested, no
problems,
implemented
for QFT)
Phase 1
MRJINTRO &
MJ01
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase 1
QFT Instrument Wording
displayed below. Equivalent text was
used for MJREF, CCREF, HEREF, and
MEREF.
[IF AL01 = REF] The answers that
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have [IF QD01‐5 THEN FILL
5, IF QD01=9 THEN FILL 4] or more
drinks on the same occasion? By
‘occasion,’ we mean at the same
time or within a couple of hours of
each other.
Initial Test Wording
people give about their use of alcohol
are important. Remember, your
answers will be kept confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a sip
or two from a drink.
Notes/Testing Results
this question was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is, since
[DATEFILL], on how many days did
you have [IF QD01‐5 THEN FILL 5, IF
QD01=9 THEN FILL 4] or more drinks
on the same occasion? By ‘occasion,’
we mean at the same time or within a
couple of hours of each other.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have 5 or more drinks on
the same occasion? By ‘occasion,’
we mean at the same time or within
a couple of hours of each other.
MRJINTRO
The next questions
are about marijuana and hashish.
Marijuana is also called pot or grass.
Marijuana is usually smoked, either
in cigarettes, called joints, or in a
pipe. It is sometimes cooked in
food. Hashish is a form of marijuana
that is also called “hash.” It is
usually smoked in a pipe. Another
form of hashish is hash oil.
MJ01 The next question is about
marijuana and hashish. Marijuana is
also called pot, grass, or weed.
Marijuana is usually smoked, either in
cigarettes, called “joints”, in cigars,
called “blunts”, or in a pipe. It is
sometimes cooked in food. Hashish is
a form of marijuana that is also called
“hash.” It is usually smoked in a pipe.
Another form of hashish is hash oil.
This item was revised to
ask females about
drinking 4 or more drinks
on the same occasion in
the Alcohol section.
Previously, this
information was
obtained in the
Consumption of Alcohol
module.
For the cognitive
interviewing we initially
tested a version of the
lifetime marijuana
question as part of the
abbreviated core. This
question incorporated
the introductory
language and referenced
only one “next question”
The next questions are about
marijuana and hashish. Marijuana is
also called pot or grass. Marijuana
is usually smoked, either in
cigarettes, called joints, or in a pipe.
It is sometimes cooked in food.
Hashish is a form of marijuana that
is also called “hash.” It is usually
smoked in a pipe. Another form of
hashish is hash oil.
6
QFT Variable
(Testing
Outcome)
Phase
Introduced
HE01
Phase 1
(question
revisions
were tested;
no advantage
was found;
dropped from
QFT)
HALINTRO
Phase 1
(question was
revised to
administer
abbreviated
version of
core drugs
module for
cognitive
interviewing
only.)
QFT Instrument Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Initial Test Wording
Notes/Testing Results
since subsequent
Have you ever, even once, used
questions to lifetime use
marijuana or hashish, or smoked part were not included. The
or all of a cigar with marijuana in it,
term “weed” was
included since it is a
called a “blunt”?
commonly used term for
marijuana. Additionally,
the cognitive
interviewing tested
inclusion of “blunts” in
the main marijuana
module. Combining
blunts with main
marijuana use was
desired because of
interest in streamlining
the instrument and
consolidating marijuana
use in one module.
Have you ever, even once, used
This next question is about heroin.
Alternate terms for
heroin?
heroin, “black tar heroin”
Have you ever, even once, used
and “cheese” were
heroin, including “black tar heroin” or tested but were not
included in the QFT since
“cheese”?
cognitive interview
respondents did not tend
to recognize them.
The next questions are about
The introductory text to
The next questions are about
substances called hallucinogens. These hallucinogens was
substances called hallucinogens.
drugs often cause people to see or
removed as part of
These drugs often cause people to
creating an abbreviated
see or experience things that are not experience things that are not real.
core drugs module for
real.
the cognitive
Press [ENTER] to continue.
interviewing. Since only
A list of some common
lifetime use was asked
hallucinogens is shown below.
about each substance,
These and many other substances
that people use as hallucinogens are
the full list of
often known by street names, and
hallucinogens was not
2012 Question Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Have you ever, even once, used
heroin?
The next questions are about
substances called hallucinogens.
These drugs often cause people to
see or experience things that are not
real.
A list of some common
hallucinogens is shown below.
These and many other substances
that people use as hallucinogens are
often known by street names, and
7
QFT Variable
(Testing
Outcome)
Phase
Introduced
LS01i
(no change
tested)
Phase 1
LS01j
(existing
question
moved to
new module;
no wording
changes)
Phase 1
QFT Instrument Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Ketamine, also called “Special K” or
“Super K”
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine
Foxy, also called 5‐MeO‐DIPT
Salvia divinorum
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine, or
Initial Test Wording
Notes/Testing Results
included prior to the list
of lifetime use questions.
The QFT wording
included examples of all
Hallucinogens asked
about in the module,
including the three
substances that were
added to the module.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
2012 Question Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
1
Yes
2
No
DK/REF
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine,
AMT, also called alpha‐
methyltryptamine, or
8
QFT Variable
(Testing
Outcome)
LS01k
(existing
question
moved to
new module;
no wording
changes)
INHINTRO
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
QFT Instrument Wording
Foxy, also called 5‐MeO‐DIPT?
Initial Test Wording
Notes/Testing Results
Phase 1
Have you ever, even once, used
Salvia divinorum?
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
Phase 1
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
These next questions are about
liquids, sprays, and gases that people
sniff or inhale to get high or to make
them feel good.
Please do not include times when you
inhaled a substance accidentally —
such as when painting, cleaning an
oven, or filling a car with gasoline. The
questions use the word “inhalant” to
include any substances that you
sniffed or inhaled for kicks or to get
high
Press [ENTER] to continue.
A streamlined version of
the inhalants
introduction was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
Foxy, also called 5‐MeO‐DIPT?
Have you ever, even once, used
Salvia divinorum?
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
9
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Felt‐tip pens, felt‐tip markers, or
magic markers
Spray paints
Computer keyboard cleaner, also
known as air duster
Other aerosol sprays
Press [ENTER] to continue.
Have you ever, even once, inhaled
felt‐tip pens, felt‐tip markers, or
magic markers for kicks or to get
high?
IN01h1 (new
question,
tested, no
problems)
Phase 1
IN01ii (new
question
tested,
problems
found,
revised for
QFT)
Phase 1
Have you ever, even once, inhaled
computer keyboard cleaner, also
known as air duster, for kicks or to
get high?
ME01
(SD17a)
(existing
question
moved to
new module,
Phase 1
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
because of references to
felt‐tip pens, felt‐tip
markers, and magic
markers in the write‐in
data in inhalants in the
annual survey.
In the past 12 months, have you
This question was added
because of references to
inhaled canned air for kicks or to get
computer keyboard
high?
cleaner and canned air in
the write‐in data in
inhalants in the annual
survey. In the cognitive
interviewing, participants
often did not know what
“canned air” was. New
terms were selected
based on cognitive
interviewing results.
No changes between initial testing and A new module was
created for
final QFT wording.
methamphetamine in
the core drugs modules.
Items measuring
methamphetamine use
2012 Question Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Spray paints
Other aerosol sprays
Press [ENTER] to continue.
Not included in 2012
Not included in 2012
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
10
QFT Variable
(Testing
Outcome)
no changes)
Phase
Introduced
ME02
(question
moved to
new module,
no changes)
MELAST3
(question
moved to
new module,
no changes)
MEFRAME3
(question
moved to
new module,
no changes)
Phase 2
MERAVE
(question
moved to
new module,
no changes)
MEMONAVE
(question
moved to
new module,
minor
changes)
MEWKAVE
QFT Instrument Wording
injected.
Have you ever, even once, used
methamphetamine?
How old were you the first time you
used methamphetamine?
Initial Test Wording
Notes/Testing Results
were moved from the
Stimulants module to
this new module.
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
2012 Question Wording
injected. Have you ever, even once,
used Methamphetamine?
SDME01 How old were you the first
time you used Methamphetamine?
Phase 2
How long has it been since you last
used methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
SD17b How long has it been since
you last used Methamphetamine?
Phase 2
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you’ve
used it?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On how many days in the past 12
months did you use
methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On average, how many days did you
use methamphetamine each month
during the past 12 months?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
Methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you
used Methamphetamine?
On how many days in the past 12
months did you use
Methamphetamine?
On average, how many days each
month during the past 12 months
did you use Methamphetamine?
Phase 2
On average, how many days did you
No changes between initial testing and A new module was
On average, how many days each
11
QFT Variable
(Testing
Outcome)
(question
moved to
new module,
no changes)
ME06 (new
question,
tested, no
changes)
INTROPR
(new
question
tested, no
changes)
PR01‐PR03
PR08
PR10
ST01‐ST04
ST05
ST06‐ST07
(question
tested,
problems
identified,
question
revised)
Phase
Introduced
Phase 2
QFT Instrument Wording
use methamphetamine each week
during the past 12 months?
[IF MELAST3 =1 OR MERECDK = 1
OR MERECRE = 1] Think specifically
about the past 30 days, from
[DATEFILL] up to and including
today. During the past 30 days, on
how many days did you use
methamphetamine?
Phase 1
These next questions are about any
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as
aspirin, Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
PR01
NOTE: Example text from PR01 is
ST01, ST02, displayed below. Equivalent text was
ST03, ST04, used for all listed variables from pain
relievers, stimulants, sedatives, and
ST06, &
tranquilizers modules.
ST07 –
Usability
Please look at the names and
pictures of the pain relievers shown
PR02,
below. Please note that some forms
PR03,
of these pain relievers may look
PR08,
different from the pictures, but you
PR10, &
should include any form that you
ST05 –
have used.
Phase 1
PROGRAMMER: DISPLAY PILLS HERE
TR01 &
FOR VICODIN, LORTAB, LORCET,
SV01 –
AND HYDROCODONE.
Phase 2
In the past 12 months, which, if any,
Initial Test Wording
final QFT wording.
Notes/Testing Results
created for
methamphetamine in
the core drugs modules.
No changes between initial testing and As part of creating a new
final QFT wording.
methamphetamine
module, a new 30‐day
methamphetamine use
variable was created.
2012 Question Wording
week during the past 12 months did
you use Methamphetamine?
Not included in 2012
These next questions are about the
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as aspirin,
Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription pain
relievers module.
PR01A. Please look at the pictures of
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in
either of these ways?
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
PR01A1. [IF PR01A = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
Not included in 2012
Initial versions of the
prescription drugs
screener questions were
written to capture
misuse only. Subsequent
to usability testing, the
question was revised to
capture first all types of
use then follow up
specific substances
endorsed with questions
about misuse.
Not included in 2012
12
QFT Variable
(Testing
Outcome)
PR04
PR06‐PR07
TR04 (new
question,
problems
found,
question
revised.)
Phase
Introduced
PR04,
PR06, &
PR07 –
Phase 1
TR04 –
Phase 2
QFT Instrument Wording
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Initial Test Wording
did you use in the past 12
months in either of these
ways?
Notes/Testing Results
2012 Question Wording
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
Please look at the names and pictures
NOTE: Example text from PR04 is
displayed below. Equivalent text was of the pain relievers shown below.
used for all listed variables from pain
relievers, stimulants, sedatives, and PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER, ULTRACET,
tranquilizers modules.
RYZOLT, AND TRAMADOL.
Please look at the names and
pictures of the pain relievers shown In the past 12 months, which, if any,
below. Remember, some forms of
of these pain relievers have you used?
these pain relievers may look
different from the pictures, but you To select more than one drug from the
list, press the space bar between each
should include any form that you
number you have typed. When you
have used.
have finished, press [ENTER].
PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER,
ULTRACET, RYZOLT, AND
TRAMADOL.
In the past 12 months, which, if any,
of these pain relievers have you
Original test of wording
Not included in 2012
yielded results that
participants were asking
whether certain
prescription drugs should
be included even though
they looked different
from what was in the
pictures. Reminder
language was added to
every 2‐4 screens to
remind respondents that
the pictures may look
different from the form
pictured but that they
should still be included.
13
QFT Variable
(Testing
Outcome)
Phase
Introduced
PR05
(question
tested in
cognitive
interviewing,
problem
found, minor
changes
made)
Phase 1
PR11
TR06
ST06
SV06 (Change
PR11 &
ST06‐
Phase 1
QFT Instrument Wording
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Please look at the names and
pictures of the pain relievers shown
below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine 3 or 4
2
Codeine pills
95
I have not used any of
these pain relievers in the past 12
months
DK/REF
In the past 12 months, have you
used any other prescription pain
reliever?
Initial Test Wording
Please look at the names and pictures
of the pain relievers shown below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you used?
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine
2
Codeine pills
95
I have not used any of these
pain relievers in the past 12 months
DK/REF
Notes/Testing Results
2012 Question Wording
Initially, “Tylenol with
Not included in 2012
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
No changes between initial testing and The prescription drugs
final QFT wording.
modules were revised to
ask first about all types
of use. Respondents
Not included in 2012
14
QFT Variable
(Testing
Outcome)
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
TR06 &
Remember, do not include “over‐
SV06‐
the‐counter” pain relievers such as
Phase 2
aspirin, Tylenol, Advil, or Aleve.
PR12
TR07
ST07
SV07 (Change
tested, no
problems
found,
implemented
in QFT.)
PR12 &
ST07—
Phase 1
TR07 &
SV07—
Phase 2
INTROTR
INTROSV
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 2
Initial Test Wording
Notes/Testing Results
received follow up
questions about misuse
only of specific drugs
that were used. This
particular item was
administered when no
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
No changes between initial testing and The prescription drugs
[IF PR12MON = 2] Have you ever,
final QFT wording.
modules were revised to
even once, used any prescription
ask first about all types
pain reliever?
of use. Respondents
received follow up
Remember, do not include “over‐
questions about misuse
the‐counter” pain relievers such as
only of specific drugs
aspirin, Tylenol, Advil, or Aleve.
that were used. This
particular question was
asked when no 12‐month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
These questions were
NOTE: Example text from INTROTR is These next questions are about the
added to introduce the
use of prescription tranquilizers.
displayed below. Semi‐equivalent
Tranquilizers are usually prescribed to new prescription
text was used for INTROSV.
relax people, to calm people down, to tranquilizer and sedative
screener modules. Minor
These next questions are about any relieve anxiety, or to relax muscle
wording revisions were
spasms. Some people call
use of prescription tranquilizers.
made after testing.
Tranquilizers are usually prescribed tranquilizers “nerve pills.”
to relax people, to calm people
down, to relieve anxiety, or to relax Press [ENTER] to continue.
muscle spasms. Some people call
tranquilizers “nerve pills.”
2012 Question Wording
Not included in 2012
These next questions ask about the
use of tranquilizers. Tranquilizers
are usually prescribed to relax
people, to calm people down, to
relieve anxiety, or to relax muscle
spasms. Some people call
tranquilizers ‘nerve pills.’
Ask your interviewer to show you
Card B.
15
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTROST
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 1
PRL01
TRL01
STL01
SVL01
(Change
tested, no
problems
found,
implemented
in QFT.)
PRL01 &
STL01—
Phase 1
TRL01 &
SVL01—
Phase 2
PRY01—
Usability
QFT Instrument Wording
Press [ENTER] to continue.
These next questions are about any
use of prescription stimulants.
People sometimes take these drugs
for attention deficit disorders, to
lose weight, or to stay awake.
Please do not include “over‐the‐
counter” stimulants such as
Dexatrim, No‐Doz, Hydroxycut, or 5‐
Hour Energy.
Press [ENTER] to continue.
Initial Test Wording
Notes/Testing Results
2012 Question Wording
These next questions are about the
use of prescription stimulants. People
sometimes take these drugs for
attention deficit disorders, to lose
weight, or to stay awake. Please do
not include “over‐the‐counter”
stimulants such as Dexatrim or No‐
Doz.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription stimulants
screener module. Minor
wording revisions were
made after testing to
include additional
examples.
These next questions ask about the
use of drugs such as amphetamines
that are known as stimulants,
‘uppers,’ or ‘speed.’ People
sometimes take these drugs to lose
weight, to stay awake, or for
attention deficit disorders. We are
not interested in the use of “over‐
the‐counter” stimulants such as
Dexatrim or No‐Doz that can be
bought in drug stores or grocery
stores without a doctor’s
prescription.
Ask your interviewer to show you
Card C.
NOTE: Example text from PR05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
Have you ever, even once, used any
other prescription pain reliever,
besides the ones shown on Card A,
when it was not prescribed for you
or that you took only for the
experience or feeling it caused?
No changes between initial testing and The prescription drugs
NOTE: Example text from PRL01 is
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[IF PR12=1] Have you ever, even
once, used any prescription pain
that were used. This
reliever in any way a doctor did not
particular question was
direct you to use it?
asked when 12 month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed
but no misuse was ever
endorsed when follow up
questions were asked
about specific drugs.
NOTE: Example text from PRY01 is
PR01C. Please look at the pictures of The revised prescription [IF PR04 = 1] Which of the pain
16
QFT Variable
(Testing
Outcome)
PRY04
PRY24
PRY32—
PRY34
PRY37
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR01=1] In the past 12 months,
did you use Vicodin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in any
of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
1
Yes
2
No
DK/REF
PR01C1. [IF PR01C = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
did you use in the past 12
months in any of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
Notes/Testing Results
drugs questions ask
about misuse of drugs
endorsed for any type of
12 month use in the
screener. The initial
wording of the third
component of the
definition of misuse,
“Just for the effect it has
on you – not for its
intended medical use”
did not help test
participants identify
misuse so it was revised
to the final QFT wording.
Also, the wording in the
2012 survey only
collected estimates of
lifetime use of individual
prescriptions, while the
new questions collect
past year use of
individual drugs.
2012 Question Wording
relievers shown below the red line
on Card A have you used when they
were not prescribed for you or that
you took only for the experience or
feeling they caused?
To select more than one drug from
the list, press the space bar between
each number you type. When you
have finished, press [ENTER].
4
Codeine
5
Demerol
6
Dilaudid
7
Fioricet
8
Fiorinal
9
Hydrocodone
10
Methadone
11
Morphine
12
OxyContin
13
Phenaphen with Codeine
14
Propoxyphene
15
SK‐65
16
Stadol
17
Talacen
18
Talwin
19
Talwin NX
20
Tramadol
21
Ultram
DK/REF
17
QFT Variable
(Testing
Outcome)
PRY05‐PRY17
PRY19
PRY20—
PRY23
PRY25—
PRY33
PRY35—
PRY36
PRY38—
PPRY39
TRY01—
TRY18
STY01—
STY23
SVY01—
SVY16
(Change
tested,
wording
Phase
Introduced
Phase 1
QFT Instrument Wording
[IF PR02=1] In the past 12 months,
did you use OxyContin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
1
Vicodin
2
Lortab
3
Lorcet
4
Hydrocodone
DK/REF
NOTE: Example text from PRY05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR02=1] In the past 12 months, did
you use OxyContin in any way a doctor
did not direct you to use it?
Notes/Testing Results
Cognitive testing
identified repetitive
nature of prescription
drugs module questions
with long definitions of
misuse and raised
concerns about
respondent fatigue. As a
result, the definition was
displayed as optional text
available by pressing F2
key on some screens.
2012 Question Wording
How long has it been since you last
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
18
QFT Variable
(Testing
Outcome)
improvement
identified,
implemented
in QFT.)
PRY18
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY28
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY01a—
PRY39a
TRY01a—
TRY18a
STY01a—
STY23a
SVY01a—
SVY16a
(Change
tested,
wording
improvement
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Please look at the pain relievers
shown in Box 1 above the red line
on Card A.
Have you ever, even once, used
Darvocet, Darvon, or Tylenol with
codeine that was not prescribed for
you or that you took only for the
experience or feeling it caused?
Not included in 2012
Phase 1
[IF PR05=1] In the past 12 months,
did you use Tylenol with codeine 3
or 4 in any way a doctor did not
direct you to use it?
[IF PR05=1] In the past 12 months, did
you use Tylenol with codeine in any
way a doctor did not direct you to use
it?
Initially, “Tylenol with
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
Phase 1
[IF PR07=4] In the past 12 months,
did you use fentanyl in any way a
doctor did not direct you to use it?
[IF PR07=4] In the past 12 months, did
you use fentanyl or “China Girl” in any
way a doctor did not direct you to use
it?
The street name for
fentanyl, “China Girl”
was not known to
cognitive interview
participants so it was
dropped from the QFT.
PRY01a‐
PRY39a &
STY01a‐
STY23a‐
Phase 1
TRY01a‐
TRY18a &
SVY01a‐
SVY16a‐
Phase 2
[IF PRY01=1] How old were you when
NOTE: Example text from PR01a is
displayed below. Equivalent text was you first used Vicodin in a way a
used for all listed variables from pain doctor did not direct you to use it?
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRFIRSTFLAG=1] Please think
about the first time you ever used
Vicodin in a way a doctor did not
direct you to use it.
[IF PRY01=1] How old were you
when you first used Vicodin in a way
a doctor did not direct you to use
it?
An introduction was
added to transition from
asking about whether
groups of drugs were
misused to asking about
the specifics of that
misuse.
NOTE: Example text from PR06 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
How old were you the first time you
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
19
QFT Variable
(Testing
Outcome)
identified,
implemented
in QFT.)
PRY01b—
PRY39b
TRY01b—
TRY02b
STY01b
SVY01b—
SVY02b
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01c—
PRY02c
TRY01c—
TRY02c
STY01c
SVY01c—
SVY02c
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRY01b—
PRY39b &
STY01b‐
Phase 1
TRY01b—
TRY02b &
SVY01b‐
Phase 2
PRY01c—
PRY02c &
STY01c—
Phase 1
TRY01c—
TRY02c &
SVY01c—
SVY02c—
Phase 2
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Not included in 2012
NOTE: Example text from PRY01b is No changes between initial testing and The prescription drugs
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[AGE1STPR1 = CURNTAGE AND
DATE OF INTERVIEW < DOB OR IF
that were used. This
AGE1STPR1 = CURNTAGE ‐ 1 AND
particular question was
DATE OF INTERVIEW ≥ DOB] Did you
asked when 12 month
first use Vicodin in a way a doctor
use of a prescription drug
did not direct you to use it in
was endorsed with first
[CURRENT YEAR ‐ 1] or [CURRENT
use within at
respondent’s current
YEAR]?
age. No change was
made to the item as a
result of cognitive
testing.
Not included in 2012
No changes between initial testing
The prescription drugs
NOTE: Example text from PRY01c is
modules were revised to
displayed below. Equivalent text was and final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
[IF AGE1STPR2 = CURNTAGE ‐ 1
only of specific drugs
AND DATE OF INTERVIEW < DOB]
that were used. This
Did you first use Lortab in a way a
particular question was
doctor did not direct you to use it in
asked when 12 month
[CURRENT YEAR ‐ 2] or [CURRENT
use of a prescription drug
YEAR ‐ 1]?
was endorsed at
respondent’s age one
year younger than their
current age. No change
was made to the item as
a result of cognitive
20
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRY01d—
PRY02d
TRY01d—
TRY02d
STY01d
SVY01d—
SVY02d
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01d‐
PRY02d &
STY01d—
Phase 1
TRY01d‐
TRY02d &
SVY01d‐
SVY02d—
Phase 2
PRM02DKRE
TRM02DKRE
STM02DKRE
SVM02DKRE
(Change
tested, no
problems
found,
implemented
in QFT.)
PRM02DKR
E &
STM02DKR
E— Phase
1
TRM02DKR
E &
SVM02DKR
E—Phase 2
QFT Instrument Wording
Initial Test Wording
No changes between initial testing
NOTE: Example text from PRY01d is
displayed below. Equivalent text was and final QFT wording.
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRYFU1 NE 0 AND PRJANFLAG1 =
0 AND PRDECFLAG1=0 AND
PR1MTHFLAG1=0]
[IF PRYFU1 = CURRENT YEAR OR
CURRENT YEAR ‐1] Earlier, you
reported that you first used Vicodin
in a way a doctor did not direct you
to use it when you were
[AGE1STPR1] years old. Based on
your date of birth, you turned
[AGE1STPR1] in [FILL WITH
MONTH/YEAR FOR AGE1STPR1
BASED ON DOB].
[IF PRYFU1 NE 0] In what month in
[PRYFU1] did you first use Vicodin in
a way a doctor did not direct you to
use it?
No changes between initial testing and
NOTE: Example text from
final QFT wording.
PRM02DKRE is displayed below.
Equivalent text was used for
TRM02DKRE, STM02DKRE, &
SVM02DKRE.
[IF PRM02 = DK/REF] What is your
best estimate of the number of days
you used [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] during the past 30
days?
Notes/Testing Results
testing.
This question asked
recent initiates of
prescription drug misuse
about the month of first
use of in order to meet a
redesign goal of
capturing more precise
data on first use among
recent initiates of drugs.
No change was made to
the item as a result of
cognitive testing.
No changes were made
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse about
individual drugs are not
collected. This question
was introduced to
measure frequency of
misuse of all misused
substances.
2012 Question Wording
Not included in 2012
Not included in 2012
21
QFT Variable
(Testing
Outcome)
PRM03
TRM03
STM03
SVM03
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRM03 &
STM03—
Phase 1
TRM03 &
SVM03—
Phase 2
PRY40
TRY19
STY24
SVY19
PRY40 &
STY24—
Phase 1
TRY19 &
SVY19—
Phase 2
PRY41
STY25
TRY20
SVY18
Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY41 &
STY25—
Phase 1
TRY20 &
SVY18—
Phase 2
QFT Instrument Wording
NOTE: Example text from PRM03 is
displayed below. Equivalent text was
used for TRM03, STM03, & SVM03.
[IF ALC30USE = 1 and PRMISUSE30 =
1)] During the past 30 days did you
use [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] while you were
drinking alcohol or within a couple
of hours of drinking?
NOTE: Example text from PRY40 is
displayed below. Equivalent text was
used for TRY19, STY24, & SVY19.
[IF PR11=1] In the past 12 months,
did you use any [IF PR11=1 AND
PRYRCOUNT > 1 FILL “other”]
prescription pain reliever in a way a
doctor did not direct you to use it?
Initial Test Wording
Notes/Testing Results
No changes between initial testing and No changes were made
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse with alcohol
are collected in the
Consumption of Alcohol
module. Here it is asked
in the relevant
prescription drug
module.
2012 Question Wording
Think again about this last time you
drank any alcoholic beverages, when
you had [CA01 FILL] [drink/drinks].
Did you also use [DRUGFILL] while
you were drinking or within a couple
of hours of drinking?
No changes between initial testing and No changes were made
Not included in 2012
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of past year misuse
about individual drugs
are not collected. This
question was introduced
to measure details of
misuse of all misused
substances.
[IF PRMISUSE12=1] Which of these
This question was added Not included in 2012
NOTE: Example text from PRY41 is
to capture more detail
displayed below. Equivalent text was statements describe your use of
about which ways the
[PRNAMEFILL] at any time in the past
used for STY25, TRY20, & SVY18.
respondents misused
12 months? To select more than one
statement, press the space bar
specific drugs. The “at
least once” wording was
[IF PRMISUSE12=1] Which of these between the numbers.
confusing to some
statements describe your use of
[PRNAMEFILL] at any time in the
1
At least once, I used
respondents in the first
past 12 months?
[PRNAMEFILL] without a
round of cognitive
prescription of my own.
interviewing, so it was
To select more than one statement, 2
At least once, I used
removed for subsequent
press the space bar between the
[PRNAMEFILL] in greater
rounds of cognitive
numbers. When you have finished,
amounts, more often, or for
interviewing and for the
press [ENTER].
longer than it was prescribed. QFT. Additional methods
3
At least once, I used
of misuse were also
22
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
I used [PRNAMEFILL]
without a prescription of
my own.
2
I used [PRNAMEFILL] in
greater amounts than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
3
I used [PRNAMEFILL] more
often than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
4
I used [PRNAMEFILL] for
longer than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
5
I used [PRNAMEFILL] in
some other way a doctor
did not direct me to use
[PRNUMFILL]
NOTE: Example text from PRY42b is
displayed below. Equivalent text was
used for STY26b, TRY42B, & SVY19B.
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
1
PRY42B
STY26B
TRY42B
SVY19B
(Change
tested,
PRY42B &
STY26B—
Phase 1
TRY42B &
SVY19B—
Phase 2
Initial Test Wording
[PRNAMEFILL] in some other
way a doctor did not direct
me to use [PRNUMFILL]
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
the last time you used [PRLASTFILL2]
in any way a doctor did not direct you
to use it.
Notes/Testing Results
identified and added to
the response options.
Response categories
were streamlined and
reduced as a result of
cognitive interviewing.
2012 Question Wording
Not included in 2012
23
QFT Variable
(Testing
Outcome)
wording
improvement
identified,
implemented
in QFT.)
PRY42C
STY26C
TRY21C
SVY19C
Phase
Introduced
PRY42C &
STY26C—
Phase 1
QFT Instrument Wording
the last time you used [PRLASTFILL2]
in any way a doctor did not direct
you to use [FILL WITH “them” IF
RPRY42A=19. ELSE FILL WITH “it”].
[IF PRMISUSE12 = 1] How did you
get the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more
than one doctor
3
I stole the [PRLASTFILL]
from a doctor’s office,
clinic, hospital, or
pharmacy
4
I got the [PRLASTFILL] from
a friend or relative for free
5
I bought the [PRLASTFILL]
from a friend or relative
6
I took the [PRLASTFILL]
from a friend or relative
without asking
7
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
8
I got the [PRLASTFILL] in
some other way
DK/REF
NOTE: Example text from PRY42c is
displayed below. Equivalent text was
used for TRY21C & SVY19C.
Initial Test Wording
[IF PRMISUSE12 = 1] How did you get
the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more than
one doctor
3
I wrote a fake prescription for
the [PRLASTFILL]
4
I stole the [PRLASTFILL] from
a doctor’s office, clinic,
hospital, or pharmacy
5
I got the [PRLASTFILL] from a
friend or relative for free
6
I bought the [PRLASTFILL]
from a friend or relative
7
I took the [PRLASTFILL] from a
friend or relative without
asking
8
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
9
I bought the [PRLASTFILL] on
the Internet
10
I got the [PRLASTFILL] in some
other way
DK/REF
Notes/Testing Results
No changes between initial testing and Wording and fills in this
final QFT wording.
question were adapted
to redesign conventions.
No changes were made
2012 Question Wording
Not included in 2012
24
QFT Variable
(Testing
Outcome)
(Change
tested, no
problems
found,
implemented
in QFT.)
(PRHOSPYR1
&
STHOSPYR1)
(Change
tested,
problems
found,
dropped from
QFT)
(PRHOSPYR2
&
STHOSPYR2)
(Change
tested,
Phase
Introduced
QFT Instrument Wording
TRY21C &
SVY19C—
[IF PRY42B=5 AND CALCAGE >=18]
Phase 2
You reported that you got the
[PRLASTFILL] from a friend or
relative for free. How did your
friend or relative get the
[PRLASTFILL]?
Phase 1
Items dropped from questionnaire.
Phase 1
Item dropped from questionnaire.
Initial Test Wording
Notes/Testing Results
as a result of the
cognitive interviewing.
2012 Question Wording
NOTE: Example text from PRHOSPYR1
is displayed below. Equivalent text was
used for STHOSPYR1.
[IF PR12MON=1 AND (PR11=1 OR
PRYRCOUNT > 1)] The computer
recorded that, in the past 12 months,
you used [PRHOSPFILL].
In the past 12 months, did you use [IF
PR11=1 AND PRYRCOUNT=1, THEN
FILL WITH PRHOSPFILL. IF PRYRCOUNT
> 1 THEN FILL WITH “any of these
prescription pain relievers”] only
when you were in the hospital? That
would include staying in the hospital
as an inpatient or being treated in an
emergency room.
[IF PR12MON=1 AND PR11 NE 1 AND
PRYRCOUNT = 1] In the past 12
months, did you use [PRHOSPFILL]
only when you were in the hospital?
That would include staying in the
hospital as an inpatient or being
treated in an emergency room.
NOTE: Example text from PRHOSPYR2
is displayed below. Equivalent text was
used for STHOSYR2.
[IF PRHOSPYR1=1 AND PRYRCOUNT >
A question about
legitimate use of
prescription drugs only
while in the hospital was
added to the instrument
during the cognitive
interviewing in order to
account for this very
specific kind of use that
respondents might not
be sure how to count.
The question confused
several respondents so it
was eliminated.
Not included in 2012
This follow up to the
previous question
assessed which
prescription drugs were
used only in a hospital
Not included in 2012
25
QFT Variable
(Testing
Phase
Outcome)
Introduced
problems
found,
dropped from
QFT)
PRYMOTIV
Phase 2
QFT Instrument Wording
Initial Test Wording
1] Which of these pain relievers did
you use only when you were in the
hospital in the past 12 months?
(LIST DRUGS ANSWERED AS YES IN
PR01‐PR11. NUMBER RESPONSE
OPTIONS SEQUENTIALLY STARTING AT
1, BUT MAINTAIN UNIQUE CODES FOR
EACH DRUG. IF PR11=1, DISPLAY
"Another prescription pain reliever"
AS THE LAST CATEGORY. )
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
NOTE: Example text from PRYMOTIV [IF PRMISCOUNT > 1 OR
is displayed below.
(PRMISCOUNT = 1 AND PRY40 NE 1
AND PRDKREFFLAG=1)]
[IF PRMISCOUNT > 1 OR
Now think about the last time you
(PRMISCOUNT = 1 AND PRY40 NE 1
used [PRLASTFILL2] in any way a
AND PRDKREFFLAG=1)]
doctor did not direct you to use [FILL
Now think about the last time you
WITH ‘them” IF RPRY42A=19. ELSE
used [PRLASTFILL2] in any way a
FILL WITH “it”].
doctor did not direct you to use [FILL
WITH ‘them” IF RPRY42A=19. ELSE
What were the reasons you used
[PRLASTFILL2] that time? To select
FILL WITH “it”].
more than one reason from the list,
press the space bar between each
What were the reasons you used
number you have typed. When you
[PRLASTFILL2] that time? To select
more than one reason from the list, have finished, press [ENTER].
press the space bar between each
number you have typed. When you
1
To relieve physical pain
have finished, press [ENTER].
2
To relax or relieve tension
3
To experiment or to see what
1
To relieve physical pain
[IF RPRY42A=19 THEN
2
To relax or relieve tension
“they’re” ELSE “it’s”] like
Notes/Testing Results
setting. It was dropped
along with the lead‐in
question due to
confusion.
This question was added
to gather data about
motivation for misuse of
Pain Relievers.
2012 Question Wording
Not included in 2012
26
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
4
To feel good or get high
To experiment or to see
what [IF RPRY42A=19 THEN 5
To help with my sleep
“they’re” ELSE “it’s”] like
6
To increase or decrease the
4
To feel good or get high
effect(s) of some other drug
5
To help with my sleep
Because I am “hooked” or I
7
6
To help me with my
have to have [IF RPRY42A=19
feelings or emotions
THEN “them” ELSE “it”]
7
To increase or decrease the
effect(s) of some other drug 8
I used [IF RPRY42A=19 THEN
8
Because I am “hooked” or I
“them” ELSE “it”] for some
have to have [IF
other reason
RPRY42A=19 THEN “them”
DK/REF
ELSE “it”]
9
I used [IF RPRY42A=19
THEN “them” ELSE “it”] for
some other reason
DK/REF
[IF SVMISCOUNT > 1 OR
[IF SVMISCOUNT > 1 OR
(SVMISCOUNT=1 AND SVY17 NE 1
(SVMISCOUNT=1 AND SVY17 NE 1
AND SVDKREFFLAG=1)] Now think
AND SVDKREFFLAG=1)] Now think
about the last time you used
about the last time you used
[SVLASTFILL2] in any way a doctor
[SVLASTFILL2] in any way a doctor did
did not direct you to use it.
not direct you to use it.
What were the reasons you used
What were the reasons you used
[SVLASTFILL2] that time? To select
[SVLASTFILL2] that time? To select
more than one reason from the list, more than one reason from the list,
press the space bar between each
press the space bar between each
number you have typed. When you number you have typed. When you
have finished, press [ENTER].
have finished, press [ENTER].
1
To relax or relieve tension
1. To relax or relieve tension
2. To experiment or to see
2
To experiment or to see what
what it’s like
it’s like
3. To feel good or get high
3
To feel good or get high
4. To help with my sleep
4
To help with my sleep
5. To help me with my
5
To increase or decrease the
feelings or emotions
Notes/Testing Results
2012 Question Wording
3
TRMOTIV
SVMOTIV
Phase 1
This question was added
to gather data about
motivation for misuse of
Tranquilizers and
Sedatives.
Not included in 2012
27
QFT Variable
(Testing
Outcome)
STYMOTIV
Phase
Introduced
Phase 2
QFT Instrument Wording
6. To increase or decrease the
effect(s) of some other
drug
7. Because I am “hooked” or I
have to have it
8. I used it for some other
reason
DK/REF
Initial Test Wording
effect(s) of some other drug
6
Because I am “hooked” or I
have to have it
7
I used it for some other
reason
DK/REF
Notes/Testing Results
[IF STMISCOUNT > 1 OR
(STMISCOUNT = 1 AND STY26NE 1
AND STDKREFFLAG=1)]Now think
about the last time you used
[STLASTFILL2] in any way a doctor
did not direct you to use [FILL WITH
‘them” IF RSTY42A=5. ELSE FILL
WITH “it”]
What were the reasons you used
[STLASTFILL2] that time? To select
more than one reason from the list,
press the space bar between each
number you have typed. When you
have finished, press [ENTER].
1. To help me lose weight
2. To help me
concentrate
3. To help me be alert or
stay awake
4. To help me study
5. To experiment or to
see what [IF
RSTY42A=5 THEN
“they’re” ELSE “it’s”]
like
No changes between initial testing and This question was added
to gather data about
final QFT wording.
motivation for misuse of
Stimulants.
2012 Question Wording
Not included in 2012
28
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRMOTOT
TRMOTOT
STMOTOT
SVMOTOT
Phase 2
PRYMOT1
Phase 2
QFT Instrument Wording
6. To feel good or get
high
7. To increase or
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. I used [IF RSTY42A=5
THEN “them” ELSE “it”]
for some other reason
DK/REF
NOTE: Example text from PRMOTOT
is displayed below. Equivalent text
was used for TRMOTOT, STMOTOT,
& SVMOTOT.
[IF PRYMOTIV=9] Please type in the
other reason you used
[PRLASTFILL2] that time. When you
have finished typing your answer,
press [ENTER] to go to the next
question.
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN PRYMOTIV] Which was
the main reason you used
[PRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1 To relieve physical pain
2 To relax or relieve tension
3 To experiment or to see
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF PRYMOTIV=8] Please type in the
reason you used [PRLASTFILL2] that
time. When you have finished typing
your answer, press [ENTER] to go to
the next question.
This question was added
to gather other, specify
data about motivation
for misuse of
prescription drugs.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN PRYMOTIV] Which was the
main reason you used [PRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1
To relieve physical pain
2
To relax or relieve tension
3
To experiment or to see what
This question was added
to gather data about the
primary motivation for
misuse of pain relievers.
Edits to logic were made
as a result of cognitive
testing.
Not included in 2012
29
QFT Variable
(Testing
Outcome)
Phase
Introduced
TRYMOT1
SVYMOT1
Phase 2
STYMOT1
Phase 2
QFT Instrument Wording
what it’s like
4 To feel good or get high
5 To help me with my
feelings or emotions
6 It helps with my sleep
7 To increase or decrease the
effect(s) of some other
drug
8 Because I am “hooked” or I
have to have it
9 [IF PRYMOTIV=9] The other
reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN TRYMOTIV] Which was
the main reason you used
[TRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to
see what it’s like
3. To feel good or get
high
4. To help me with my
feelings or emotions
5. It helps with my sleep
6. To increase or
decrease the effect(s)
of some other drug
7. Because I am “hooked”
or I have to have it
8. [IF PRYMOTIV=9] The
other reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
4
5
6
7
Initial Test Wording
it’s like
To feel good or get high
It helps with my sleep
To increase or decrease the
effect(s) of some other drug
Because I am “hooked” or I
have to have it
Notes/Testing Results
2012 Question Wording
DK/REF
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN TRYMOTIV] Which was the
main reason you used [TRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to see
what it’s like
3. To feel good or get high
4. To help with my sleep
5. To increase or decrease
the effect(s) of some
other drug
6. Because I am “hooked”
or I have to have it
7. I used it for some other
reason
DK/REF
This question was added
to gather data about the
primary motivation for
misuse of tranquilizers
and stimulants. Edits to
logic were made as a
result of cognitive
testing.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐9
This question was added
Not included in 2012
30
QFT Variable
(Testing
Outcome)
STY25A
(Change
tested, minor
wording
problems
found,
revisions
Phase
Introduced
Phase 1
QFT Instrument Wording
Initial Test Wording
CHOSEN IN STYMOTIV]
CHOSEN IN STYMOTIV] Which
Which was the main reason
was the main reason you
you used [STLASTFILL2] that
used [STLASTFILL2] that time?
time?
PROGRAMMER: FILL AS RESPONSE
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE
OPTIONS ONLY THOSE
CHOSEN IN STYMOTIV
1.
To help me lose weight
CHOSEN IN STYMOTIV
1. To help me lose weight 2.
To help me concentrate
2. To help me
3.
To help me be alert or stay
concentrate
awake
3. To help me be alert or
4.
To help me study
stay awake
5.
To experiment or to see what
4. To help me study
[IF RSTY42A=5 THEN “they’re”
5. To experiment or to
see what [IF
ELSE “it’s”]like
RSTY42A=5 THEN
6.
To feel good or get high
“they’re” ELSE
7.
To increase or decrease the
“it’s”]like
effect(s) of some other drug
6. To feel good or get
8.
Because I am “hooked” or I
high
have to have [IF RSTY42A=5
7. To increase or
THEN “them” ELSE “it”]
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. [IF STMOTIV=9] The
other reason I reported
[IF STMISUSE12=1] At any time in
[IF STMISUSE12=1] At any time in the
the past 12 months, did you ever
past 12 months, did you ever use a
use a needle to inject
needle to inject [STNAMEFILL] in any
[STNAMEFILL]?
way a doctor did not direct you to use
[STNUMFILL]?
Notes/Testing Results
to gather data about the
primary motivation for
misuse of sedatives. Edits
to logic were made as a
result of cognitive
testing.
Question moved from
Special Drugs to
Stimulants main module.
As a result of cognitive
testing, the wording of
the question was
streamlined.
2012 Question Wording
Not included in 2012
31
QFT Variable
(Testing
Phase
Outcome)
Introduced
QFT Instrument Wording
implemented
in QFT.)
SD01 (Change Phase 2
The last questions were about
tested, no
prescription drugs. The next
problems
question is about non‐prescription
found,
cough or cold medicines, also known
implemented
as “over‐the‐counter” medicines.
in QFT.)
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
SD02(Change Phase 2
[IF SD01 = 1] How long has it been
tested, no
since you last took one of these
problems
cough or cold medicines to get high?
found,
implemented
in QFT.)
SD05/SD15
Phase 2
Have you ever, even once, used a
(Change
needle to inject any other drug that
tested, no
was not prescribed for you?
problems
found,
implemented
in QFT.)
SD10a
Phase 2
Have you ever, even once, used a
(Change
needle to inject
tested, no
methamphetamine?
problems
found,
implemented
in QFT.)
MJMM
(Change
tested, no
problems
found,
implemented
Phase 2
Initial Test Wording
Notes/Testing Results
No changes between initial testing and Wording introducing the
final QFT wording.
over the counter
medicine questions was
tailored to redesign
conventions and
question order.
No changes between initial testing and Collected recency data
final QFT wording.
for misuse of cough and
cold medicines.
No changes between initial testing and In order to maintain
final QFT wording.
conventions of revised
misuse wording, deleted
the words, “only for the
experience or feeling it
caused.”
No changes between initial testing and To maintain consistency
final QFT wording.
across modules, and to
limit follow up questions
to data collected in the
stand alone
methamphetamine
module, removed
reference to Desoxyn
and Methedrine.
Questions about medical
[IF (MJLAST3 = 1 ‐ 2 OR MJRECDK = 1 [IF MJ01=1 OR MJREF=1] Was any of
‐ 2 OR MJRECRE = 1 ‐ 2) OR BL03=2 ] your marijuana use recommended by use of marijuana were
a doctor?
added to the redesign
Earlier, you reported using
instrument. After
marijuana in the past year. Was any
cognitive interviewing,
of your marijuana use in the past 12
the wording was tailored
months recommended by a doctor?
2012 Question Wording
The last two questions were about
prescription drugs. The next
question is about non‐prescription
cough or cold medicines, also known
as over the counter medicines.
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
Not included in 2012
Have you ever, even once, used a
needle to inject any other drug that
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Have you ever, even once, used a
needle to inject Methamphetamine,
Desoxyn, or Methedrine when it
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Not included in 2012
32
QFT Variable
(Testing
Outcome)
in QFT.)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
MJMM01
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
[IF MJMM=1] Was all of your
[IF MJMM=1] Was all of your
marijuana use in the past 12 months marijuana use recommended by a
recommended by a doctor?
doctor?
DRPR
DRTR
DRST
DRSV
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
NOTE: Example text from DRPR is
displayed below. Equivalent text was
used for DRTR, DRST, & DRSV.
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next
questions. Remember, we are only
interested in prescription pain
relievers that you used in any way a
doctor did not direct you to. Earlier
the computer recorded that in the
past 12 months you used [IF
PRMISCOUNT=1 FILL PRFILL2][IF
PRMISCOUNT>=2 FILL WITH “the
pain relievers listed below” ] in a
way a doctor did not direct you to
use [PRNUMFILL].
[IF PRMISCOUNT>=2 FILL WITH
DRUG NAMES FROM PRY01‐PRY39
BELOW. USE MULTIPLE COLUMNS
AS NEEDED. IF PRY40 = 1, ADD
"Some other prescription pain
reliever".]
The next questions refer to [IF
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested in
your use of prescription pain relievers
that were not prescribed for you or
that you used only for the experience
or feeling they caused.
Press [ENTER] to continue.
Notes/Testing Results
to 12 month users of
marijuana only.
Questions about medical
use of marijuana were
added to the redesign
instrument. After
cognitive interviewing,
the wording was tailored
to 12 month users of
marijuana only.
Wording in the
Substance Dependence
and Abuse section was
updated to reflect
revised wording
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
2012 Question Wording
Not included in 2012
IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested
in your use of prescription pain
relievers that were not prescribed
for you or that you used only for the
experience or feeling they caused.
Press [ENTER] to
continue.
33
QFT Variable
(Testing
Outcome)
Phase
Introduced
DRST05
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
Height
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase 1
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
PRY40 NE 1 AND PRMISCOUNT=1
FILL PRFILL2 as a prescription pain
reliever; IF PRY40 = 1 AND
PRMISCOUNT=1 FILL WITH “this
other prescription pain reliever”; IF
PRMISCOUNT>=2 FILL WITH “these
as prescription pain relievers”].
Press [ENTER] to continue.
[IF DRST04 = 1] Were you able to
No changes between initial testing and Wording in the Drug
keep to the limits you set, or did you final QFT wording.
Dependence and Abuse
section was updated to
often use prescription stimulants
reflect revised wording
more than you intended to?
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
HLTH04 This question asks about
HLTHNEW01
About how tall are
Height and weight
your height.
you, without shoes? First, please type questions were added as
part of the redesign in
in the number of feet, then press
[ENTER].
order to capture more
To answer in feet and inches, press
details on physical health
1. To answer in meters and
To answer using meters and
of respondents. Several
centimeters, press 2. To answer in
centimeters, press F3.
versions of these
inches only, press 3. To answer in
questions were tested in
centimeters only, press 4. Then
press [ENTER] to continue.
______feet [RANGE: 2‐8]
order to make it easier
DK/REF
for respondents to
1
I would rather answer in
answer in either metric
feet and inches
HLTHNEW01a (IF HLTHNEW01 NE
or US measurements.
2
I would rather answer in
DK/RF) Next, please type in the
Edits were also made so
meters and centimeters
number of inches and then press
that respondents could
3
I would rather answer only [ENTER].
answer only in inches or
centimeters.
in inches
4
I would rather answer only ______inches [RANGE: 0‐11]
in centimeters
DK/REF
DK/REF
2012 Question Wording
[IF DRPR04 = 1] Were you able to
keep to the limits you set, or did you
often use prescription pain relievers
more than you intended to?
Not included in 2012
34
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
HLTH05 [IF HLTH04=1] About how
tall are you, without shoes? First,
please type in the number of feet,
then press [ENTER].
______feet [RANGE: 0‐8]
DK/REF
HLTH06 [IF HLTH04 = 1 OR 3 AND
HLTH05 NE DK/RF] Please type in
the number of inches and then press
[ENTER].
______inch(es) [RANGE: 0‐110]
DK/REF
HLTH07 [IF HLTH04=2] About how
tall are you, without shoes? First,
please type in the number of
meters, then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTH08 [IF HLTH04 = 2 OR 4AND
HLTH07 NE DK/REF] Please type in
the number of centimeters and then
press [ENTER].
______centimeter(s) [RANGE:0‐300]
DK/REF
Initial Test Wording
PROGRAMMER: PLEASE DISPLAY
HLTHNEW01 AND HLTHNEW01a ON
THE SAME SCREEN.
HLTHNEW01b (IF HLTHNEW01=DK)
You may also report your height using
meters and centimeters. About how
tall are you, without shoes? First,
please type in the number of meters,
then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTHNEW01c [IF HLTHNEW01b NE
BLANK] Next, please type in the
number of centimeters and then press
[ENTER].
______centimeters [RANGE:0‐275]
DK/REF
PROGRAMMER: PLEASE DISPLAY
THESE HLTHNEW01b AND
HLTHNEW01c ON THE SAME SCREEN.
Notes/Testing Results
2012 Question Wording
35
QFT Variable
(Testing
Outcome)
Weight
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase
Introduced
Phase 1
QFT Instrument Wording
HLTH09 The next question asks
about your weight.
To answer in pounds, press 1. To
answer in kilograms, press 2. Then
press [ENTER] to continue.
1
I would rather answer in
pounds
2
I would rather answer in
kilograms
DK/REF
HLTH10 [IF HLTH09=1 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of pounds and then press [ENTER].
________pounds [RANGE: 50‐550]
DK/REF
HLTH12 [IF HLTH11=2 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of kilograms and then press
[ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTH13 [IF HLTH02=1 AND
HLTH09=1] About how much did
you weigh before you got pregnant?
Please type in the number of pounds
and then press [ENTER].
Initial Test Wording
HLTHNEW02
(IF HLTH01 NE 1)
About how much do you weigh?
Please type in the number of pounds
and then press [ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02A (IF HLTHNEW02=DK)
You may also report your weight using
kilograms. About how much do you
weigh? Please type in the number of
kilograms and then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTHNEW02p (IF HLTH01 = 1) About
how much did you weigh before you
got pregnant? Please type in the
number of pounds and then press
[ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02pp (IF HLTHNEW02p
=DK) You may also report your weight
using kilograms. About how much did
you weigh before you got pregnant?
Please type in the number of
kilograms and then press [ENTER].
Notes/Testing Results
2012 Question Wording
Height and weight
Not included in 2012
questions were added as
part of the redesign in
order to capture more
details on physical health
of respondents. Several
versions of these
questions were tested in
order to make it easier
for respondents to
answer in either metric
or US measurements.
36
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTHNEW03
(HLTH19)
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 1
HLTH22
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QFT Instrument Wording
________pounds [RANGE: 50‐550]
DK/REF
HLTH14 [IF HLTH02=1 AND HLTH09
=2] About how much did you weigh
before you got pregnant? Please
type in the number of kilograms and
then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
During the past 12 months, how
many times have you visited a
doctor, nurse, physician assistant or
nurse practitioner about your own
health at a doctor’s office, a clinic,
or some other place?
[((IF HLTH16>=1 AND (HLTH17=1 OR
HLTH19>=1)) OR (HLTH17=1 AND
(HLTH16>=1 OR HLTH19>=1 OR
HLTH19DK>=1)) OR ((HLTH19>=1 OR
HLTH19DK>=1) AND (HLTH16>=1 OR
HLTH17=1))) AND (ALLAST3=1 OR 2
OR ALRECDK=1 OR 2 OR ALRECRE=1
OR 2)] Please think about all of the
talks you have had with a doctor or
other health care professional
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussions you may
Initial Test Wording
Notes/Testing Results
2012 Question Wording
_______ kilograms [RANGE: 22‐275]
DK/REF
DK/REF
During the past 12 months, how many
times have you visited a doctor, nurse,
physician assistant or other health
care professional about your own
health at a doctor’s office, a clinic, or
some other place?
Do not include times you were
hospitalized overnight, visits to
hospital emergency rooms, home
visits by a health care professional,
dental visits, or telephone calls.
No changes between initial testing and
final QFT wording.
Questions about health
Not included in 2012
provider visits were
added as part of the
redesign. Instructions
about how to count visits
were deemed
unnecessary or confusing
and removed.
Not included in 2012
Questions were tested
about discussing alcohol
& drug use with a doctor.
No changed were made
as a result of testing.
37
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTH26othr Phase 2
(Change
tested, no
problems
found,
implemented
in QFT.)
HLTHNEW11 Phase 1
(HLTH29)
(Change
tested, no
problems
found,
QFT Instrument Wording
Initial Test Wording
have had in person with a doctor or
other health professional about your
alcohol use.
[((IF HLTH16>=1 AND HLTH17 NE 1
AND HLTH19<1 AND HLTH19DK<1)
OR (HLTH17 =1 AND HLTH16 = 0
AND HLTH19<1 AND HLTH19DK<1)
OR ((HLTH19 >=1 OR HLTH19DK>=1)
AND HLTH16 = 0 AND HLTH17 NE 1))
AND (ALLAST3=1 OR 2 OR
ALRECDK=1 OR 2 OR ALRECRE=1 OR
2)] Please think about [VISITFILL]
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussion you may
have had in person with a doctor or
other health professional about your
alcohol use.
To select more than one statement,
press the space bar between each
number you type. When you have
finished, press [ENTER].
No change.
(IF HLTH26=30) Please tell me which
other kind of cancer you had.
[IF HLTH25=9 AND HLTH28 AND
HLTH28a AND HLTH28b AND
HLTH28c AND HLTH28c AND
HLTH28d AND HLTH28e AND
HLTH28f AND HLTH28g AND
HLTH28h AND HLTH28i AND
(IF HLTHNEW08=9) Did you have
cancer during the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
38
QFT Variable
(Testing
Outcome)
implemented
in QFT.)
Phase
Introduced
HLTHNEW12a Phase 1
(HLTH31)
(Change
tested, no
problems
found,
implemented
in QFT.)
Disability
NA
Items (QD55‐
QD61) Not
tested
QFT Instrument Wording
HLTH28j AND HLTH28k AND
HLTH28l AND HLTH28m AND
HLTH28n AND HLTH28o AND
HLTH28p AND HLTH28q AND
HLTH28r AND HLTH28s AND
HLTH28t AND HLTH28u AND
HLTH28v AND HLTH28w AND
HLTH28x AND HLTH28y AND
HLTH28z AND HLTH28aa AND
HLTH28bb AND HLTH28cc NE
CALCAGE] Did you have cancer
during the past 12 months?
[IF HLTH25=1 AND HLTH30 NE
CALCAGE] Did you have any kind of
heart condition or heart disease in
the past 12 months?
QD55 How well do you speak
English?
QD56 Are you deaf or do you
have serious difficulty hearing?
QD57 Are you blind or do you
have serious difficulty seeing, even
when wearing glasses?
QD58 Because of a physical,
mental or emotional condition, do
you have serious difficulty
concentrating, remembering, or
making decisions?
QD59 Do you have serious
Initial Test Wording
(IF HLTHNEW08=1) Did you have any
kind of heart condition or heart
disease in the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about health
conditions were added.
Only logistical/spec
changes were made as a
result of the cognitive
testing.
Not included in 2012
These items were added
in response to feedback
received about the
survey.
Not included in 2012
39
QFT Variable
(Testing
Outcome)
Phase
Introduced
QD17 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
difficulty walking or climbing stairs?
QD60 Do you have difficulty
dressing or bathing?
QD61 [IF CURNTAGE >14]
Because of a physical, mental or
emotional condition, do you have
difficulty doing errands alone such
as visiting a doctors’ office or
shopping?
No changes between initial testing and This question was moved
The next questions are about
final QFT wording.
to ACASI. No changes
school. Are you now attending or
were made as a result of
are you currently enrolled in school?
testing.
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
QD26 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
No changes between initial testing and This question was moved
[IF CURNTAGE = 15 OR OLDER] The
final QFT wording.
to ACASI.
next questions are about working.
Explanatory/example
Did you work at a job or business at
text was added as
any time last week? By last week, I
optional text available
mean the week beginning on
with pressing the F2 key.
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
No changes were made
as a result of testing.
Press F2 to hear more information
about what types of work to include.
Military
QD10d For this question, please
NA
These items were added
2012 Question Wording
The next questions are about
school. Are you now attending or
are you currently enrolled in school?
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
1
YES
2
NO
DK/REF
[IF CURNTAGE = 15 OR OLDER] The
next questions are about working.
Did you work at a job or business at
any time last week? By last week, I
mean the week beginning on
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
1
YES
2
NO
DK/REF
Not included in 2012
40
QFT Variable
(Testing
Outcome)
Family Items
(QD10d &
QD10e) Not
tested
Phase
Introduced
PROXYINT
NA
NOPROX
(HINSINT)
(revisions
Phase 2
QFT Instrument Wording
include all persons in your
immediate family, whether or not
they live with you. Is anyone in your
immediate family currently serving
in the United States military? Press
F2 to see and hear definitions of
“immediate family” and “military.”
QD10e Which member or members
of your immediate family are
currently in the United States
military? Press F2 to see and hear
definitions of “immediate family.”
1 My spouse
2 Unmarried partner
3 My mother
4 My father
5 My son or sons
6 My daughter or
daughters
7 My brother or brothers
8 My sister or sisters
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you [IF FAMILY MEMBERS IN
ROSTER FILL “and your family”]
receive. This information will help in
planning health care services and
finding ways to lower costs of care.
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR
HASJOIN= 2 OR DK/REF] I’m going to
Initial Test Wording
Notes/Testing Results
in response to feedback
received about the
survey.
2012 Question Wording
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you receive. (This information will
help in planning health care services
and finding ways to lower costs of
care.)
New language was added Not included in 2012
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR HASJOIN = to assist the interviewer
with transitioning back to
2 OR DK/REF] I’m going to give the
Revised wording after
Phase 2 testing to
encourage nominations
of proxies when
necessary.
41
QFT Variable
(Testing
Outcome)
tested,
problems
found,
changes
implemented
in QFT)
QHI03
(Wording
revised for
accuracy
after testing,
implemented
in QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
give the computer back to you so
that you can complete the last part
of the interview on your own.
[IF (QP03 = 2 OR DK/REF) OR (QP04
= 2 OR DK/REF) OR (HASJOIN = 2 OR
DK/REF) ADD: Since your [QP02 FILL]
is not available, please answer these
next questions the best you can.]
[IF NEWPROX = 2 AND HASJOIN=1]
I’m going to give the computer to
you so that you can complete the
last part of the interview on your
own.
Please put on the headphones.
When you are ready, let me know.
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
There are certain programs that
cover active duty and retired career
military personnel and their
dependents and survivors and also
disabled veterans and their
dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to see and hear more
information about these programs.
CHAMPUS stands for Civilian
Health and Medical Program of
Initial Test Wording
computer back to you so that you can
complete the last part of the interview
on your own. When you are ready, let
me know.
[IF QP03 = 2 OR DK/REF OR QP04 = 2
OR DK/REF OR HASJOIN = 2 OR DK/REF
ADD THIS TEXT PRIOR TO THE
QUESTION: Since your [QP02 FILL] is
not available, please answer these
next questions the best you can.]
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
Notes/Testing Results
ACASI for the final
questions. Language was
tailored to situations
where a proxy might be
required to answer the
income series. As a
result of testing,
revisions were made to
make the proxy
transition happen more
smoothly.
2012 Question Wording
There are certain programs that cover
active duty and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to hear more information
about these programs.
CHAMPUS stands for Comprehensive
Health and Medical Plan for the
Wording was revised for
ACASI administration,
and later to accurately
note program names.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
(These programs cover active duty
and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.)
1
YES
2
NO
DK/REF
INTERVIEWER NOTE:
42
QFT Variable
(Testing
Outcome)
INTROINC
(revisions
tested, minor
problems
found,
changes
made for
QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
the Uniformed Services. It
provides health care in private
facilities for dependents of
military personnel on active duty
or retired for reasons other than
disability. In some areas, this
may be known as TRICARE.
Initial Test Wording
Uniformed Services. It provides health
care in private facilities for
dependents of military personnel on
active duty or retired for reasons
other than disability. In some areas,
this may be known as TRICARE.
CHAMPVA stands for Comprehensive
Health and Medical Plan of the
CHAMPVA stands for Civilian
Veterans Administration. It provides
Health and Medical Program of
health care for the spouse,
the Department of Veterans
dependents, or survivors of a veteran
Affairs. It provides health care
who has a total, permanent service‐
for the spouse, dependents, or
survivors of a veteran who has a connected disability.
total, permanent service‐
The VA provides medical assistance to
connected disability.
veterans of the Armed Forces,
particularly those with service‐
The VA provides medical
connected ailments.
assistance to veterans of the
Armed Forces, particularly those
with service‐connected ailments. Military health care refers to health
care available to active duty personnel
and their dependents.
Military health care refers to
health care available to active
duty personnel and their
dependents.
[IF NO FAMILY MEMBERS IN ROSTER]
[IF NO FAMILY MEMBERS IN
These next questions are about the
ROSTER] These next questions are
kinds and amounts of income that you
about the kinds and amounts of
receive.
income that you receive.
[IF ONE FAMILY MEMBER IN ROSTER [IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] These next
AND HASJOIN NE 1 ] These next
questions are about the kinds and
questions are about the kinds and
amounts of income that you and
amounts of income that you and your
your [FAMILY RELATIONSHIP FILL]
[FAMILY RELATIONSHIP FILL] receive.
receive.
Notes/Testing Results
2012 Question Wording
CHAMPUS stands for
Comprehensive Health and Medical
Plan for the Uniformed Services. It
provides health care in private
facilities for dependents of military
personnel on active duty or retired
for reasons other than disability. In
some areas, this may be known as
TRICARE.
CHAMPVA stands for
Comprehensive Health and Medical
Plan of the Veterans Administration.
It provides health care for the
spouse, dependents, or survivors of
a veteran who has a total,
permanent service‐connected
disability.
Military health care refers to health
care available to active duty
personnel and their dependents; in
addition, the VA provides medical
assistance to veterans of the Armed
Forces, particularly those with
service‐connected ailments.
Wording was revised for [IF NO FAMILY MEMBERS IN
ACASI administration. As ROSTER] These next questions are
a result of the cognitive
about the kinds and amounts of
income that you receive.
testing, changes were
made to make the
[IF ONE FAMILY MEMBER IN ROSTER
process go more
AND HASJOIN NE 1 ] These next
smoothly.
questions are about the kinds and
amounts of income that you and
your [FAMILY RELATIONSHIP FILL]
receive.
43
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you and your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, you and [IF
QD01=5 FILL his, QD01 = 9 FILL
her][FAMILY RELATIONSHIP FILLS]
living here – receive.
[PROGRAMMER NOTE: THE PROXY
SHOULD NOT APPEAR IN [FAMILY
RELATIONSHIP FILLS]. ALSO, USE
‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF HASJOIN NE 1] These questions
refer to the calendar year [CURRENT
YEAR ‐ 1] rather than to the past 12
Initial Test Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
you receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] These
next questions are about the kinds
and amounts of income that you, your
[FAMILY RELATIONSHIP FILLS] living
here receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN=1] These
next questions are about the kinds
and amounts of income that [SAMPLE
MEMBER] and [IF QD01=5 FILL his,
QD01 = 9 FILL her] family – that is,
your [SAMPLE MEMBER POSS]
[FAMILY RELATIONSHIP FILLS] living
here – receive.
These questions refer to the calendar
year [CURRENT YEAR ‐ 1] rather than
to the past 12 months that were
referred to in some earlier questions.
The calendar year [CURRENT YEAR ‐ 1]
would be from January 1st, [CURRENT
YEAR ‐ 1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
Notes/Testing Results
2012 Question Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you, your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, your [SAMPLE
MEMBER POSS] [FAMILY
RELATIONSHIP FILLS] living here –
receive.
These questions refer to the
calendar year [CURRENT YEAR ‐ 1]
rather than to the past 12 months
that were referred to in some earlier
questions. The calendar year
[CURRENT YEAR ‐ 1] would be from
January 1st, [CURRENT YEAR ‐ 1],
through December 31st, [CURRENT
YEAR ‐ 1].
PRESS [ENTER] TO CONTINUE
44
QFT Variable
(Testing
Outcome)
QI12AN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
months that were referred to in
some earlier questions. The calendar
year [CURRENT YEAR ‐ 1] would be
from January 1st, [CURRENT YEAR ‐
1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Any other kind of non‐
monetary welfare or public
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family member
living here receive any type of welfare
or public assistance?
# OF MONTHS RECEIVED ASSISTANCE:
[RANGE: 1 ‐ 12]
DK/REF
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
45
QFT Variable
(Testing
Outcome)
QI12BN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
assistance
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance, not including food stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance,
not including food stamps?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
46
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTRTINN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QI21B
(Change
tested, no
problems
Phase 2
QFT Instrument Wording
Any other kind of non‐
monetary welfare or public
assistance
Below is a list of some other sources
of income. When I ask you the next
questions, please consider these as
well as the other sources asked
about in earlier questions.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
Press [Enter] to continue.
[IF QI20N = 1 OR QI20NREF = 1] Of
these income groups, which
category best represents [SAMPLE
MEMBER POSS] total personal
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Below is a list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we just
talked about.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
Wording was revised for
ACASI administration.
No changes were made
as a result of cognitive
testing.
HAND R SHOWCARD 16a. Here is a
list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we
just talked about.
INTERVIEWER: PLEASE READ THIS
ALOUD TO THE RESPONDENT AS
HE/SHE FOLLOWS ALONG ON THE
SHOWCARD.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
[IF QI20N = 1] Of these income groups,
which category best represents
[SAMPLE MEMBER POSS] total
personal income during [CURRENT
Income categories were
revised to adjust for
inflation and the
question was revised for
[IF QI20N = 1] ENTER NUMBER THAT
BEST REPRESENTS (R’S/SAMPLE
MEMBER’S) TOTAL PERSONAL
INCOME DURING [CURRENT YEAR ‐
47
QFT Variable
(Testing
Outcome)
found,
implemented
in QFT.)
INTROFI1
(Change
tested, minor
problems
found,
changes
made for
QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
income during [CURRENT YEAR ‐ 1]?
1
$20,000 ‐ $24,999
2
$25,000 ‐ $29,999
3
$30,000 ‐ $34,999
4
$35,000 ‐ $39,999
5
$40,000 ‐ $44,999
6
$45,000 ‐ $49,999
7
$50,000 ‐ $74,999
8
$75,000 ‐ $99,999
9
$100,000 ‐ $149,999
10
$150,000 or more
DK/REF
Initial Test Wording
YEAR ‐ 1]?
21
$20,000 ‐ $24,999
22
$25,000 ‐ $29,999
$30,000 ‐ $34,999
23
24
$35,000 ‐ $39,999
25
$40,000 ‐ $44,999
26
$45,000 ‐ $49,999
27
$50,000 ‐ $74,999
28
$75,000 ‐ $99,999
29
$100,000 ‐ $149,999
30
$150,000 or more
DK/REF
[IF MORE THAN ONE FAMILY MEMBER
[IF MORE THAN ONE FAMILY
IN ROSTER AND IF QI21B NE 29]
MEMBER IN ROSTER AND IF QI21B
NE 30]
Next, we would like to know about the
total family income from all sources
Next, we would like to know about
during [CURRENT YEAR ‐ 1] before
the total family income from all
sources during [CURRENT YEAR ‐ 1] taxes and other deductions.
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN NE 1 ] We would like
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
you to combine everyone’s income – that is, yours and that of your [FAMILY
that is, yours and that of your
RELATIONSHIP FILL].
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN=1] We would like you to
AND HASJOIN=1] We would like you combine everyone’s income – that is,
to combine everyone’s income –
[SAMPLE MEMBER POSS] and yours.
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] We
[IF AT LEAST TWO FAMILY
would like you to combine everyone’s
MEMBERS IN ROSTER AND HASJOIN income – that is, yours and that of
Notes/Testing Results
ACASI administration. No
changes were made as a
result of cognitive
testing.
2012 Question Wording
1].
21
22
23
24
25
26
27
28
29
$20,000 ‐ $24,999
$25,000 ‐ $29,999
$30,000 ‐ $34,999
$35,000 ‐ $39,999
$40,000 ‐ $44,999
$45,000 ‐ $49,999
$50,000 ‐ $74,999
$75,000 ‐ $99,999
$100,000 OR MORE
DK/REF
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration.
Edits were made to
family relationship fills to
result in a more natural
sentence structure.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
48
QFT Variable
(Testing
Outcome)
QI23A
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is, yours,
[SAMPLE MEMBER POSS] and that of
[IF QD01 = 5 FILL his, QD = 9 FILL
her] [FAMILY RELATIONSHIP FILLS]
living here. [PROGRAMMER NOTE:
THE PROXY SHOULD NOT APPEAR IN
[FAMILY RELATIONSHIP FILLS]. ALSO,
USE ‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
Initial Test Wording
your [FAMILY RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN = 1] We
would like you to combine everyone’s
income — that is, [SAMPLE MEMBER
POSS] and that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here.
Please include all of the sources of
income that we just talked about.
Notes/Testing Results
2012 Question Wording
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, yours and that of your [FAMILY
RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
testing.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
49
QFT Variable
(Testing
Outcome)
QI23B
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
YEAR – 1] – that is, your [SAMPLE
MEMBER POSS] and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 = 9
FILL her] [FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
Initial Test Wording
that is, your [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
Notes/Testing Results
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
2012 Question Wording
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
Collapsed from multiple 2012
variables due to change in modes.
50
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, [SAMPLE
MEMBER POSS] and yours?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your [FAMILY
RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 =9
FILL her][FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
Initial Test Wording
Notes/Testing Results
income during [CURRENT YEAR – 1] –
testing.
that is, yours and that of your [FAMILY
RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, [SAMPLE MEMBER POSS] and
yours?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[FAMILY RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
2012 Question Wording
51
QFT Variable
(Testing
Outcome)
Phase
Introduced
CELL1
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
CELL2
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
(MJE01 –
MJE70)
(entire
existing
module
dropped)
LU22 – LU26
(existing
question
dropped)
None
None
QFT Instrument Wording
Initial Test Wording
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Is there at least one telephone at
Is there at least one telephone at this
this address that is not a cell phone? address that is not a cell phone?
Do you or anyone at this address
have a working cell phone?
Do you or anyone at this address have
a working cell phone?
Notes/Testing Results
2012 Question Wording
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
The last question has to do with
telephones in your household. How
many different telephone numbers
do you have in this household?
Please don’t include cellular phones
in your answer. Also, don’t count
business numbers or extensions
with the same number.
INTERVIEWER NOTE:
Do not include phone lines that are
used only for fax machines and/or
Internet access.
Not included in 2012
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
2012 instrument items or modules removed from consideration for the QFT
Module dropped from QFT.
Module dropped from QFT.
These questions were
outdated in terms of the
current marijuana
market. They were
removed to make room
for other important
topics.
Item dropped from QFT.
Item dropped from QFT.
Dropped “which came
first” questions from the
Prior Substance Use
module to make room for
Market Information for Marijuana
Module
LU22
Earlier, the computer
recorded that you started using
alcohol and cigarettes when you
were [AFUFILL] years old. Which
52
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
SEN04/YE04
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
CA12‐CA14d
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Notes/Testing Results
additions to the
Prescription Drug
modules.
This question had little
analytic value and was
dropped to make room
for more important
topics.
The Alcohol module now
measures gender specific
binge drinking
prevalence, so the ‘4 or
more drinks’ questions
were deleted.
2012 Question Wording
did you use first?
LU23 Earlier, the computer
recorded that you started using
marijuana and cigarettes when you
were [AFUFILL] years old. Which
did you use first?
LU24 Earlier, the computer
recorded that you started using
alcohol and marijuana when you
were [AFUFILL] years old. Which
did you use first?
LU25 Earlier, the computer
recorded that you started using
alcohol, cigarettes and marijuana
when you were [AFUFILL] years old.
Which did you use first?
LU26 After first using [FILL
LU25], which of these did you use
next?
How many times have you moved in
the past 5 years?
CA12 Have you ever had 4 or more
drinks on the same occasion?
CA13 During the past 30 days, that
is, since [DATEFILL], on how many
days did you have 4 or more drinks
on the same occasion?
CA14 How old were you the first
53
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
Industry and
Occupation
Questions –
INOC01 –
INOC06
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these questions
due to low levels of
analytic utility and
difficulties administering
these questions in ACASI
Household
Roster –
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these items due
to low analytic utility
2012 Question Wording
time you had 4 or more drinks on
the same occasion?
INOC01
[IF QD26 =
1 OR QD27 = 1] In what kind of
business or industry do you work?
That is, what product is made or
what service is offered?
INOC02 [IF QD26=1 OR QD27=1
AND INOC01 NE DK/REF] HAND R
SHOWCARD 7. Which of these
categories best describes the
business or industry in which you
work?
INOC02M What do they make?
INOC02T What do they sell?
INOC03Please describe the business
or industry in which you work.
INOC04 What kind of work do you
do? That is, what is your
occupation?
INOC05 What are your most
important activities or duties in that
job?
INOC06 Which of these categories
best describes the business in which
you work?
Is (s)he your biological, step‐,
adoptive, or foster [FILL
54
QFT Variable
(Testing
Outcome)
FTHRTYPE,
MTHRTYPE,
SONTYPE,
DAUTYPE
(existing
question
dropped)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
RELATIONSHIP]?
55
NATIONAL SURVEY ON DRUG
USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY
REPORT—DRAFT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009 & 0211838
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 17, 2014
This page is intentionally blank.
2015 NATIONAL SURVEY ON
DRUG USE AND HEALTH
QUESTIONNAIRE REDESIGN
PRETESTING SUMMARY REPORT
Contract Nos. 283-2004-00022 & HHSS283200800004C
RTI Project Nos. 0209009.486.014 & 0211838.108.006.004
Authors:
Project Director:
Elizabeth Dean
Brian Head
Patricia LeBaron
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
January 2014
Acknowledgments
This publication was developed for the Substance Abuse and Mental Health Services
Administration (SAMHSA), Center for Behavioral Health Statistics and Quality (CBHSQ), by
RTI International (a trade name of Research Triangle Institute), Research Triangle Park, North
Carolina, under Contract Nos. 283-2004-00022 and HHSS283200800004C. Significant
contributors at SAMHSA include Dicy Painter, Joel Kennet, and Peggy Barker. Significant
contributors at RTI include Larry Kroutil and Doug Currivan.
ii
Table of Contents
Chapter
Page
1.
Overview ..............................................................................................................................1
1.1
Background of the Redesign ....................................................................................1
1.2
Goals of the Pretesting .............................................................................................3
1.2.1 Usability Testing ..........................................................................................3
1.2.2 Cognitive Interviewing Phase 1 ...................................................................4
2.
Usability Testing ..................................................................................................................7
2.1
Data Collection and Participant Profiles ..................................................................7
2.1.1 Description of Recruitment Methods ...........................................................7
2.1.2 Respondent Characteristics across Usability Testing ..................................7
2.1.3 Data Collection Procedures..........................................................................8
2.2
Key Findings ............................................................................................................9
2.2.1 Logistical Issues with Answering the "Yes/No" or "Enter All That
Apply" Questions .........................................................................................9
2.2.2 Ability to Use the Drug Images ...................................................................9
2.2.3 Screen Layout Preferences ...........................................................................9
2.2.4 Issues with the Question Text from a Usability Standpoint ........................9
2.3
Changes Implemented ..............................................................................................9
3.
Cognitive Interviewing: Phase 1 ........................................................................................11
3.1
Data Collection and Participant Profiles ................................................................11
3.1.1 Description of Recruitment Methods .........................................................11
3.1.2 Respondent Characteristics across Phase 1................................................11
3.1.3 Data Collection Procedures........................................................................12
3.2
Key Findings ..........................................................................................................13
3.2.1 Identification of Prescription Drugs...........................................................13
3.2.2 Performance of the Misuse Criteria ...........................................................13
3.2.3 Other Issues ................................................................................................14
3.3
Changes Implemented ............................................................................................14
4.
Cognitive Interviewing: Phase 2 ........................................................................................17
4.1
Data Collection and Participant Profiles ................................................................17
4.1.1 Description of Recruitment Methods .........................................................17
4.1.2 Participant Characteristics .........................................................................17
4.1.3 Data Collection Procedures........................................................................18
4.2
Key Findings ..........................................................................................................19
4.2.1 Demographic Items ....................................................................................19
4.2.2 Prescription Drugs and Use/Misuse ...........................................................20
4.2.3 Tobacco, Illicit, and Special Drugs ............................................................21
4.3
Changes Implemented ............................................................................................22
4.3.1 Phase 2, Round 1........................................................................................22
4.3.2 Phase 2, Round 2........................................................................................23
iii
Table of Contents (continued)
Chapter
Page
5.
Discussion ..........................................................................................................................27
5.1
Summary of Major Changes ..................................................................................27
5.2
Usability Testing Advantages ................................................................................27
5.3
Recommendations for Future Pretests ...................................................................28
6.
References ..........................................................................................................................29
iv
List of Attachments
Attachment
Page
A.
Phase 1.1 Summary of Findings ..................................................................................... A-1
B.
Phase 1.2 Summary of Findings ......................................................................................B-1
C.
Phase 1.3 Summary of Findings ......................................................................................C-1
D.
Phase 2.1 Summary of Findings ..................................................................................... D-1
E.
Phase 2.2 Summary of Findings ...................................................................................... E-1
F.
Variable Wording Crosswalk ........................................................................................... F-1
v
This page is intentionally blank.
vi
List of Tables
Table
Page
1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants ........ 8
2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants ....... 12
3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants ....... 18
4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens ......................... 22
5.
List of Screens Presented for the Full and Abbreviated Tutorials .................................... 25
vii
This page is intentionally blank.
viii
1. Overview
1.1
Background of the Redesign
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance
Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S.
civilian, noninstitutionalized population aged 12 and older. The conduct of the NSDUH is
paramount in meeting a critical objective of SAMHSA's mission to maintain current data on the
prevalence of substance use in the United States. In order to continue producing current data,
SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ) must update the
NSDUH periodically to reflect changing substance use and mental health issues. These updates
are necessary because substance use in the United States is a dynamic phenomenon that changes
with time, demographic shifts, economic prosperity, and most importantly, availability of
substances.
CBHSQ plans to redesign the NSDUH for the 2015 survey year to achieve two main
goals: (1) revise the questionnaire to address changing policy and research data needs, and
(2) modify the survey methodology to improve the quality of estimates and the efficiency of data
collection and processing.
The questionnaire will be revised to improve questions that cause known or suspected
problems with data that are collected using the current NSDUH questionnaire. New content that
addresses current data needs will be added. Revisions that are designed to reduce errors
associated with usability problems in the design and layout of the computer-assisted interviewing
(CAI) instrument are planned. The changes include electronic calendars, revised prescription
drug modules, revised front end demographics, a revised smokeless tobacco module, revised
questions for measuring binge alcohol use, a revised hallucinogens module, new questions for
lifetime use of specific inhalants, a new methamphetamine module, a revised special drugs
module, a revised consumption of alcohol module, and a revised back end demographics section.
Specifically, the following are highlights of issues for the modules for prescription drugs and
health care and for interviewer-administered questions that will be addressed by the redesigned
questionnaire.
•
In the current interview procedures, interviewers and respondents mark a printed calendar
with the start of the 30-day and 12-month reference periods. In the redesigned
questionnaire, respondents will be shown electronic versions of these calendars that note
the start of these reference periods, and they will receive a tutorial about these electronic
calendars. The electronic calendars also will appear periodically to remind respondents of
reference periods, and respondents can use a function key to review the calendars at any
time during the self-administered modules.
1
•
The content of the specific prescription drugs in the core prescription drug modules was
last updated in 1999, based on prevalence information from 1998. Since that time, despite
minor updates to the questionnaire, several prescription drugs included in NSDUH have
been discontinued or are otherwise no longer legally available, and recently approved
prescription drugs with abuse potential are not included in the core modules. All
questions measuring misuse of prescription drugs will be moved into the prescription
drug modules. In addition, a review of the drugs that are included in these modules will
result in recommendations for drugs to be added or dropped based on their availability.
•
The current NSDUH questionnaire asks only about misuse of prescription drugs. A
revised structure will be tested that first uses a screener module to ask about any use of
prescription drugs. Respondents who report any use pf prescription drugs will then
receive questions determining whether any of that use constituted misuse.
•
The current NSDUH definition of misuse (i.e., use of drugs that were not prescribed for
you or that you took only for the experience or feeling they caused) combines a behavior
and a motivation. It also does not include overuse of prescribed medication. The revised
definition will focus on behaviors that constitute misuse and will refer to use "in any way
a doctor did not direct you to use it/them." Overuse (i.e., use in greater amounts, more
often, or longer than directed) will be added to the examples of behaviors that constitute
misuse.
•
Except for misuse of OxyContin® in the current pain relievers module and misuse of
methamphetamine in the stimulants module, misuse of specific prescription-type
psychotherapeutic drugs in the current questionnaire is limited to the lifetime period.
Measurement of past year misuse is limited to misuse of any drugs within a
psychotherapeutic category (e.g., pain relievers). However, information about more
recent misuse of specific prescription drugs (e.g., in the past year) is of greater interest to
policymakers and stakeholders who use the NSDUH prescription drug data. Therefore,
questions will measure past year misuse of individual prescription drugs.
•
In the current questionnaire, methamphetamine is included in the module for misuse of
prescription stimulants, and respondents are asked in the stimulants module about misuse
of methamphetamine. However, most methamphetamine that is used in the United States
is manufactured illegally rather than by the pharmaceutical industry. Respondents also
may fail to report methamphetamine use in the context of questions about misuse of
prescription drugs. Therefore, the redesigned questionnaire will include a separate
module for methamphetamine, and the questions in the stimulants module will ask about
misuse of prescription stimulants.
•
For the current questionnaire, respondents are handed printed "pill cards" with pictures of
prescription drugs in a given module to aid respondents in recalling their misuse of
specific drugs. NSDUH incurs the expense of printing pill cards for interviewers who are
new to the survey or reprinting the pill cards when new dosages become available for
some prescription drugs. Also, respondents do not always ask interviewers for the pill
cards when they reach the prescription drug questions. Therefore, electronic drug images
to be displayed on the laptop computer screen will replace the printed pill cards. The
electronic images also will include examples of prescription drugs other than pills (e.g.,
patches, vials).
2
•
The current health care module of the questionnaire is limited in scope. Questions
measuring height, weight, a revised list of health conditions, and details about diagnoses
of those health conditions will be tested and added to the health care module.
•
The current survey includes questions about emergency room visits, hospitalizations,
substance abuse treatment utilization, and mental health service utilization, but does not
include questions about outpatient primary care service utilization or opportunities for
intervention with substance users in primary care settings. Questions related to these
topics will be tested and included in the health care module of the questionnaire.
•
Although audio computer-assisted self-interviewing (ACASI) is used for most of the
sensitive questions, respondents may be likely to consider some questions in intervieweradministered sections to be sensitive (e.g., income). Pretesting efforts will explore the
feasibility of moving health insurance and income questions to an ACASI mode.
Proposed revisions will improve the precision of estimates that exhibit properties
associated with measurement error. Revisions will also be designed to reduce errors associated
with usability problems in the design and layout of the CAI instrument.
1.2
Goals of the Pretesting
The pretesting for the 2015 NSDUH questionnaire redesign had three components:
Usability Testing (Prescription Drugs), Phase 1 (Prescription Drugs) Cognitive Interviewing, and
Phase 2 (Redesigned Modules) Cognitive Interviewing. The purpose of each of these three
components is discussed in the following sections.
1.2.1
Usability Testing
Usability testing was conducted in the first stage of redesign pretesting. Prior to the
usability testing, questions for misuse of pain relievers and stimulants were reworded, producing
two new versions of these questions. One of these versions was a "three-criteria" bulleted list that
asked participants about using a set of prescription pain relievers and stimulants "in any of these
ways" in the past 12 months:
•
without a prescription of your own,
•
in greater amounts, more often, or longer than you were told to take it, or
•
just for the effect it had on you—not for its intended medical use.
A second version ("two criteria") eliminated one criterion from the three-criteria list: use
"in greater amounts, more often, or longer that you were told to take it," but used the same
wording for the remaining two criteria.
In addition, the usability instrument tested the placement of pill images above or below
the question text in order to test the new electronic drug images. Participants were presented with
a total of four versions of the questions. Two versions of the three-criteria questions featured
images above and below the questions. There were also two versions of the two-criteria
questions with images above and below the questions. In addition to soliciting feedback on the
criteria items, usability testing sought to accomplish three primary goals:
3
1. observe participant use of and reaction to different options for proposed laptop
computer screen layouts for the screeners and modules for prescription pain relievers
and prescription stimulants;
2. collect participant feedback and recommendations for improving the design of the
laptop computer screen layouts for the screeners and modules for prescription pain
relievers and prescription stimulants; and
3. collect participant feedback on the electronic reference date calendars and electronic
drug images.
1.2.2
Cognitive Interviewing Phase 1
The second stage of redesign pretesting, cognitive interviewing, occurred in two phases.
The first phase of interviewing focused on redesigned prescription drug modules and a
redesigned health module. The second phase continued to test these modules but expanded the
scope to include the full redesigned NSDUH instrument.
Phase 1 cognitive interviewing took place over three rounds and tested an instrument
with changes to the prescription pain relievers, prescription stimulants, and health modules. The
Phase 1 redesigned instrument included an abbreviated core set of substance use questions that
measured lifetime and 30-day use. The following changes were reflected in the Phase 1
abbreviated core questionnaire:
•
Smokeless tobacco sections (chewing tobacco, snuff, and the new product, snus) were
combined into one section.
•
Hallucinogen items currently included in special drugs for ketamine, tryptamines (DMT,
AMT, and "Foxy"), and Salvia divinorum were moved from the noncore special drugs
module to the core hallucinogens module.
•
New inhalants questions for lifetime use of markers and air duster were added to the core
inhalants module.
•
Prescription pain relievers and prescription stimulants screeners focused on any use of
specific prescription drugs in the past 12 months (i.e., medical use or misuse). Misuse of
prescription pain relievers and stimulants in the corresponding main modules was defined
as use "in any way a doctor did not direct you to use it/them" and was based on the
examples from the three-criteria bulleted list that was implemented during usability
testing. Respondents could be reminded about these criteria by pressing a function key to
see a pop-up screen with the bulleted list. Electronic drug images were placed below the
introductory question text, and some images showed examples other than pills.
•
New health module questions were added.
•
The electronic reference date calendar was featured throughout the instrument.
The goals of Phase 1 cognitive interviewing were to test new and revised question
wordings and response options. This testing included gauging participants' reactions to key terms
and concepts and detecting any issues with question comprehension. A test of the electronic
reference date calendar was also a Phase 1 goal.
4
1.2.3 Cognitive Interviewing Phase 2
The cognitive interviewing in Phase 2 took place over two rounds and was designed to be
a broader test of changes to the instrument. Phase 2, round 1 included all of the modules that
were tested in Phase 1. In addition, the following revisions were made for Phase 2:
•
New items were added to core demographics, including new military veteran questions.
•
Screeners and modules for tranquilizers and sedatives were added to the instrument and
tested.
•
The definition of binge drinking was changed for female respondents to include four or
more drinks on an occasion in the past 30 days.
•
Questions about needle use in the noncore special drugs module were reworded, and
questions about use of methamphetamine and prescription stimulants with a needle were
moved to the corresponding core modules.
•
New questions about medical use of marijuana were added to the noncore blunts module.
•
Education, health insurance, and income were moved to the ACASI portion of the
interview. In addition, the highest level response category for income was revised.
•
A new module introduced proxy respondents to the ACASI.
Phase 2, round 2 included the fully (nonabbreviated) redesigned instrument, with all core
modules, and incorporated audio recordings for the ACASI portion.
The goals of Phase 2 cognitive interviewing included retesting changes that were made to
question wording, routing, and response options following Phase 1. Testing the content that was
new to the cognitive interviewing protocols in Phase 2 was also a goal.
5
This page is intentionally blank.
6
2. Usability Testing
2.1
Data Collection and Participant Profiles
2.1.1
Description of Recruitment Methods
Participants were recruited for usability testing via a recruitment notice that was posted
on Craigslist. Persons who called RTI International in response to the recruitment notice were
administered a series of screening questions. The screening questions included items about the
caller's age and location to establish eligibility for the study; to be eligible, a person needed to be
aged 18 or older and located within 50 miles of Research Triangle Park, NC. The screening
questions also collected information on other demographic characteristics (e.g., gender,
education), computer use, and any use of prescription pain relievers and stimulants in the past 12
months. Questions about use of pain relievers and stimulants were included to gauge participants'
familiarity with the general types of drugs that were included in the usability testing. However,
the usability testing did not focus on participants' actual behavior with respect to prescription
drug use.
2.1.2
Respondent Characteristics across Usability Testing
Table 1 provides information on characteristics of the eight usability participants. The
number of participants was evenly divided between males and females. Participants' ages ranged
from 28 to 60. Among the four participants in the 26 to 34 age category, there were two
participants aged 30 years, one participant who was aged 28, and another aged 32. Among the
three participants aged 50 or older, ages ranged from 53 to 60. There was only one participant in
the 35 to 49 age group. Six participants reported some education beyond high school, while two
participants did not.
As might be expected, because participants were recruited from Craigslist, all reported
using a computer for personal use every day. However, only half reported that most of what they
do at work is done on a computer. Two participants reported using a computer at work less than
daily or never; the screening interviewer was instructed to choose "never" for computer use at
work if participants did not use a computer at work or if they did not have a job.
Five participants reported using prescription pain relievers for any reason in the past 12
months, and one reported using prescription stimulants in that period. None reported use of both
classes of prescription drugs in the past 12 months. Although the screening question included
items about use of prescription pain relievers or stimulants in the past 12 months, use of these
prescription drugs was not an eligibility requirement for the study. Consequently, two
participants reported not using either of these classes of prescription drugs in the past 12 months.
7
Table 1.
Selected Characteristics of Final Prescription Drug Redesign Usability Participants
Participant Characteristics
Gender
Male
Female
Total
Age Group
18 to 25
26 to 34
35 to 49
50 or Older
Total
Highest Grade Completed
High School or Less
Beyond High School
Total
Computer Use at Home/For Personal Use
Every Day
Computer Use at Work
Never
Sometimes but Not Every Day
Every Day but Not All Day
Most Work Is on a Computer
Total
2.1.3
Number
4
4
8
0
4
1
3
8
2
6
8
8
1
1
2
4
8
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures that were approved by RTI's Institutional Review Board. Parental permission was not
an issue for usability testing because all participants were adults.
Participants were then presented with a booklet containing a series of tasks. Participants
were shown the task instructions but not the follow-up probes, which were read aloud by the
interviewer. The follow-up questions provided breaks between sets of usability tasks. In the first
two tasks, all participants were shown unlabeled pictures of the pain relievers or stimulants that
were included in the interview. The aim of these tasks was to test participant identification of
prescription drugs solely from the drug images.
In the remaining tasks, participants were presented with hypothetical scenarios that were
read aloud by the interviewer. Participants were instructed to answer the questions on the laptop
computer as though the behavior described in a given scenario was true for them. Some scenarios
tested participants' identification of drugs by name, and others tested participants' ability to
identify drugs based on pictures in the booklet and the corresponding on-screen images. The
interviewer asked participants to read the questions aloud according to how they appeared on the
computer screen and then asked them to indicate how they thought the question should be
answered based on the scenario they were given. To obtain preliminary information about the
cognitive processing of the question, the interviewer also asked why participants thought a
question should be answered a certain way based on the scenario.
8
Although each interview was allowed 90 minutes, interviews generally only required
about 1 hour to complete, including the introduction, informed consent, usability test questions
and tasks, and incentive disbursement. Participants were given a $60 Visa gift card for
completing the interview.
2.2
Key Findings
2.2.1
Logistical Issues with Answering the "Yes/No" or "Enter All That Apply"
Questions
No participants appeared to have difficulty with answering the yes/no questions.
However, some participants had difficulty with the "enter all that apply" questions when the
numbering of response categories continued from the previous question, instead of resuming the
numbering of the first category with 1.
2.2.2
Ability to Use the Drug Images
Some scenarios in the test booklets presented situations in which a drug was shown as a
picture rather than being mentioned by name. Participants were able to correctly answer the
relevant survey question by matching the on-screen drug image to the drug image in the test
booklet. One participant commented specifically about how well laid out the on-screen images
were. These results suggest that participants can successfully use the on-screen images as a recall
tool.
2.2.3
Screen Layout Preferences
Participants as a whole did not have a strong preference for whether the drug images
were placed above or below the question text. Those who expressed a preference were fairly
evenly divided between the two types of layouts.
2.2.4
Issues with the Question Text from a Usability Standpoint
From a usability standpoint, no major issues were identified with respect to whether it
was easier for participants to answer the two-criteria or three-criteria questions. In particular, no
one commented that the three-criteria layout appeared to be too "busy" on the computer screen.
One participant recommended a larger point size for the on-screen font, particularly to aid older
participants in reading the questions.
Some participants talked about the repetitive nature of the questions, but no one explicitly
mentioned that the three-criteria questions were overly repetitive. Rather, comments about the
repetitive nature of the questions appeared to be caused by participants being presented with the
same basic questions four different ways (i.e., with the drug images above or below the questions
and with the two-criteria or three-criteria wording).
2.3
Changes Implemented
Based on the results of the usability testing, either image layout was deemed acceptable
to participants. It was decided that the drug images would be displayed at the top rather than that
the bottom of the screen, and that the introductory question text would appear above the drug
9
images. In addition, response option values for "enter all that apply" questions would be reset to
1 on each relevant questionnaire screen. For example, if there are five drugs displayed per
screen, each will be numbered as 1 through 5, as opposed to being labeled 1 through 5 and then 6
through 10. This will address the issue with entering responses that participants identified with
the "enter all that apply" questions during usability testing.
10
3. Cognitive Interviewing: Phase 1
3.1
Data Collection and Participant Profiles
3.1.1
Description of Recruitment Methods
Phase 1 cognitive interviewing included three rounds of interviews. Interviews were
conducted in Research Triangle Park, NC; Chicago, IL; and Washington, DC. Advertisements
were placed on Craigslist, and flyers were distributed to outpatient substance abuse treatment
programs. In the first round of Phase 1 cognitive interviews, only adults were interviewed. In the
second and third rounds, adolescents were also interviewed. In all three rounds, the recruitment
advertisements targeted past year users of prescription pain relievers and prescription stimulants.
Licit or illicit use was not specified in the advertisements. There were two versions of the
recruitment advertisements for rounds 2 and 3. One version was for adults, and one version was
for adolescents aged 12 to 17.
Persons who responded to the recruitment advertisements were administered a screener
over the telephone to assess their eligibility for the study. The screener took approximately 5
minutes to administer. It included questions about how callers learned about the study (to
identify the source of recruitment), age, gender, place of residence, education, physical
limitations, prior research study participation, and use of prescription pain relievers or stimulants
(for any reason) in the past 12 months. Recruitment was balanced to include people who had
used prescription pain relievers or stimulants in the past 12 months as well as nonusers.
Procedures for obtaining permission from a parent or legal guardian to interview adolescents are
described in Section 3.1.3.
In addition, persons who have had considerable prior experience with survey research
may bias the data and conclusions. Therefore, persons were deemed ineligible if they had
participated in more than one prior research study in the past 12 months (regardless of who
conducted the study or the content) or in any prior RTI research study in this period.
3.1.2
Respondent Characteristics across Phase 1
Table 2 presents an overview of selected demographic characteristics of Phase 1
cognitive interview participants. Out of 40 respondents in Phase 1, 10 were adolescents. As
noted previously, these 10 adolescents were interviewed in rounds 2 and 3 only. Seven of the 10
adolescent respondents were interviewed in round 2. Recruitment was balanced across gender,
with 18 female respondents. In Phase 1, 13 of the respondents were recruited from substance
abuse treatment programs. These 13 substance abuse treatment patients were interviewed in
rounds 1 and 2; recruitment in round 3 was limited to persons from the general population.
Thirty respondents had used prescription pain relievers in the past 12 months and 18 had used
prescription stimulants in the past 12 months. Although not shown in the table, of the 30 adults,
15 had a high school diploma, GED, or lower education. The other 15 had attended at least some
college.
11
Table 2.
Selected Demographic Characteristics of Phase 1 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)1
Female
Enrolled in Treatment Program (Currently or
within Past 12 Months)
12 Month User of Any Prescription Pain
Reliever
12 Month User of Any Prescription Stimulant
Chicago, IL
Washington, DC
Research Triangle Park, NC
1
Round 1
Round 2
Round 3
Total
16
0
7
8
5
7
5
5
9
3
6
0
30
10
18
13
14
7
9
30
7
5
5
6
4
2
3
7
7
3
4
5
18
10
12
18
Mean age not available.
3.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across all three rounds
of Phase 1 data collection. The interview was conducted using the computer-assisted Blaise
interview. The cognitive interviewer first administered introductory questions verifying the
respondent's age and gender. The interviewer introduced the laptop functions to the respondent
and described the feature of the electronic reference date calendar. The respondent was then
asked to conduct the next section of the interview using a self-administered design. In Phase 1,
audio files were not used in the first two rounds, so respondents read questions on the screens
themselves and then answered the questions. In round 3, respondents could listen to audio
recordings of the questions. Interviewers administered think-aloud probes and follow-up probes
to collect data on the cognitive properties of specific questions. The following modules were
tested in all rounds in Phase 1:
•
An abbreviated ensemble screener of drugs
•
Pain relievers screener
•
Stimulants screener
•
Pain relievers main module
•
Stimulants main module
•
Health module
The protocol largely remained the same for rounds 1 and 2. After round 2, slight changes
were made to the cognitive interview protocol. Changes included the following:
•
adding probes for questions that were identified as problematic in the first rounds of
interviewing,
12
•
dropping three categories describing sources for obtaining prescription drugs, and
•
editing selected question logic, question wording, and response options.
Probes were deleted for questions that no longer necessitated testing in the third round.
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
3.2
Key Findings
Summaries of the findings and recommendations from rounds 1, 2, and 3 of Phase 1
cognitive interviewing can be found in Appendices B, C, and D. These summaries also include
recommendations for edits to increase respondent comprehension of key items.
Key findings from Phase 1 interviews can be arranged into three broad categories:
(1) identification of prescription drugs, (2) performance of the misuse criteria, and (3) other
issues not related to prescription drugs. The subsections that follow cover each of these key
findings categories in turn. Changes that were made as a result of these findings are covered in
Section 3.3.
3.2.1
Identification of Prescription Drugs
In most cases, participants recognized the prescription drugs that they took by name.
However, some participants did find the drug images to be helpful, particularly if participants
were unfamiliar with certain prescription drugs.
In the screener, several screens showed brand name prescription drugs (e.g., Vicodin®)
and the generic equivalent (e.g., hydrocodone); "generic" was shown in parentheses next to the
generic drug name. There was some uncertainty about the "generic" term associated with some
drugs. Some participants reported that their doctors would write the brand-name drug on the
prescription, but the pharmacy would substitute the generic. Another issue concerns the number
of generic drug manufacturers and the variety of generic equivalents of specific drugs that may
be dispensed when prescriptions are filled. It would not be realistic to try to include exhaustive
examples of images for generic drugs. Addition of the word "generic" to the drug images in
round 3 (i.e., in addition to its appearance in the response category for the generic) appeared to
be helpful.
Another issue with identifying prescription drugs involved making respondents aware
that the questions applied to any form regardless of its appearance or whether they used the drug
in pill form or another form. Some respondents neglected to report use of liquid forms of the
prescription drug, especially if the images for these particular drugs showed only pills. Other
respondents mentioned using the drug, but in a pill form that was not shown on the screen. An
edit was made to the instrument to address this issue. This edit is described with PR01 in Section
3.3.
3.2.2
Performance of the Misuse Criteria
The wording "in any way a doctor did not direct you to use it" differentiated between
misuse of prescription drugs and use with a prescription and as directed. With few exceptions,
participants who used prescription drugs only with a prescription of their own and as prescribed
13
could determine that they did not use prescription drugs "in any way a doctor did not direct you
to use [them]." Similarly, persons who misused prescription drugs could determine that their use
constituted use "in any way a doctor did not direct you to use it," even if they used prescription
drugs in ways not explicitly listed as examples of misuse.
Later in the module, a question asked respondents which of the specific components of
misuse applied to them. This decomposition question puzzled some respondents. In round 1, the
criterion of overuse was presented as a single response (i.e., use in greater amounts, more often,
or longer than I was told to take it). Some round 1 respondents had trouble determining whether
they should choose this second category or the third one (use in some other way a doctor did not
direct me to use it). Specifically, some respondents thought that they needed to have engaged in
all three of the behaviors that were listed in the second category in order to choose it. Three
scenarios for misuse also arose during the interviews that were not listed as examples in the
instrument. These were (1) using with alcohol, (2) asking a doctor to prescribe a larger dosage of
prescription pills than was needed, and (3) buying prescriptions in other countries where
prescriptions are not needed for purchase.
3.2.3
Other Issues
Respondents had trouble understanding the use of canned air or air duster as an inhalant.
Although no respondents had used canned air as an inhalant, there were difficulties identifying
the product by that term. Problems were identified with respondents entering height and weight
in the new health module, but these were resolved with minor revisions to the screens. For
questions about the number of outpatient doctor visits, participants had difficulty determining
what did or did not constitute a doctor visit. Summaries of the findings and recommendations
from rounds 1, 2, and 3 of Phase 1 cognitive interviewing can be found in Appendices B, C, and
D. These summaries also include recommendations for edits to increase respondent
comprehension of key items.
3.3
Changes Implemented
As a result of all rounds of Phase 1 cognitive interviewing, several changes were made to
the instrument. These are listed below and are accompanied by the variable name.1
•
IN01ii—The word "air duster" was added to this question about canned air in the
inhalants section for increased comprehension.
•
PR01—The text in the introduction to each prescription drug screener was revised to
include "Please note that not all forms, dosages, shapes, or colors of these pain relievers
are shown on these screens, but you should include any form that you have used." This
sentence would convey that respondents should consider forms of the drug other than
pills (e.g., liquid forms), even if the image showed only pill forms.
•
PR07—Reference to the term "China Girl" was removed for fentanyl because none of the
respondents knew it, and fentanyl was the only prescription drug with a slang name.
1
Variables in the health module were renumbered during pretesting. Variable names as they appear in the
questionnaire field test (QFT) instrument are in parentheses.
14
•
PRHOSPYR1—This question about use of prescription drugs only in the hospital was
dropped because many respondents had difficulty answering it.
•
The calendar icon, which reminds respondents how to access the electronic reference date
calendar, was made more visible to encourage respondents to use it.
•
Audio was removed for the answer choices for height and weight because those are
included in the questions.
•
HLTHNEW03 (HLTH19)—This question, which asks the respondent how many times
he or she has visited a doctor in the past year, was revised to streamline the text.
•
HLTHNEW11 (HLTH29) & 12a (HLTH31)—Specs about specific health conditions
were changed to skip questions about whether respondents had specific conditions in the
past 12 months if the age at first diagnosis equals current age.
15
This page is intentionally blank.
16
4. Cognitive Interviewing: Phase 2
4.1
Data Collection and Participant Profiles
4.1.1
Description of Recruitment Methods
Phase 2 of cognitive interviewing included two rounds of interviews. For both rounds of
interviews, recruitment advertisements were placed on Craigslist in Research Triangle Park, NC;
Chicago, IL; and Washington, DC.
The Phase 2 cognitive interview research questions influenced the recruitment goals.
During Phase 2, methodologists sought to test the transition from asking questions about health
insurance and income using an interviewer-administered mode to using ACASI. Because
respondents can nominate a proxy to answer these questions on their behalf, new questions were
developed to provide this proxy with a tutorial on how to use the computer and the interviewing
program. Transition statements were also developed to inform the proxy about their role in the
interview. To test this process and these new screens, RTI recruited adolescents to serve as
respondents and their parents to serve as proxies. In some cases, the parents went on to complete
a full interview after serving as a proxy respondent for their child.
Three versions of the advertisement were posted: (1) a version targeting military
veterans, (2) a version targeting parent-teen pairs, and (3) a final version targeting adolescents
whose parent would participate in a short interview as a proxy for health insurance and income
questions.
Prospective participants who responded to the advertisement were administered a
screening questionnaire. The screener took, on average, approximately 5 minutes to administer.
A variety of demographic questions were asked of prospective participants, including questions
about age, gender, educational attainment, military status, and use of prescription drugs in the
previous 12 months. Answers to screener questions, in addition to the location at which they
would be interviewed and the study population to which they belonged (military veteran, parentteen pair, adolescent with proxy interview, and drug use history), were used to select participants
and reach recruitment targets to the extent that was logistically possible. Because reporting illicit
or prescription drug use was not an eligibility requirement in the first round, few first round
participants were heavy or expert drug users. Therefore, in the second round of interviews, RTI
targeted individuals who had used any of the four psychotherapeutic prescriptions or
methamphetamine and those who had received drug treatment in the past 12 months. For these
reasons, the final sample of participants represents a diverse set of demographic characteristics
and experiences.
4.1.2
Participant Characteristics
Table 3 presents a summary of selected demographic characteristics of Phase 2 cognitive
interview participants. For some demographics, such as age and gender, participants were well
distributed across interviewing rounds. For other characteristics of interest, the distribution
reflects changes in recruiting goals between rounds. Reasons for changes to recruitment goals are
discussed subsequently.
17
Participants ranged in age from 12 to 68. A total of 24 adults participated in Phase 2
interviews, with a relatively even distribution between rounds (round 1 = 10; round 2 = 14). The
mean age of adult participants was 42 years. Among the 16 adolescents, 10 participated in the
first round, and 6 participated in the second round of interviews. The mean age of adolescent
participants was 15 years. Approximately half of the sample was female (i.e., regardless of age).
Among adults who provided information on educational attainment, a majority had at least some
education beyond high school (not shown in Table 3).
Interviews in Phase 2, round 1 included a relatively large number of military veterans,
and it included a much small number of individuals with a history of heavy drug use. As
discussed in Section 4.2, veterans exhibited no problems answering the questions of interest. As
a result, the recruitment goals shifted in the second round of interviews to target individuals with
more drug experience.
Table 3.
Selected Demographic Characteristics of Phase 2 Cognitive Interview Participants
Participant Characteristic
Adult (18 or Older)1
Adolescent (12 to 17)2
Female
Veteran
Enrolled in Treatment Program (Currently or Within
Past 12 Months)
High Use Participant*
Chicago, IL
Washington, DC
Raleigh, NC
Round 1
10
10
9
5
2
0
4
3
13
Round 2
14
6
12
2
5
11*
5
5
10
Total
24
16
21
7
7
11
9
8
23
1
Mean age = 42.
Mean age = 15.
* The category "high use" was created for Phase 2, round 2 to refer to individuals who had experience with more
than one or two prescription drugs. This enabled recruitment of individuals who had a history with heavy drug
use.
2
4.1.3
Data Collection Procedures
Prior to each interview, informed consent was obtained from participants pursuant to
procedures approved by RTI's Institutional Review Board. Parental permission was collected in
addition to adolescent assent, when necessary.
The methods of cognitive interview administration were the same across the two rounds
of Phase 2 data collection. These procedures were also the same as those described in Section
3.1.3. Audio was available to participants in Phase 2. Interviewers administered think-aloud
probes and follow-up probes to collect data on the cognitive properties of specific questions.
In round 1, a selected set of modules was administered to participants. In round 2, the
entire proposed redesigned instrument was administered to participants. In both rounds, probes
were inserted following items of interest.
18
After each interview, adult respondents were given $40 Visa gift cards for participation.
Adolescent respondents were given $30 Visa gift cards.
4.2
Key Findings
Summaries of the findings and recommendations from rounds 1 and 2 of Phase 2
cognitive interviewing can be found in Appendices D and E. These summaries also include
recommendations for edits to increase respondent comprehension of key items. Key findings
from Phase 2 interviews can be arranged into three broad categories: (1) findings related to the
front- and back-end demographics sections, (2) findings related to prescription drug use and
misuse, and (3) findings related to substances other than prescription drugs. The following
subsections cover each of these key findings categories in turn. Changes that were made as a
result of these findings are discussed in Section 4.3.
4.2.1
Demographic Items
As noted in Section 4.2.1, round 1 interviews included a recruitment focus on military
veterans. In general, veterans were able to understand terms and phrases in the way in which they
were intended. Veterans understood the phrase "full time in the Reserves" (item V2a) despite the
fact that none of them reported ever being in the Reserves. Most participants were also able to
correctly define a combat zone when methodologists probed about the meaning of the term
(QD10c).
For a few participants, the new education question (QD11) was problematic. Interviewer
probing revealed that the order of the response categories was confusing. The question asked
about the highest grade that was completed, and response categories 1 to 11 included language to
that effect. Consequently, a participant originally understood an answer of "12" to mean
finishing the 12th grade. Revisions, as described in Section 4.3.2, were made to address this
item.
A majority of participants understood the questions in the back-end demographics
module. For example, participants correctly understood, despite being on summer break, that
they should report currently attending school (QD17). They also understood, when asked what
grade they were currently attending, to report the grade they are planning to enter after summer
vacation comes to a close. No problems were encountered when defining testing for drugs or
alcohol as part of the job hiring process (QD49), and what the term "random" means in the
context of random drug testing (QD50). One back-end demographic question that RTI noticed
could be moderately confusing to respondents was the item about sources of income
(INTRTINN). In the past this item was interviewer administered. The transition to an ACASI
administration resulted in awkward question wording and thus was revised to account for the
mode.
Most participants were able to answer a question about their height (HLTH04) with no
problems. However, it became clear in one interview that some respondents might not be able to
answer the question in the formats that were available (feet/inches or meters/centimeters). Some
respondents will be able to answer only in inches or only in centimeters because this is often how
height is measured in medical evaluations. Revisions to the available methods of entering height
were therefore implemented.
19
Income branching questions (QI20N, QI21A, and QI21B) were generally understood by
participants. One branch question (QI20N) asks respondents whether they make over or under an
income threshold. Contingent on their answer, respondents are taken to QI21A (for incomes
below $20,000) or QI21B (for incomes over $20,000). The numeric categories carrying over
from QI21A to QI21B was confusing to one participant. That is, there were "20" categories in
the first branch of questions that the participant was not routed to and had not seen. The question
she was routed to (QI21B) began categories at "21." No other question began a list of numeric
categories in such a way, which led to the initial confusion about how to answer the question.
The response category labels were revised to begin at 1.
The instrument tested in Phase 2, round 2 included two transitions from interviewer
administration to self-administration. In both rounds of interviewing, most respondents who were
serving as a proxy believed the proxy tutorial was simple to follow and understand. In a couple
of instances, however, it was apparent that the transition between modes could be improved. In
response, edits were made to these items, as explained in Section 4.3.1.
4.2.2
Prescription Drugs and Use/Misuse
A number of issues related to prescription drugs continued to be explored in the Phase 2
cognitive interviews. Highlighted findings include how participants understood concepts such as
generic, use, misuse, and "not all forms shown"; what motivated participants to misuse
prescriptions; and participants' ability to recall their age at first misuse of prescription drugs.
Relevant findings are discussed in this section, and corresponding changes that were
implemented are discussed in Section 4.3.
In most cases, the concepts that were explored were understood by participants as had
been intended. Most participants were able to identify generic drugs as the less expensive
alternative to a brand name drug. A small proportion of participants showed less clarity about the
chemical equivalency of generic and brand name drugs and that generics also require a
prescription.
In both rounds of interviews, participants shared a similar understanding of the concept of
misuse of prescription drugs. However, questions about misuse are administered to participants
who report the use of prescription drugs. Therefore, this finding must be measured against the
finding that there was less clarity among participants about the concept of use. The intended
understanding of "use" for prescription drugs starting at PR01 was use of any kind. Because the
first question about prescription drug use followed a long list of questions about illicit drug use,
some participants thought that the prescription drug screener questions were asking only about
the misuse of prescription drugs. Conversely, several participants thought the screener questions
were asking only about use of prescription drugs in a way that they were prescribed.
In addition to determining whether participants interpreted the concept of misuse in the
intended manner, the SAMHSA/RTI instrumentation team aimed to learn more about how
participants recalled their first misuse and their motivations for misuse. Questions that measured
motivations for misuse were added for the first time in Phase 2. No problems were apparent in
recalling the age at which misuse first occurred.
20
One cognitive issue was uncovered when the motivation items were tested. This finding
revealed an absence of a specific category for misusing prescription drugs to deal with emotions.
It also became apparent that logic and skip patterns in this series of questions needed editing. A
description of these revisions begins in Section 4.3.1.
Throughout the testing, instructions to respondents that indicated that not all forms of
pills were shown on the screen underwent further modification. Despite this iterative testing,
problems persisted. For instance, only morphine pills were shown on the screen as examples for
this drug. Respondents had difficulty determining whether to report use of morphine in liquid
form that they received by injection or intravenously. In response, a photo of a vial of liquid
morphine was added to this screen, along with the reminder about the forms that drugs can take.
4.2.3
Tobacco, Illicit, and Special Drugs
Nonprescription drug items were also of interest in Phase 2 cognitive interviews. Of
particular interest were new items about new cigarette naming conventions, substances that were
now asked about in the hallucinogen module, and whether participants understood questions
about medical marijuana.
Previous versions of the NSDUH questionnaire asked about the type of cigarettes current
smokers smoked most often (e.g., lights, ultra lights, mediums, or full-flavored cigarettes).
Recent legislation banned cigarette manufacturers from using terms on packaging that referred to
cigarette strength. As a result, many manufacturers migrated to using a color-coded system as an
alternative naming convention. In an attempt to continue to track trends in the types of cigarettes
most smoked, a new question (CGCOLOR) was developed.
In cognitive interviews, the SAMHSA/RTI instrumentation team aimed to determine how
well current smokers could recall the package color of the cigarettes they smoke most often. Half
of those who reported smoking in the past 30 days (current smokers) reported an issue with
answering this question. Participants reported that cigarette packaging includes multiple colors,
indicating that participants failed to understand the purpose of the question about package color.
The multiple colors on packaging left some participants unsure if they should pick the main
color, background color, or color of the text on the packaging. As a result, this question was
dropped from the specifications.
New hallucinogen questions were also of special interest in Phase 2 interviews.
Previously, these questions were housed within the special drugs module. The redesigned
questionnaire included them in the hallucinogens module. A primary objective of testing these
items in Phase 2 was to determine the extent to which participants would be familiar with these
drugs. Table 4 shows the number of participants who had heard of each of these drugs. As can be
seen in the table, participants were most familiar with ketamine.
21
Table 4.
Counts of Phase 2 Participants Who Had Heard of New Hallucinogens
Number of Participants Who Had
Heard of the Drug
14
8
9
Type of Hallucinogen
Ketamine (LS01i)
DMT, AMT, or Foxy (LS01j)
Salvia Divinorum (LS01k)
All of the participants who were routed to a new item about medical marijuana (MJMM)
demonstrated an understanding of the concept of medical marijuana that was consistent with
what was intended in the question.
4.3
Changes Implemented
4.3.1
Phase 2, Round 1
After Phase 2, round 1 interviews, RTI and SAMHSA met to discuss findings of the
interviews. Decisions were made with regard to what changes were needed and where further
attention should be placed during the next round of interviews. A bulleted list of the changes is
provided, along with a brief review of the reason for each change.
•
QI21B—It was learned in cognitive interviewing that this item had the potential to
confuse respondents since the categories carried over from another income branch
question that respondents who are administered QI21B would not see. Categories were
renumbered starting at 1 to be consistent with other items in the instrument.
•
INTRO1—A sentence about the respondent being able to read the questions in the
ACASI system was included in this item. It was deemed during the meeting between RTI
and SAMHSA that it was unnecessary. It was subsequently removed.
•
HLTH04—It was determined in cognitive interviews that some respondents might have a
difficult time answering a question about their height in the formats provided (feet and
inches or meters and centimeters). Two new options were added to the question. These
options allow respondents to indicate a preference to answer the question about height in
only inches or only centimeters.
•
QD26—During the meeting after Phase 2, round 1, SAMHSA and RTI agreed that the F2
options for more information could be improved by tailoring them to the question on
which they appear. The more information option for QD26 was tailored to indicate that a
respondent could ask for more information about the types of work to include when
answering about working in the past week.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, SVYMOTIV, and
STYMOTIV)—During the meeting after Phase 2, round 1, RTI and SAMHSA reviewed
the questions about motivations for misuse of prescription drugs. Resolutions were
reached on question wording and logic. Several decisions were made on the question
logic. These include the decision to ask relevant motivation questions (PRYMOTIV,
TRYMOTIV, SVYMOTIV, and STYMOTIV) of all respondents who reported
prescription drug misuse in the previous 12 months, the decision to ask about a main
reason when respondents reported multiple motivations for misuse (PRYMOT1,
22
TRYMOT1, SVYMOT1, and STYMOT1), and the decision not to allow respondents to
select the "other" reason as their main reason for misuse.
•
"Not all forms" language—During Phase 2, round 1 interviews, issues continued to be
found with the phrasing of the concept that "not all forms" of drugs are displayed in the
drug images. It was decided during the meeting after Phase 2, round 1 that revised
wording would be tested for Phase 2, round 2. The change can be characterized as one
from negative phrasing ("not all forms…are shown") to a more neutral phrasing ("some
forms…may look different").
•
PRIPROX and PRIPROXR—To reduce burden for respondents who previously
completed an interview or proxy interview, it was decided to recreate filter questions.
Two items were created. One allows the field interviewer (FI) to indicate the respondent
previously completed the tutorial, if they are aware of this fact (PRIPROX). If the FI is
unsure or cannot recall whether a respondent has previously completed the tutorial, a
question is asked of the respondent (PRIPROXR). A response indicating previous
completion of a tutorial routes the respondent to a transition screen where the FI informs
the respondent about the following section (TOPROXY).
4.3.2
Phase 2, Round 2
A number of changes to question wording, question logic, and response options resulted
from the Phase 2, round 2 interviews as well.
•
QD11—RTI learned in cognitive interviews that the order of categories could be
confusing to some respondents. Categories 12 and 13 were reversed so that category 12
now represents a high school diploma and 13 means 12th grade but no diploma.
•
CGCOLOR—About half of cognitive interview participants were unable to understand
the question was asking about the new color code system cigarette manufacturers have
implemented. It was decided that this question should not be included in the
specifications.
•
INTROPR (introduction to the pain relievers screener, which is the first prescription drug
module)—Through interviews it was determined that it was unclear to participants that
prescription drug questions were asking about any kind of use. To emphasize that point,
the word "any" was inserted in the first sentence of introductions to prescription drug
modules (INTROPR, INTROTR, INTROSV, and INTROST) so that it reads "These next
questions are about any use of prescription [drug type]."
•
PR06 (use of pain relievers containing morphine)—Several participants were unclear that
nonpill forms of morphine should be included in their response. The SAMHSA/RTI
instrumentation team added to this screen the reminder to include all forms of the pain
reliever that one has used. It was also decided that an additional image showing a vial of
morphine would be added to the screen. Including this image should prompt respondents
to include liquid forms of morphine they received by injection or intravenously.
•
Motivation for misuse items (PRYMOTIV, TRYMOTIV, and SVYMOTIV)—A
participant responded that he misused a prescription drug to help him deal with his
emotions. None of the existing response options were appropriate. A response category
"to help me with my feelings or emotions" was added.
23
•
Other reasons for misuse (PRMOTOT, TRMOTOT, SVMOTOT, STMOTOT)—It was
unclear to a participant that the instruction involved typing in the "other reason" for
misusing a prescription drug. The word "other" was inserted into the sentence so that it is
clear that respondents should type in the reason they were thinking of when they
indicated another reason in PRYMOTIV, TRYMOTIV, SVYMOTIV, or STYMOTIV.
•
Other reason is the main reason for misuse (PRYMOT1, TRYMOT1, SVYMOT1, and
STYMOT1)—It was previously decided not to allow respondents to indicate that the
other reason was the main reason for their misuse. However, some respondents chose to
list the other reason as their main reason. It was decided to add a category "the other
reason I reported" to account for this.
•
DRPR—This question asks respondents to think about the pain relievers that they
previously reported misusing. Respondents with heavy levels of prescription drug use
were either not clear which pills were pain relievers or which they reported using versus
misusing. The question was edited to include a list of misused prescription pills for
reference. Wording was also edited to consistently refer to misuse.
•
QI12AN and QI12BN—Some cognitive interview participants had difficulty
understanding the types of welfare or public assistance to which these questions referred.
A bulleted list of the types of assistance respondents should include in their answers was
added to this question.
•
INTRTINN—This question was previously administered by an interviewer. Wording in
that question referred to the sources of income "that we just talked about." This wording
was changed, in case it was awkward to use when the computer is reading the questions.
•
Proxy tutorial—During cognitive interviews, it became clear that it was burdensome to
have respondents answer all of the ACASI tutorial items in their interview if they
previously served as a proxy for another respondent. As a result, an abbreviated tutorial
was created for use in the Questionnaire Field Test. This tutorial will ensure that
respondents are presented with information they did not previously receive when they
served as a proxy. Table 5 shows the screens that are a part of the full and abbreviated
front-end and back-end tutorials.
•
Tutorial transition—During Phase 2, interviewers observed that there was no screen for
transitioning the computer from the interviewer to the respondent. Additional instructions
were needed for interviewers to turn the computer over to the respondent when a proxy
was not used in the interview. Such instructions were added to HINSINT, which was
renamed from NOPROX.
•
Calendar—Where appropriate, the SAMHSA/RTI instrumentation team added reminders
for the respondent to press a function key to access either the 30-day or the 12-month
reference date calendars. These reminders were added to most screens in the instrument
that asked the respondent to report behavior in the past month or year. Where reminders
were not included, the calendars themselves were added.
•
INTROINC—In the final paragraph, this item informs respondents that the upcoming
questions refer to the past calendar year rather than the past 12 months that were referred
to in previous questions. However, this would be confusing to proxy respondents because
they were not administered a battery of questions using the 12-month reference period.
24
Skip logic was added to the item to ensure that proxy respondents are not shown the
paragraph referring to a calendar year.
Table 5.
List of Screens Presented for the Full and Abbreviated Tutorials
Full Front-End Tutorial
IntroAcasi1
IntroAcasi3
IntroAcasi4
IntroAcasi2
HeadPhone
INTRO1
INTRO2
Abbreviated Front-End
Tutorial
Back-End Tutorial
IntrAcasi1b
IntrAcasi3b
IntrAcasi4b
IntroAcasi2
Nohphone
INTRO3
GOTDOG
EYECOLOR
ALLAPPLY
NUMBER
BACKUP
PLAYINFO
rangeerr
calendar
calendr2
calendr3
ANYQUES
EYECOLRB
ALLAPPLY
NUMBERB
BACKUPB
PLAYINFB
Rangeerr
Calendar
calendr2
calendr3
ANYQUES
ANYQUES2
25
This page is intentionally blank.
26
5. Discussion
5.1
Summary of Major Changes
The redesigned instrument was tested across a total of six rounds and variations
throughout the redesign pretesting procedures; these rounds included one round of usability
testing, three rounds of cognitive interviewing to test redesigned prescription drug modules, and
two rounds of instrument cognitive interviewing for the total redesigned instrument. Usability
testing was conducted in October 2009. Cognitive interviewing was conducted from January
2011 through February 2012. Findings from these pretests informed many questionnaire
changes. The major changes included
•
deciding upon the "top of screen" layout, use of the wording "in any way not directed by
a doctor" to describe misuse, and making respondents aware of the three misuse criteria
wording for the prescription drug questions;
•
finalizing new wordings to reference generic drugs and that not all forms of prescription
drugs may be pictured on a given screen;
•
removing questions about hospital use of prescription drugs;
•
improving the display of the calendar icon that reminds respondents that the electronic
calendar is available through a function key;
•
refining questions in the health module about height and weight for ease of selfadministration; and
•
adding a transition to the proxy tutorial.
A crosswalk of question text that is included in the main NSDUH questionnaire and the
pretesting questionnaires and ultimately is recommended for inclusion in the field test
questionnaire is included as Attachment F. This document serves as a reference to track question
development across the pretesting phases.
5.2
Usability Testing Advantages
Conducting a series of usability tests followed by increasingly in-depth cognitive
interviews allows for a detailed, iterative examination of potential questionnaire problems. The
usability testing identified early on that respondents did not have problems with the screen layout
and that people understood the purpose and function of the onscreen pictures. One advantage of
conducting a small round of usability testing was that it allowed the study team to eliminate
some errors that might have burdened cognitive interviewing and clouded the findings of those
interviews. A significant recurring theme of comprehending "generic" medications was first
observed in the usability testing. Cognitive interviewers, as a result, knew to probe the questions
about generic drugs for additional issues in subsequent rounds.
27
5.3
Recommendations for Future Pretests
Given the advantages of the usability testing, RTI recommends including it in future
rounds of pretesting. Usability testing is especially adept at blending testing of new equipment
and screen layouts and features. Now that the reference date calendars and drug images are part
of the CAI instrument, there may be usability implications of transitioning to new hardware.
With the addition of a new tablet device that has the potential to display show cards and other FI
materials, usability testing is recommended for any respondent and FI interaction with the tablet.
The recurring themes of the redesigned cognitive and usability testing suggest the need to
investigate the following issues for the field test:
•
The prescription drugs sections are longer. However, use of prescription drugs in the past
12 months was a criterion for recruiting cognitive interview participants. Consequently,
the administration time for the prescription drug questions and the overall administration
time may be shorter with a broader cross-section of respondents in the field test.
Nevertheless, it will be important to track the average interview times during the field test
to identify sections that may be still too long.
•
It will be important for field test interviewers to report notes and experiences on the
proxy transition into the second ACASI portion of the interview. This is a situation that is
likely to be dependent on many variables that are present within the field (rather than lab)
setting.
•
It will be important for the instrumentation team to track reports of usability concerns
with the electronic calendars.
•
Some questions that were administered smoothly in the cognitive testing lab may be
awkward with the wider variety of respondents in the field. Field interviewers will be
encouraged to note awkward wording and choppy or missing transitions.
Overall, iterative rounds of pretesting, combinations of usability testing and cognitive
interviewing, and revisions to instruments and protocols in response to findings have resulted in
an instrument that is well positioned to be tested in a field setting.
28
6. References
Office of Applied Studies. (2008, February 14). The NSDUH Report: Use of specific
hallucinogens: 2006. Rockville, MD: Substance Abuse and Mental Health Services
Administration.
29
Appendix B
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 1 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 1 Results
I.
Highlights
Respondent Characteristics
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain relievers.
Some reported noticing behaviors that concerned them, like needing to take more
medication because the drugs were having less effect for pain relief. They
stopped taking the medicine, lived with the pain for a while, and then would
resume taking the medicine as prescribed. Others in this group reported not liking
the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Identification of Prescription Drugs
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same question.
Some participants in particular reported that their doctors would write the brandname drug on the prescription but the pharmacy would substitute the generic.
Therefore, one issue for remaining rounds would be assisting persons in reporting
February 14, 2011
2
what they actually took, regardless of the name of the drug that was written on a
prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular color.
Additional options also may need to be considered for clarifying for respondents
that pictures may not show all possible examples of a particular pill, or forms
other than pills.
Performance of the Nonmedical Use Criteria
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for stimulants.
An issue for persons who obtained drugs outside of the United States without a
prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
February 14, 2011
3
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
MJ01 – Most participants were able to correctly identify what a blunt was.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
February 14, 2011
4
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
thinking that Vidodan and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
February 14, 2011
5
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
PR03 – One person used Darvocet in the past 12 months.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
February 14, 2011
6
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
PR10 – No one used any of these pain relievers.
February 14, 2011
7
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
ST02 – None of the participants had used any of these drugs.
ST03 - None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
February 14, 2011
8
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
February 14, 2011
9
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
February 14, 2011
10
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
February 14, 2011
11
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
February 14, 2011
12
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
February 14, 2011
13
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
February 14, 2011
14
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
February 14, 2011
15
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
February 14, 2011
16
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
February 14, 2011
17
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
February 14, 2011
18
Appendix C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 2 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 2 Results
I.
Highlights
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
May 9, 2011
2
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
May 9, 2011
3
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
May 9, 2011
4
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
II.
Ensemble Screener Findings
CG17REV – Of the 16 participants who responded to this question, 4 have heard
of snus and 3 correctly identified what it was. The fourth person thought that it
goes up your nose.
Round 2: No probe.
AL01 – Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions included being fruity, girly, ready-made and
flavored. Most participants compared them to wine coolers, and 2 participants
mentioned that 4 Loko is similar. These same two participants identified Sparks
as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
MJ01 – Most participants were able to correctly identify what a blunt was.
Round 2: No probe.
CC01 – Descriptions of crack included: smoked, more addictive, costs less,
crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Five participants reported use of cocaine or crack. There were no
inconsistencies in their narrative of use.
May 9, 2011
5
Round 2: No probe.
HE01 – Only 1 participant had heard of cheese, while most had heard of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
SD17a – Almost all participants had heard of methamphetamine. They had no
other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
HALINTRO – Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included: marijuana, cocaine, crack, heroin,
alcohol, crystal meth, and Ambien. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens
will become a problem here, we should ask them to specify which “other” hallucinogens they
have used. However, use of drugs other than hallucinogens would not be determined until the
May 9, 2011
6
data are processed. The NSDUH data editing procedures since 1999 continue to treat
respondents as hallucinogen users, even if they specify use of drugs other than hallucinogens.
INHINTRO: Most participants were able to correctly indentify examples of
inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
IN01a: Responses about a definition of ‘for kicks or to get high’ included: to
escape, for a euphoric feeling, excess use, stupor, for fun or a desired effect,
recreational, another frame of mind, to feel good, get outside yourself, heightened
senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
IN01e & h: Two participants reported using an inhalant, but correctly decided
against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Most participants had not heard about inhaling markers and pens to get
high. Many were incredulous that this actually happens, and none reported doing
it. However, 2 people had heard of this. (We may observe greater awareness of
this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
IN01ii: There was a good deal of confusion about canned air. Many participants
referred to this as ‘air duster’. A Google search of ‘air duster’ revealed a number
of products by that name. One participant thought that canned air had to do with
filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics
air duster” to make the meaning clearer.
May 9, 2011
7
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it
is not clear whether people who have inhaled these products will be able to identify the
substance by the examples given, and that people who have inhaled other aerosol products
(but not these) will be able to answer correctly that they have not used these inhalants. Google
results show that this type of product also is referred to as gas duster and the brand name Dust
Off. We should not continue to edit this question without feedback from users of aerosol
products about whether these are appropriate examples. Feedback from users of aerosol
products also would help to evaluate whether these users can correctly distinguish between use
of these and other aerosol products.
IN01l: Based on observations, SAMHSA requested that the question be revised so
that it no longer references substances that were “listed.” (This revision also
applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Some descriptions of over-the-counter
drugs included: lower dose, pick it off the shelves, don’t alter your state of mind,
not as strong, can’t get high off of them, bought without doctor's permission, and
off the shelf. Prescriptions were described as: containing narcotics, higher dose,
controlled substance, bought from the pharmacist, stronger, gives you a high,
more powerful, more dangerous, easier to abuse, more expensive, harder to get,
need a prescription from a doctor with a signature on it, narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
PR01: Participants reported using a multiple drugs on this list. Two participants
reported thinking that Lorcet and Percocet were the same thing. Another reported
May 9, 2011
8
thinking that Vicodin and hydrocodone were the same, along with Norco. He
reported taking Norco as opposed to hydrocodone, but reported the hydrocodone.
One participant was given the pain reliever in the hospital. Participants were able
to distinguish medical and nonmedical use in their narratives of use. For instance,
one participant reported taking the pain relievers as prescribed on the first three
days, but then took nine pills in one day. He acknowledged that he did not take
them as prescribed. Participants discussed using the pain relievers for recreation,
following the instructions for use, or stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs
and their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to
determine whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being
shown on the screen.
PR02: 11 participants reported using these drugs. One participant said that he
thought that Percocet and Lorcet were the same thing, but the names on the
bottles were different, so he must be wrong. All participants were able to answer
the question based just on the names, although 3 reported that the pictures were
May 9, 2011
9
helpful. One reported that his prescription is for Percocet, but his pharmacy gives
him generic oxycodone for insurance reasons, so he sometimes gets confused
about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
PR03 – One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
PR04 – One person had used Ultram and tramadol. Another used tramadol when
he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
PR05 – Seven people had used Tylenol with codeine. One participant called this
a Tylenol 3. Another reported that he also used Tylenol 4, which is the pill with
the 4 on it. The FDA's Center for Drug Evaluation and Research lists these as
"Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as
Tylenol with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
PR06 – 2 participants reported using morphine in the past 12 months. Others
reported use, but added that this was not in the past 12 months. Participants
distinguished between the liquid and pill forms of morphine, reporting they had it
in an IV.
May 9, 2011
10
Recommendation: Consider whether an image of a liquid morphine ampule should be shown
in the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
PR07 – Participants were familiar with a number of these drugs. Some reported
lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
PR08 – 2 participants reported use of Suboxone. One was using it to stop going
through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
PR09 – 2 participants reported use of Dilaudid. One said that he used this only in
the hospital and asked if that counted. Two participants had used methadone, at
least one of which was at a methadone clinic under supervision. One used
Demerol and said that the picture helped him identify it. Finally, one person
reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12
May 9, 2011
11
months (i.e., not just in pill form) should be reported in PR09. 2. Consider whether
instructions need to include reports of any medications that people took while in the hospital.
However, receiving dosages of pills in a hospital is a different experience than receiving a
prescription that will be filled for use at home. Participants may be unable to identify which
pills they took in a hospital, or to recall that they took pills at all. Do we anticipate that use of
pain relievers (or specific pain relievers) will be significantly underestimated if we do not
instruct respondents to report pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the
module. Perhaps it should read: “Remember, not all forms of these pain relievers may be
shown on the screens.
PR10 – No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
PR11 – Other pain relievers that participants reported that they used in the past 12
months included non-steroidal anti-inflammatory drugs (NSAIDS) such as
naprosyn or diclofenac (brand name Cataflam). Other drugs that were reported
included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
May 9, 2011
12
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
PRHOSPYR1 – Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in
where you used this (these) drug(s). This might draw more attention to the fact that the
question is not just confirming earlier reports of the drug. Another alternative would be to
ask respondents whether they used any of these drugs when they were in the hospital
(rather than “only” when they were in the hospital). If use of specific prescription drugs in
the hospital is reported, ask which drugs the respondents took when they were not in the
hospital. Questions to capture information about use of prescription drugs only in the
hospital may be dropped if respondents continue to have difficulty answering them.
IV.
Stimulant Screener
INTROST – Most participants had heard of No Doz and Dexatrim. 3 people
mentioned 5 Hour Energy as an example of a stimulant. One person mentioned
Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
ST01 – One participant replied that he was not sure which one to report, because
Dexedrine is the brand name, used by the company while Dextroamphetamine is
the chemical name. He was also familiar with the mix of amphetamine and
dextroamphetamine. One participant reported Vyvanse here as a
dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
May 9, 2011
13
ST02 – None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
ST03 - None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
ST04 – Two of the participants used Phentermine. One bought it in Mexico,
while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
ST05 – One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
ST06 – One participant reported getting Meridia in Mexico. (NOTE: This is no
longer available in the US).
Round 2: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Participants did not report using any stimulants in their lifetimes. One
participant later reported that her son was prescribed medicine for ADD. She
took one of his pills so that she could see how it affected him. However, she
neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
May 9, 2011
14
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
V.
Pain Reliever Module Findings
PRL01- In describing examples of using a prescription pain reliever in a way not
directed by a doctor, two participants mentioned using a prescription pain reliever
with other drugs or with alcohol. One specifically reported taking pain relievers
with a glass of wine because “they just work better” that way. Otherwise,
participants described ways of use not directed by a doctor that are included in the
definition, such as:
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining
pain relievers with other drugs or alcohol as a way of using pain relievers that is not directed
by a doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
May 9, 2011
15
PRY01 – All participants routed to this question about Vicodin understood that
we were asking this question to get at abuse of prescription pain relievers or
getting them off the street. Many participants used the word “abuse” when asked
what this question was asking. Examples of such behavior include not using it as
prescribed, taking it without a prescription, ‘over doing it,’ getting high, or selling
them. When asked to provide a narrative of the last time participants used
Vicodin, two participants described taking a prescribed medicine more than the
doctor said to because it was needed to make the pain better. A third participant
described having a prescription for dental pain, but using it more than prescribed
because of liking the feeling it caused. Another had a prescription but was abusing
the Vicodin and Norco highly at the time. One participant mentioned taking their
prescription with alcohol. Two participants mentioned using it without a
prescription. One of these used it only as a last resort because nothing else was
available to feed an addiction.
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them
to use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
PRY01a – Participants reported numerous mechanisms by which they
remembered their age at first use, including:
May 9, 2011
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
16
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
PRY02b – One participant chose the year because it was not in 2011, the past
couple of weeks. Another just knew the year was 2009.
PRY01d – One participant reported that the calendar helped with choosing the
month.
PRY02 – The one participant who was asked what this question about Lortab was
getting at reported, “the addictiveness of prescription drugs,” and “how easy it is
to abuse them.” Two participants were asked to describe the last time they used
Lortab. One reported using someone else’s prescription and the other reported
using it when heroin was unavailable.
PRY02a – The two participants who received this question distinctly remembered
events associated with the first time they used Lortab (finding grandmother’s pills
and hanging out with friends). One reported that it was someone else’s
prescription and the other reported snorting it.
PRY02c – One participant realized he had the year wrong when answering this
question. He mixed up how old he was last summer.
PRY02d – The participant remembered the month was July or August because of
how hot it was outside.
PRY03 – The one participant who was asked what this question about Lorcet was
getting at described snorting it as using it a way it was not prescribed, so as to get
an immediate high. Two participants answered this question as “no” because they
used this as prescribed, following directions as needed for pain.
PRY04 – Two participants were asked to describe what this question about
hydrocodone was getting at. One reported using it recreationally, not following
the doctor’s instructions, taking more than necessary, or snorting it. Another
described a new type of misuse: asking for a larger dose at time of prescription
with the knowledge that a large dose was not really necessary. This participant
still endorsed use of hydrocodone “in a way a doctor did not direct you to use it.”
When asked to describe the last time the pill was used, one participant reported
May 9, 2011
17
getting it for a sore throat in the ER and another reported getting the pills from an
acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as
examples of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
PRY04a – Participants did not have trouble reporting their age, although one
wanted to see a calendar for month of last use. When asked how they were
misusing the pills, one reported taking 9 in one day and another reported taking it
with alcohol. A third participant in answering this question expressed some
confusion about Vicodin and hydrocodone. This person reported thinking that
they were the same thing and appeared to be confused by the fact that we asked
about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth
date. It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
PRY05 - One participant described using OxyContin with a prescription. Her
mother kept it locked away and gave it to her at allotted times; otherwise, she
would have abused it. Another participant reported using it once when a friend
provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
PRY05a – One participant remembered the age at first use easily because it was in
the summer, “after my birthday.” Another had a hard time determining whether it
was at age 19 or age 20. It was after high school but before college. When using
OxyContin at that time, one participant reported using someone else’s
prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
May 9, 2011
18
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
PRY06 – Participants who were asked about what this question about Percocet is
getting at understood it to mean abuse. When asked about how they used it in a
way not directed by a doctor, one reported popping and snorting them, but with a
prescription. Another reported using another’s prescription. One participant used
it but did not report using it in any way not directed by a doctor. The Percocet
made this person sleepy and nauseated, and they did not care to finish the whole
prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
PRY06a – Participants remembered their ages by associating it with events in
their lives: sophomore year in college, a friend visiting from Florida, the diagnosis
of migraines. Another had no idea of the age at first use. Participants reported
first using Percocet in a way a doctor did not direct them to use it by using it
without a prescription, mixing it, popping it, and taking it “more than I should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
PRY09 - When asked what this question about oxycodone is getting at, one
participant who did not answer affirmatively said she was scared of it and asked,
“Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
PRY09a – Participants did not have trouble remembering their ages, but one
asked, “Do you really think that people remember the months and the years?”
PRY18 – When describing use of Tylenol with codeine “in any way a doctor did
not direct you to use it,” examples included: using someone else’s prescription
May 9, 2011
19
because their hydrocodone ran out and a friend gave them something to tide them
over (stave off withdrawal) and taking more than prescribed.
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
PRY18a – Participants did not have trouble remembering their age of first use.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY24 – One participant described using an IV drip of morphine in the hospital,
and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
PRY28a – One participant in particular had trouble remembering when he turned
specific ages for first use of fentanyl “in any way a doctor did not direct you to
use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – A participant described their use of Suboxone “in any way a doctor did
not direct you to use it” as a backup for when this person could not get into a
methadone clinic. This made it relatively easy for this participant to recall the age
and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – A participant who answered this question affirmatively recalled using
Demerol when he had run out of codeine and/or Vicodin. A friend with chronic
pain shared the Demerol with him to extend his prescription. He was able to
recall his age by the month and year and by his birth date.
May 9, 2011
20
PRY33 – A participant remembered using Dilaudid in the hospital as an injection
for back pain. He reported using this according to doctor instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
PRY34 – A participant routed to this question answered it as “no.” The person
had not used methadone “in any way a doctor did not direct you to use it” because
the person attends a methadone treatment program every day and receives the
methadone there as directed.
PRL02 – One participant reported using a pain reliever with alcohol as a type of
use not directed by a doctor in response to this question.
PRM02DKRE – The participant who received this question about their “best
guess” thought their answer was accurate.
PRM03 – Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said because it was yesterday, and another
knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
PRY41 – One participant understood this question but volunteered that the
wording “at least once” threw her off. She thought that the answers would say the
same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
May 9, 2011
21
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
PRY42B – Participants did not have trouble remembering how they got pain
relievers the last time. One additional way of getting a prescription pain reliever
reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the
current versions of these questions for SAMHSA to use in evaluating categories for
combination or deletion.
VI.
Stimulants Module Findings
STL01 – A participant reported not having a prescription for phentermine, but in
Mexico, they did not need one. This person pointed out that you can buy
May 9, 2011
22
prescription drugs in Mexican drug stores and airports. Technically in Mexico
they were not prescription drugs.
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in
any way a doctor did not direct you to use it,” and how to instruct respondents to count or not
count this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – When describing last use of Adderall, a participant reported that a friend
provided it in 2009. There was no prescription and they were mixing it with other
things.
STY01a – A participant reported remembering the exact night in 2009 hanging
out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY03a – A participant remembered their age at first use of Dexedrine very well
because it was associated with a recent move and new classmates/friends.
STY20 – The same participant who got phentermine in another country described
it here. Another participant reported using phentermine strictly to use weight, as
directed, and answered STY20 as “no.”
STY25 – The same user of phentermine mentioned getting it outside the U.S.
Another participant reported snorting as a third way of use.
Phase 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY26b – One participant had been prescribed stimulants for ADD while in
rehab. When the participant emerged from rehab, this person told the doctor
about having a prescription for stimulants and was able to get another prescription
easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
May 9, 2011
23
VII.
Health Module
HLTHNEW01 – 5 participants had difficulty entering their height into the
computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom
of the task bar. Participants also did not read the screen about entering inches. They were
still looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
HLTHNEW01a – Participants were not accustomed to seeing the question mark
that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – No problems entering weight. See HLTHNEW01 results for
recommendations.
Round 2: See HLTHNEW01 results for recommendations.
HLTH05 – One participant reported 8. Upon probing, he said that he was in the
hospital for 8 days and 7 nights and had missed the part of the question that
specified nights.
HLTHNEW03 – A few participants included dental appointments in their
estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
May 9, 2011
24
HLTHNEW03 – Most participants had a reasonable understanding about what a
health care professional was. One participant would not count a chiropractor, or a
person at the eye care center because they have not been to medical school. One
participant said that there were none at the drug treatment center, while another
reported that visits to the methadone clinic would count. Other participants
expressed doubt about including specialists and physical therapists. One
participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a
number of constructs, such as a reference period, thinking about your own health versus
someone else’s, the definition of a health care professional, and exclusions to the rule. The
respondent burden has the potential to be high, as does the measurement error.
HLTHNEW05 – There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows,
but there should be a linkage between the series of questions.
HLTHNEW06 – One participant reported that his dentist asked him about his
alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04
because the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06
focuses only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
May 9, 2011
25
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order of
the health conditions in this question.
HLTHNEW20 – One participant asked if we were referring to the past 12 months
or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – All three participants who reported high blood pressure were
confused by this question. One participant said that he has been told that it was a
little high, but was not sure that this was a chronic state. One participant said that
after the first time, the doctor knew that he had it and they talked about it, but he
didn’t tell him that he had it again. He said that he was on medicine that was
controlling it. The third participant said that they discuss his high blood pressure,
but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an
awkward stopping point, and does not seem to flow after the last few questions. Recommend
adding another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – After the stimulant main module, one participant reported that
many of the questions are redundant. However, no other participants made any
comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes:
Round 2: SAMHSA observers gave a reminder for RTI cognitive interviewers to be
sure to administer spontaneous probes in the cognitive interviewing in situations
where interviewers think it is necessary.
Informed Consent Statement:
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
May 9, 2011
26
likely to be small, the IRB may still require prospective respondents to be informed of
this.
May 9, 2011
27
Attachment C
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 1 Round 3 Results
NSDUH Prescription Drug Redesign Cognitive Interviewing Round 3 Results
I.
Highlights
General Themes (All Rounds)
Listed below is a summary of overall themes that emerged from all three rounds
of cognitive interviewing; these themes are not necessarily in the order of
question administration. Specific findings are discussed in further detail in the
remainder of the report.
Participants generally were able to recognize the prescription drugs by name,
by using the pictures, or by using both.
Although more questions were required, asking separately about any use of
specific prescription drugs in the past 12 months and nonmedical use of the
drugs that were used in that period simplified the cognitive task for
participants.
There was some uncertainty about the “generic” term associated with some
drugs. Addition of the word “generic” to the pill images in Round 3 appeared
to be helpful.
The wording “in any way a doctor did not direct you to use it” differentiated
between medical and nonmedical users of prescription drugs. With few
exceptions, participants who used prescription drugs only for medical reasons
could determine that they did not use prescription drugs “in any way a doctor
did not direct you to use (them).” Similarly, nonmedical users could determine
that their use constituted use “in any way a doctor did not direct you to use it,”
even if they used prescription drugs in ways not explicitly listed as examples
of nonmedical use.
Further testing is needed for questions about use of canned air as an inhalant
and injection of stimulants.
Revision of the questions about height and weight (HLTHNEW01 and
HLTHNEW02 series) following Round 1 helped to improve participant
understanding and the task of answering these questions.
For questions about the number of outpatient doctor visits (HLTHNEW03 and
HLTHNEWDK), participants had some difficulty in determining what did or
did not constitute a doctor visit, and in determining their number of visits.
August 17, 2011
2
Respondent Characteristics (Round 1)
Cognitive interviewers reported that the respondents were quite sophisticated in
their knowledge of the subject. They were well-versed in what it means to misuse
prescription drugs according to the examples and wordings given in the
instrument, even if they themselves did not report misuse of prescription drugs.
They also had a high awareness of which drugs they took (see below).
Three types of respondents emerged during these interviews. Treatment clients
were heavy drug users who were well aware of the potential for abuse and
addiction with using prescription pain relievers. Several were aware of people
crushing and then snorting pain relievers. Some also reported misuse of certain
pain relievers when heroin or other preferred drugs were not available.
Many general population users had chronic pain, and struggled between a desire
to relieve this pain and a desire not to become dependent on the pain
relievers. Some reported noticing behaviors that concerned them, like needing to
take more medication because the drugs were having less effect for pain
relief. They stopped taking the medicine, lived with the pain for a while, and then
would resume taking the medicine as prescribed. Others in this group reported
not liking the effects of the drugs, such as nausea or feeling too disoriented.
A third participant profile revealed that of the recreational user. For example,
Vicodin might be considered better for partying than for pain relief. Another
example involved use of Adderall and Vicodin on weekends in combination with
alcohol to enhance a person's partying experiences.
Although most participants were knowledgeable about pain relievers, heavy past
year stimulant users were missing from this group of participants. Therefore one
option to consider for Round 2 recruitment efforts is to place a greater focus on
recruitment of past year stimulant users, especially for adults, since Round 1 has
provided considerable information about adults' experiences with pain relievers.
Respondent Characteristics (Round 2)
The respondent profile among Round 2 participants was not as clear. Overall, 12
respondents participated in Round 2. Of these, 7 were adolescents and 5 were
adults. The participant pool did not seem to contain too many heavy drug users,
despite 5 participants having been recruited from drug treatment centers.
Respondent Characteristics (Round 3)
Overall, 12 respondents participated in Round 3, including 3 adolescents and 9
adults. Each of these participants was recruited from the general public. A few of
the participants were heavy drug users. Two reported use of multiple pain
relievers in the past 12 months, although one of these participants reported
August 17, 2011
3
misusing only one pain reliever. Another participant was a recovering heroin
addict who was intimately familiar with a wide variety of prescription drugs.
Identification of Prescription Drugs (Round 1)
Participants often recognized the drugs by name, although some did find the pill
images to be helpful. One issue that sometimes caused confusion was the
inclusion of brand-name drugs and the generic equivalents in the same
question. Some participants in particular reported that their doctors would write
the brand-name drug on the prescription but the pharmacy would substitute the
generic. Therefore, one issue for remaining rounds would be assisting persons in
reporting what they actually took, regardless of the name of the drug that was
written on a prescription.
Another issue with generic drugs concerns the number of manufacturers of
generics and the variety of generic equivalents of specific drugs that may be
dispensed when people fill prescriptions. It would not be realistic to try to include
exhaustive examples of pill images for generic drugs. Nevertheless, additional
feedback in Rounds 2 and 3 could be helpful for identifying if important examples
of certain generic drugs are being missed, such as pills of a particular
color. Additional options also may need to be considered for clarifying for
respondents that pictures may not show all possible examples of a particular pill,
or forms other than pills.
Identification of Prescription Drugs (Round 2)
Respondents were able to identify most of the drugs by either name or picture.
Echoing Round 1 findings, participants often recognized the drugs by name,
although some did find the pill images to be helpful. Despite the inclusion of the
word ‘generic’ after the drug name, respondents continued to be confused by the
distinction between brand name and generic drugs. Some respondents were
unaware that the brand name and the generic were two different drugs. Other
respondents thought they should report both because they were the same drug.
Some knew that they were different, but were unsure whether the pharmacist
substituted a generic while filling their prescription for the brand name. Round 3
cognitive interviews should continue to test respondents’ understanding of generic
drugs.
Respondents did not provide feedback on particular pill images of generics. This
was based primarily on confusion about the pill names.
Identification of Prescription Drugs (Round 3)
Respondents were able to identify most of the drugs either by name or picture.
Consistent with findings in earlier rounds, participants often recognized the drugs
by name, although some did find the pill images to be helpful. The Round 3
instrument added the word ‘generic’ to the pill image in order to help respondents
August 17, 2011
4
understand this term. Many respondents understood that generic forms of pills are
not the name brand, and are often cheaper. Some respondents thought that these
generics were “weaker.” Some respondents reported being confused about what
the specific generic pill was a generic of, but seemed to understand that pills had
generic forms and that pharmacies would occasionally substitute generics for
brand name drugs.
Performance of the Nonmedical Use Criteria (Round 1)
Based on the examples given and use of the term "in any way that a doctor did not
direct you to use it," participants generally were able to determine without much
difficulty whether they used these prescription drugs as prescribed or in a manner
that constituted nonmedical use. When participants gave information about how
they recalled information such as how old they were when they first used a
particular prescription drug nonmedically, they commonly cited examples of
getting medication from someone else's prescription, using a lot of the
medication, doubling up on dosages, or taking it at parties or when "hanging out"
with friends.
In the decomposition questions, however, some participants had trouble
determining whether they should choose the second category (use in greater
amounts, more often, or longer than I was told to take it) or the third one (use in
some other way a doctor did not direct me to use it). The specific issue is
whether participants see these two categories as distinct or overlapping. Because
the second criterion includes three ways in which persons can overuse
prescription medication, another issue is whether participants recognize that they
should endorse this category if some but not all of these characteristics apply.
Three scenarios for misuse arose during the interviews that were not listed as
examples in the instrument. These were (1) use with alcohol, (2) asking for a
larger dosage of prescription pills than was needed, and (3) buying prescriptions
in other countries where prescriptions are not needed. An important question for
the first two of these is whether respondents would recognize these behaviors as
use of a prescription drug "in any way a doctor did not direct you to use it" even if
these examples are not explicitly listed. In addition, use in combination with
alcohol may be a criterion for use of pain relievers, tranquilizers, and sedatives "in
any way a doctor did not direct you to use it" but not necessarily for
stimulants. An issue for persons who obtained drugs outside of the United States
without a prescription is whether this should be counted as nonmedical use if the
prescriptions were obtained legally, even if these drugs would require a
prescription in the United States. If this should not be counted as nonmedical use,
the related issue is whether and how to indicate this to respondents.
August 17, 2011
5
Performance of the Nonmedical Use Criteria (Round 2)
Participants continued to be able to determine without much difficulty whether
they used these prescription drugs as prescribed or in a manner that constituted
nonmedical use. Most respondents were able to clearly differentiate between
medical and nonmedical use. At least one respondent who was unsure about how
to classify his use resolved the issue after making use of the F2 reminder for the
ways of nonmedical use. In Round 3, use of the F2 reminder should be noted.
For the most part, new questions asking whether the respondent used certain
prescription drugs only while in the hospital performed well. In a few cases,
however, respondents made mistakes in reporting this. Suggestions for rewording
the question are included later in the summary.
Performance of the Nonmedical Use Criteria (Round 3)
Most respondents were able to differentiate between medical and nonmedical use.
Two respondents misreported their use as medical when it was not. One
respondent took a pill for longer than he had been told to take it, but did not
initially think of that as nonmedical use. Another participant did not think that
she had misused a pill because it was prescribed, because so many of the pills that
she takes are not prescribed. However, she took the pills more often than was
prescribed, and therefore reported misuse. One respondent reported using
“Tylenol 3” without a prescription, but he had really used over-the-counter (OTC)
Tylenol.
While decomposing her use, one participant reported that she had misused the
pills by taking the pills in greater amounts than prescribed and more often than
prescribed. Upon probing, it was discovered that she was describing the same
behavior.
Other Issues (Round 2)
The inclusion of adolescents in the sample for Round 2 has raised respondent
reports of being familiar with using magic markers as an inhalant. Respondents
seem to understand the question and have no problem reporting their behavior.
Despite the inclusion of the term “air duster,” many respondents were unfamiliar
with the term. Further discussion of this item is included later in the summary.
Changes to the blood pressure questions in the health module have eliminated
confusion with these items.
Changes to the height and weight items in the health module have diminished
confusion and usability issues. Further recommendations for these items are
included later in the summary.
August 17, 2011
6
Other Issues (Round 3)
Asking additional probes about familiarity with different names for air duster did
not reveal a term that was more familiar to participants.
The audio pauses in the middle of the questions to allow respondents to look at
the pictures of the pain relievers and stimulants. In some cases, the pause was too
long, while in other cases, the pause was too short.
Respondents did not have any problems with the F2 key, which is used to pull up
additional information about ways of misuse. On occasion, respondents used this
button to aid in their response process.
II.
Ensemble Screener Findings
CG17REV – Round 1: Of the 16 participants who responded to this question, 4
have heard of snus and 3 correctly identified what it was. The fourth person
thought that it goes up your nose.
Round 2: No probe.
Round 3: No probe.
AL01 – Round 1: Most, but not all, participants had heard of the alcopops that
were listed in the protocol. Descriptions included being fruity, girly, ready-made
and flavored. Most participants compared them to wine coolers, and 2
participants mentioned that 4 Loko is similar. These same two participants
identified Sparks as being caffeinated as well as alcoholic.
Round 2: Most, but not all, participants had heard of the alcopops that were listed
in the protocol. Descriptions include being fruity, sugary, carbonated, girlie, and
a mixture between soda and alcohol. Comparisons were made to wine coolers,
sweet wine, malt beverages, and 4 Loko.
Round 3: No probe.
MJ01 – Round 1: Most participants were able to correctly identify what a blunt
was.
Round 2: No probe.
Round 3: No probe.
CC01 – Round 1: Descriptions of crack included: smoked, more addictive, costs
less, crystalline substance smoked in a pipe, rock, baking soda is added. Cocaine:
August 17, 2011
7
snorted, powder, inject, pure. Two participants thought that there was no
difference between cocaine and crack.
CC01 – Round 1: Five participants reported use of cocaine or crack. There were
no inconsistencies in their narrative of use.
Round 2: No probe.
Round 3: No probe.
HE01 – Round 1: Only 1 participant had heard of cheese, while most had heard
of black tar.
Round 2: No respondents had heard of cheese. Most had heard of black tar.
Round 3: No probe.
SD17a – Round 1: Almost all participants had heard of methamphetamine. They
had no other names to suggest.
Round 2: Almost all participants had heard of methamphetamine. One
respondent reported that marijuana dipped in meth was called a dipper, but a
Google search revealed that this is marijuana dipped in PCP.
Round 3: No probe.
HALINTRO – Round 1: Most participants were able to correctly indentify
examples of hallucinogens. Incorrect examples included: marijuana, cocaine,
crack, heroin, alcohol, crystal meth, and Ambien. Participants included a number
of substances that can make one hallucinate, regardless of whether that substance
was a hallucinogen.
Round 2: Incorrect examples of hallucinogens included marijuana, Percocet,
crack, and “uppers and downers.” Other respondents were able to correctly
identify examples of hallucinogens. Participants included a number of substances
that can make one hallucinate, regardless of whether that substance was a
hallucinogen.
Round 3: Most participants were able to correctly indentify examples of
hallucinogens. Incorrect examples included marijuana, crack, paint,
markers, hairspray, depression medication, Percocet, and weed laced with
embalming fluid.
Recommendation: Include the list of hallucinogens in HALINTRO, as in the current CAI
instrument.
August 17, 2011
8
Action Item: Look at 2010 data for LS01a-LS01f and refusal follow-ups to determine the
numbers of respondents who got different LSFILL patterns.
Peggy also noted that inclusion of the three new hallucinogens will affect the Prior Substance
Use module – i.e., to add these three new substances and follow the same pattern as for LSD,
PCP, and Ecstasy. It is fine to include this in Round 2.2.
LS01: Round 1: No probe.
Round 2: R was not sure whether to report use of PCP if he crushed it up and
laced a blunt with it. He typically does not think of this as use of a blunt, but
ultimately reported it.
Round 3: No probe.
LS01h1: Round 1: No probe.
Round 2: R wanted to report use of OxyContin here. She asked whether
hallucinogens and narcotics were the same thing. After discussion, she decided to
not report her use here. Another R reported his use of marijuana here, despite the
fact that this drug had already been asked about.
Recommendation: If we think that respondents’ reports of lifetime use of non-hallucinogens will
become a problem here, we should ask them to specify which “other” hallucinogens they have
used. However, use of drugs other than hallucinogens would not be determined until the data
are processed. The NSDUH data editing procedures since 1999 continue to treat respondents as
hallucinogen users, even if they specify use of drugs other than hallucinogens.
Round 3: No probe.
INHINTRO: Round 1: Most participants were able to correctly indentify
examples of inhalants.
Round 2: Most participants were able to correctly identify examples of inhalants.
However, one respondent reported cocaine.
Round 3: Most participants were able to correctly identify examples of
inhalants. One reported computer spray duster.
IN01a: Round 1: Responses about a definition of ‘for kicks or to get high’
included: to escape, for a euphoric feeling, excess use, stupor, for fun or a desired
effect, recreational, another frame of mind, to feel good, get outside yourself,
heightened senses, for abuse, alter mood, because of boredom. Some participants
differentiated between "for kicks" and "to get high." For these participants, "For
August 17, 2011
9
kicks" implied more casual use focused on having fun or doing it out of boredom.
"To get high" implied less causal use focused on escape or to avoid emotions.
Round 2: Responses about a definition of ‘for kicks or to get high’ included: for
the fun of it, alter your state of mind, recreational, getting intoxicated, mood
changing, and get a buzz.
Round 3: Definitions included for fun, trying to get a buzz, doing it
recreationally, just to do it.
Recommendation: Respondents understand this wording and it should continue to be used in
the question.
IN01e & h: Round 1: Two participants reported using an inhalant, but correctly
decided against reporting this because it was not ‘for kicks or to get high.’ One
accidentally smelled some gas that had spilled at the gas station, and the other had
laughing gas at the dentist.
IN01h1: Round 1: Most participants had not heard about inhaling markers and
pens to get high. Many were incredulous that this actually happens, and none
reported doing it. However, 2 people had heard of this. (We may observe greater
awareness of this behavior among adolescents.)
Round 2: One respondent had used a Sharpie for kicks or to get high. He said
that he smelled markers in class and they gave him a headache. Other
respondents were familiar with the use of markers to get high. Three respondents
had not heard of it at all.
Round 3: No respondents reported inhaling markers or felt tip pens.
IN01ii: Round 1: There was a good deal of confusion about canned air. Many
participants referred to this as ‘air duster’. A Google search of ‘air duster’
revealed a number of products by that name. One participant thought that canned
air had to do with filling flat tires. Two confused this with whippits.
Recommendation: Revise this question. Perhaps add the words “air duster” or “electronics air
duster” to make the meaning clearer.
Round 2: Confusion about IN01ii persisted. Three respondents correctly
identified the product. Others reported the substance as Swiffer, furniture polish,
air freshener, or spray paint.
Recommendation: Despite a broad range of responses about what this question is referring to,
no one thus far has (correctly or incorrectly) reported using this type of inhalant. Therefore, it is
not clear whether people who have inhaled these products will be able to identify the substance
August 17, 2011
10
by the examples given, and that people who have inhaled other aerosol products (but not these)
will be able to answer correctly that they have not used these inhalants. Google results show
that this type of product also is referred to as gas duster and the brand name Dust Off. We
should not continue to edit this question without feedback from users of aerosol products about
whether these are appropriate examples. Feedback from users of aerosol products also would
help to evaluate whether these users can correctly distinguish between use of these and other
aerosol products.
Round 3: Four respondents had never heard of any alternate terms for
canned air, including computer duster, canned computer duster, canned air
duster, and Dust Off. Two respondents had heard of all of these names.
Three respondents had heard of computer duster. No respondents reported
inhaling this.
IN01l: Round 1: Based on observations, SAMHSA requested that the question be
revised so that it no longer references substances that were “listed.” (This revision
also applies to LS01h but does not apply to pain relievers and stimulants.)
III.
Pain Reliever Screener Findings
INTROPR: Round 1: Participants seemed to understand the difference between
over-the-counter drugs and prescription drugs. Some descriptions of over-thecounter drugs included: lower dose, pick it off the shelves, don’t alter your state
of mind, not as strong, can’t get high off of them, bought without doctor's
permission, and off the shelf. Prescriptions were described as: containing
narcotics, higher dose, controlled substance, bought from the pharmacist,
stronger, gives you a high, more powerful, more dangerous, easier to abuse, more
expensive, harder to get, need a prescription from a doctor with a signature on it,
narcotics.
Round 2: Participants seemed to understand the difference between over-thecounter drugs and prescription drugs. Descriptions of the two were similar to
those provided in Round 1.
SAMHSA also provided feedback about how respondents will know whether they
are in sections of the interview asking about pain relievers, tranquilizers,
stimulants, or sedatives.
Recommendation: Consider a way to assist respondents in knowing which prescription drug
section of the interview they are in, such as including a label at the top of the screen.
NOTE: Screen shots were sent to SAMHSA on May 31, 2011 that showed section headings for
pain relievers. SAMHSA made the decision on June 14 not to add headings to the CAI.
August 17, 2011
11
Round 3: No probe.
PR01: Round 1: Participants reported using a multiple drugs on this list. Two
participants reported thinking that Lorcet and Percocet were the same thing.
Another reported thinking that Vicodin and hydrocodone were the same, along
with Norco. He reported taking Norco as opposed to hydrocodone, but reported
the hydrocodone. One participant was given the pain reliever in the hospital.
Participants were able to distinguish medical and nonmedical use in their
narratives of use. For instance, one participant reported taking the pain relievers
as prescribed on the first three days, but then took nine pills in one day. He
acknowledged that he did not take them as prescribed. Participants discussed
using the pain relievers for recreation, following the instructions for use, or
stopping use early when pain eased.
Recommendations: Discuss the merits of further differentiating between brand name drugs and
their generic equivalents.
Feedback on pill images of hydrocodone products: One participant reported that
the pictures of hydrocodone looked the same, but the ones she took were yellow.
Recommendation: Investigate whether we get similar feedback in Rounds 2 and 3 to determine
whether to identify a suitable picture of a yellow hydrocodone pill for the field test.
Participants were all able to correctly fit their use in the 12 month reference
period.
Round 2: Respondents generally did not have problems answering this question.
However, one respondent was confused about the inclusion of the word “generic”
following hydrocodone. She asked if it had another name. Participants were all
able to correctly fit their use in the 12 month reference period.
Respondents reported two different understandings of the sentence about not
including all forms of the drug on the screen. Some respondents correctly
reported thinking that there could be different forms, such as a liquid or a capsule
or additional milligrams, which were not listed on the screen. Other respondents
understood that not all pain relievers were listed, including those that have
different names.
SAMHSA also noted that this wording seemed odd. The intent is for respondents
to think about other forms of the drugs that are not shown that they may have
used.
Recommendation: Consider revising the statement about not all forms of the drugs being shown
on the screen.
August 17, 2011
12
Round 3: Respondents were able to correctly identify which of these pills
they had taken in the past 12 months. They reported that the pictures and
drug names helped them with this identification.
Similar to the findings of Round 2, respondents reported two different
understandings of the sentence about not including all forms of the drug on
the screen. Three respondents correctly reported thinking that there could
be different forms, such as a liquid or a capsule or additional dosages, which
were not listed on the screen. Other respondents understood that not all pain
relievers were listed, including those that have different names.
Duplicating the labeling of generic pills on the screen appeared to increase
visibility of the term. Most respondents noticed this and most had a general
understanding of its meaning. However, respondents did not necessarily
understand that the generic on the screen was a generic for the other name
brand pills on the screen.
Respondents correctly understood the 12 month reference period.
Recommendation: Consider expanding the sentence about not showing all forms of the drugs
on the screen to provide examples of other forms, such as liquids or patches.
Add "such as different shapes, colors, or dosages." There is less worry about liquids because
respondents are likely to have gotten these only in the hospital.
PR02: Round 1: 11 participants reported using these drugs. One participant said
that he thought that Percocet and Lorcet were the same thing, but the names on
the bottles were different, so he must be wrong. All participants were able to
answer the question based just on the names, although 3 reported that the pictures
were helpful. One reported that his prescription is for Percocet, but his pharmacy
gives him generic oxycodone for insurance reasons, so he sometimes gets
confused about this. He reported taking the oxycodone.
Feedback on pill images of oxycodone products: One participant reported that the
oxycodone she took looked like pills in the picture, but hers were bigger. Hers
were the size of the Percodan. Another reported that she took the fat, white
Percocets, and there were no white Percocets in the picture.
Recommendation: Continue to investigate whether we get similar feedback in Rounds 2 and 3
about discrepancies between pills taken and pill images shown online for oxycodone products.
Round 2: Two respondents were confused about which drug they had taken. One
respondent was unsure whether he took Percocet or Percodan. The other
August 17, 2011
13
respondent was unsure whether she took OxyContin or the generic oxycodone.
She guessed that the pharmacist would have given her the generic.
Round 3: Respondents did not have any problem identifying which pills they
had used. Five respondents had used Percocet.
PR03 – Round 1: One person used Darvocet in the past 12 months.
Round 2: No respondents reported using these drugs.
Round 3: One person used Darvocet in the past 12 months.
PR04 – Round 1: One person had used Ultram and tramadol. Another used
tramadol when he ran out of a prescription for oxycodone.
Round 2: No respondents reported using these drugs.
Round 3: One respondent had used Ultram, but in a liquid form in an IV.
Two respondents used Tramadol.
PR05 – Round 1: Seven people had used Tylenol with codeine. One participant
called this a Tylenol 3. Another reported that he also used Tylenol 4, which is the
pill with the 4 on it. The FDA's Center for Drug Evaluation and Research lists
these as "Tylenol with Codeine No. 3" and "Tylenol with Codeine No. 4."
Recommendation: Consider changing the question to refer to Tylenol with Codeine as Tylenol
with codeine Number 3 and Tylenol with codeine Number 4.
Round 2: Respondents were able to recognize the revised term of Tylenol 3 or 4.
Four respondents used these drugs. One respondent chose both the Tylenol and
the codeine, even though he was referring to the same instance of use. He thought
that it was a trick and reported both because codeine was in both pills and he took
codeine. Another respondent asked if Tylenol with Codeine was regular Tylenol.
Round 3: Respondents continued to recognize the names of Tylenol 3 and 4
and to refer to the pill by these names. Six respondents reported using this.
However, one adolescent respondent was actually referring to the OTC
Tylenol and mistakenly believed that this was what was being asked.
Another adolescent respondent reported taking a red and white gel tab OTC
Tylenol. Based on the pictures that were shown on the screen, he decided not
to report this use.
PR06 – Round 1: 2 participants reported using morphine in the past 12 months.
Others reported use, but added that this was not in the past 12 months.
August 17, 2011
14
Participants distinguished between the liquid and pill forms of morphine,
reporting they had it in an IV.
Recommendation: Consider whether an image of a liquid morphine ampule should be shown in
the field test as an example of morphine.
Round 2: No respondents reported using these drugs.
Round 3: One participant reported using both morphine and MS Contin.
The morphine was used in IV form in the hospital.
PR07 – Round 1: Participants were familiar with a number of these drugs. Some
reported lifetime use, but none in the past 12 months.
Feedback on images of Fentanyl products: Participants provided positive
feedback about the images. Among the benefits cited were comments such as,
“The pictures were really helpful. I looked at the pictures first. There were two
whose names were unfamiliar, so I needed the pictures. If I hadn’t seen the
pictures, I might have confused them with another drug.” “It looked like this
(points at Actiq) but it was spelled like that (points at Fentanyl).” “The pictures
were very useful. I used “China Girl” once. And the picture helps clarify what it
looked like. Especially when comparing fentora and Fentanyl. If the pictures
weren’t there, I might have mistaken fentora for Fentanyl because the names are
similar.” “The packaging helps.” 2 participants also said that Fentanyl comes in
a lollipop too. (NOTE: A picture of the Fentanyl in lollipop form was used in
place of Actiq because of time and cost involved in acquiring a special image of
Actiq.)
Recommendation: Consider whether to include the lollipop form as an example of Fentanyl.
Round 2: No respondents reported using these drugs. Some respondents had
heard of China White, but did not know that it was Fentanyl. Most respondents
reported that they had never heard of China Girl. Respondents had not heard of
Sublimaze either.
Round 3: One respondent used the Fentanyl patch. Three respondents
reported hearing the name China White as another name for Fentanyl. One
of these respondents reported the alternate name as White China. No
respondents had heard of any of the other names that we probed about, nor
had they heard of Sublimaze.
Two participants, when asked, said that the lettering on the package of the
Duragesic was hard to read. Others reported that the pictures were of
limited use because they knew that had not used these drugs.
August 17, 2011
15
Recommendation: Get rid of "China Girl." This is the only prescription drug
with a slang name.
PR08 – Round 1: 2 participants reported use of Suboxone. One was using it to
stop going through withdrawal from pain reliever addiction.
Round 2: One respondent used Suboxone because his dealer did not have
OxyContin.
Round 3: One respondent used Suboxone and Subutex. He used one in
rehab for heroin addiction and another on his own to avoid a relapse into
heroin use.
PR09 – Round 1: 2 participants reported use of Dilaudid. One said that he used
this only in the hospital and asked if that counted. Two participants had used
methadone, at least one of which was at a methadone clinic under supervision.
One used Demerol and said that the picture helped him identify it. Finally, one
person reported that the pictures helped him decide that he had taken Opana ER as
opposed to Opana.
Recommendations: 1. Given the likely variations in the appearance of methadone that is
dispensed in liquid form, consider how to clarify that any use of methadone in the past 12 months
(i.e., not just in pill form) should be reported in PR09. 2. Consider whether instructions need to
include reports of any medications that people took while in the hospital. However, receiving
dosages of pills in a hospital is a different experience than receiving a prescription that will be
filled for use at home. Participants may be unable to identify which pills they took in a hospital,
or to recall that they took pills at all. Do we anticipate that use of pain relievers (or specific
pain relievers) will be significantly underestimated if we do not instruct respondents to report
pain relievers that they took in the hospital?
Round 2: One respondent ended up reporting use of Demerol, but was confused
about whether this should be reported. She was administered this as an anesthetic
in the hospital, but did not see a liquid form on the screen. When the statement
about not all forms being shown on the screen was pointed out, she said that she
did not know that statement applied to this screen, since it was not on the screen.
Recommendation: Edit the sentence to make it clear that it applies to all screens in the module.
Perhaps it should read: “Remember, not all forms of these pain relievers may be shown on the
screens.
Round 3: One respondent reported being very addicted to Dilaudid.
Another respondent said that the pictures of the drugs were an excellent
confirmation to his thinking that he took this drug.
August 17, 2011
16
PR10 – Round 1: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting. One
participant said that they would not have reported use of Percocet had they only
seen the pictures.
Round 2: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the combination
of the drug names and the pictures of the drugs aided them in their reporting.
Round 3: No one used any of these pain relievers.
In response to probe questions about the utility of the pillcard images and the
names of the pills, participants were generally in agreement that the
combination of the drug names and the pictures of the drugs aided them in
their reporting.
PR11 – Round 1: Other pain relievers that participants reported that they used in
the past 12 months included non-steroidal anti-inflammatory drugs (NSAIDS)
such as naprosyn or diclofenac (brand name Cataflam). Other drugs that were
reported included Lyrica (pregabalin) and Robaxin (methocarbamol). One person
mentioned liquid methadone as an example of something that people might report,
but this person had not actually taken it. One participant reported Gabatin, which
does not appear to be a drug. Perhaps she meant Gabapentin.
Round 2: One respondent reported taking a Klonopin, which is not a pain
reliever. Another mentioned that people would report sleeping pills, ADHD
drugs, and penicillin here.
Round 3: Respondents reported that people might report a number of other
pills at this question, including Valium (tranquilizer), Lexapro
(antidepressant), Narco (pain reliever), Paxil (antidepressant), Celexa
(antidepressant), and neurontin (anticonvulsant). [Note: neurontin and
Gabapentin are equivalent.]
PRHOSPYR1 – Round 2: Four respondents reported their answer to this question
incorrectly. Of these, 2 had indeed only taken a particular drug in the hospital but
answered no. One had taken the same drug both in the hospital and at home, but
reported yes. Finally, one respondent missed the phrase “when you were in the
August 17, 2011
17
hospital” and said, yes it is correct that she had taken all of these drugs.
Recommendation: This question should be edited to facilitate respondent understanding.
Although the reasons for confusion seem varied, respondents may be thinking that this
question is a summary question that recaps drugs that they have used, and ignore the
additional construct of use only in the hospital. Perhaps we should edit as follows: The
computer recorded that, in the past 12 months, you used [FILL]. We are interested in where
you used this (these) drug(s). This might draw more attention to the fact that the question is
not just confirming earlier reports of the drug. Another alternative would be to ask
respondents whether they used any of these drugs when they were in the hospital (rather than
“only” when they were in the hospital). If use of specific prescription drugs in the hospital is
reported, ask which drugs the respondents took when they were not in the hospital. Questions
to capture information about use of prescription drugs only in the hospital may be dropped if
respondents continue to have difficulty answering them.
NOTE: SAMHSA made the decision to drop the hospital questions for Round 3.
IV.
Stimulant Screener
INTROST – Round 1: Most participants had heard of No Doz and Dexatrim. 3
people mentioned 5 Hour Energy as an example of a stimulant. One person
mentioned Hydroxycut.
Round 2: Most respondents were familiar with at least one of the stimulants.
Round 3: Most respondents were familiar with at least one of the stimulants
that were listed. Many of the respondents had heard of all four.
Recommendation: Keep this list of over the counter stimulants.
ST01 – Round 1: One participant replied that he was not sure which one to
report, because Dexedrine is the brand name, used by the company while
Dextroamphetamine is the chemical name. He was also familiar with the mix of
amphetamine and dextroamphetamine. One participant reported Vyvanse here as
a dextroamphetamine, but changed her answer when she got to ST05. One
participant said that the pictures of Adderall helped her decide whether to choose
the extended release or the original.
Round 2: One respondent used Adderall while another used Adderall XR. The
XR user used the pictures to identify the drug. No one had heard about mixed
amphetamine-dextroamphetamine pills.
August 17, 2011
18
Round 3: One respondent used Adderall while another used Adderall XR.
These respondents had no problem answering the question.
ST02 – Round 1: None of the participants had used any of these drugs.
Round 2: Four respondents had used these drugs. One respondent was looking
for extended-release Ritalin and did not recognize that to be Ritalin LA.
Round 3: One respondent used Concerta in the past 12 months. He stated
that the coating on it made it difficult to dissolve under the tongue.
ST03 - Round 1: None of the participants had used any of these drugs.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST04 – Round 1: Two of the participants used Phentermine. One bought it in
Mexico, while the other was prescribed it at a diet clinic.
Round 2: None of the participants had used any of these drugs.
Round 3: None of the participants had used any of these drugs.
ST05 – Round 1: One participant used Vyvanse.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Phentermine, which then helped with
recalling the name.
Round 2: Three respondents had used these drugs and were able to identify them.
In response to probe questions about the utility of the pillcard images and the
names of the pills, a few participants reported that the pictures were more helpful
because they were not familiar with the names of the drugs, as a result of not
using them. One recognized the picture of Concerta, which then helped with
recalling the name.
Round 3: One participant used Vyvanse for ADHD.
Some respondents reported that drug names were helpful while answering
these questions. Others reported that the pictures were helpful because the
names were really long. There were no pictures that were not helpful.
August 17, 2011
19
ST06 – Round 1: One participant reported getting Meridia in Mexico. (NOTE:
This is no longer available in the US).
Round 2: No others were reported.
Round 3: No others were reported.
STHOSPYR1 : Round 2: Respondents were able to answer this question.
ST07 – Round 1: Participants did not report using any stimulants in their
lifetimes. One participant later reported that her son was prescribed medicine for
ADD. She took one of his pills so that she could see how it affected him.
However, she neglected to report that use in ST07.
Feedback on pill images – One participant requested that the pills should be
bigger so that participants could read what is written on them. Another reported
that the writing and numbers on the pills should be bigger because that is how
people on the street recognize what they are buying. A third participant pointed
out the potential for confusing Wellbutrin, Buspar, and Buprenorphine. She
recommended including the generic names with the brand names.
Recommendation: Investigate adding a zoom function to enlarge the pill images. Two issues to
consider are inclusion of keystroke commands (to avoid the need for participants to use the
touchpad) and whether simple enlargement of the existing images will improve respondents'
ability to identify the writing on the pills.
Round 2: One respondent recommended placing the numbers next to the pictures
of the drug names to make the question easier to answer.
Round 3: In response to the question about lifetime use of stimulants, one
respondents noted that we did not include Adipex (which is a form of
phentermine).
A couple of respondents noted that it would be helpful for the numbers on
the pill images to be bolder and more distinct.
V.
Pain Reliever Module Findings
PRL01- Round 1: In describing examples of using a prescription pain reliever in
a way not directed by a doctor, two participants mentioned using a prescription
pain reliever with other drugs or with alcohol. One specifically reported taking
pain relievers with a glass of wine because “they just work better” that way.
Otherwise, participants described ways of use not directed by a doctor that are
included in the definition, such as:
August 17, 2011
20
Taking someone else’s prescription
Using the prescription too much/more than a doctor prescribed
Getting it from a source other than a doctor
Snorting a pill.
One participant recommended including “norco” on the list because people may
not know that it is the brand name for hydrocodone.
Except for the issue of use in combination with alcohol or other drugs and the
recommendation to include “norco,” this feedback suggests that Round 1
participants who were routed to PRL01 correctly understood ways of using pain
relievers that were not directed by a doctor to encompass relevant components of
nonmedical use.
Recommendation: Determine if additional participants in Rounds 2 or 3 endorse combining pain
relievers with other drugs or alcohol as a way of using pain relievers that is not directed by a
doctor.
Round 2: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
Round 3: All respondents that were routed to this question understood its
meaning and were able to answer it without difficulty.
PRY01 – Round 1: All participants routed to this question about Vicodin
understood that we were asking this question to get at abuse of prescription pain
relievers or getting them off the street. Many participants used the word “abuse”
when asked what this question was asking. Examples of such behavior include
not using it as prescribed, taking it without a prescription, ‘over doing it,’ getting
high, or selling them. When asked to provide a narrative of the last time
participants used Vicodin, two participants described taking a prescribed medicine
more than the doctor said to because it was needed to make the pain better. A
third participant described having a prescription for dental pain, but using it more
than prescribed because of liking the feeling it caused. Another had a prescription
but was abusing the Vicodin and Norco highly at the time. One participant
mentioned taking their prescription with alcohol. Two participants mentioned
using it without a prescription. One of these used it only as a last resort because
nothing else was available to feed an addiction.
August 17, 2011
21
Two participants described in their narrative use that would not be described as
misuse. One reported using the pain reliever even though the pain had lessened to
the point where it might have been managed without taking that drug:
“They gave me a week’s worth. The pain got less and less. It got to the point that
I could take OTC medicine. I hate taking pain medicine. I was supposed to take
them as needed, which was once a day. I took all of them.”
Another took less than the prescribed dosage:
“I took it as prescribed but I lowered the dose, I was supposed to take it 3 times a
day but I only took it 1 time at night before I went to sleep because I didn’t like
the way it made me feel. Taking it with a muscle relaxer it gave me that sluggish
feeling. House could have burned down and I would have been with it.”
Recommendation: Determine if additional cognitive interview participants in Rounds 2 or 3
consider taking less than the prescribed dosage as use in any way a doctor did not direct them to
use a medication.
Round 2: One respondent reported being administered Vicodin in the hospital. Other
respondents reported getting the drug from a dealer or friends.
Round 3: In response to a probe about what the question was trying to measure,
three respondents said it was to measure abuse. Other mentions include, “using it
without doctor’s orders,” “not taking it any way to cause me to become dependent,”
and “not taking it for other things.” Other responses include:
“If you didn’t follow your prescription”
“If someone gives you their Percocet, if you take it more often or for fun”
“Are you using a prescription drug that wasn’t prescribed by your doctor or
any doctor?”
Respondents who took Vicodin in the past 12 months reported a combination of use
and misuse. Some respondents took pills in greater amounts than prescribed
because of the pain.
PRY01a – Round 1: Participants reported numerous mechanisms by which they
remembered their age at first use, including:
It was the first time using the drug this way
Referencing anchoring events (friendships, injury, senior year in high school,
getting a driver’s license, etc.)
August 17, 2011
22
Recency of the event (it was in the last year).
When participants described the first time they used Vicodin, they mentioned
taking too much, taking it without a prescription, doctor/pharmacy shopping, and
mixing it with other drugs.
Round 2: Respondents had similar retrieval strategies compared with Round 1.
Round 3: One respondent reported just remembering the age of first misuse.
PRY02b – Round 1: One participant chose the year because it was not in 2011,
the past couple of weeks. Another just knew the year was 2009.
PRY01d – Round 1: One participant reported that the calendar helped with
choosing the month.
PRY02 – Round 1: The one participant who was asked what this question about
Lortab was getting at reported, “the addictiveness of prescription drugs,” and
“how easy it is to abuse them.” Two participants were asked to describe the last
time they used Lortab. One reported using someone else’s prescription and the
other reported using it when heroin was unavailable.
PRY02a – Round 1: The two participants who received this question distinctly
remembered events associated with the first time they used Lortab (finding
grandmother’s pills and hanging out with friends). One reported that it was
someone else’s prescription and the other reported snorting it.
PRY02c – Round 1: One participant realized he had the year wrong when
answering this question. He mixed up how old he was last summer.
PRY02d – Round 1: The participant remembered the month was July or August
because of how hot it was outside.
PRY03 – Round 1: The one participant who was asked what this question about
Lorcet was getting at described snorting it as using it a way it was not prescribed,
so as to get an immediate high. Two participants answered this question as “no”
because they used this as prescribed, following directions as needed for pain.
PRY04 – Round 1: Two participants were asked to describe what this question
about hydrocodone was getting at. One reported using it recreationally, not
following the doctor’s instructions, taking more than necessary, or snorting it.
Another described a new type of misuse: asking for a larger dose at time of
prescription with the knowledge that a large dose was not really necessary. This
participant still endorsed use of hydrocodone “in a way a doctor did not direct you
August 17, 2011
23
to use it.” When asked to describe the last time the pill was used, one participant
reported getting it for a sore throat in the ER and another reported getting the pills
from an acquaintance/friend who said they would help with withdrawal.
Recommendation: Consider including asking for a larger dosage or number of pills as examples
of use “in a way a doctor did not direct you to use it.”
Round 2: Respondents reported various types of use and misuse. One respondent
used it in the hospital. Another was taking it with alcohol and taking as many as
he could get his hands on.
Round 3: Two respondents had used hydrocodone in the past 12 months.
One respondent received a pill from his mother. Another was abusing the
drug because of addiction.
PRY04a – Round 1: Participants did not have trouble reporting their age,
although one wanted to see a calendar for month of last use. When asked how
they were misusing the pills, one reported taking 9 in one day and another
reported taking it with alcohol. A third participant in answering this question
expressed some confusion about Vicodin and hydrocodone. This person reported
thinking that they were the same thing and appeared to be confused by the fact
that we asked about them separately.
Recommendation: For the next round of cognitive interviewing, insert a probe about whether
participants understand why they have the month response options they do. This instrument
includes a tailoring of response options of months of use based upon current age and birth date.
It would be helpful to see if this is confusing to respondents.
Round 2: Respondents reported understanding why only certain months were
shown in the response options.
Round 3: Respondents did not have any problems remembering their age at
first misuse.
PRY05 - Round 1: One participant described using OxyContin with a
prescription. Her mother kept it locked away and gave it to her at allotted times;
otherwise, she would have abused it. Another participant reported using it once
when a friend provided it.
Round 2: One respondent correctly reported thinking that the question was asking
about taking a prescription that was not prescribed for you or asking a friend for
some pills. The respondents also mentioned selling the drug.
Round 3: One respondent took this as directed.
August 17, 2011
24
PRY05a – Round 1: One participant remembered the age at first use easily
because it was in the summer, “after my birthday.” Another had a hard time
determining whether it was at age 19 or age 20. It was after high school but
before college. When using OxyContin at that time, one participant reported
using someone else’s prescription and mixing it with alcohol.
Round 2: One respondent reported that “in a way a doctor didn’t direct you to”
could mean: doctor didn’t prescribe it for you, doctor said to swallow not snort,
taking more pills than doctor prescribed, giving or sharing with friends. Another
reported taking someone else’s pills.
Round 3: Another participant associated use of this drug with a relationship
that he had. When he started using this, he was injecting it.
PRY06 – Round 1: Participants who were asked about what this question about
Percocet is getting at understood it to mean abuse. When asked about how they
used it in a way not directed by a doctor, one reported popping and snorting them,
but with a prescription. Another reported using another’s prescription. One
participant used it but did not report using it in any way not directed by a doctor.
The Percocet made this person sleepy and nauseated, and they did not care to
finish the whole prescription.
Round 2: When asked if he needed to see the ways of use, the respondent replied,
“I do not need to see the ways because it included that way I used it. I didn’t have
a prescription for it.”
Round 3: Five respondents reported using this prescription. One
respondent did not report misuse when he took one pill for pain that had
been prescribed to him for an earlier procedure. He did not report this
because the bottle said the pills were for pain, and they had been prescribed
to him.
PRY06a – Round 1: Participants remembered their ages by associating it with
events in their lives: sophomore year in college, a friend visiting from Florida, the
diagnosis of migraines. Another had no idea of the age at first use. Participants
reported first using Percocet in a way a doctor did not direct them to use it by
using it without a prescription, mixing it, popping it, and taking it “more than I
should.”
Round 2: An older respondent reported not having any idea of his age when he
first used Percocet nonmedically. He initially thought that the question was
asking about the past 12 months. After probing, he said that this would take him
back to his early 30s or late 20s. He ended up choosing Don’t Know.
August 17, 2011
25
Round 3: One participant had some trouble remembering her age at first
abuse. She estimated the age.
PRY09 - Round 1: When asked what this question about oxycodone is getting at,
one participant who did not answer affirmatively said she was scared of it and
asked, “Are people abusing the prescribed pills?” She showed the interviewer her
oxycodone pill and it was different from the pill cards. It said 5 & 12 on it and
did not have an M on it. One participant described use “in any way a doctor did
not direct you to use it” here as taking more than was prescribed. Another
reported getting oxycodone because Percocet and Vicodin were not available, so
she bought oxycodone from a drug dealer.
Round 2: One respondent interpreted the question as asking if she took more or
less than was prescribed.
Round 3: One respondent reported using this as directed. Two others took
this without a prescription.
One respondent asked for a calendar. She did not notice the calendar icon
on the top of the screen.
Recommendation: Make the icon for the calendar bolder or more visible.
PRY09a – Round 1: Participants did not have trouble remembering their ages,
but one asked, “Do you really think that people remember the months and the
years?”
Round 3: One participant estimated her age at first abuse.
PRY10: Round 3: One participant reported taking Darvocet as directed.
She stated that she did not abuse this.
PRY13: Round 3: One participant reported taking Ultram as directed. She
stated that she did not abuse this.
PRY17: Round 3: Three participants had used Tramadol in the past 12
months. One received a prescription for a broken wrist, one received a
prescription for pain, and the third did not have a prescription. Neither
respondent with prescriptions abused the drug.
PRY18 – Round 1: When describing use of Tylenol with codeine “in any way a
doctor did not direct you to use it,” examples included: using someone else’s
prescription because their hydrocodone ran out and a friend gave them something
to tide them over (stave off withdrawal) and taking more than prescribed.
August 17, 2011
26
Round 2: One person took one and had a bad reaction so did not take any more.
Another had a prescription and took them as prescribed. A third respondent
reported doubling up and taking pills sooner than advised.
Round 3: Six respondents had used Tylenol with codeine. One respondent
noted that he did not think that he had misused it because he had a
prescription, but after looking at the ways of misuse he was taking more than
were prescribed.
PRY18a – Round 1: Participants did not have trouble remembering their age of
first use.
Round 3: One participant said that she did not know her age at first misuse.
PRY19a - Round 2: One respondent benchmarked his use around his friend’s
death from overdose. He received the pills from his friend.
PRY22 – Round 3: One participant got this prescription from a doctor and
took the pills as directed.
PRY24 – Round 1: One participant described using an IV drip of morphine in the
hospital, and reported that he used this as prescribed.
Round 2: One respondent reported abusing this around the same time he started
using other drugs.
Recommendation: In future cognitive interview rounds, probe to determine if hospital use of a
prescription drug is included in reports of prescription drug use by participants.
Round 3: One respondent received a couple of morphine pills from a friend
and took them all at once.
PRY24a – Round 3: This respondent did not have any trouble remembering
age at first abuse.
PRY28 – Round 3: One respondent received the fentanyl patch at the
hospital.
PRY28a – Round 1: One participant in particular had trouble remembering when
he turned specific ages for first use of fentanyl “in any way a doctor did not direct
you to use it.” At this question, he reported forgetting his age because it’s 2011
(meaning early 2011 and he is still adjusting to the calendar change).
PRY29 – Round 1: A participant described their use of Suboxone “in any way a
doctor did not direct you to use it” as a backup for when this person could not get
August 17, 2011
27
into a methadone clinic. This made it relatively easy for this participant to recall
the age and the month and year.
Round 2: One participant bought this from a drug dealer in the park. He said that
he broke it up into little pieces in order to make it last longer. He acknowledged
that doctors do not tell you to do that.
Round 3: One participant used this in the methadone clinic and then
subsequently bought it from a drug dealer to help with heroin addiction.
PRY31: Round 2: The same respondent who reported use of Suboxone also
reported use of buprenorphine. He stated that this was the same as Suboxone. He
reported using both substances, but was thinking of the same instance.
PRY32 – Round 1: A participant who answered this question affirmatively
recalled using Demerol when he had run out of codeine and/or Vicodin. A friend
with chronic pain shared the Demerol with him to extend his prescription. He
was able to recall his age by the month and year and by his birth date.
Round 3: One participant used the Demerol as prescribed.
PRY33 – Round 1: A participant remembered using Dilaudid in the hospital as
an injection for back pain. He reported using this according to doctor
instructions.
Round 2: A participant initially reported using Dilaudid in the past 12 months.
Upon reaching this question, he said that he remembered that he was cold shaking
the drug but it was more than 12 month ago.
Round 3: This participant goes to the hospital to get Dilaudid, because she is
addicted to it. If she cannot get Dilaudid, she will seek out other drugs.
PRY33a – Round 3: The first time she used it, she had a prescription but she
used more than that.
PRY34 – Round 1: A participant routed to this question answered it as “no.” The
person had not used methadone “in any way a doctor did not direct you to use it”
because the person attends a methadone treatment program every day and
receives the methadone there as directed.
PRL02 – Round 1: One participant reported using a pain reliever with alcohol as
a type of use not directed by a doctor in response to this question.
PRM02DKRE – Round 1: The participant who received this question about their
“best guess” thought their answer was accurate.
August 17, 2011
28
PRM03 – Round 1: Two participants reported how they were able to remember
whether or not they were drinking alcohol. One said because it was yesterday,
and another knew they had used the alcohol to increase the effects of the opiates.
Round 2: Two participants reported how they were able to remember whether or
not they were drinking alcohol. One said that she would use alcohol and Vicodin
together with a friend. The other said that she never used these together.
Round 3: No respondents had taken the pills and drank at the same time.
One respondent was not sure how to report this, because she drank within a
week of taking the pills and was not sure how long alcohol stays in your
system. She ultimately decided to answer ‘no’.
PRY41 – Round 1: One participant understood this question but volunteered that
the wording “at least once” threw her off. She thought that the answers would say
the same thing without the text “at least once” and that it would be easier to
read/understand. Another participant reported not being able to endorse option #2
(used in greater amounts, more often or for longer than it was required) because
only part of what was listed applied to this person. The participant used in greater
amounts and more often, but not longer.
One participant reported wanting to select answer #3 because this person
“doubled up” on the prescription. That is, “doubling up” would fall into the
“some other way” category. When probed, this person agreed that “doubling up”
would also fall into category #2. Another participant considered choosing #3
along with #2 because the person used alcohol while taking the pills, which the
person considered a way of using in a way not directed by a doctor. Other
participants considered crushing and snorting as an example that applies to
category #3.
Recommendations: Remove the “at least once” wording. Determine in future interviews
whether participants think that all three of the aspects of misuse in option #2 need to apply in
order to endorse the option. Continue to monitor whether participants see options #2 and #3 as
distinct or overlapping and any difficulty they have in deciding which answer(s) to choose.
Round 2: Respondents all agreed that they should report a behavior, even if it
only happened once. One respondent who had only taken 1 pill reported that.
Respondents mentioned that they should include all relevant information.
Recommendation: Keep the question as is, because it is performing well without the ‘at least
once’ wording.
August 17, 2011
29
Round 2: Respondents did not have any problem with this question. One
respondent wanted to answer with “Some other way,” but decided that ways 1-4
described his use adequately. One respondent did report “Some other way.”
Round 3: One participant reported “some other way.” She included this
because she injected the drug instead of taking it by mouth. She also picked
the other four response options. After probing, she stated that she was
counting the same behavior of taking more drugs than were prescribed as
both number 2 and 3.
PRY42B – Round 1: Participants did not have trouble remembering how they got
pain relievers the last time. One additional way of getting a prescription pain
reliever reported was getting them out of the country. However, this reason is not
commonly endorsed in “OTHER, Specify” data. In 2009, for example, 5
respondents reported that the “other way” they obtained pain relievers they used
nonmedically in the past 30 days was by obtaining them outside of the U.S. Only
one respondent in 2009 reported this as the way that he/she obtained the pain
relievers the last time in the past 12 months.
Recommendation: Consider the addition of getting pills outside the U.S. to this list.
Round 2: Respondents did not have any trouble remembering how they got pain
relievers the last time. One respondent tried to report another way, which was
stealing them from a friend. He then realized that stealing was option #7. No
respondents mentioned getting the drugs from another country.
One respondent also was confused by the categories in the questions about how
respondents or friends/family members obtained prescription drugs.
Recommendation: Larry will compile frequencies of responses to these categories in the current
versions of these questions for SAMHSA to use in evaluating categories for combination or
deletion.
Round 3: One respondent said that she got it from more than one doctor
because she would get her prescriptions at the ER, and there are multiple
doctors that prescribe them.
VI.
Stimulants Module Findings
STL01 – Round 1: A participant reported not having a prescription for
phentermine, but in Mexico, they did not need one. This person pointed out that
you can buy prescription drugs in Mexican drug stores and airports. Technically
in Mexico they were not prescription drugs.
August 17, 2011
30
Recommendation: Continue to examine this issue in Round 2 to determine whether to provide
clarification on buying prescriptions in other countries, whether that should count as use “in any
way a doctor did not direct you to use it,” and how to instruct respondents to count or not count
this as use “in any way a doctor did not direct you to use” a prescription drug.
Round 2: Respondents did not report use in other countries. Two respondents
reported lifetime (but not past year) nonmedical use of stimulants.
STY01 – Round 1: When describing last use of Adderall, a participant reported
that a friend provided it in 2009. There was no prescription and they were mixing
it with other things.
Round 3: The participant answered “no.” The participant had a
prescription for ADHD and last took Adderall this morning, according to the
prescription.
STY01a – Round 1: A participant reported remembering the exact night in 2009
hanging out with a particular girl.
Round 2: The respondent had been using it a lot in the past year, and did not have
difficulty remembering that.
STY02 – Round 3: The respondent took Adderall XR for ADHD. He had a
prescription.
STY03a – Round 1: A participant remembered their age at first use of Dexedrine
very well because it was associated with a recent move and new
classmates/friends.
STY08 – Round 3: This respondent did not abuse Concerta.
STY20 – Round 1: The same participant who got phentermine in another country
described it here. Another participant reported using phentermine strictly to lose
weight, as directed, and answered STY20 as “no.”
STY25 – Round 1: The same user of phentermine mentioned getting it outside
the U.S. Another participant reported snorting as a third way of use.
Round 2: One respondent answered using it without and prescription and in
another way that a doctor did not direct him to use it. He used the drugs
recreationally. It was not clear that he was correct in answering “some other
way.”
STY25a and STY25b: No respondents in any of the three rounds reported
using stimulants with a needle in the past 12 months.
August 17, 2011
31
STY26b – Round 1: One participant had been prescribed stimulants for ADD
while in rehab. When the participant emerged from rehab, this person told the
doctor about having a prescription for stimulants and was able to get another
prescription easily.
One participant thought it would be easier for most participants to see the ways of
use on all the screens, even though as a user in recovery, she was well informed
about misuse.
VII.
Health Module
HLTHNEW01 – Round 1: 5 participants had difficulty entering their height into
the computer.
Recommendation: Revise this question. Perhaps move the sentence about entering in metric
sooner in the question. Add the phrase ‘then press Enter’ to the end of the instructions about
entering metric. One participant recommended adding the words ‘press Enter’ at the bottom of
the task bar. Participants also did not read the screen about entering inches. They were still
looking at the task bar, and entered the inches upon seeing the value label.
Round 2: Two respondents had difficulty here. They entered 1 to answer in feet
and inches but then did not hit enter.
Recommendation: Add the words, ‘and then press Enter’ to the question.
Round 3: No respondents had any difficulty entering their height or weight.
Recommendation: Questions are functioning properly. Keep as is.
HLTHNEW01a – Round 1: Participants were not accustomed to seeing the
question mark that results from pressing F3.
Recommendation: Revise the entry field to say Inch(es) so that participants can enter 0 or 1.
HLTHNEW02 – Round 1: No problems entering weight. See HLTHNEW01
results for recommendations.
Round 2: See HLTHNEW01 results for recommendations.
Action Item: Do not play the audio for the answer choices for height and weight because those
are included in the questions.
HLTH05 – Round 1: One participant reported 8. Upon probing, he said that he
was in the hospital for 8 days and 7 nights and had missed the part of the question
that specified nights.
August 17, 2011
32
HLTHNEW03 – Round 1: A few participants included dental appointments in
their estimates, despite the instruction to not do so. Most participants were pretty
confident about their answers. Those participants with infrequent appointments
or with many appointments were less sure, and estimated the response as opposed
to using recall and count strategies.
Round 2: Respondents seemed to correctly exclude dentist visits here. Most
participants were confident about their answers. Those participants with
infrequent appointments or with many appointments were less sure, and estimated
the response as opposed to using recall and count strategies. However, with the
removal of the response options, one respondent reported don’t know. He was
then able to answer the follow up question, where the response options were
provided.
HLTHNEW03 – Round 1: Most participants had a reasonable understanding
about what a health care professional was. One participant would not count a
chiropractor, or a person at the eye care center because they have not been to
medical school. One participant said that there were none at the drug treatment
center, while another reported that visits to the methadone clinic would count.
Other participants expressed doubt about including specialists and physical
therapists. One participant said only doctors counted as health care professionals.
Recommendation: Assess the level of accuracy that is needed in this question. Is each of the
response categories used in analysis, or can they be collapsed? This question includes a number
of constructs, such as a reference period, thinking about your own health versus someone else’s,
the definition of a health care professional, and exclusions to the rule. The respondent burden
has the potential to be high, as does the measurement error.
Round 3: Depending upon the extent of visiting doctor offices, respondents
struggled with accurately counting the number of visits. One respondent
initially included dental visits despite the instruction not to.
Action Item: Delete the explanatory text from HLTHNEW03. Consider moving the
instructions to appear before the question, and include a probe.
HLTHNEW05 – Round 1: There were no probes asked about this question.
Recommendation: Use logic from HLTHNEW04. It could be that the doctor already knows, but
there should be a linkage between the series of questions.
HLTHNEW06 – Round 1: One participant reported that his dentist asked him
about his alcohol use. He then asked if a dentist was included. Instructions in
HLTHNEW03 say to exclude the dentist, which may have caused confusion. One
August 17, 2011
33
participant did not include questions asked upon visits to the emergency room.
He stated that his doctor does not ask, because his doctor knows him. Another
participant reported that he did not know how to answer and had a number of
issues with the question. He filled out a form in the waiting room. The form said,
How many drinks do you have per week. He had difficulty mapping this question
to the responses of How much do you drink and How often do you drink. He did
not have a discussion with the doctor.
Recommendation: 1. Change the Round 2 specs to only ask this question of participants who
used alcohol in the past 30 days (ALC30USE=1). Participants who did not use alcohol in the
past 30 days will not receive this question. 2. Ask HLTHNEW06 before HLTHNEW04 because
the mention of a form is conditioning participants. 3. Include logic for responses to
HLTHNEW04b in logic for HLTHNEW06. 4. Make more apparent that HLTHNEW06 focuses
only on in-person communication.
Round 2: Respondents did not have problems with this item. One respondent
asked if being asked to cut down on alcohol was the same as being told to quit
drinking. Another stated that doctors who would ask this would not have a
history with their patients.
Round 3: Respondents did not have a problem with this question. Many
reported that doctors asked if they drink and then how often and how much.
HLTHNEW08, Round 2 – Peggy will follow up about any revision to the order
of the health conditions in this question.
HLTHNEW11 & 12a – Round 3: The respondent was first diagnosed at their
current age, but the follow up question asked whether they had cancer or
heart disease in the past 12 months.
Recommendation: Change the specs to skip questions about whether respondents had specific
conditions in the past 12 months if the age at first diagnosis equals their current age.
Action Item: Implement this change.
HLTHNEW20 – Round 1: One participant asked if we were referring to the past
12 months or ever.
Recommendation: Include a reference period in this question.
HLTHNEW20a – Round 1: All three participants who reported high blood
pressure were confused by this question. One participant said that he has been
told that it was a little high, but was not sure that this was a chronic state. One
participant said that after the first time, the doctor knew that he had it and they
August 17, 2011
34
talked about it, but he didn’t tell him that he had it again. He said that he was on
medicine that was controlling it. The third participant said that they discuss his
high blood pressure, but he was not sure how to answer the question.
Recommendation: Change the wording of this question. The BRFSS asks, “Are you currently
taking medicine for your high blood pressure?”
Round 2: There was no confusion about this question.
HLTHNEW21 - Round 1: There were no probes asked about this question.
Recommendation: Asking the STD question as the last question in the interview is an awkward
stopping point, and does not seem to flow after the last few questions. Recommend adding
another few questions in here for a better transition.
VIII.
Additional Issues
Respondent Burden – Round 1: After the stimulant main module, one participant
reported that many of the questions are redundant. However, no other participants
made any comments about finding the overall burden of the interview to be difficult.
Round 2: No comments about burden were made.
Probes: Round 2: SAMHSA observers gave a reminder for RTI cognitive
interviewers to be sure to administer spontaneous probes in the cognitive interviewing
in situations where interviewers think it is necessary.
Round 2: SAMHSA raised the issue about whether the informed consent statement
might be modified for, “There’s a small chance that someone else might hear your
answers.” Liz will ask RTI’s IRB about modifying this sentence. Although this risk is
likely to be small, the IRB may still require prospective respondents to be informed of
this.
Round 3: No additional issues.
August 17, 2011
35
Attachment D
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 1 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.1 Results
December 7, 2011
Respondent Characteristics
During the first round of Phase 2 interviewing, survey methodologists interviewed twenty
respondents. Of these twenty, ten were adolescents and ten were adults. Almost all respondents
in the first round of Phase 2 interviewing were recruited from the general population. A couple
of respondents were recruited from drug treatment centers in the Raleigh, North Carolina area.
In order to be recruited in the cognitive interview sample, respondents did not necessarily need to
report use of illicit or prescription drugs in the past 12 months during the telephone screener.
Therefore, most respondents were casual users or often non users of the drugs that were tested in
this round. Few were expert or heavy users.
Round 2.1 tested the transition from the questionnaire asking health insurance and income items
using interviewer administration to using a self-administered mode. Particularly, transitioning to
a proxy respondent who completes the health insurance and income items on behalf of the
respondent using self-administration warranted thorough testing. Interviewers and parents
provided valuable feedback about this process.
In general, proxy respondents were able to successfully answer questions about adolescent health
insurance and family income. The ACASI tutorial worked well, and proxy respondents reported
understanding the questions as well as why they were being asked to answer them. Individual
findings are listed below.
In addition, new questions measured details of military service and status. In order to thoroughly
test these new items, round 2.1 interviewed five veterans of the military. These veterans
provided feedback on the new items in the front end CAPI section of the interview.
Specific feedback on individual items is detailed below.
Veteran Questions
V2a – Respondents understood the use of the phrase “full time in the Reserves.” None of these
respondents reported ever being in the Reserves.
QD10c – Respondents were able to correctly define a combat zone. Among the definitions were
“being in combat,” “being in the fighting,” and Iraq and Kuwait. One respondent said that she
did not serve in a combat zone as women were not allowed to do so when she was in the
military. Another respondent incorrectly reported that he served in a combat zone. He had a
dangerous assignment of guarding prisons in the United States, so he reported yes. After
repeating the question, the respondent acknowledged that he should respond ‘no.’ Respondents
identified with the mention of hazard pay in this question.
2
Hallucinogens
LS01i – Eight respondents had heard of Ketamine. Most reported hearing about it on TV.
LS01j – Three respondents had heard of DMT and one had heard of Foxy. No respondents
reported use.
LS01k – One respondent thought that salvia divinorum was a substitute for marijuana. Another
thought that it was synthetic marijuana. A third person reported hearing of the substance.
Inhalants Screener
IN01ii – No respondents reported use of computer keyboard cleaner just to get high. Fewer than
half of the respondents knew about the spray. A few respondents had heard the term “spray
computer cleaner” or “compressed gas duster” before. One respondent suggested identifying the
spray by its brand names. Others did not have many recommendations for other words used to
describe the spray.
Methamphetamine Module
ME01 – Three respondents in the sample reported lifetime use of methamphetamine. One
respondent reported use of it twenty years ago and one respondent reported using it for two years
three years ago. No respondents reported 12-month use of methamphetamines. Those that
reported lifetime use did not have any problems with the module.
Prescription Drugs
PR01 – One respondent thought that we were only interested in misuse of prescription pain
relievers at this question. A couple of respondents were unaware that the next eight screens were
going to ask about other pain relievers and therefore wanted to report their use here.
Recommendation: Continue to measure whether asking about illicit use of substances before
pain reliever use is priming the respondent to only think about misuse.
Recommendation: Discuss whether we should convey to respondents that more than one screen
asks about pain relievers.
Generic Pills
Although most respondents understood that generic drugs are a cheaper version of name brand
drugs, they were not always clear on what the name brand equivalents of the generic drugs
pictured were. Two respondents reported that generic drugs are over the counter.
3
Not All Forms
The statement that not all forms of a particular drug may be shown on a screen continues to
cause problems for respondents. Many respondents reported thinking that it was referring to
pain relievers with a different name than the one shown on the screen. Others correctly thought
that we were acknowledging that certain pills might take various forms, not all of which were
pictured. A couple of respondents said that they did not understand the statement. Despite the
statement about not all forms, one respondent did not report intravenous use of morphine
because only the pills were shown on the screen.
Recommendation
Evaluate whether new probes should be used to test these questions in Round 2.2 or if the
wording should be further revised.
Pill images
Respondents that had used the drugs report that the pill images aid in their recall. Those that
report not using the pill images often report not using any pills.
Tranquilizers and Sedatives
No new issues were detected with the inclusion of the tranquilizer and sedatives module.
TR06 – When asked whether there were any other tranquilizers that respondents would report, a
few were mentioned. Paxil, Valium, promethazine (phenergen), and trazedone were reported by
respondents.
INTROSV – Respondents had for the most part heard of Benadryl, while fewer respondents had
heard of the other sedatives. Suggestions for other OTC sedatives included Adderall, ibuprofen,
motion sickness pills and melatonin.
Definitions of misuse
Respondents were able to distinguish between use and misuse of prescription drugs and reported
their use correctly.
Special Drugs
SD01 – Most respondents who named examples mentioned “Nyquil.” Other examples were
Sudafed, Tylenol, aspirin, Motrin, Benadryl, and Claritin. Many mentioned “cough syrup.” One
R he’d heard of ‘purple stuff.’ He wasn’t specific and probing lead to him discussing some drink
that a professional football player had been caught with that caused a suspension or fine. He said
he thought it was codeine or Dimetapp or Sudafed like substance. (Note: he is probably referring
to prescription strength cough syrup here.) Only 2 respondents had used OTC medicines. One
4
had taken Benadryl and another had taken Nyquil while he was in bootcamp because he was not
able to drink alcohol at the time.
SD21 – One respondent suggested that “the hospital” be added to this item.
Health
HLTH04 – One respondent only knew he was 54 inches tall, and he did not know how to convert
it to feet and inches.
HLTH19 – Four respondents expressed not knowing whether the dentist should count. None
counted the visits initially, but considered including them after a probe. Of the four, three
decided to count them and one did not.
Recommendation: Wording about not including dentist visits was recently deleted from the
specifications for this item. RTI and SAMHSA should discuss whether this is still appropriate.
HLTH22 – Four respondents reported doctors discussing their drinking with them.
Back End Demographics
QD17 – Fifteen respondents said that they would report being in school even if they were on
summer break. One respondent said he did not know and another said he would say “no” even if
he was just on summer vacation.
QD18 – Respondents were asked a hypothetical question about this item. Most were asked if
they would report being in 7th or 8th grade, if the question was administered during the summer
after they had finished 7th grade. Only one of 7 people probed said “7th grade”. Two participants
correctly reported about their current situation, saying that they would report the grade that they
were “going into.”
QD26 – One adolescent was confused as to whether this question asked about the different
places he had worked or the number of bosses he’d had. Another adolescent was not sure
whether she should count babysitting. She decided that she would NOT count it.
QD49 - One respondent had trouble with ‘DK’ for items QD47 & QD48. She had forgotten that
she needed to use the function keys to indicate this as her answer. She answered ‘DK’ to both
QD47/48. Another participant said that was a difficult question to answer because it depended
on the job one was applying to. Only certain job classifications are subject to testing, which at
her employer (a local government) were ones classified as ‘drug sensitive.’ These jobs are ones
like police officers, welfare workers, or anyone who drives a government vehicle. They do not
test for other job classifications, like the one she is in.
QD50 – The 4 Rs who were probed all mentioned that this question was asking about being
tested “at random.”
5
QD54 – One adolescent R mentioned she spent half the week in one home and half the week in
the other, then asked if she should count “the one I spend the most time in?” It was not hard for
her to choose the one she spent the most time in.
Proxy Tutorial
Two respondents had trouble with entering a “1” to answer the first question in the tutorial.
Recommendation: Change the first question in the tutorial to a “Press Enter to continue” screen.
Two respondents thought it was odd to be asked about the teenager’s income, since they did not
work.
In the income branching questions, one R tried to enter “1” for the first option, but the
appropriate number was “21.” R recognized her mistake and entered “21.”
Recommendation: Change the response options for this question to begin with 1, as opposed to
21.
One respondent complained that the tutorial “went into too much detail.” R thought it was too
slow and instead of letting his intuition take over it bogged him down. He mentioned reading the
script that was for the interviewer (on the demo screens where the interviewer points to the keys,
etc.) and said that “The script was for you but I was thinking of what it said rather than what it
was trying to teach me.”
Additional Issues
In order to reduce respondent burden, a filter question could be added to identify proxies who
have already been introduced to the interview program, whether during their own interview or
while serving as a proxy for another respondents.
Additional instructions and introductions are needed in the back end demographics section. In
cases where a proxy is not nominated, the screens do not clearly indicate which questions are
interviewer administered and which are self-administered.
6
Appendix E
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 2 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.2 Results
February 29, 2011
Respondent Characteristics
Twenty participants were interviewed during the second round of Phase 2 Questionnaire
Redesign Cognitive Interviewing. Summary information about the participants is presented in the
table below. This table shows that the sample was comprised largely of adults, females, and
those who had significant experience with drugs.
NSDUH Cognitive Interviewing Round 2.2
Participant Information Summary (n = 20)
n
Adults
14
Adult mean age = 39
Adolescents
6
Adolescent mean age = 14
Females
12
Enrolled in treatment program
5
(currently or within past 12 months)
High use participants
11
All participants were recruited from the general population using online ads that were placed in
Chicago, IL; Washington, D.C., and Research Triangle Park, NC. However, since most users
interviewed in the first round were casual users, the second round targeted heavy drug users and
individuals who had recently received treatment. Of the 14 adult interviews, 5 were currently
enrolled or had been enrolled in a treatment program in the past 12 months and 11 were
categorized as ‘high use’ participants. High use was defined as having used multiple prescription
drugs.
As in the previous round, Round 2.2 tested the second transition from interviewer to ACASI
administration for proxy responses to health insurance and income items. Feedback from parents
and the interviewers about the transitions was provided. All proxy respondents were able to
successfully transition and answer proxy questions. Findings about the process are provided
below.
Some questions were tested for the first time in Round 2.2. These questions asked respondents
about the color of the pack of cigarettes they smoked and about their motivations for misusing
prescription drugs. By recruiting ‘high use’ participants we were able to thoroughly test these
new items.
Specific feedback on individual items is provided below.
2
Veteran Questions
QD10c – Recruitment in Round 2.2 did not focus on Veterans as heavily compared to Round 2.1.
Therefore, only one participant was routed to this question. She correctly defined a combat zone.
She initially indicated that a combat zone could be a “zone even in Peacetime” but clarified that
it meant “places where you are taking on fire.”
Front End Demographics
QD11 – Participants were generally able to answer this question with no trouble. Two
participants initially failed to understand that the question asked about the highest grade finished,
but ultimately they realized the meaning of the question and changed their answers to the correct
one. One participant had trouble with the numeric categories. She answered “12” but then
changed her answer to “15.” When the interviewer probed to learn more about the change from
not finishing high school to taking some college credits the participant realized she should have
selected “13” to indicate earning a high school diploma. A scripted probe lead to one final
noteworthy comment about this question. One participant had taken credit courses during her
service in the military which she wanted to report. Since no option was available for having
taken military training/courses she realized her best option was to answer “13-regular high
school diploma.”
Recommendation: This question asks about the highest grade completed. Response categories
are generally the same as the number of the grade. However, the form of category 12 is markedly
different. If a respondent answers ‘12’ they are indicating that they did not receive a high school
diploma. Respondents who do not thoroughly review the text accompanying the response options
may be inclined to answer ‘12’ to indicate they have a high school diploma. Therefore, we
should consider changing the order of the response options so that this inclination will not invite
measurement error. An alternative would be to add a question about receiving a high school
diploma if the respondent answers 12.
Tobacco
CGCOLOR – Eight of the participants reported smoking cigarettes in the past 30 days. Of these,
four considered the question to be “self-explanatory” and found it easy to recall the color of the
pack. The other half of participants, however, noted complicating issues with the question. Two
found the question somewhat confusing because packs of cigarettes have more than one color on
the packaging. One of these participants noted that he wasn’t sure whether he should or could
select more than one color or if he should “just pick the main color.” A similar comment was
made by another participant who said this question made her think of the two different colors on
the packaging—the background color (black) and the color of the text (blue). One participant,
who previously indicated the question was self-explanatory, said that his brand of cigarettes
3
(Newport) only comes in green packaging. Upon probing he elaborated that the shade of the pack
indicates the strength of the cigarette—lighter colors indicate lighter and darker colors indicate
fuller flavors. Finally, one other participant was unable to answer the question because he had
been given a single cigarette from someone else and wasn’t sure what color the pack was.
Respondents did not necessarily understand that this question was asking about the strength of
the cigarette and its association with color.
Recommendation: Consider whether language could be added to clarify what we mean by
“color” in the question. We should continue to attempt to reach the tobacco consultant about
these questions.
CGMENTH1 – One participant volunteered that this question was difficult for her to answer.
She smokes Camel Crush cigarettes, which allow the smoker to choose between regular and
menthol flavors. The cigarettes are by default regular flavor. By squeezing a point on the
cigarette a smoker can activate a menthol flavoring. The participant sometimes smokes them
only as non-menthol, other times she smokes them only as menthol, and occasionally she
switches from non-menthol to menthol at some midpoint.
CG34 – One participant indicated this question was difficult for him to answer. He had only used
cigars to make blunts, but had never smoked tobacco cigars. He noted this prior to answering the
question. The participant indicated he would answer “yes” to this question because it does not
specify what the cigar had in it.
MJ01 – One participant indicated this question was difficult to answer because she was not sure
what was meant by “used.” She pointed out that the alcohol questions were very specific about
what counted as drinking (a sip or two does not count), but we were less specific about
marijuana. She had taken one toke of a marijuana cigarette in her lifetime and was not sure that
counted as “used.”
Hallucinogens
LS01i – Six participants had heard of Ketamine. There was variety in the places people had
heard about Ketamine, what participants thought the drug was intended for, and the effects the
drug has on a user. Several participants indicated they thought it was a prescription drug for
animals. One thought it was the “date rape drug.” One participant indicated she had heard about
it from kids in her neighborhood and another had heard about it on television. Only one
participant had tried ketamine. He used at a rave “back when he partied with some white boys.”
He said it is “like meth but it’s for people who don’t want to do meth” and that it “makes you
feel like [expletive] in the morning.”
LS01j – One participant had heard of all three, one had heard of AMT and Foxy, one had heard
of DMT only, and one had heard of Foxy only. In addition, one participant reported using DMT
with the same group with which he had used Ketamine. He said it had a similar effect as
4
marijuana. He decided to not use DMT anymore and to instead “just use weed” because he got
the same effect from it and marijuana was from a plant.
LS01k –Six participants had heard of salvia. One had heard of it on television, one from a movie,
and four did not specify where they had heard of it. Two of the participants said saliva is similar
to marijuana, while one said that when you smoke it you “lose your mind for 20 minutes.” Two
indicated it was legal and one said he thought you could buy it at smoke shops.
Inhalants Screener
IN01ii – Findings on this item were similar to those in Phase 2.1 interviews. None of the
participants reported use of computer keyboard cleaner just to get high. Fewer than half of the
participants knew about the spray and of these only two made comments which reveal they
understood it could be used to get high. Seven participants had heard the term “spray computer
cleaner” or “compressed gas duster” before. One participant said he had heard it called by the
brand name “Dust Off.” And, one participant thought that computer keyboard cleaner might be
called “alcohol” perhaps suggesting the participant thinks that is a main ingredient.
Methamphetamine Module
ME01 – Just one participant in this round of interviews reported lifetime use of
methamphetamine. It was during a time when a close relative of the participant was diagnosed
with cancer. Someone the participant knew had experience with methamphetamine and
suggested it could help with relaxation. The participant reported no issues with the module.
Generic Pills
Each of the participants characterized generic drugs as a less expensive version of a brand name
drug. Three participants made comments about the quality of generics compared to name brand
drugs, two of which questioned the effectiveness of generics compared to name brand drugs.
Not All Forms
There continue to be problems associated with the statement that not all forms of a drug may be
shown on a screen. Participants in this round clearly thought this statement referred to how pills
look, but did not seem to understand that forms also included other modes of drug
administration—i.e., injectable drugs. For example, many of the participants indicated that the
statement meant that generic alternatives might not be presented when the brand name drug was
pictured, that not all shapes and/or colors would be shown, and that the text imprinted on the pill
might look different than what is shown. None of the participants mentioned the possibility that
drugs could be something other than a pill, and as mentioned below two participants noted
trouble with answering PR06 because morphine is often administered through an IV.
5
Recommendation: Discuss modifying the language to clarify that “not all forms” means both
the color/shape and mode of administration. This language has been tested and modified over
many rounds of interviewing, so we should review previous revisions and findings when
exploring whether to revise it further. See also the recommendation following PR06.
Pain Relievers
PR01 – It was evident in this round of interviewing that there is confusion about what kind of use
these questions are about. Four participants thought we were interested in any kind of use (use or
misuse), one thought we were only interested in misuse, and seven thought the question was
asking only about prescribed use. Upon probing one participant paged back to INTROPR and
pointed out that it only talks about “use of prescription pain relievers.” To him this suggested he
should only think about use that is prescribed.
Recommendation: Consider revising the following sentence: “These next questions are about
the use of prescription pain relievers” to say “These next questions are about any use of
prescription pain relievers.” Bolding ‘any’ may reinforce that we are measuring both use and
misuse. This probe was added to the instrument to address concerns that respondents would think
that we were only measuring illicit use of prescriptions, because most of the previous substances
that were asked about are illicit.
PR06 – Two participants had difficulty understanding that non-pill forms counted in this
question, which asks about morphine.. One asked whether “injections counted” and the other had
had morphine through an IV drip and initially failed to report it.
Recommendation: Consider adding a reminder on this screen that not all forms of the drugs
may be shown on the screen, because morphine is very likely to be administered in liquid form.
PRY42B – One participant volunteered that she had a problem with only being allowed to select
one way for the last time she used a prescription pain reliever. She said that she got the drug in
multiple ways. By forcing her select one way the respondent said we were forcing her to provide
a socially desirable answer, because she would select the way that did not make her look like a
drug addict.
Recalling age of first misuse
Participants used landmark events to recall dates of first use. For example, one participant
recalled first use of Xanax by recalling the group of people he was “hanging around with” during
that period of his life and also recalled his approximate age when spending time with this group.
Three other participants recalled first use because it was at a time of stress associated with life
changes, or medical events they or their family members experienced.
6
Definitions of misuse
Participants had no problems making the distinction between use and misuse of prescription
drugs. Misuse was correctly reported. One respondent reported that she sold her prescription
drugs, which should be considered misuse. Upon probing she indicated that a doctor did not tell
her to sell her pills.
Ways in which participants misused prescription drugs
Only one participant mentioned misusing a prescription in a way different than defined in the
section introductions. She indicated she used less of the drug than prescribed. She believed this
way fit in the “in any other way a doctor did not direct” category. Most other participants
indicated they had used the drug without a prescription. However, several made a point to say at
different junctures that they “knew” how a doctor would prescribe it and their use was consistent
with that “knowledge.”
Motivations for misuse
Participants were generally able to answer questions about the reasons for use the last time they
had used a drug in a way a doctor had not prescribed, and in a similar manner most had no
problem identifying the main reason for this misuse. A few issues were noted, however.
PRYMOTIV – Of the participants routed to this item, three mentioned either an issue with or
problem in answering questions about the reasons for misuse. One participant had previously
indicated she had used a prescription in a way a doctor had not directed her to by using less than
prescribed. She noted that there was no response alternative that fit her situation. One respondent
noted that he took the pills to escape his emotions, and inidicated that a response option about
dealing with emotions should be included.
PRMOTOT – When one participant selected “some other reason” for PRYMOTIV and was
presented PRMOTOT he was unclear the question was asking him to specify the “other reason.”
Recommendation: Revise the wording to say, Please type in the other readon you used [RX
DRUG] that time. This revision inserts “other” into the sentence.
PRMOT1 – Participants did not display or report any problems answering this item. However,
one interviewer noted that currently we do not allow “some other reason” from PRMOTOT to be
selected as the main reason in PRMOT1.
Recommendation: Discuss including “some other reason.” This will require developing an
alternative audio fill for the “some other reason” such as “The other reason you reported.”
Alternatively, we could acknowledge to respondents that they can only pick from reasons 1-7
7
from PRMOTOT as the main reason, but we would need to consider the limitations of this
approach.
TRYMOTIV – Of the participants routed to this item, one again noted that the response
categories did not fit her situation because she answered “yes” to misuse, but had actually used
less than a doctor had directed her to. Another participant mentioned difficulty answering this
item because he did not think his depression fit in any of the categories.
SVYMOTIV – One participant again noted that he thought we should have a response alternative
that captured dealing with emotions. The other participant who was routed to this item was able
to successfully match his answer to a response alternative.
Special Drugs
SD01 – Consistent with findings from Phase 2.1 interviews, popular examples of OTC drugs for
use just to get high were “Nyquil,” “Benadryl,” and the generic terms such as “cough syrup,” or
“cough medicine.” Others mentioned were “mouthwash,” “paragesic,” “triple-C,” “No-doze,”
“Sudafed,” “Vicks,” Robitussin,” “Formula 44,” “sleep aids,” “Tylenol,” and “Tylenol PM.” One
participant mentioned “purple drink” which seemed to be similar to a substance a participant in a
Phase 2.1 interview mentioned. However, we believe a prescription drug may be involved with
this substance. One other participant mentioned “red juice,” describing an OTC syrup.
Blunts
MJMM – It was clear to all five of the participants who were routed to this item that the question
was asking about marijuana prescribed by a doctor. In response to the probe, all five mentioned
prescribed use. Two participants differentiated this use from use with marijuana bought from
“someone else” or “someone on the street.”
Substance Dependence and Abuse
DRPR -- One participant reported heavy use of prescription pain relievers. Some of this was
misuse, but much of it was prescribed use. It was evident she was not clear this question was
asking about misuse.
Recommendation: Consider listing drugs the respondent has misused to clarify which drugs
respondents should be thinking of while answering this question. This will also provide a
reminder about which drugs are pain relievers, as opposed to other classes of drugs.
Recommendation: Confirm that these questions should use the clause “or that you used only for
the experience or feeling they caused” as there seemed to be some disagreement or confusion
about this.
8
Drug Treatment
TX04c – One participant was unsure whether AA should be counted for this item.
Recommendation: Discuss moving the AA item earlier in the sequence of treatment questions
so that it comes first.
Health
HLTH04 – None of the participants reported or displayed any problems answering about their
height. One participant was unsure how to continue after typing in his answer for “feet.” He
asked, “Do I press enter?”
HLTH18 – One adolescent participant was unclear whether the question included going with
someone else to the hospital for their inpatient stay.
HLTH19 – Generally speaking, participants listed and counted valid medical professionals to
report an answer. One reported a visit to a dentist.
HLTH22 – Three participants reported doctors discussing their drinking with them. These
reports included only being asked on a form, being asked anytime one saw a new doctor, and a
general discussion about “overindulgence” that was unrelated to actual use.
HLTH25 – One participant listed a number of health conditions, but upon probing indicated that
he was reporting times his doctor had asked or discussed these with him. It seems he may have
confused language in HLTH22 with that in HLTH25—that is language about a doctor discussing
these issues with him.
QI12AN – Two participants had difficulty understanding this question. Each had trouble
understanding what types of welfare or public assistance should be reported.
Recommendation: Add a bulleted list of the types of welfare respondents should include when
reporting.
Back End Demographics
QD17 – Nine participants said that they would correctly report being in school even if they were
on summer break. One said he was answer “no” even if he was on spring or summer break.
QD18 – A hypothetical question was asked of participants about being between 7th and 8th
grades. One participant incorrectly said she would say the 7th grade.
QD07 – One participant was unhappy with the omission of a response category for cohabiting.
He said he has been in a committed relationship for approximately four years and thinks the
category “have never been married” does not accurately reflect his situation.
9
QD26 – None of the participants had trouble answering this question about whether they had
worked at a job or business in the past week.
QD49 - Participants did not have any problems understanding this item about whether their
workplace had drug or alcohol testing as part of the hiring process.
QD50 – One participant was probed on this item and he correctly understood what was meant by
drug testing “random basis.” He said it means “out of the blue…like names out of hat and you
pick one.”
PERAGEYR – One participant was unable to answer this question about the age of each
household member because he did not know his parents’ ages.
Income
INTRTINN – This item about sources of income may be confusing to some respondents since
the item is now self-administered whereas it was previously interviewer administered.
Recommendation: Consider modifying language for this item since it is now self-administered
where as it was previously interviewer administered. For example, use of the term “we” in the
clause “those we just talked about” might be awkward since the respondent is interacting with a
computer.
Proxy Tutorial
In general, proxy participants for this round found the tutorial “easy” or “straightforward.” One
proxy participant asked for clarification about whether she could proceed before reviewing all
instructions. In another interview, an interviewer noted that for proxy respondents who have
previously completed an interview or proxy interview there was no transition screen making for
an awkward transition to the ACASI questions.
Recommendation: Add a screen at the beginning of the front-end proxy tutorial that allows
respondents who have previously completed a NSDUH interview or proxy interview to complete
an abbreviated rather than full tutorial.
Recommendation: We should add one more screen to the proxy tutorial transition that indicates
that the interviewer should turn the computer over to a proxy who has already completed the
tutorial elsewhere.
Recommendation: INTROINC says, These questions refer to the calendar year 2011 rather
than to the past 12 months that were referred to in some earlier questions. We recommend not
using this paragraph when there is a proxy, because they have not answered a large battery of
questions with a reference period of the past 12 months. QHI14 is the only question in the health
insurance and income items that asks about the past 12 months. This question asks about the
number of months in the past 12 that the sample person was without health insurance.
10
Additional notes
The Substance Dependence and Abuse section, starting at screen INTRODR, only requires
having smoked ‘all or part of a cigarette’ one time in the past 30 days for a respondent to receive
questions about abuse. One respondent had only smoked one cigarette and many questions did
not apply to him.
Recommendation: Consider whether there should be a higher barrier of entry to this and similar
sections of the Dependence module.
Currently the calendar and calendar reminders are only provided in the core items.
Recommendation: Add calendars and calendar reminders throughout the instrument.
11
Appendix F
Variable Wording Crosswalk
Attachment F. Variable Wording Crosswalk
QFT Variable
(Testing
Outcome)
V2b
(new
question,
tested, no
problems)
QD05 (not
tested,
implemented
in QFT)
QD10c
(new
Phase
Introduced
QFT Instrument Wording
Phase 2
[IF QD10 = 2] Are you currently
serving full‐time in a Reserve
component? Full‐time service does
not include annual training for the
Reserves or National Guard.
NA
Phase 2
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
GUAMANIAN OR
CHAMORRO
6
SAMOAN
7
OTHER PACIFIC ISLANDER
8
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
9
OTHER (SPECIFY)
[IF QD10a=1] Did you ever serve on
active duty in the United States
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
to attempt to better
identify respondents in
the Reserves.
2012 Question Wording
Not included in 2012
QD05 HAND R SHOWCARD 2 Which
of these groups describes you? Just
give me the number or numbers from
the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE (AMERICAN
INDIAN INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
After testing, added two
new race categories to
this question in response
to feedback.
No changes between initial testing and This question was added
final QFT wording.
to better determine if
Not included in 2012
2
QFT Variable
(Testing
Outcome)
question,
tested, no
problems)
QD11
(modified
question;
tested;
problems
found;
revised for
final QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
Armed Forces or Reserve
components in a military combat
zone or an area where you drew
imminent danger pay or hostile fire
pay?
HAND R SHOWCARD 5. What is the
highest grade or year of school you
have completed? Just give me the
number from the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING
COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 REGULAR HIGH SCHOOL
DIPLOMA
13 12TH GRADE, NO DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT,
BUT NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
Initial Test Wording
Notes/Testing Results
respondents who
indicate that they are on
active duty are actually in
a Reserve component.
2012 Question Wording
What is the highest grade or year of
school you have completed?
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE, NO DIPLOMA
13 REGULAR HIGH SCHOOL
DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT, BUT
NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
The original question
used in 2012 had no
category for completing
the 12th grade without
receiving a diploma. This
category was inserted as
response #12, but during
testing it was found that
the order of categories
could be confusing to
some respondents. As a
result of testing,
categories 12 and 13
were reversed so that
category 12 now
represents a high school
diploma and 13 means
12th grade, but no
diploma.
HAND R SHOWCARD 4. What is
the highest grade or year of school
you have completed?
Please tell me the number from
the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NEVER ATTENDED SCHOOL
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE COMPLETED
13 COLLEGE OR UNIVERSITY /
1ST YEAR COMPLETED
14 COLLEGE OR UNIVERSITY /
2ND YEAR COMPLETED
15 COLLEGE OR UNIVERSITY /
3RD YEAR COMPLETED
16 COLLEGE OR UNIVERSITY /
4TH YEAR COMPLETED
3
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCOLOR)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
QFT Instrument Wording
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE,
MD, DDS, DVM, LLB, JD)
DK/REF
Item dropped from questionnaire.
Initial Test Wording
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE, MD,
DDS, DVM, LLB, JD)
DK/REF
CGCOLOR [IF (CG11 = 1‐2 OR CG11 =
4 OR CG11 = 7 OR CG11 = 10 OR CG11
= 12 OR CG11 = 14 OR CG11 = 17‐21
OR CG11 = 24‐25 OR CG11a = 31 AND
CG13 = 1) OR IF (RCG11 = 1‐2 OR
RCG11 = 4 OR RCG11 = 7 OR RCG11 =
10 OR RCG11 = 12 OR RCG11 = 14 OR
RCG11 = 17‐21 OR RCG11 = 24‐25 OR
RCG11a = 31 AND RCG13 = 1) OR IF
(RRCG11 = 1‐2 OR RRCG11 = 4 OR
RRCG11 = 7 OR RRCG11 = 10 OR
RRCG11 = 12 OR RRCG11 = 14 OR
RRCG11 = 17‐21 OR RRCG11 = 24‐25
OR RRCG11a = 31 AND RRCG13 = 1]
What color is the pack of
[CG11/CG11a/RCG11/RCG11a/RRCG1
1/RRCG11a FILL] cigarettes you
smoked most often during the past 30
days?
1
Blue
2
Green
3
Gold
4
Red
5
Silver
6
White
Notes/Testing Results
2012 Question Wording
17 COLLEGE OR UNIVERSITY /
5TH OR HIGHER YEAR
COMPLETED
DK/REF
Not included in 2012
Recent legislation
banned cigarette
manufacturers from
using terms on packaging
that referred to cigarette
strength. In an attempt
to continue to track
trends in the types of
cigarettes most smoked,
a new question was
developed to capture
data based on the new
cigarette color code
system. Most
respondents had a
problem with the
question and reported
that the cigarette
packaging included
multiple colors,
indicating that
participants failed to
understand the purpose
of the question.
4
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCLR2)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
CG25
Phase 1
(CG17REV)
(question
revised in
order to
administer
abbreviated
core for
cognitive
interviewing;
new wording
tested and
implemented
)
ALREF
Phase 1
QFT Instrument Wording
Item dropped from questionnaire.
Initial Test Wording
7
Some other color
DK/REF
CGCLR2 [IF CGCOLOR = 7] Please
type in the color of the pack of
cigarettes you smoked most often
during the past 30 days.
The next questions are about your
use of “smokeless” tobacco such as
snuff, dip, chewing tobacco, or
“snus.”
Have you ever used “smokeless”
tobacco, even once?
Have you ever, even once, tried any
“smokeless” tobacco such as snuff,
dip, chewing tobacco, or “snus”?
NOTE: Example text from ALREF is
ALREF [IF AL01 = REF] The answers
Notes/Testing Results
This follow‐up question
to an answer of “Some
other color” to CGCOLOR
was removed along with
the initial question due
to participant
misunderstanding of the
question.
An abbreviated core
module was used for
cognitive interviewing.
The version of the
smokeless tobacco
question that was used
eliminated the
introductory wording,
“The next questions are
about your use of . . .”
since only the lifetime
use of smokeless tobacco
was assessed. The
revised wording that was
tested and maintained
combined snuff and
chewing tobacco, since
incidence use rates of
these are quite low. In
addition, a new product
called “snus” has
recently emerged on the
market and it is also
included in the QFT
wording. No changes to
the question were made
as a result of the testing.
A streamlined version of
2012 Question Wording
Not included in 2012
These next questions are about your
use of snuff, sometimes called dip.
Snuff is a finely ground form of
tobacco that usually comes in a
container called a tin. You can use
snuff by placing a pinch or dip in
your mouth between your lip and
gum or between your cheek and
gum. Snuff can also be inhaled
through the nose. Snuff is sold in
both loose form and in ready‐to‐use
packets.
Have you ever used snuff, even
once?
[IF AL01 = REF] The answers that
5
QFT Variable
(Testing
Outcome)
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
AL08
(new
question,
tested, no
problems,
implemented
for QFT)
Phase 1
MRJINTRO &
MJ01
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase 1
QFT Instrument Wording
displayed below. Equivalent text was
used for MJREF, CCREF, HEREF, and
MEREF.
[IF AL01 = REF] The answers that
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have [IF QD01‐5 THEN FILL
5, IF QD01=9 THEN FILL 4] or more
drinks on the same occasion? By
‘occasion,’ we mean at the same
time or within a couple of hours of
each other.
Initial Test Wording
people give about their use of alcohol
are important. Remember, your
answers will be kept confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a sip
or two from a drink.
Notes/Testing Results
this question was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is, since
[DATEFILL], on how many days did
you have [IF QD01‐5 THEN FILL 5, IF
QD01=9 THEN FILL 4] or more drinks
on the same occasion? By ‘occasion,’
we mean at the same time or within a
couple of hours of each other.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have 5 or more drinks on
the same occasion? By ‘occasion,’
we mean at the same time or within
a couple of hours of each other.
MRJINTRO
The next questions
are about marijuana and hashish.
Marijuana is also called pot or grass.
Marijuana is usually smoked, either
in cigarettes, called joints, or in a
pipe. It is sometimes cooked in
food. Hashish is a form of marijuana
that is also called “hash.” It is
usually smoked in a pipe. Another
form of hashish is hash oil.
MJ01 The next question is about
marijuana and hashish. Marijuana is
also called pot, grass, or weed.
Marijuana is usually smoked, either in
cigarettes, called “joints”, in cigars,
called “blunts”, or in a pipe. It is
sometimes cooked in food. Hashish is
a form of marijuana that is also called
“hash.” It is usually smoked in a pipe.
Another form of hashish is hash oil.
This item was revised to
ask females about
drinking 4 or more drinks
on the same occasion in
the Alcohol section.
Previously, this
information was
obtained in the
Consumption of Alcohol
module.
For the cognitive
interviewing we initially
tested a version of the
lifetime marijuana
question as part of the
abbreviated core. This
question incorporated
the introductory
language and referenced
only one “next question”
The next questions are about
marijuana and hashish. Marijuana is
also called pot or grass. Marijuana
is usually smoked, either in
cigarettes, called joints, or in a pipe.
It is sometimes cooked in food.
Hashish is a form of marijuana that
is also called “hash.” It is usually
smoked in a pipe. Another form of
hashish is hash oil.
6
QFT Variable
(Testing
Outcome)
Phase
Introduced
HE01
Phase 1
(question
revisions
were tested;
no advantage
was found;
dropped from
QFT)
HALINTRO
Phase 1
(question was
revised to
administer
abbreviated
version of
core drugs
module for
cognitive
interviewing
only.)
QFT Instrument Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Initial Test Wording
Notes/Testing Results
since subsequent
Have you ever, even once, used
questions to lifetime use
marijuana or hashish, or smoked part were not included. The
or all of a cigar with marijuana in it,
term “weed” was
included since it is a
called a “blunt”?
commonly used term for
marijuana. Additionally,
the cognitive
interviewing tested
inclusion of “blunts” in
the main marijuana
module. Combining
blunts with main
marijuana use was
desired because of
interest in streamlining
the instrument and
consolidating marijuana
use in one module.
Have you ever, even once, used
This next question is about heroin.
Alternate terms for
heroin?
heroin, “black tar heroin”
Have you ever, even once, used
and “cheese” were
heroin, including “black tar heroin” or tested but were not
included in the QFT since
“cheese”?
cognitive interview
respondents did not tend
to recognize them.
The next questions are about
The introductory text to
The next questions are about
substances called hallucinogens. These hallucinogens was
substances called hallucinogens.
drugs often cause people to see or
removed as part of
These drugs often cause people to
creating an abbreviated
see or experience things that are not experience things that are not real.
core drugs module for
real.
the cognitive
Press [ENTER] to continue.
interviewing. Since only
A list of some common
lifetime use was asked
hallucinogens is shown below.
about each substance,
These and many other substances
that people use as hallucinogens are
the full list of
often known by street names, and
hallucinogens was not
2012 Question Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Have you ever, even once, used
heroin?
The next questions are about
substances called hallucinogens.
These drugs often cause people to
see or experience things that are not
real.
A list of some common
hallucinogens is shown below.
These and many other substances
that people use as hallucinogens are
often known by street names, and
7
QFT Variable
(Testing
Outcome)
Phase
Introduced
LS01i
(no change
tested)
Phase 1
LS01j
(existing
question
moved to
new module;
no wording
changes)
Phase 1
QFT Instrument Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Ketamine, also called “Special K” or
“Super K”
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine
Foxy, also called 5‐MeO‐DIPT
Salvia divinorum
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine, or
Initial Test Wording
Notes/Testing Results
included prior to the list
of lifetime use questions.
The QFT wording
included examples of all
Hallucinogens asked
about in the module,
including the three
substances that were
added to the module.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
2012 Question Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
1
Yes
2
No
DK/REF
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine,
AMT, also called alpha‐
methyltryptamine, or
8
QFT Variable
(Testing
Outcome)
LS01k
(existing
question
moved to
new module;
no wording
changes)
INHINTRO
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
QFT Instrument Wording
Foxy, also called 5‐MeO‐DIPT?
Initial Test Wording
Notes/Testing Results
Phase 1
Have you ever, even once, used
Salvia divinorum?
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
Phase 1
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
These next questions are about
liquids, sprays, and gases that people
sniff or inhale to get high or to make
them feel good.
Please do not include times when you
inhaled a substance accidentally —
such as when painting, cleaning an
oven, or filling a car with gasoline. The
questions use the word “inhalant” to
include any substances that you
sniffed or inhaled for kicks or to get
high
Press [ENTER] to continue.
A streamlined version of
the inhalants
introduction was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
Foxy, also called 5‐MeO‐DIPT?
Have you ever, even once, used
Salvia divinorum?
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
9
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Felt‐tip pens, felt‐tip markers, or
magic markers
Spray paints
Computer keyboard cleaner, also
known as air duster
Other aerosol sprays
Press [ENTER] to continue.
Have you ever, even once, inhaled
felt‐tip pens, felt‐tip markers, or
magic markers for kicks or to get
high?
IN01h1 (new
question,
tested, no
problems)
Phase 1
IN01ii (new
question
tested,
problems
found,
revised for
QFT)
Phase 1
Have you ever, even once, inhaled
computer keyboard cleaner, also
known as air duster, for kicks or to
get high?
ME01
(SD17a)
(existing
question
moved to
new module,
Phase 1
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
because of references to
felt‐tip pens, felt‐tip
markers, and magic
markers in the write‐in
data in inhalants in the
annual survey.
In the past 12 months, have you
This question was added
because of references to
inhaled canned air for kicks or to get
computer keyboard
high?
cleaner and canned air in
the write‐in data in
inhalants in the annual
survey. In the cognitive
interviewing, participants
often did not know what
“canned air” was. New
terms were selected
based on cognitive
interviewing results.
No changes between initial testing and A new module was
created for
final QFT wording.
methamphetamine in
the core drugs modules.
Items measuring
methamphetamine use
2012 Question Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Spray paints
Other aerosol sprays
Press [ENTER] to continue.
Not included in 2012
Not included in 2012
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
10
QFT Variable
(Testing
Outcome)
no changes)
Phase
Introduced
ME02
(question
moved to
new module,
no changes)
MELAST3
(question
moved to
new module,
no changes)
MEFRAME3
(question
moved to
new module,
no changes)
Phase 2
MERAVE
(question
moved to
new module,
no changes)
MEMONAVE
(question
moved to
new module,
minor
changes)
MEWKAVE
QFT Instrument Wording
injected.
Have you ever, even once, used
methamphetamine?
How old were you the first time you
used methamphetamine?
Initial Test Wording
Notes/Testing Results
were moved from the
Stimulants module to
this new module.
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
2012 Question Wording
injected. Have you ever, even once,
used Methamphetamine?
SDME01 How old were you the first
time you used Methamphetamine?
Phase 2
How long has it been since you last
used methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
SD17b How long has it been since
you last used Methamphetamine?
Phase 2
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you’ve
used it?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On how many days in the past 12
months did you use
methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On average, how many days did you
use methamphetamine each month
during the past 12 months?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
Methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you
used Methamphetamine?
On how many days in the past 12
months did you use
Methamphetamine?
On average, how many days each
month during the past 12 months
did you use Methamphetamine?
Phase 2
On average, how many days did you
No changes between initial testing and A new module was
On average, how many days each
11
QFT Variable
(Testing
Outcome)
(question
moved to
new module,
no changes)
ME06 (new
question,
tested, no
changes)
INTROPR
(new
question
tested, no
changes)
PR01‐PR03
PR08
PR10
ST01‐ST04
ST05
ST06‐ST07
(question
tested,
problems
identified,
question
revised)
Phase
Introduced
Phase 2
QFT Instrument Wording
use methamphetamine each week
during the past 12 months?
[IF MELAST3 =1 OR MERECDK = 1
OR MERECRE = 1] Think specifically
about the past 30 days, from
[DATEFILL] up to and including
today. During the past 30 days, on
how many days did you use
methamphetamine?
Phase 1
These next questions are about any
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as
aspirin, Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
PR01
NOTE: Example text from PR01 is
ST01, ST02, displayed below. Equivalent text was
ST03, ST04, used for all listed variables from pain
relievers, stimulants, sedatives, and
ST06, &
tranquilizers modules.
ST07 –
Usability
Please look at the names and
pictures of the pain relievers shown
PR02,
below. Please note that some forms
PR03,
of these pain relievers may look
PR08,
different from the pictures, but you
PR10, &
should include any form that you
ST05 –
have used.
Phase 1
PROGRAMMER: DISPLAY PILLS HERE
TR01 &
FOR VICODIN, LORTAB, LORCET,
SV01 –
AND HYDROCODONE.
Phase 2
In the past 12 months, which, if any,
Initial Test Wording
final QFT wording.
Notes/Testing Results
created for
methamphetamine in
the core drugs modules.
No changes between initial testing and As part of creating a new
final QFT wording.
methamphetamine
module, a new 30‐day
methamphetamine use
variable was created.
2012 Question Wording
week during the past 12 months did
you use Methamphetamine?
Not included in 2012
These next questions are about the
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as aspirin,
Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription pain
relievers module.
PR01A. Please look at the pictures of
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in
either of these ways?
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
PR01A1. [IF PR01A = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
Not included in 2012
Initial versions of the
prescription drugs
screener questions were
written to capture
misuse only. Subsequent
to usability testing, the
question was revised to
capture first all types of
use then follow up
specific substances
endorsed with questions
about misuse.
Not included in 2012
12
QFT Variable
(Testing
Outcome)
PR04
PR06‐PR07
TR04 (new
question,
problems
found,
question
revised.)
Phase
Introduced
PR04,
PR06, &
PR07 –
Phase 1
TR04 –
Phase 2
QFT Instrument Wording
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Initial Test Wording
did you use in the past 12
months in either of these
ways?
Notes/Testing Results
2012 Question Wording
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
Please look at the names and pictures
NOTE: Example text from PR04 is
displayed below. Equivalent text was of the pain relievers shown below.
used for all listed variables from pain
relievers, stimulants, sedatives, and PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER, ULTRACET,
tranquilizers modules.
RYZOLT, AND TRAMADOL.
Please look at the names and
pictures of the pain relievers shown In the past 12 months, which, if any,
below. Remember, some forms of
of these pain relievers have you used?
these pain relievers may look
different from the pictures, but you To select more than one drug from the
list, press the space bar between each
should include any form that you
number you have typed. When you
have used.
have finished, press [ENTER].
PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER,
ULTRACET, RYZOLT, AND
TRAMADOL.
In the past 12 months, which, if any,
of these pain relievers have you
Original test of wording
Not included in 2012
yielded results that
participants were asking
whether certain
prescription drugs should
be included even though
they looked different
from what was in the
pictures. Reminder
language was added to
every 2‐4 screens to
remind respondents that
the pictures may look
different from the form
pictured but that they
should still be included.
13
QFT Variable
(Testing
Outcome)
Phase
Introduced
PR05
(question
tested in
cognitive
interviewing,
problem
found, minor
changes
made)
Phase 1
PR11
TR06
ST06
SV06 (Change
PR11 &
ST06‐
Phase 1
QFT Instrument Wording
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Please look at the names and
pictures of the pain relievers shown
below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine 3 or 4
2
Codeine pills
95
I have not used any of
these pain relievers in the past 12
months
DK/REF
In the past 12 months, have you
used any other prescription pain
reliever?
Initial Test Wording
Please look at the names and pictures
of the pain relievers shown below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you used?
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine
2
Codeine pills
95
I have not used any of these
pain relievers in the past 12 months
DK/REF
Notes/Testing Results
2012 Question Wording
Initially, “Tylenol with
Not included in 2012
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
No changes between initial testing and The prescription drugs
final QFT wording.
modules were revised to
ask first about all types
of use. Respondents
Not included in 2012
14
QFT Variable
(Testing
Outcome)
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
TR06 &
Remember, do not include “over‐
SV06‐
the‐counter” pain relievers such as
Phase 2
aspirin, Tylenol, Advil, or Aleve.
PR12
TR07
ST07
SV07 (Change
tested, no
problems
found,
implemented
in QFT.)
PR12 &
ST07—
Phase 1
TR07 &
SV07—
Phase 2
INTROTR
INTROSV
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 2
Initial Test Wording
Notes/Testing Results
received follow up
questions about misuse
only of specific drugs
that were used. This
particular item was
administered when no
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
No changes between initial testing and The prescription drugs
[IF PR12MON = 2] Have you ever,
final QFT wording.
modules were revised to
even once, used any prescription
ask first about all types
pain reliever?
of use. Respondents
received follow up
Remember, do not include “over‐
questions about misuse
the‐counter” pain relievers such as
only of specific drugs
aspirin, Tylenol, Advil, or Aleve.
that were used. This
particular question was
asked when no 12‐month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
These questions were
NOTE: Example text from INTROTR is These next questions are about the
added to introduce the
use of prescription tranquilizers.
displayed below. Semi‐equivalent
Tranquilizers are usually prescribed to new prescription
text was used for INTROSV.
relax people, to calm people down, to tranquilizer and sedative
screener modules. Minor
These next questions are about any relieve anxiety, or to relax muscle
wording revisions were
spasms. Some people call
use of prescription tranquilizers.
made after testing.
Tranquilizers are usually prescribed tranquilizers “nerve pills.”
to relax people, to calm people
down, to relieve anxiety, or to relax Press [ENTER] to continue.
muscle spasms. Some people call
tranquilizers “nerve pills.”
2012 Question Wording
Not included in 2012
These next questions ask about the
use of tranquilizers. Tranquilizers
are usually prescribed to relax
people, to calm people down, to
relieve anxiety, or to relax muscle
spasms. Some people call
tranquilizers ‘nerve pills.’
Ask your interviewer to show you
Card B.
15
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTROST
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 1
PRL01
TRL01
STL01
SVL01
(Change
tested, no
problems
found,
implemented
in QFT.)
PRL01 &
STL01—
Phase 1
TRL01 &
SVL01—
Phase 2
PRY01—
Usability
QFT Instrument Wording
Press [ENTER] to continue.
These next questions are about any
use of prescription stimulants.
People sometimes take these drugs
for attention deficit disorders, to
lose weight, or to stay awake.
Please do not include “over‐the‐
counter” stimulants such as
Dexatrim, No‐Doz, Hydroxycut, or 5‐
Hour Energy.
Press [ENTER] to continue.
Initial Test Wording
Notes/Testing Results
2012 Question Wording
These next questions are about the
use of prescription stimulants. People
sometimes take these drugs for
attention deficit disorders, to lose
weight, or to stay awake. Please do
not include “over‐the‐counter”
stimulants such as Dexatrim or No‐
Doz.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription stimulants
screener module. Minor
wording revisions were
made after testing to
include additional
examples.
These next questions ask about the
use of drugs such as amphetamines
that are known as stimulants,
‘uppers,’ or ‘speed.’ People
sometimes take these drugs to lose
weight, to stay awake, or for
attention deficit disorders. We are
not interested in the use of “over‐
the‐counter” stimulants such as
Dexatrim or No‐Doz that can be
bought in drug stores or grocery
stores without a doctor’s
prescription.
Ask your interviewer to show you
Card C.
NOTE: Example text from PR05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
Have you ever, even once, used any
other prescription pain reliever,
besides the ones shown on Card A,
when it was not prescribed for you
or that you took only for the
experience or feeling it caused?
No changes between initial testing and The prescription drugs
NOTE: Example text from PRL01 is
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[IF PR12=1] Have you ever, even
once, used any prescription pain
that were used. This
reliever in any way a doctor did not
particular question was
direct you to use it?
asked when 12 month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed
but no misuse was ever
endorsed when follow up
questions were asked
about specific drugs.
NOTE: Example text from PRY01 is
PR01C. Please look at the pictures of The revised prescription [IF PR04 = 1] Which of the pain
16
QFT Variable
(Testing
Outcome)
PRY04
PRY24
PRY32—
PRY34
PRY37
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR01=1] In the past 12 months,
did you use Vicodin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in any
of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
1
Yes
2
No
DK/REF
PR01C1. [IF PR01C = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
did you use in the past 12
months in any of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
Notes/Testing Results
drugs questions ask
about misuse of drugs
endorsed for any type of
12 month use in the
screener. The initial
wording of the third
component of the
definition of misuse,
“Just for the effect it has
on you – not for its
intended medical use”
did not help test
participants identify
misuse so it was revised
to the final QFT wording.
Also, the wording in the
2012 survey only
collected estimates of
lifetime use of individual
prescriptions, while the
new questions collect
past year use of
individual drugs.
2012 Question Wording
relievers shown below the red line
on Card A have you used when they
were not prescribed for you or that
you took only for the experience or
feeling they caused?
To select more than one drug from
the list, press the space bar between
each number you type. When you
have finished, press [ENTER].
4
Codeine
5
Demerol
6
Dilaudid
7
Fioricet
8
Fiorinal
9
Hydrocodone
10
Methadone
11
Morphine
12
OxyContin
13
Phenaphen with Codeine
14
Propoxyphene
15
SK‐65
16
Stadol
17
Talacen
18
Talwin
19
Talwin NX
20
Tramadol
21
Ultram
DK/REF
17
QFT Variable
(Testing
Outcome)
PRY05‐PRY17
PRY19
PRY20—
PRY23
PRY25—
PRY33
PRY35—
PRY36
PRY38—
PPRY39
TRY01—
TRY18
STY01—
STY23
SVY01—
SVY16
(Change
tested,
wording
Phase
Introduced
Phase 1
QFT Instrument Wording
[IF PR02=1] In the past 12 months,
did you use OxyContin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
1
Vicodin
2
Lortab
3
Lorcet
4
Hydrocodone
DK/REF
NOTE: Example text from PRY05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR02=1] In the past 12 months, did
you use OxyContin in any way a doctor
did not direct you to use it?
Notes/Testing Results
Cognitive testing
identified repetitive
nature of prescription
drugs module questions
with long definitions of
misuse and raised
concerns about
respondent fatigue. As a
result, the definition was
displayed as optional text
available by pressing F2
key on some screens.
2012 Question Wording
How long has it been since you last
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
18
QFT Variable
(Testing
Outcome)
improvement
identified,
implemented
in QFT.)
PRY18
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY28
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY01a—
PRY39a
TRY01a—
TRY18a
STY01a—
STY23a
SVY01a—
SVY16a
(Change
tested,
wording
improvement
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Please look at the pain relievers
shown in Box 1 above the red line
on Card A.
Have you ever, even once, used
Darvocet, Darvon, or Tylenol with
codeine that was not prescribed for
you or that you took only for the
experience or feeling it caused?
Not included in 2012
Phase 1
[IF PR05=1] In the past 12 months,
did you use Tylenol with codeine 3
or 4 in any way a doctor did not
direct you to use it?
[IF PR05=1] In the past 12 months, did
you use Tylenol with codeine in any
way a doctor did not direct you to use
it?
Initially, “Tylenol with
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
Phase 1
[IF PR07=4] In the past 12 months,
did you use fentanyl in any way a
doctor did not direct you to use it?
[IF PR07=4] In the past 12 months, did
you use fentanyl or “China Girl” in any
way a doctor did not direct you to use
it?
The street name for
fentanyl, “China Girl”
was not known to
cognitive interview
participants so it was
dropped from the QFT.
PRY01a‐
PRY39a &
STY01a‐
STY23a‐
Phase 1
TRY01a‐
TRY18a &
SVY01a‐
SVY16a‐
Phase 2
[IF PRY01=1] How old were you when
NOTE: Example text from PR01a is
displayed below. Equivalent text was you first used Vicodin in a way a
used for all listed variables from pain doctor did not direct you to use it?
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRFIRSTFLAG=1] Please think
about the first time you ever used
Vicodin in a way a doctor did not
direct you to use it.
[IF PRY01=1] How old were you
when you first used Vicodin in a way
a doctor did not direct you to use
it?
An introduction was
added to transition from
asking about whether
groups of drugs were
misused to asking about
the specifics of that
misuse.
NOTE: Example text from PR06 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
How old were you the first time you
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
19
QFT Variable
(Testing
Outcome)
identified,
implemented
in QFT.)
PRY01b—
PRY39b
TRY01b—
TRY02b
STY01b
SVY01b—
SVY02b
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01c—
PRY02c
TRY01c—
TRY02c
STY01c
SVY01c—
SVY02c
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRY01b—
PRY39b &
STY01b‐
Phase 1
TRY01b—
TRY02b &
SVY01b‐
Phase 2
PRY01c—
PRY02c &
STY01c—
Phase 1
TRY01c—
TRY02c &
SVY01c—
SVY02c—
Phase 2
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Not included in 2012
NOTE: Example text from PRY01b is No changes between initial testing and The prescription drugs
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[AGE1STPR1 = CURNTAGE AND
DATE OF INTERVIEW < DOB OR IF
that were used. This
AGE1STPR1 = CURNTAGE ‐ 1 AND
particular question was
DATE OF INTERVIEW ≥ DOB] Did you
asked when 12 month
first use Vicodin in a way a doctor
use of a prescription drug
did not direct you to use it in
was endorsed with first
[CURRENT YEAR ‐ 1] or [CURRENT
use within at
respondent’s current
YEAR]?
age. No change was
made to the item as a
result of cognitive
testing.
Not included in 2012
No changes between initial testing
The prescription drugs
NOTE: Example text from PRY01c is
modules were revised to
displayed below. Equivalent text was and final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
[IF AGE1STPR2 = CURNTAGE ‐ 1
only of specific drugs
AND DATE OF INTERVIEW < DOB]
that were used. This
Did you first use Lortab in a way a
particular question was
doctor did not direct you to use it in
asked when 12 month
[CURRENT YEAR ‐ 2] or [CURRENT
use of a prescription drug
YEAR ‐ 1]?
was endorsed at
respondent’s age one
year younger than their
current age. No change
was made to the item as
a result of cognitive
20
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRY01d—
PRY02d
TRY01d—
TRY02d
STY01d
SVY01d—
SVY02d
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01d‐
PRY02d &
STY01d—
Phase 1
TRY01d‐
TRY02d &
SVY01d‐
SVY02d—
Phase 2
PRM02DKRE
TRM02DKRE
STM02DKRE
SVM02DKRE
(Change
tested, no
problems
found,
implemented
in QFT.)
PRM02DKR
E &
STM02DKR
E— Phase
1
TRM02DKR
E &
SVM02DKR
E—Phase 2
QFT Instrument Wording
Initial Test Wording
No changes between initial testing
NOTE: Example text from PRY01d is
displayed below. Equivalent text was and final QFT wording.
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRYFU1 NE 0 AND PRJANFLAG1 =
0 AND PRDECFLAG1=0 AND
PR1MTHFLAG1=0]
[IF PRYFU1 = CURRENT YEAR OR
CURRENT YEAR ‐1] Earlier, you
reported that you first used Vicodin
in a way a doctor did not direct you
to use it when you were
[AGE1STPR1] years old. Based on
your date of birth, you turned
[AGE1STPR1] in [FILL WITH
MONTH/YEAR FOR AGE1STPR1
BASED ON DOB].
[IF PRYFU1 NE 0] In what month in
[PRYFU1] did you first use Vicodin in
a way a doctor did not direct you to
use it?
No changes between initial testing and
NOTE: Example text from
final QFT wording.
PRM02DKRE is displayed below.
Equivalent text was used for
TRM02DKRE, STM02DKRE, &
SVM02DKRE.
[IF PRM02 = DK/REF] What is your
best estimate of the number of days
you used [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] during the past 30
days?
Notes/Testing Results
testing.
This question asked
recent initiates of
prescription drug misuse
about the month of first
use of in order to meet a
redesign goal of
capturing more precise
data on first use among
recent initiates of drugs.
No change was made to
the item as a result of
cognitive testing.
No changes were made
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse about
individual drugs are not
collected. This question
was introduced to
measure frequency of
misuse of all misused
substances.
2012 Question Wording
Not included in 2012
Not included in 2012
21
QFT Variable
(Testing
Outcome)
PRM03
TRM03
STM03
SVM03
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRM03 &
STM03—
Phase 1
TRM03 &
SVM03—
Phase 2
PRY40
TRY19
STY24
SVY19
PRY40 &
STY24—
Phase 1
TRY19 &
SVY19—
Phase 2
PRY41
STY25
TRY20
SVY18
Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY41 &
STY25—
Phase 1
TRY20 &
SVY18—
Phase 2
QFT Instrument Wording
NOTE: Example text from PRM03 is
displayed below. Equivalent text was
used for TRM03, STM03, & SVM03.
[IF ALC30USE = 1 and PRMISUSE30 =
1)] During the past 30 days did you
use [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] while you were
drinking alcohol or within a couple
of hours of drinking?
NOTE: Example text from PRY40 is
displayed below. Equivalent text was
used for TRY19, STY24, & SVY19.
[IF PR11=1] In the past 12 months,
did you use any [IF PR11=1 AND
PRYRCOUNT > 1 FILL “other”]
prescription pain reliever in a way a
doctor did not direct you to use it?
Initial Test Wording
Notes/Testing Results
No changes between initial testing and No changes were made
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse with alcohol
are collected in the
Consumption of Alcohol
module. Here it is asked
in the relevant
prescription drug
module.
2012 Question Wording
Think again about this last time you
drank any alcoholic beverages, when
you had [CA01 FILL] [drink/drinks].
Did you also use [DRUGFILL] while
you were drinking or within a couple
of hours of drinking?
No changes between initial testing and No changes were made
Not included in 2012
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of past year misuse
about individual drugs
are not collected. This
question was introduced
to measure details of
misuse of all misused
substances.
[IF PRMISUSE12=1] Which of these
This question was added Not included in 2012
NOTE: Example text from PRY41 is
to capture more detail
displayed below. Equivalent text was statements describe your use of
about which ways the
[PRNAMEFILL] at any time in the past
used for STY25, TRY20, & SVY18.
respondents misused
12 months? To select more than one
statement, press the space bar
specific drugs. The “at
least once” wording was
[IF PRMISUSE12=1] Which of these between the numbers.
confusing to some
statements describe your use of
[PRNAMEFILL] at any time in the
1
At least once, I used
respondents in the first
past 12 months?
[PRNAMEFILL] without a
round of cognitive
prescription of my own.
interviewing, so it was
To select more than one statement, 2
At least once, I used
removed for subsequent
press the space bar between the
[PRNAMEFILL] in greater
rounds of cognitive
numbers. When you have finished,
amounts, more often, or for
interviewing and for the
press [ENTER].
longer than it was prescribed. QFT. Additional methods
3
At least once, I used
of misuse were also
22
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
I used [PRNAMEFILL]
without a prescription of
my own.
2
I used [PRNAMEFILL] in
greater amounts than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
3
I used [PRNAMEFILL] more
often than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
4
I used [PRNAMEFILL] for
longer than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
5
I used [PRNAMEFILL] in
some other way a doctor
did not direct me to use
[PRNUMFILL]
NOTE: Example text from PRY42b is
displayed below. Equivalent text was
used for STY26b, TRY42B, & SVY19B.
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
1
PRY42B
STY26B
TRY42B
SVY19B
(Change
tested,
PRY42B &
STY26B—
Phase 1
TRY42B &
SVY19B—
Phase 2
Initial Test Wording
[PRNAMEFILL] in some other
way a doctor did not direct
me to use [PRNUMFILL]
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
the last time you used [PRLASTFILL2]
in any way a doctor did not direct you
to use it.
Notes/Testing Results
identified and added to
the response options.
Response categories
were streamlined and
reduced as a result of
cognitive interviewing.
2012 Question Wording
Not included in 2012
23
QFT Variable
(Testing
Outcome)
wording
improvement
identified,
implemented
in QFT.)
PRY42C
STY26C
TRY21C
SVY19C
Phase
Introduced
PRY42C &
STY26C—
Phase 1
QFT Instrument Wording
the last time you used [PRLASTFILL2]
in any way a doctor did not direct
you to use [FILL WITH “them” IF
RPRY42A=19. ELSE FILL WITH “it”].
[IF PRMISUSE12 = 1] How did you
get the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more
than one doctor
3
I stole the [PRLASTFILL]
from a doctor’s office,
clinic, hospital, or
pharmacy
4
I got the [PRLASTFILL] from
a friend or relative for free
5
I bought the [PRLASTFILL]
from a friend or relative
6
I took the [PRLASTFILL]
from a friend or relative
without asking
7
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
8
I got the [PRLASTFILL] in
some other way
DK/REF
NOTE: Example text from PRY42c is
displayed below. Equivalent text was
used for TRY21C & SVY19C.
Initial Test Wording
[IF PRMISUSE12 = 1] How did you get
the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more than
one doctor
3
I wrote a fake prescription for
the [PRLASTFILL]
4
I stole the [PRLASTFILL] from
a doctor’s office, clinic,
hospital, or pharmacy
5
I got the [PRLASTFILL] from a
friend or relative for free
6
I bought the [PRLASTFILL]
from a friend or relative
7
I took the [PRLASTFILL] from a
friend or relative without
asking
8
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
9
I bought the [PRLASTFILL] on
the Internet
10
I got the [PRLASTFILL] in some
other way
DK/REF
Notes/Testing Results
No changes between initial testing and Wording and fills in this
final QFT wording.
question were adapted
to redesign conventions.
No changes were made
2012 Question Wording
Not included in 2012
24
QFT Variable
(Testing
Outcome)
(Change
tested, no
problems
found,
implemented
in QFT.)
(PRHOSPYR1
&
STHOSPYR1)
(Change
tested,
problems
found,
dropped from
QFT)
(PRHOSPYR2
&
STHOSPYR2)
(Change
tested,
Phase
Introduced
QFT Instrument Wording
TRY21C &
SVY19C—
[IF PRY42B=5 AND CALCAGE >=18]
Phase 2
You reported that you got the
[PRLASTFILL] from a friend or
relative for free. How did your
friend or relative get the
[PRLASTFILL]?
Phase 1
Items dropped from questionnaire.
Phase 1
Item dropped from questionnaire.
Initial Test Wording
Notes/Testing Results
as a result of the
cognitive interviewing.
2012 Question Wording
NOTE: Example text from PRHOSPYR1
is displayed below. Equivalent text was
used for STHOSPYR1.
[IF PR12MON=1 AND (PR11=1 OR
PRYRCOUNT > 1)] The computer
recorded that, in the past 12 months,
you used [PRHOSPFILL].
In the past 12 months, did you use [IF
PR11=1 AND PRYRCOUNT=1, THEN
FILL WITH PRHOSPFILL. IF PRYRCOUNT
> 1 THEN FILL WITH “any of these
prescription pain relievers”] only
when you were in the hospital? That
would include staying in the hospital
as an inpatient or being treated in an
emergency room.
[IF PR12MON=1 AND PR11 NE 1 AND
PRYRCOUNT = 1] In the past 12
months, did you use [PRHOSPFILL]
only when you were in the hospital?
That would include staying in the
hospital as an inpatient or being
treated in an emergency room.
NOTE: Example text from PRHOSPYR2
is displayed below. Equivalent text was
used for STHOSYR2.
[IF PRHOSPYR1=1 AND PRYRCOUNT >
A question about
legitimate use of
prescription drugs only
while in the hospital was
added to the instrument
during the cognitive
interviewing in order to
account for this very
specific kind of use that
respondents might not
be sure how to count.
The question confused
several respondents so it
was eliminated.
Not included in 2012
This follow up to the
previous question
assessed which
prescription drugs were
used only in a hospital
Not included in 2012
25
QFT Variable
(Testing
Phase
Outcome)
Introduced
problems
found,
dropped from
QFT)
PRYMOTIV
Phase 2
QFT Instrument Wording
Initial Test Wording
1] Which of these pain relievers did
you use only when you were in the
hospital in the past 12 months?
(LIST DRUGS ANSWERED AS YES IN
PR01‐PR11. NUMBER RESPONSE
OPTIONS SEQUENTIALLY STARTING AT
1, BUT MAINTAIN UNIQUE CODES FOR
EACH DRUG. IF PR11=1, DISPLAY
"Another prescription pain reliever"
AS THE LAST CATEGORY. )
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
NOTE: Example text from PRYMOTIV [IF PRMISCOUNT > 1 OR
is displayed below.
(PRMISCOUNT = 1 AND PRY40 NE 1
AND PRDKREFFLAG=1)]
[IF PRMISCOUNT > 1 OR
Now think about the last time you
(PRMISCOUNT = 1 AND PRY40 NE 1
used [PRLASTFILL2] in any way a
AND PRDKREFFLAG=1)]
doctor did not direct you to use [FILL
Now think about the last time you
WITH ‘them” IF RPRY42A=19. ELSE
used [PRLASTFILL2] in any way a
FILL WITH “it”].
doctor did not direct you to use [FILL
WITH ‘them” IF RPRY42A=19. ELSE
What were the reasons you used
[PRLASTFILL2] that time? To select
FILL WITH “it”].
more than one reason from the list,
press the space bar between each
What were the reasons you used
number you have typed. When you
[PRLASTFILL2] that time? To select
more than one reason from the list, have finished, press [ENTER].
press the space bar between each
number you have typed. When you
1
To relieve physical pain
have finished, press [ENTER].
2
To relax or relieve tension
3
To experiment or to see what
1
To relieve physical pain
[IF RPRY42A=19 THEN
2
To relax or relieve tension
“they’re” ELSE “it’s”] like
Notes/Testing Results
setting. It was dropped
along with the lead‐in
question due to
confusion.
This question was added
to gather data about
motivation for misuse of
Pain Relievers.
2012 Question Wording
Not included in 2012
26
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
4
To feel good or get high
To experiment or to see
what [IF RPRY42A=19 THEN 5
To help with my sleep
“they’re” ELSE “it’s”] like
6
To increase or decrease the
4
To feel good or get high
effect(s) of some other drug
5
To help with my sleep
Because I am “hooked” or I
7
6
To help me with my
have to have [IF RPRY42A=19
feelings or emotions
THEN “them” ELSE “it”]
7
To increase or decrease the
effect(s) of some other drug 8
I used [IF RPRY42A=19 THEN
8
Because I am “hooked” or I
“them” ELSE “it”] for some
have to have [IF
other reason
RPRY42A=19 THEN “them”
DK/REF
ELSE “it”]
9
I used [IF RPRY42A=19
THEN “them” ELSE “it”] for
some other reason
DK/REF
[IF SVMISCOUNT > 1 OR
[IF SVMISCOUNT > 1 OR
(SVMISCOUNT=1 AND SVY17 NE 1
(SVMISCOUNT=1 AND SVY17 NE 1
AND SVDKREFFLAG=1)] Now think
AND SVDKREFFLAG=1)] Now think
about the last time you used
about the last time you used
[SVLASTFILL2] in any way a doctor
[SVLASTFILL2] in any way a doctor did
did not direct you to use it.
not direct you to use it.
What were the reasons you used
What were the reasons you used
[SVLASTFILL2] that time? To select
[SVLASTFILL2] that time? To select
more than one reason from the list, more than one reason from the list,
press the space bar between each
press the space bar between each
number you have typed. When you number you have typed. When you
have finished, press [ENTER].
have finished, press [ENTER].
1
To relax or relieve tension
1. To relax or relieve tension
2. To experiment or to see
2
To experiment or to see what
what it’s like
it’s like
3. To feel good or get high
3
To feel good or get high
4. To help with my sleep
4
To help with my sleep
5. To help me with my
5
To increase or decrease the
feelings or emotions
Notes/Testing Results
2012 Question Wording
3
TRMOTIV
SVMOTIV
Phase 1
This question was added
to gather data about
motivation for misuse of
Tranquilizers and
Sedatives.
Not included in 2012
27
QFT Variable
(Testing
Outcome)
STYMOTIV
Phase
Introduced
Phase 2
QFT Instrument Wording
6. To increase or decrease the
effect(s) of some other
drug
7. Because I am “hooked” or I
have to have it
8. I used it for some other
reason
DK/REF
Initial Test Wording
effect(s) of some other drug
6
Because I am “hooked” or I
have to have it
7
I used it for some other
reason
DK/REF
Notes/Testing Results
[IF STMISCOUNT > 1 OR
(STMISCOUNT = 1 AND STY26NE 1
AND STDKREFFLAG=1)]Now think
about the last time you used
[STLASTFILL2] in any way a doctor
did not direct you to use [FILL WITH
‘them” IF RSTY42A=5. ELSE FILL
WITH “it”]
What were the reasons you used
[STLASTFILL2] that time? To select
more than one reason from the list,
press the space bar between each
number you have typed. When you
have finished, press [ENTER].
1. To help me lose weight
2. To help me
concentrate
3. To help me be alert or
stay awake
4. To help me study
5. To experiment or to
see what [IF
RSTY42A=5 THEN
“they’re” ELSE “it’s”]
like
No changes between initial testing and This question was added
to gather data about
final QFT wording.
motivation for misuse of
Stimulants.
2012 Question Wording
Not included in 2012
28
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRMOTOT
TRMOTOT
STMOTOT
SVMOTOT
Phase 2
PRYMOT1
Phase 2
QFT Instrument Wording
6. To feel good or get
high
7. To increase or
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. I used [IF RSTY42A=5
THEN “them” ELSE “it”]
for some other reason
DK/REF
NOTE: Example text from PRMOTOT
is displayed below. Equivalent text
was used for TRMOTOT, STMOTOT,
& SVMOTOT.
[IF PRYMOTIV=9] Please type in the
other reason you used
[PRLASTFILL2] that time. When you
have finished typing your answer,
press [ENTER] to go to the next
question.
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN PRYMOTIV] Which was
the main reason you used
[PRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1 To relieve physical pain
2 To relax or relieve tension
3 To experiment or to see
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF PRYMOTIV=8] Please type in the
reason you used [PRLASTFILL2] that
time. When you have finished typing
your answer, press [ENTER] to go to
the next question.
This question was added
to gather other, specify
data about motivation
for misuse of
prescription drugs.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN PRYMOTIV] Which was the
main reason you used [PRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1
To relieve physical pain
2
To relax or relieve tension
3
To experiment or to see what
This question was added
to gather data about the
primary motivation for
misuse of pain relievers.
Edits to logic were made
as a result of cognitive
testing.
Not included in 2012
29
QFT Variable
(Testing
Outcome)
Phase
Introduced
TRYMOT1
SVYMOT1
Phase 2
STYMOT1
Phase 2
QFT Instrument Wording
what it’s like
4 To feel good or get high
5 To help me with my
feelings or emotions
6 It helps with my sleep
7 To increase or decrease the
effect(s) of some other
drug
8 Because I am “hooked” or I
have to have it
9 [IF PRYMOTIV=9] The other
reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN TRYMOTIV] Which was
the main reason you used
[TRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to
see what it’s like
3. To feel good or get
high
4. To help me with my
feelings or emotions
5. It helps with my sleep
6. To increase or
decrease the effect(s)
of some other drug
7. Because I am “hooked”
or I have to have it
8. [IF PRYMOTIV=9] The
other reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
4
5
6
7
Initial Test Wording
it’s like
To feel good or get high
It helps with my sleep
To increase or decrease the
effect(s) of some other drug
Because I am “hooked” or I
have to have it
Notes/Testing Results
2012 Question Wording
DK/REF
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN TRYMOTIV] Which was the
main reason you used [TRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to see
what it’s like
3. To feel good or get high
4. To help with my sleep
5. To increase or decrease
the effect(s) of some
other drug
6. Because I am “hooked”
or I have to have it
7. I used it for some other
reason
DK/REF
This question was added
to gather data about the
primary motivation for
misuse of tranquilizers
and stimulants. Edits to
logic were made as a
result of cognitive
testing.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐9
This question was added
Not included in 2012
30
QFT Variable
(Testing
Outcome)
STY25A
(Change
tested, minor
wording
problems
found,
revisions
Phase
Introduced
Phase 1
QFT Instrument Wording
Initial Test Wording
CHOSEN IN STYMOTIV]
CHOSEN IN STYMOTIV] Which
Which was the main reason
was the main reason you
you used [STLASTFILL2] that
used [STLASTFILL2] that time?
time?
PROGRAMMER: FILL AS RESPONSE
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE
OPTIONS ONLY THOSE
CHOSEN IN STYMOTIV
1.
To help me lose weight
CHOSEN IN STYMOTIV
1. To help me lose weight 2.
To help me concentrate
2. To help me
3.
To help me be alert or stay
concentrate
awake
3. To help me be alert or
4.
To help me study
stay awake
5.
To experiment or to see what
4. To help me study
[IF RSTY42A=5 THEN “they’re”
5. To experiment or to
see what [IF
ELSE “it’s”]like
RSTY42A=5 THEN
6.
To feel good or get high
“they’re” ELSE
7.
To increase or decrease the
“it’s”]like
effect(s) of some other drug
6. To feel good or get
8.
Because I am “hooked” or I
high
have to have [IF RSTY42A=5
7. To increase or
THEN “them” ELSE “it”]
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. [IF STMOTIV=9] The
other reason I reported
[IF STMISUSE12=1] At any time in
[IF STMISUSE12=1] At any time in the
the past 12 months, did you ever
past 12 months, did you ever use a
use a needle to inject
needle to inject [STNAMEFILL] in any
[STNAMEFILL]?
way a doctor did not direct you to use
[STNUMFILL]?
Notes/Testing Results
to gather data about the
primary motivation for
misuse of sedatives. Edits
to logic were made as a
result of cognitive
testing.
Question moved from
Special Drugs to
Stimulants main module.
As a result of cognitive
testing, the wording of
the question was
streamlined.
2012 Question Wording
Not included in 2012
31
QFT Variable
(Testing
Phase
Outcome)
Introduced
QFT Instrument Wording
implemented
in QFT.)
SD01 (Change Phase 2
The last questions were about
tested, no
prescription drugs. The next
problems
question is about non‐prescription
found,
cough or cold medicines, also known
implemented
as “over‐the‐counter” medicines.
in QFT.)
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
SD02(Change Phase 2
[IF SD01 = 1] How long has it been
tested, no
since you last took one of these
problems
cough or cold medicines to get high?
found,
implemented
in QFT.)
SD05/SD15
Phase 2
Have you ever, even once, used a
(Change
needle to inject any other drug that
tested, no
was not prescribed for you?
problems
found,
implemented
in QFT.)
SD10a
Phase 2
Have you ever, even once, used a
(Change
needle to inject
tested, no
methamphetamine?
problems
found,
implemented
in QFT.)
MJMM
(Change
tested, no
problems
found,
implemented
Phase 2
Initial Test Wording
Notes/Testing Results
No changes between initial testing and Wording introducing the
final QFT wording.
over the counter
medicine questions was
tailored to redesign
conventions and
question order.
No changes between initial testing and Collected recency data
final QFT wording.
for misuse of cough and
cold medicines.
No changes between initial testing and In order to maintain
final QFT wording.
conventions of revised
misuse wording, deleted
the words, “only for the
experience or feeling it
caused.”
No changes between initial testing and To maintain consistency
final QFT wording.
across modules, and to
limit follow up questions
to data collected in the
stand alone
methamphetamine
module, removed
reference to Desoxyn
and Methedrine.
Questions about medical
[IF (MJLAST3 = 1 ‐ 2 OR MJRECDK = 1 [IF MJ01=1 OR MJREF=1] Was any of
‐ 2 OR MJRECRE = 1 ‐ 2) OR BL03=2 ] your marijuana use recommended by use of marijuana were
a doctor?
added to the redesign
Earlier, you reported using
instrument. After
marijuana in the past year. Was any
cognitive interviewing,
of your marijuana use in the past 12
the wording was tailored
months recommended by a doctor?
2012 Question Wording
The last two questions were about
prescription drugs. The next
question is about non‐prescription
cough or cold medicines, also known
as over the counter medicines.
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
Not included in 2012
Have you ever, even once, used a
needle to inject any other drug that
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Have you ever, even once, used a
needle to inject Methamphetamine,
Desoxyn, or Methedrine when it
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Not included in 2012
32
QFT Variable
(Testing
Outcome)
in QFT.)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
MJMM01
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
[IF MJMM=1] Was all of your
[IF MJMM=1] Was all of your
marijuana use in the past 12 months marijuana use recommended by a
recommended by a doctor?
doctor?
DRPR
DRTR
DRST
DRSV
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
NOTE: Example text from DRPR is
displayed below. Equivalent text was
used for DRTR, DRST, & DRSV.
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next
questions. Remember, we are only
interested in prescription pain
relievers that you used in any way a
doctor did not direct you to. Earlier
the computer recorded that in the
past 12 months you used [IF
PRMISCOUNT=1 FILL PRFILL2][IF
PRMISCOUNT>=2 FILL WITH “the
pain relievers listed below” ] in a
way a doctor did not direct you to
use [PRNUMFILL].
[IF PRMISCOUNT>=2 FILL WITH
DRUG NAMES FROM PRY01‐PRY39
BELOW. USE MULTIPLE COLUMNS
AS NEEDED. IF PRY40 = 1, ADD
"Some other prescription pain
reliever".]
The next questions refer to [IF
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested in
your use of prescription pain relievers
that were not prescribed for you or
that you used only for the experience
or feeling they caused.
Press [ENTER] to continue.
Notes/Testing Results
to 12 month users of
marijuana only.
Questions about medical
use of marijuana were
added to the redesign
instrument. After
cognitive interviewing,
the wording was tailored
to 12 month users of
marijuana only.
Wording in the
Substance Dependence
and Abuse section was
updated to reflect
revised wording
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
2012 Question Wording
Not included in 2012
IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested
in your use of prescription pain
relievers that were not prescribed
for you or that you used only for the
experience or feeling they caused.
Press [ENTER] to
continue.
33
QFT Variable
(Testing
Outcome)
Phase
Introduced
DRST05
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
Height
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase 1
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
PRY40 NE 1 AND PRMISCOUNT=1
FILL PRFILL2 as a prescription pain
reliever; IF PRY40 = 1 AND
PRMISCOUNT=1 FILL WITH “this
other prescription pain reliever”; IF
PRMISCOUNT>=2 FILL WITH “these
as prescription pain relievers”].
Press [ENTER] to continue.
[IF DRST04 = 1] Were you able to
No changes between initial testing and Wording in the Drug
keep to the limits you set, or did you final QFT wording.
Dependence and Abuse
section was updated to
often use prescription stimulants
reflect revised wording
more than you intended to?
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
HLTH04 This question asks about
HLTHNEW01
About how tall are
Height and weight
your height.
you, without shoes? First, please type questions were added as
part of the redesign in
in the number of feet, then press
[ENTER].
order to capture more
To answer in feet and inches, press
details on physical health
1. To answer in meters and
To answer using meters and
of respondents. Several
centimeters, press 2. To answer in
centimeters, press F3.
versions of these
inches only, press 3. To answer in
questions were tested in
centimeters only, press 4. Then
press [ENTER] to continue.
______feet [RANGE: 2‐8]
order to make it easier
DK/REF
for respondents to
1
I would rather answer in
answer in either metric
feet and inches
HLTHNEW01a (IF HLTHNEW01 NE
or US measurements.
2
I would rather answer in
DK/RF) Next, please type in the
Edits were also made so
meters and centimeters
number of inches and then press
that respondents could
3
I would rather answer only [ENTER].
answer only in inches or
centimeters.
in inches
4
I would rather answer only ______inches [RANGE: 0‐11]
in centimeters
DK/REF
DK/REF
2012 Question Wording
[IF DRPR04 = 1] Were you able to
keep to the limits you set, or did you
often use prescription pain relievers
more than you intended to?
Not included in 2012
34
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
HLTH05 [IF HLTH04=1] About how
tall are you, without shoes? First,
please type in the number of feet,
then press [ENTER].
______feet [RANGE: 0‐8]
DK/REF
HLTH06 [IF HLTH04 = 1 OR 3 AND
HLTH05 NE DK/RF] Please type in
the number of inches and then press
[ENTER].
______inch(es) [RANGE: 0‐110]
DK/REF
HLTH07 [IF HLTH04=2] About how
tall are you, without shoes? First,
please type in the number of
meters, then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTH08 [IF HLTH04 = 2 OR 4AND
HLTH07 NE DK/REF] Please type in
the number of centimeters and then
press [ENTER].
______centimeter(s) [RANGE:0‐300]
DK/REF
Initial Test Wording
PROGRAMMER: PLEASE DISPLAY
HLTHNEW01 AND HLTHNEW01a ON
THE SAME SCREEN.
HLTHNEW01b (IF HLTHNEW01=DK)
You may also report your height using
meters and centimeters. About how
tall are you, without shoes? First,
please type in the number of meters,
then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTHNEW01c [IF HLTHNEW01b NE
BLANK] Next, please type in the
number of centimeters and then press
[ENTER].
______centimeters [RANGE:0‐275]
DK/REF
PROGRAMMER: PLEASE DISPLAY
THESE HLTHNEW01b AND
HLTHNEW01c ON THE SAME SCREEN.
Notes/Testing Results
2012 Question Wording
35
QFT Variable
(Testing
Outcome)
Weight
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase
Introduced
Phase 1
QFT Instrument Wording
HLTH09 The next question asks
about your weight.
To answer in pounds, press 1. To
answer in kilograms, press 2. Then
press [ENTER] to continue.
1
I would rather answer in
pounds
2
I would rather answer in
kilograms
DK/REF
HLTH10 [IF HLTH09=1 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of pounds and then press [ENTER].
________pounds [RANGE: 50‐550]
DK/REF
HLTH12 [IF HLTH11=2 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of kilograms and then press
[ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTH13 [IF HLTH02=1 AND
HLTH09=1] About how much did
you weigh before you got pregnant?
Please type in the number of pounds
and then press [ENTER].
Initial Test Wording
HLTHNEW02
(IF HLTH01 NE 1)
About how much do you weigh?
Please type in the number of pounds
and then press [ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02A (IF HLTHNEW02=DK)
You may also report your weight using
kilograms. About how much do you
weigh? Please type in the number of
kilograms and then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTHNEW02p (IF HLTH01 = 1) About
how much did you weigh before you
got pregnant? Please type in the
number of pounds and then press
[ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02pp (IF HLTHNEW02p
=DK) You may also report your weight
using kilograms. About how much did
you weigh before you got pregnant?
Please type in the number of
kilograms and then press [ENTER].
Notes/Testing Results
2012 Question Wording
Height and weight
Not included in 2012
questions were added as
part of the redesign in
order to capture more
details on physical health
of respondents. Several
versions of these
questions were tested in
order to make it easier
for respondents to
answer in either metric
or US measurements.
36
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTHNEW03
(HLTH19)
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 1
HLTH22
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QFT Instrument Wording
________pounds [RANGE: 50‐550]
DK/REF
HLTH14 [IF HLTH02=1 AND HLTH09
=2] About how much did you weigh
before you got pregnant? Please
type in the number of kilograms and
then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
During the past 12 months, how
many times have you visited a
doctor, nurse, physician assistant or
nurse practitioner about your own
health at a doctor’s office, a clinic,
or some other place?
[((IF HLTH16>=1 AND (HLTH17=1 OR
HLTH19>=1)) OR (HLTH17=1 AND
(HLTH16>=1 OR HLTH19>=1 OR
HLTH19DK>=1)) OR ((HLTH19>=1 OR
HLTH19DK>=1) AND (HLTH16>=1 OR
HLTH17=1))) AND (ALLAST3=1 OR 2
OR ALRECDK=1 OR 2 OR ALRECRE=1
OR 2)] Please think about all of the
talks you have had with a doctor or
other health care professional
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussions you may
Initial Test Wording
Notes/Testing Results
2012 Question Wording
_______ kilograms [RANGE: 22‐275]
DK/REF
DK/REF
During the past 12 months, how many
times have you visited a doctor, nurse,
physician assistant or other health
care professional about your own
health at a doctor’s office, a clinic, or
some other place?
Do not include times you were
hospitalized overnight, visits to
hospital emergency rooms, home
visits by a health care professional,
dental visits, or telephone calls.
No changes between initial testing and
final QFT wording.
Questions about health
Not included in 2012
provider visits were
added as part of the
redesign. Instructions
about how to count visits
were deemed
unnecessary or confusing
and removed.
Not included in 2012
Questions were tested
about discussing alcohol
& drug use with a doctor.
No changed were made
as a result of testing.
37
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTH26othr Phase 2
(Change
tested, no
problems
found,
implemented
in QFT.)
HLTHNEW11 Phase 1
(HLTH29)
(Change
tested, no
problems
found,
QFT Instrument Wording
Initial Test Wording
have had in person with a doctor or
other health professional about your
alcohol use.
[((IF HLTH16>=1 AND HLTH17 NE 1
AND HLTH19<1 AND HLTH19DK<1)
OR (HLTH17 =1 AND HLTH16 = 0
AND HLTH19<1 AND HLTH19DK<1)
OR ((HLTH19 >=1 OR HLTH19DK>=1)
AND HLTH16 = 0 AND HLTH17 NE 1))
AND (ALLAST3=1 OR 2 OR
ALRECDK=1 OR 2 OR ALRECRE=1 OR
2)] Please think about [VISITFILL]
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussion you may
have had in person with a doctor or
other health professional about your
alcohol use.
To select more than one statement,
press the space bar between each
number you type. When you have
finished, press [ENTER].
No change.
(IF HLTH26=30) Please tell me which
other kind of cancer you had.
[IF HLTH25=9 AND HLTH28 AND
HLTH28a AND HLTH28b AND
HLTH28c AND HLTH28c AND
HLTH28d AND HLTH28e AND
HLTH28f AND HLTH28g AND
HLTH28h AND HLTH28i AND
(IF HLTHNEW08=9) Did you have
cancer during the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
38
QFT Variable
(Testing
Outcome)
implemented
in QFT.)
Phase
Introduced
HLTHNEW12a Phase 1
(HLTH31)
(Change
tested, no
problems
found,
implemented
in QFT.)
Disability
NA
Items (QD55‐
QD61) Not
tested
QFT Instrument Wording
HLTH28j AND HLTH28k AND
HLTH28l AND HLTH28m AND
HLTH28n AND HLTH28o AND
HLTH28p AND HLTH28q AND
HLTH28r AND HLTH28s AND
HLTH28t AND HLTH28u AND
HLTH28v AND HLTH28w AND
HLTH28x AND HLTH28y AND
HLTH28z AND HLTH28aa AND
HLTH28bb AND HLTH28cc NE
CALCAGE] Did you have cancer
during the past 12 months?
[IF HLTH25=1 AND HLTH30 NE
CALCAGE] Did you have any kind of
heart condition or heart disease in
the past 12 months?
QD55 How well do you speak
English?
QD56 Are you deaf or do you
have serious difficulty hearing?
QD57 Are you blind or do you
have serious difficulty seeing, even
when wearing glasses?
QD58 Because of a physical,
mental or emotional condition, do
you have serious difficulty
concentrating, remembering, or
making decisions?
QD59 Do you have serious
Initial Test Wording
(IF HLTHNEW08=1) Did you have any
kind of heart condition or heart
disease in the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about health
conditions were added.
Only logistical/spec
changes were made as a
result of the cognitive
testing.
Not included in 2012
These items were added
in response to feedback
received about the
survey.
Not included in 2012
39
QFT Variable
(Testing
Outcome)
Phase
Introduced
QD17 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
difficulty walking or climbing stairs?
QD60 Do you have difficulty
dressing or bathing?
QD61 [IF CURNTAGE >14]
Because of a physical, mental or
emotional condition, do you have
difficulty doing errands alone such
as visiting a doctors’ office or
shopping?
No changes between initial testing and This question was moved
The next questions are about
final QFT wording.
to ACASI. No changes
school. Are you now attending or
were made as a result of
are you currently enrolled in school?
testing.
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
QD26 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
No changes between initial testing and This question was moved
[IF CURNTAGE = 15 OR OLDER] The
final QFT wording.
to ACASI.
next questions are about working.
Explanatory/example
Did you work at a job or business at
text was added as
any time last week? By last week, I
optional text available
mean the week beginning on
with pressing the F2 key.
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
No changes were made
as a result of testing.
Press F2 to hear more information
about what types of work to include.
Military
QD10d For this question, please
NA
These items were added
2012 Question Wording
The next questions are about
school. Are you now attending or
are you currently enrolled in school?
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
1
YES
2
NO
DK/REF
[IF CURNTAGE = 15 OR OLDER] The
next questions are about working.
Did you work at a job or business at
any time last week? By last week, I
mean the week beginning on
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
1
YES
2
NO
DK/REF
Not included in 2012
40
QFT Variable
(Testing
Outcome)
Family Items
(QD10d &
QD10e) Not
tested
Phase
Introduced
PROXYINT
NA
NOPROX
(HINSINT)
(revisions
Phase 2
QFT Instrument Wording
include all persons in your
immediate family, whether or not
they live with you. Is anyone in your
immediate family currently serving
in the United States military? Press
F2 to see and hear definitions of
“immediate family” and “military.”
QD10e Which member or members
of your immediate family are
currently in the United States
military? Press F2 to see and hear
definitions of “immediate family.”
1 My spouse
2 Unmarried partner
3 My mother
4 My father
5 My son or sons
6 My daughter or
daughters
7 My brother or brothers
8 My sister or sisters
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you [IF FAMILY MEMBERS IN
ROSTER FILL “and your family”]
receive. This information will help in
planning health care services and
finding ways to lower costs of care.
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR
HASJOIN= 2 OR DK/REF] I’m going to
Initial Test Wording
Notes/Testing Results
in response to feedback
received about the
survey.
2012 Question Wording
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you receive. (This information will
help in planning health care services
and finding ways to lower costs of
care.)
New language was added Not included in 2012
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR HASJOIN = to assist the interviewer
with transitioning back to
2 OR DK/REF] I’m going to give the
Revised wording after
Phase 2 testing to
encourage nominations
of proxies when
necessary.
41
QFT Variable
(Testing
Outcome)
tested,
problems
found,
changes
implemented
in QFT)
QHI03
(Wording
revised for
accuracy
after testing,
implemented
in QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
give the computer back to you so
that you can complete the last part
of the interview on your own.
[IF (QP03 = 2 OR DK/REF) OR (QP04
= 2 OR DK/REF) OR (HASJOIN = 2 OR
DK/REF) ADD: Since your [QP02 FILL]
is not available, please answer these
next questions the best you can.]
[IF NEWPROX = 2 AND HASJOIN=1]
I’m going to give the computer to
you so that you can complete the
last part of the interview on your
own.
Please put on the headphones.
When you are ready, let me know.
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
There are certain programs that
cover active duty and retired career
military personnel and their
dependents and survivors and also
disabled veterans and their
dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to see and hear more
information about these programs.
CHAMPUS stands for Civilian
Health and Medical Program of
Initial Test Wording
computer back to you so that you can
complete the last part of the interview
on your own. When you are ready, let
me know.
[IF QP03 = 2 OR DK/REF OR QP04 = 2
OR DK/REF OR HASJOIN = 2 OR DK/REF
ADD THIS TEXT PRIOR TO THE
QUESTION: Since your [QP02 FILL] is
not available, please answer these
next questions the best you can.]
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
Notes/Testing Results
ACASI for the final
questions. Language was
tailored to situations
where a proxy might be
required to answer the
income series. As a
result of testing,
revisions were made to
make the proxy
transition happen more
smoothly.
2012 Question Wording
There are certain programs that cover
active duty and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to hear more information
about these programs.
CHAMPUS stands for Comprehensive
Health and Medical Plan for the
Wording was revised for
ACASI administration,
and later to accurately
note program names.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
(These programs cover active duty
and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.)
1
YES
2
NO
DK/REF
INTERVIEWER NOTE:
42
QFT Variable
(Testing
Outcome)
INTROINC
(revisions
tested, minor
problems
found,
changes
made for
QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
the Uniformed Services. It
provides health care in private
facilities for dependents of
military personnel on active duty
or retired for reasons other than
disability. In some areas, this
may be known as TRICARE.
Initial Test Wording
Uniformed Services. It provides health
care in private facilities for
dependents of military personnel on
active duty or retired for reasons
other than disability. In some areas,
this may be known as TRICARE.
CHAMPVA stands for Comprehensive
Health and Medical Plan of the
CHAMPVA stands for Civilian
Veterans Administration. It provides
Health and Medical Program of
health care for the spouse,
the Department of Veterans
dependents, or survivors of a veteran
Affairs. It provides health care
who has a total, permanent service‐
for the spouse, dependents, or
survivors of a veteran who has a connected disability.
total, permanent service‐
The VA provides medical assistance to
connected disability.
veterans of the Armed Forces,
particularly those with service‐
The VA provides medical
connected ailments.
assistance to veterans of the
Armed Forces, particularly those
with service‐connected ailments. Military health care refers to health
care available to active duty personnel
and their dependents.
Military health care refers to
health care available to active
duty personnel and their
dependents.
[IF NO FAMILY MEMBERS IN ROSTER]
[IF NO FAMILY MEMBERS IN
These next questions are about the
ROSTER] These next questions are
kinds and amounts of income that you
about the kinds and amounts of
receive.
income that you receive.
[IF ONE FAMILY MEMBER IN ROSTER [IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] These next
AND HASJOIN NE 1 ] These next
questions are about the kinds and
questions are about the kinds and
amounts of income that you and
amounts of income that you and your
your [FAMILY RELATIONSHIP FILL]
[FAMILY RELATIONSHIP FILL] receive.
receive.
Notes/Testing Results
2012 Question Wording
CHAMPUS stands for
Comprehensive Health and Medical
Plan for the Uniformed Services. It
provides health care in private
facilities for dependents of military
personnel on active duty or retired
for reasons other than disability. In
some areas, this may be known as
TRICARE.
CHAMPVA stands for
Comprehensive Health and Medical
Plan of the Veterans Administration.
It provides health care for the
spouse, dependents, or survivors of
a veteran who has a total,
permanent service‐connected
disability.
Military health care refers to health
care available to active duty
personnel and their dependents; in
addition, the VA provides medical
assistance to veterans of the Armed
Forces, particularly those with
service‐connected ailments.
Wording was revised for [IF NO FAMILY MEMBERS IN
ACASI administration. As ROSTER] These next questions are
a result of the cognitive
about the kinds and amounts of
income that you receive.
testing, changes were
made to make the
[IF ONE FAMILY MEMBER IN ROSTER
process go more
AND HASJOIN NE 1 ] These next
smoothly.
questions are about the kinds and
amounts of income that you and
your [FAMILY RELATIONSHIP FILL]
receive.
43
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you and your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, you and [IF
QD01=5 FILL his, QD01 = 9 FILL
her][FAMILY RELATIONSHIP FILLS]
living here – receive.
[PROGRAMMER NOTE: THE PROXY
SHOULD NOT APPEAR IN [FAMILY
RELATIONSHIP FILLS]. ALSO, USE
‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF HASJOIN NE 1] These questions
refer to the calendar year [CURRENT
YEAR ‐ 1] rather than to the past 12
Initial Test Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
you receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] These
next questions are about the kinds
and amounts of income that you, your
[FAMILY RELATIONSHIP FILLS] living
here receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN=1] These
next questions are about the kinds
and amounts of income that [SAMPLE
MEMBER] and [IF QD01=5 FILL his,
QD01 = 9 FILL her] family – that is,
your [SAMPLE MEMBER POSS]
[FAMILY RELATIONSHIP FILLS] living
here – receive.
These questions refer to the calendar
year [CURRENT YEAR ‐ 1] rather than
to the past 12 months that were
referred to in some earlier questions.
The calendar year [CURRENT YEAR ‐ 1]
would be from January 1st, [CURRENT
YEAR ‐ 1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
Notes/Testing Results
2012 Question Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you, your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, your [SAMPLE
MEMBER POSS] [FAMILY
RELATIONSHIP FILLS] living here –
receive.
These questions refer to the
calendar year [CURRENT YEAR ‐ 1]
rather than to the past 12 months
that were referred to in some earlier
questions. The calendar year
[CURRENT YEAR ‐ 1] would be from
January 1st, [CURRENT YEAR ‐ 1],
through December 31st, [CURRENT
YEAR ‐ 1].
PRESS [ENTER] TO CONTINUE
44
QFT Variable
(Testing
Outcome)
QI12AN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
months that were referred to in
some earlier questions. The calendar
year [CURRENT YEAR ‐ 1] would be
from January 1st, [CURRENT YEAR ‐
1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Any other kind of non‐
monetary welfare or public
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family member
living here receive any type of welfare
or public assistance?
# OF MONTHS RECEIVED ASSISTANCE:
[RANGE: 1 ‐ 12]
DK/REF
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
45
QFT Variable
(Testing
Outcome)
QI12BN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
assistance
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance, not including food stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance,
not including food stamps?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
46
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTRTINN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QI21B
(Change
tested, no
problems
Phase 2
QFT Instrument Wording
Any other kind of non‐
monetary welfare or public
assistance
Below is a list of some other sources
of income. When I ask you the next
questions, please consider these as
well as the other sources asked
about in earlier questions.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
Press [Enter] to continue.
[IF QI20N = 1 OR QI20NREF = 1] Of
these income groups, which
category best represents [SAMPLE
MEMBER POSS] total personal
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Below is a list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we just
talked about.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
Wording was revised for
ACASI administration.
No changes were made
as a result of cognitive
testing.
HAND R SHOWCARD 16a. Here is a
list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we
just talked about.
INTERVIEWER: PLEASE READ THIS
ALOUD TO THE RESPONDENT AS
HE/SHE FOLLOWS ALONG ON THE
SHOWCARD.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
[IF QI20N = 1] Of these income groups,
which category best represents
[SAMPLE MEMBER POSS] total
personal income during [CURRENT
Income categories were
revised to adjust for
inflation and the
question was revised for
[IF QI20N = 1] ENTER NUMBER THAT
BEST REPRESENTS (R’S/SAMPLE
MEMBER’S) TOTAL PERSONAL
INCOME DURING [CURRENT YEAR ‐
47
QFT Variable
(Testing
Outcome)
found,
implemented
in QFT.)
INTROFI1
(Change
tested, minor
problems
found,
changes
made for
QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
income during [CURRENT YEAR ‐ 1]?
1
$20,000 ‐ $24,999
2
$25,000 ‐ $29,999
3
$30,000 ‐ $34,999
4
$35,000 ‐ $39,999
5
$40,000 ‐ $44,999
6
$45,000 ‐ $49,999
7
$50,000 ‐ $74,999
8
$75,000 ‐ $99,999
9
$100,000 ‐ $149,999
10
$150,000 or more
DK/REF
Initial Test Wording
YEAR ‐ 1]?
21
$20,000 ‐ $24,999
22
$25,000 ‐ $29,999
$30,000 ‐ $34,999
23
24
$35,000 ‐ $39,999
25
$40,000 ‐ $44,999
26
$45,000 ‐ $49,999
27
$50,000 ‐ $74,999
28
$75,000 ‐ $99,999
29
$100,000 ‐ $149,999
30
$150,000 or more
DK/REF
[IF MORE THAN ONE FAMILY MEMBER
[IF MORE THAN ONE FAMILY
IN ROSTER AND IF QI21B NE 29]
MEMBER IN ROSTER AND IF QI21B
NE 30]
Next, we would like to know about the
total family income from all sources
Next, we would like to know about
during [CURRENT YEAR ‐ 1] before
the total family income from all
sources during [CURRENT YEAR ‐ 1] taxes and other deductions.
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN NE 1 ] We would like
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
you to combine everyone’s income – that is, yours and that of your [FAMILY
that is, yours and that of your
RELATIONSHIP FILL].
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN=1] We would like you to
AND HASJOIN=1] We would like you combine everyone’s income – that is,
to combine everyone’s income –
[SAMPLE MEMBER POSS] and yours.
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] We
[IF AT LEAST TWO FAMILY
would like you to combine everyone’s
MEMBERS IN ROSTER AND HASJOIN income – that is, yours and that of
Notes/Testing Results
ACASI administration. No
changes were made as a
result of cognitive
testing.
2012 Question Wording
1].
21
22
23
24
25
26
27
28
29
$20,000 ‐ $24,999
$25,000 ‐ $29,999
$30,000 ‐ $34,999
$35,000 ‐ $39,999
$40,000 ‐ $44,999
$45,000 ‐ $49,999
$50,000 ‐ $74,999
$75,000 ‐ $99,999
$100,000 OR MORE
DK/REF
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration.
Edits were made to
family relationship fills to
result in a more natural
sentence structure.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
48
QFT Variable
(Testing
Outcome)
QI23A
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is, yours,
[SAMPLE MEMBER POSS] and that of
[IF QD01 = 5 FILL his, QD = 9 FILL
her] [FAMILY RELATIONSHIP FILLS]
living here. [PROGRAMMER NOTE:
THE PROXY SHOULD NOT APPEAR IN
[FAMILY RELATIONSHIP FILLS]. ALSO,
USE ‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
Initial Test Wording
your [FAMILY RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN = 1] We
would like you to combine everyone’s
income — that is, [SAMPLE MEMBER
POSS] and that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here.
Please include all of the sources of
income that we just talked about.
Notes/Testing Results
2012 Question Wording
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, yours and that of your [FAMILY
RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
testing.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
49
QFT Variable
(Testing
Outcome)
QI23B
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
YEAR – 1] – that is, your [SAMPLE
MEMBER POSS] and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 = 9
FILL her] [FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
Initial Test Wording
that is, your [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
Notes/Testing Results
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
2012 Question Wording
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
Collapsed from multiple 2012
variables due to change in modes.
50
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, [SAMPLE
MEMBER POSS] and yours?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your [FAMILY
RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 =9
FILL her][FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
Initial Test Wording
Notes/Testing Results
income during [CURRENT YEAR – 1] –
testing.
that is, yours and that of your [FAMILY
RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, [SAMPLE MEMBER POSS] and
yours?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[FAMILY RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
2012 Question Wording
51
QFT Variable
(Testing
Outcome)
Phase
Introduced
CELL1
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
CELL2
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
(MJE01 –
MJE70)
(entire
existing
module
dropped)
LU22 – LU26
(existing
question
dropped)
None
None
QFT Instrument Wording
Initial Test Wording
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Is there at least one telephone at
Is there at least one telephone at this
this address that is not a cell phone? address that is not a cell phone?
Do you or anyone at this address
have a working cell phone?
Do you or anyone at this address have
a working cell phone?
Notes/Testing Results
2012 Question Wording
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
The last question has to do with
telephones in your household. How
many different telephone numbers
do you have in this household?
Please don’t include cellular phones
in your answer. Also, don’t count
business numbers or extensions
with the same number.
INTERVIEWER NOTE:
Do not include phone lines that are
used only for fax machines and/or
Internet access.
Not included in 2012
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
2012 instrument items or modules removed from consideration for the QFT
Module dropped from QFT.
Module dropped from QFT.
These questions were
outdated in terms of the
current marijuana
market. They were
removed to make room
for other important
topics.
Item dropped from QFT.
Item dropped from QFT.
Dropped “which came
first” questions from the
Prior Substance Use
module to make room for
Market Information for Marijuana
Module
LU22
Earlier, the computer
recorded that you started using
alcohol and cigarettes when you
were [AFUFILL] years old. Which
52
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
SEN04/YE04
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
CA12‐CA14d
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Notes/Testing Results
additions to the
Prescription Drug
modules.
This question had little
analytic value and was
dropped to make room
for more important
topics.
The Alcohol module now
measures gender specific
binge drinking
prevalence, so the ‘4 or
more drinks’ questions
were deleted.
2012 Question Wording
did you use first?
LU23 Earlier, the computer
recorded that you started using
marijuana and cigarettes when you
were [AFUFILL] years old. Which
did you use first?
LU24 Earlier, the computer
recorded that you started using
alcohol and marijuana when you
were [AFUFILL] years old. Which
did you use first?
LU25 Earlier, the computer
recorded that you started using
alcohol, cigarettes and marijuana
when you were [AFUFILL] years old.
Which did you use first?
LU26 After first using [FILL
LU25], which of these did you use
next?
How many times have you moved in
the past 5 years?
CA12 Have you ever had 4 or more
drinks on the same occasion?
CA13 During the past 30 days, that
is, since [DATEFILL], on how many
days did you have 4 or more drinks
on the same occasion?
CA14 How old were you the first
53
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
Industry and
Occupation
Questions –
INOC01 –
INOC06
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these questions
due to low levels of
analytic utility and
difficulties administering
these questions in ACASI
Household
Roster –
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these items due
to low analytic utility
2012 Question Wording
time you had 4 or more drinks on
the same occasion?
INOC01
[IF QD26 =
1 OR QD27 = 1] In what kind of
business or industry do you work?
That is, what product is made or
what service is offered?
INOC02 [IF QD26=1 OR QD27=1
AND INOC01 NE DK/REF] HAND R
SHOWCARD 7. Which of these
categories best describes the
business or industry in which you
work?
INOC02M What do they make?
INOC02T What do they sell?
INOC03Please describe the business
or industry in which you work.
INOC04 What kind of work do you
do? That is, what is your
occupation?
INOC05 What are your most
important activities or duties in that
job?
INOC06 Which of these categories
best describes the business in which
you work?
Is (s)he your biological, step‐,
adoptive, or foster [FILL
54
QFT Variable
(Testing
Outcome)
FTHRTYPE,
MTHRTYPE,
SONTYPE,
DAUTYPE
(existing
question
dropped)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
RELATIONSHIP]?
55
Appendix E
NSDUH Prescription Drug Redesign Cognitive Interviewing
Phase 2 Round 2 Results
NSDUH Questionnaire Redesign Cognitive Interviewing Round 2.2 Results
February 29, 2011
Respondent Characteristics
Twenty participants were interviewed during the second round of Phase 2 Questionnaire
Redesign Cognitive Interviewing. Summary information about the participants is presented in the
table below. This table shows that the sample was comprised largely of adults, females, and
those who had significant experience with drugs.
NSDUH Cognitive Interviewing Round 2.2
Participant Information Summary (n = 20)
n
Adults
14
Adult mean age = 39
Adolescents
6
Adolescent mean age = 14
Females
12
Enrolled in treatment program
5
(currently or within past 12 months)
High use participants
11
All participants were recruited from the general population using online ads that were placed in
Chicago, IL; Washington, D.C., and Research Triangle Park, NC. However, since most users
interviewed in the first round were casual users, the second round targeted heavy drug users and
individuals who had recently received treatment. Of the 14 adult interviews, 5 were currently
enrolled or had been enrolled in a treatment program in the past 12 months and 11 were
categorized as ‘high use’ participants. High use was defined as having used multiple prescription
drugs.
As in the previous round, Round 2.2 tested the second transition from interviewer to ACASI
administration for proxy responses to health insurance and income items. Feedback from parents
and the interviewers about the transitions was provided. All proxy respondents were able to
successfully transition and answer proxy questions. Findings about the process are provided
below.
Some questions were tested for the first time in Round 2.2. These questions asked respondents
about the color of the pack of cigarettes they smoked and about their motivations for misusing
prescription drugs. By recruiting ‘high use’ participants we were able to thoroughly test these
new items.
Specific feedback on individual items is provided below.
2
Veteran Questions
QD10c – Recruitment in Round 2.2 did not focus on Veterans as heavily compared to Round 2.1.
Therefore, only one participant was routed to this question. She correctly defined a combat zone.
She initially indicated that a combat zone could be a “zone even in Peacetime” but clarified that
it meant “places where you are taking on fire.”
Front End Demographics
QD11 – Participants were generally able to answer this question with no trouble. Two
participants initially failed to understand that the question asked about the highest grade finished,
but ultimately they realized the meaning of the question and changed their answers to the correct
one. One participant had trouble with the numeric categories. She answered “12” but then
changed her answer to “15.” When the interviewer probed to learn more about the change from
not finishing high school to taking some college credits the participant realized she should have
selected “13” to indicate earning a high school diploma. A scripted probe lead to one final
noteworthy comment about this question. One participant had taken credit courses during her
service in the military which she wanted to report. Since no option was available for having
taken military training/courses she realized her best option was to answer “13-regular high
school diploma.”
Recommendation: This question asks about the highest grade completed. Response categories
are generally the same as the number of the grade. However, the form of category 12 is markedly
different. If a respondent answers ‘12’ they are indicating that they did not receive a high school
diploma. Respondents who do not thoroughly review the text accompanying the response options
may be inclined to answer ‘12’ to indicate they have a high school diploma. Therefore, we
should consider changing the order of the response options so that this inclination will not invite
measurement error. An alternative would be to add a question about receiving a high school
diploma if the respondent answers 12.
Tobacco
CGCOLOR – Eight of the participants reported smoking cigarettes in the past 30 days. Of these,
four considered the question to be “self-explanatory” and found it easy to recall the color of the
pack. The other half of participants, however, noted complicating issues with the question. Two
found the question somewhat confusing because packs of cigarettes have more than one color on
the packaging. One of these participants noted that he wasn’t sure whether he should or could
select more than one color or if he should “just pick the main color.” A similar comment was
made by another participant who said this question made her think of the two different colors on
the packaging—the background color (black) and the color of the text (blue). One participant,
who previously indicated the question was self-explanatory, said that his brand of cigarettes
3
(Newport) only comes in green packaging. Upon probing he elaborated that the shade of the pack
indicates the strength of the cigarette—lighter colors indicate lighter and darker colors indicate
fuller flavors. Finally, one other participant was unable to answer the question because he had
been given a single cigarette from someone else and wasn’t sure what color the pack was.
Respondents did not necessarily understand that this question was asking about the strength of
the cigarette and its association with color.
Recommendation: Consider whether language could be added to clarify what we mean by
“color” in the question. We should continue to attempt to reach the tobacco consultant about
these questions.
CGMENTH1 – One participant volunteered that this question was difficult for her to answer.
She smokes Camel Crush cigarettes, which allow the smoker to choose between regular and
menthol flavors. The cigarettes are by default regular flavor. By squeezing a point on the
cigarette a smoker can activate a menthol flavoring. The participant sometimes smokes them
only as non-menthol, other times she smokes them only as menthol, and occasionally she
switches from non-menthol to menthol at some midpoint.
CG34 – One participant indicated this question was difficult for him to answer. He had only used
cigars to make blunts, but had never smoked tobacco cigars. He noted this prior to answering the
question. The participant indicated he would answer “yes” to this question because it does not
specify what the cigar had in it.
MJ01 – One participant indicated this question was difficult to answer because she was not sure
what was meant by “used.” She pointed out that the alcohol questions were very specific about
what counted as drinking (a sip or two does not count), but we were less specific about
marijuana. She had taken one toke of a marijuana cigarette in her lifetime and was not sure that
counted as “used.”
Hallucinogens
LS01i – Six participants had heard of Ketamine. There was variety in the places people had
heard about Ketamine, what participants thought the drug was intended for, and the effects the
drug has on a user. Several participants indicated they thought it was a prescription drug for
animals. One thought it was the “date rape drug.” One participant indicated she had heard about
it from kids in her neighborhood and another had heard about it on television. Only one
participant had tried ketamine. He used at a rave “back when he partied with some white boys.”
He said it is “like meth but it’s for people who don’t want to do meth” and that it “makes you
feel like [expletive] in the morning.”
LS01j – One participant had heard of all three, one had heard of AMT and Foxy, one had heard
of DMT only, and one had heard of Foxy only. In addition, one participant reported using DMT
with the same group with which he had used Ketamine. He said it had a similar effect as
4
marijuana. He decided to not use DMT anymore and to instead “just use weed” because he got
the same effect from it and marijuana was from a plant.
LS01k –Six participants had heard of salvia. One had heard of it on television, one from a movie,
and four did not specify where they had heard of it. Two of the participants said saliva is similar
to marijuana, while one said that when you smoke it you “lose your mind for 20 minutes.” Two
indicated it was legal and one said he thought you could buy it at smoke shops.
Inhalants Screener
IN01ii – Findings on this item were similar to those in Phase 2.1 interviews. None of the
participants reported use of computer keyboard cleaner just to get high. Fewer than half of the
participants knew about the spray and of these only two made comments which reveal they
understood it could be used to get high. Seven participants had heard the term “spray computer
cleaner” or “compressed gas duster” before. One participant said he had heard it called by the
brand name “Dust Off.” And, one participant thought that computer keyboard cleaner might be
called “alcohol” perhaps suggesting the participant thinks that is a main ingredient.
Methamphetamine Module
ME01 – Just one participant in this round of interviews reported lifetime use of
methamphetamine. It was during a time when a close relative of the participant was diagnosed
with cancer. Someone the participant knew had experience with methamphetamine and
suggested it could help with relaxation. The participant reported no issues with the module.
Generic Pills
Each of the participants characterized generic drugs as a less expensive version of a brand name
drug. Three participants made comments about the quality of generics compared to name brand
drugs, two of which questioned the effectiveness of generics compared to name brand drugs.
Not All Forms
There continue to be problems associated with the statement that not all forms of a drug may be
shown on a screen. Participants in this round clearly thought this statement referred to how pills
look, but did not seem to understand that forms also included other modes of drug
administration—i.e., injectable drugs. For example, many of the participants indicated that the
statement meant that generic alternatives might not be presented when the brand name drug was
pictured, that not all shapes and/or colors would be shown, and that the text imprinted on the pill
might look different than what is shown. None of the participants mentioned the possibility that
drugs could be something other than a pill, and as mentioned below two participants noted
trouble with answering PR06 because morphine is often administered through an IV.
5
Recommendation: Discuss modifying the language to clarify that “not all forms” means both
the color/shape and mode of administration. This language has been tested and modified over
many rounds of interviewing, so we should review previous revisions and findings when
exploring whether to revise it further. See also the recommendation following PR06.
Pain Relievers
PR01 – It was evident in this round of interviewing that there is confusion about what kind of use
these questions are about. Four participants thought we were interested in any kind of use (use or
misuse), one thought we were only interested in misuse, and seven thought the question was
asking only about prescribed use. Upon probing one participant paged back to INTROPR and
pointed out that it only talks about “use of prescription pain relievers.” To him this suggested he
should only think about use that is prescribed.
Recommendation: Consider revising the following sentence: “These next questions are about
the use of prescription pain relievers” to say “These next questions are about any use of
prescription pain relievers.” Bolding ‘any’ may reinforce that we are measuring both use and
misuse. This probe was added to the instrument to address concerns that respondents would think
that we were only measuring illicit use of prescriptions, because most of the previous substances
that were asked about are illicit.
PR06 – Two participants had difficulty understanding that non-pill forms counted in this
question, which asks about morphine.. One asked whether “injections counted” and the other had
had morphine through an IV drip and initially failed to report it.
Recommendation: Consider adding a reminder on this screen that not all forms of the drugs
may be shown on the screen, because morphine is very likely to be administered in liquid form.
PRY42B – One participant volunteered that she had a problem with only being allowed to select
one way for the last time she used a prescription pain reliever. She said that she got the drug in
multiple ways. By forcing her select one way the respondent said we were forcing her to provide
a socially desirable answer, because she would select the way that did not make her look like a
drug addict.
Recalling age of first misuse
Participants used landmark events to recall dates of first use. For example, one participant
recalled first use of Xanax by recalling the group of people he was “hanging around with” during
that period of his life and also recalled his approximate age when spending time with this group.
Three other participants recalled first use because it was at a time of stress associated with life
changes, or medical events they or their family members experienced.
6
Definitions of misuse
Participants had no problems making the distinction between use and misuse of prescription
drugs. Misuse was correctly reported. One respondent reported that she sold her prescription
drugs, which should be considered misuse. Upon probing she indicated that a doctor did not tell
her to sell her pills.
Ways in which participants misused prescription drugs
Only one participant mentioned misusing a prescription in a way different than defined in the
section introductions. She indicated she used less of the drug than prescribed. She believed this
way fit in the “in any other way a doctor did not direct” category. Most other participants
indicated they had used the drug without a prescription. However, several made a point to say at
different junctures that they “knew” how a doctor would prescribe it and their use was consistent
with that “knowledge.”
Motivations for misuse
Participants were generally able to answer questions about the reasons for use the last time they
had used a drug in a way a doctor had not prescribed, and in a similar manner most had no
problem identifying the main reason for this misuse. A few issues were noted, however.
PRYMOTIV – Of the participants routed to this item, three mentioned either an issue with or
problem in answering questions about the reasons for misuse. One participant had previously
indicated she had used a prescription in a way a doctor had not directed her to by using less than
prescribed. She noted that there was no response alternative that fit her situation. One respondent
noted that he took the pills to escape his emotions, and inidicated that a response option about
dealing with emotions should be included.
PRMOTOT – When one participant selected “some other reason” for PRYMOTIV and was
presented PRMOTOT he was unclear the question was asking him to specify the “other reason.”
Recommendation: Revise the wording to say, Please type in the other readon you used [RX
DRUG] that time. This revision inserts “other” into the sentence.
PRMOT1 – Participants did not display or report any problems answering this item. However,
one interviewer noted that currently we do not allow “some other reason” from PRMOTOT to be
selected as the main reason in PRMOT1.
Recommendation: Discuss including “some other reason.” This will require developing an
alternative audio fill for the “some other reason” such as “The other reason you reported.”
Alternatively, we could acknowledge to respondents that they can only pick from reasons 1-7
7
from PRMOTOT as the main reason, but we would need to consider the limitations of this
approach.
TRYMOTIV – Of the participants routed to this item, one again noted that the response
categories did not fit her situation because she answered “yes” to misuse, but had actually used
less than a doctor had directed her to. Another participant mentioned difficulty answering this
item because he did not think his depression fit in any of the categories.
SVYMOTIV – One participant again noted that he thought we should have a response alternative
that captured dealing with emotions. The other participant who was routed to this item was able
to successfully match his answer to a response alternative.
Special Drugs
SD01 – Consistent with findings from Phase 2.1 interviews, popular examples of OTC drugs for
use just to get high were “Nyquil,” “Benadryl,” and the generic terms such as “cough syrup,” or
“cough medicine.” Others mentioned were “mouthwash,” “paragesic,” “triple-C,” “No-doze,”
“Sudafed,” “Vicks,” Robitussin,” “Formula 44,” “sleep aids,” “Tylenol,” and “Tylenol PM.” One
participant mentioned “purple drink” which seemed to be similar to a substance a participant in a
Phase 2.1 interview mentioned. However, we believe a prescription drug may be involved with
this substance. One other participant mentioned “red juice,” describing an OTC syrup.
Blunts
MJMM – It was clear to all five of the participants who were routed to this item that the question
was asking about marijuana prescribed by a doctor. In response to the probe, all five mentioned
prescribed use. Two participants differentiated this use from use with marijuana bought from
“someone else” or “someone on the street.”
Substance Dependence and Abuse
DRPR -- One participant reported heavy use of prescription pain relievers. Some of this was
misuse, but much of it was prescribed use. It was evident she was not clear this question was
asking about misuse.
Recommendation: Consider listing drugs the respondent has misused to clarify which drugs
respondents should be thinking of while answering this question. This will also provide a
reminder about which drugs are pain relievers, as opposed to other classes of drugs.
Recommendation: Confirm that these questions should use the clause “or that you used only for
the experience or feeling they caused” as there seemed to be some disagreement or confusion
about this.
8
Drug Treatment
TX04c – One participant was unsure whether AA should be counted for this item.
Recommendation: Discuss moving the AA item earlier in the sequence of treatment questions
so that it comes first.
Health
HLTH04 – None of the participants reported or displayed any problems answering about their
height. One participant was unsure how to continue after typing in his answer for “feet.” He
asked, “Do I press enter?”
HLTH18 – One adolescent participant was unclear whether the question included going with
someone else to the hospital for their inpatient stay.
HLTH19 – Generally speaking, participants listed and counted valid medical professionals to
report an answer. One reported a visit to a dentist.
HLTH22 – Three participants reported doctors discussing their drinking with them. These
reports included only being asked on a form, being asked anytime one saw a new doctor, and a
general discussion about “overindulgence” that was unrelated to actual use.
HLTH25 – One participant listed a number of health conditions, but upon probing indicated that
he was reporting times his doctor had asked or discussed these with him. It seems he may have
confused language in HLTH22 with that in HLTH25—that is language about a doctor discussing
these issues with him.
QI12AN – Two participants had difficulty understanding this question. Each had trouble
understanding what types of welfare or public assistance should be reported.
Recommendation: Add a bulleted list of the types of welfare respondents should include when
reporting.
Back End Demographics
QD17 – Nine participants said that they would correctly report being in school even if they were
on summer break. One said he was answer “no” even if he was on spring or summer break.
QD18 – A hypothetical question was asked of participants about being between 7th and 8th
grades. One participant incorrectly said she would say the 7th grade.
QD07 – One participant was unhappy with the omission of a response category for cohabiting.
He said he has been in a committed relationship for approximately four years and thinks the
category “have never been married” does not accurately reflect his situation.
9
QD26 – None of the participants had trouble answering this question about whether they had
worked at a job or business in the past week.
QD49 - Participants did not have any problems understanding this item about whether their
workplace had drug or alcohol testing as part of the hiring process.
QD50 – One participant was probed on this item and he correctly understood what was meant by
drug testing “random basis.” He said it means “out of the blue…like names out of hat and you
pick one.”
PERAGEYR – One participant was unable to answer this question about the age of each
household member because he did not know his parents’ ages.
Income
INTRTINN – This item about sources of income may be confusing to some respondents since
the item is now self-administered whereas it was previously interviewer administered.
Recommendation: Consider modifying language for this item since it is now self-administered
where as it was previously interviewer administered. For example, use of the term “we” in the
clause “those we just talked about” might be awkward since the respondent is interacting with a
computer.
Proxy Tutorial
In general, proxy participants for this round found the tutorial “easy” or “straightforward.” One
proxy participant asked for clarification about whether she could proceed before reviewing all
instructions. In another interview, an interviewer noted that for proxy respondents who have
previously completed an interview or proxy interview there was no transition screen making for
an awkward transition to the ACASI questions.
Recommendation: Add a screen at the beginning of the front-end proxy tutorial that allows
respondents who have previously completed a NSDUH interview or proxy interview to complete
an abbreviated rather than full tutorial.
Recommendation: We should add one more screen to the proxy tutorial transition that indicates
that the interviewer should turn the computer over to a proxy who has already completed the
tutorial elsewhere.
Recommendation: INTROINC says, These questions refer to the calendar year 2011 rather
than to the past 12 months that were referred to in some earlier questions. We recommend not
using this paragraph when there is a proxy, because they have not answered a large battery of
questions with a reference period of the past 12 months. QHI14 is the only question in the health
insurance and income items that asks about the past 12 months. This question asks about the
number of months in the past 12 that the sample person was without health insurance.
10
Additional notes
The Substance Dependence and Abuse section, starting at screen INTRODR, only requires
having smoked ‘all or part of a cigarette’ one time in the past 30 days for a respondent to receive
questions about abuse. One respondent had only smoked one cigarette and many questions did
not apply to him.
Recommendation: Consider whether there should be a higher barrier of entry to this and similar
sections of the Dependence module.
Currently the calendar and calendar reminders are only provided in the core items.
Recommendation: Add calendars and calendar reminders throughout the instrument.
11
Appendix F
Variable Wording Crosswalk
Attachment F. Variable Wording Crosswalk
QFT Variable
(Testing
Outcome)
V2b
(new
question,
tested, no
problems)
QD05 (not
tested,
implemented
in QFT)
QD10c
(new
Phase
Introduced
QFT Instrument Wording
Phase 2
[IF QD10 = 2] Are you currently
serving full‐time in a Reserve
component? Full‐time service does
not include annual training for the
Reserves or National Guard.
NA
Phase 2
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
GUAMANIAN OR
CHAMORRO
6
SAMOAN
7
OTHER PACIFIC ISLANDER
8
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
9
OTHER (SPECIFY)
[IF QD10a=1] Did you ever serve on
active duty in the United States
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
to attempt to better
identify respondents in
the Reserves.
2012 Question Wording
Not included in 2012
QD05 HAND R SHOWCARD 2 Which
of these groups describes you? Just
give me the number or numbers from
the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE (AMERICAN
INDIAN INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
QD05 HAND R SHOWCARD 2
Which of these groups describes
you? Just give me the number or
numbers from the card.
1
WHITE
2
BLACK OR AFRICAN
AMERICAN
3
AMERICAN INDIAN OR
ALASKA NATIVE
(AMERICAN INDIAN
INCLUDES NORTH
AMERICAN, CENTRAL
AMERICAN, AND SOUTH
AMERICAN INDIANS)
4
NATIVE HAWAIIAN
5
OTHER PACIFIC ISLANDER
6
ASIAN (INCLUDING: ASIAN
INDIAN, CHINESE, FILIPINO,
JAPANESE, KOREAN, AND
VIETNAMESE)
7
OTHER (SPECIFY)
After testing, added two
new race categories to
this question in response
to feedback.
No changes between initial testing and This question was added
final QFT wording.
to better determine if
Not included in 2012
2
QFT Variable
(Testing
Outcome)
question,
tested, no
problems)
QD11
(modified
question;
tested;
problems
found;
revised for
final QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
Armed Forces or Reserve
components in a military combat
zone or an area where you drew
imminent danger pay or hostile fire
pay?
HAND R SHOWCARD 5. What is the
highest grade or year of school you
have completed? Just give me the
number from the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING
COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 REGULAR HIGH SCHOOL
DIPLOMA
13 12TH GRADE, NO DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT,
BUT NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
Initial Test Wording
Notes/Testing Results
respondents who
indicate that they are on
active duty are actually in
a Reserve component.
2012 Question Wording
What is the highest grade or year of
school you have completed?
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NO SCHOOLING COMPLETED
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE, NO DIPLOMA
13 REGULAR HIGH SCHOOL
DIPLOMA
14 GED CERTIFICATE OF HIGH
SCHOOL COMPLETION
15 SOME COLLEGE CREDIT, BUT
NO DEGREE
16 ASSOCIATE’S DEGREE (FOR
EXAMPLE, AA, AS)
17 BACHELOR’S DEGREE (FOR
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
The original question
used in 2012 had no
category for completing
the 12th grade without
receiving a diploma. This
category was inserted as
response #12, but during
testing it was found that
the order of categories
could be confusing to
some respondents. As a
result of testing,
categories 12 and 13
were reversed so that
category 12 now
represents a high school
diploma and 13 means
12th grade, but no
diploma.
HAND R SHOWCARD 4. What is
the highest grade or year of school
you have completed?
Please tell me the number from
the card.
INCLUDE JUNIOR OR COMMUNITY
COLLEGE ATTENDANCE; DO NOT
INCLUDE TECHNICAL SCHOOLS
(BEAUTICIAN, MECHANIC, ETC.).
0 NEVER ATTENDED SCHOOL
1 1ST GRADE COMPLETED
2 2ND GRADE COMPLETED
3 3RD GRADE COMPLETED
4 4TH GRADE COMPLETED
5 5TH GRADE COMPLETED
6 6TH GRADE COMPLETED
7 7TH GRADE COMPLETED
8 8TH GRADE COMPLETED
9 9TH GRADE COMPLETED
10 10TH GRADE COMPLETED
11 11TH GRADE COMPLETED
12 12TH GRADE COMPLETED
13 COLLEGE OR UNIVERSITY /
1ST YEAR COMPLETED
14 COLLEGE OR UNIVERSITY /
2ND YEAR COMPLETED
15 COLLEGE OR UNIVERSITY /
3RD YEAR COMPLETED
16 COLLEGE OR UNIVERSITY /
4TH YEAR COMPLETED
3
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCOLOR)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
QFT Instrument Wording
EXAMPLE, BA, BS)
18 MASTER’S DEGREE (FOR
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE,
MD, DDS, DVM, LLB, JD)
DK/REF
Item dropped from questionnaire.
Initial Test Wording
EXAMPLE, MA, MS, MENG,
M. ED, MSW, MBA)
19 DOCTORATE DEGREE (FOR
EXAMPLE, PHD, EDD)
20 PROFESSIONAL DEGREE
BEYOND A BACHELOR’S
DEGREE (FOR EXAMPLE, MD,
DDS, DVM, LLB, JD)
DK/REF
CGCOLOR [IF (CG11 = 1‐2 OR CG11 =
4 OR CG11 = 7 OR CG11 = 10 OR CG11
= 12 OR CG11 = 14 OR CG11 = 17‐21
OR CG11 = 24‐25 OR CG11a = 31 AND
CG13 = 1) OR IF (RCG11 = 1‐2 OR
RCG11 = 4 OR RCG11 = 7 OR RCG11 =
10 OR RCG11 = 12 OR RCG11 = 14 OR
RCG11 = 17‐21 OR RCG11 = 24‐25 OR
RCG11a = 31 AND RCG13 = 1) OR IF
(RRCG11 = 1‐2 OR RRCG11 = 4 OR
RRCG11 = 7 OR RRCG11 = 10 OR
RRCG11 = 12 OR RRCG11 = 14 OR
RRCG11 = 17‐21 OR RRCG11 = 24‐25
OR RRCG11a = 31 AND RRCG13 = 1]
What color is the pack of
[CG11/CG11a/RCG11/RCG11a/RRCG1
1/RRCG11a FILL] cigarettes you
smoked most often during the past 30
days?
1
Blue
2
Green
3
Gold
4
Red
5
Silver
6
White
Notes/Testing Results
2012 Question Wording
17 COLLEGE OR UNIVERSITY /
5TH OR HIGHER YEAR
COMPLETED
DK/REF
Not included in 2012
Recent legislation
banned cigarette
manufacturers from
using terms on packaging
that referred to cigarette
strength. In an attempt
to continue to track
trends in the types of
cigarettes most smoked,
a new question was
developed to capture
data based on the new
cigarette color code
system. Most
respondents had a
problem with the
question and reported
that the cigarette
packaging included
multiple colors,
indicating that
participants failed to
understand the purpose
of the question.
4
QFT Variable
(Testing
Outcome)
Phase
Introduced
(CGCLR2)
Phase 2
(new
question;
tested;
problems
found;
dropped from
QFT)
CG25
Phase 1
(CG17REV)
(question
revised in
order to
administer
abbreviated
core for
cognitive
interviewing;
new wording
tested and
implemented
)
ALREF
Phase 1
QFT Instrument Wording
Item dropped from questionnaire.
Initial Test Wording
7
Some other color
DK/REF
CGCLR2 [IF CGCOLOR = 7] Please
type in the color of the pack of
cigarettes you smoked most often
during the past 30 days.
The next questions are about your
use of “smokeless” tobacco such as
snuff, dip, chewing tobacco, or
“snus.”
Have you ever used “smokeless”
tobacco, even once?
Have you ever, even once, tried any
“smokeless” tobacco such as snuff,
dip, chewing tobacco, or “snus”?
NOTE: Example text from ALREF is
ALREF [IF AL01 = REF] The answers
Notes/Testing Results
This follow‐up question
to an answer of “Some
other color” to CGCOLOR
was removed along with
the initial question due
to participant
misunderstanding of the
question.
An abbreviated core
module was used for
cognitive interviewing.
The version of the
smokeless tobacco
question that was used
eliminated the
introductory wording,
“The next questions are
about your use of . . .”
since only the lifetime
use of smokeless tobacco
was assessed. The
revised wording that was
tested and maintained
combined snuff and
chewing tobacco, since
incidence use rates of
these are quite low. In
addition, a new product
called “snus” has
recently emerged on the
market and it is also
included in the QFT
wording. No changes to
the question were made
as a result of the testing.
A streamlined version of
2012 Question Wording
Not included in 2012
These next questions are about your
use of snuff, sometimes called dip.
Snuff is a finely ground form of
tobacco that usually comes in a
container called a tin. You can use
snuff by placing a pinch or dip in
your mouth between your lip and
gum or between your cheek and
gum. Snuff can also be inhaled
through the nose. Snuff is sold in
both loose form and in ready‐to‐use
packets.
Have you ever used snuff, even
once?
[IF AL01 = REF] The answers that
5
QFT Variable
(Testing
Outcome)
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
AL08
(new
question,
tested, no
problems,
implemented
for QFT)
Phase 1
MRJINTRO &
MJ01
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase 1
QFT Instrument Wording
displayed below. Equivalent text was
used for MJREF, CCREF, HEREF, and
MEREF.
[IF AL01 = REF] The answers that
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have [IF QD01‐5 THEN FILL
5, IF QD01=9 THEN FILL 4] or more
drinks on the same occasion? By
‘occasion,’ we mean at the same
time or within a couple of hours of
each other.
Initial Test Wording
people give about their use of alcohol
are important. Remember, your
answers will be kept confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a sip
or two from a drink.
Notes/Testing Results
this question was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
people give us about their use of
alcohol are important to this study’s
success. We know that this
information is personal, but
remember your answers will be kept
confidential.
Please think again about answering
this question: Have you ever, even
once, had a drink of any type of
alcoholic beverage? Please do not
include times when you only had a
sip or two from a drink.
During the past 30 days, that is, since
[DATEFILL], on how many days did
you have [IF QD01‐5 THEN FILL 5, IF
QD01=9 THEN FILL 4] or more drinks
on the same occasion? By ‘occasion,’
we mean at the same time or within a
couple of hours of each other.
During the past 30 days, that is,
since [DATEFILL], on how many days
did you have 5 or more drinks on
the same occasion? By ‘occasion,’
we mean at the same time or within
a couple of hours of each other.
MRJINTRO
The next questions
are about marijuana and hashish.
Marijuana is also called pot or grass.
Marijuana is usually smoked, either
in cigarettes, called joints, or in a
pipe. It is sometimes cooked in
food. Hashish is a form of marijuana
that is also called “hash.” It is
usually smoked in a pipe. Another
form of hashish is hash oil.
MJ01 The next question is about
marijuana and hashish. Marijuana is
also called pot, grass, or weed.
Marijuana is usually smoked, either in
cigarettes, called “joints”, in cigars,
called “blunts”, or in a pipe. It is
sometimes cooked in food. Hashish is
a form of marijuana that is also called
“hash.” It is usually smoked in a pipe.
Another form of hashish is hash oil.
This item was revised to
ask females about
drinking 4 or more drinks
on the same occasion in
the Alcohol section.
Previously, this
information was
obtained in the
Consumption of Alcohol
module.
For the cognitive
interviewing we initially
tested a version of the
lifetime marijuana
question as part of the
abbreviated core. This
question incorporated
the introductory
language and referenced
only one “next question”
The next questions are about
marijuana and hashish. Marijuana is
also called pot or grass. Marijuana
is usually smoked, either in
cigarettes, called joints, or in a pipe.
It is sometimes cooked in food.
Hashish is a form of marijuana that
is also called “hash.” It is usually
smoked in a pipe. Another form of
hashish is hash oil.
6
QFT Variable
(Testing
Outcome)
Phase
Introduced
HE01
Phase 1
(question
revisions
were tested;
no advantage
was found;
dropped from
QFT)
HALINTRO
Phase 1
(question was
revised to
administer
abbreviated
version of
core drugs
module for
cognitive
interviewing
only.)
QFT Instrument Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Initial Test Wording
Notes/Testing Results
since subsequent
Have you ever, even once, used
questions to lifetime use
marijuana or hashish, or smoked part were not included. The
or all of a cigar with marijuana in it,
term “weed” was
included since it is a
called a “blunt”?
commonly used term for
marijuana. Additionally,
the cognitive
interviewing tested
inclusion of “blunts” in
the main marijuana
module. Combining
blunts with main
marijuana use was
desired because of
interest in streamlining
the instrument and
consolidating marijuana
use in one module.
Have you ever, even once, used
This next question is about heroin.
Alternate terms for
heroin?
heroin, “black tar heroin”
Have you ever, even once, used
and “cheese” were
heroin, including “black tar heroin” or tested but were not
included in the QFT since
“cheese”?
cognitive interview
respondents did not tend
to recognize them.
The next questions are about
The introductory text to
The next questions are about
substances called hallucinogens. These hallucinogens was
substances called hallucinogens.
drugs often cause people to see or
removed as part of
These drugs often cause people to
creating an abbreviated
see or experience things that are not experience things that are not real.
core drugs module for
real.
the cognitive
Press [ENTER] to continue.
interviewing. Since only
A list of some common
lifetime use was asked
hallucinogens is shown below.
about each substance,
These and many other substances
that people use as hallucinogens are
the full list of
often known by street names, and
hallucinogens was not
2012 Question Wording
Press [ENTER] to continue.
MJ01 Have you ever, even once,
used marijuana or hashish?
Have you ever, even once, used
heroin?
The next questions are about
substances called hallucinogens.
These drugs often cause people to
see or experience things that are not
real.
A list of some common
hallucinogens is shown below.
These and many other substances
that people use as hallucinogens are
often known by street names, and
7
QFT Variable
(Testing
Outcome)
Phase
Introduced
LS01i
(no change
tested)
Phase 1
LS01j
(existing
question
moved to
new module;
no wording
changes)
Phase 1
QFT Instrument Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Ketamine, also called “Special K” or
“Super K”
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine
Foxy, also called 5‐MeO‐DIPT
Salvia divinorum
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine
AMT, also called alpha‐
methyltryptamine, or
Initial Test Wording
Notes/Testing Results
included prior to the list
of lifetime use questions.
The QFT wording
included examples of all
Hallucinogens asked
about in the module,
including the three
substances that were
added to the module.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
2012 Question Wording
we can’t list them all. Please take a
moment to look at the substances
listed below so you know what kind
of drugs the next questions are
about.
LSD, also called ‘acid’
PCP, also called ‘angel dust’ or
phencyclidine
Peyote
Mescaline
Psilocybin
‘Ecstasy’, also called MDMA
Press [ENTER] to continue.
Have you ever, even once, used
Ketamine, also called “Special K” or
“Super K”?
1
Yes
2
No
DK/REF
Have you ever, even once, used any
of the following:
DMT, also called
dimethyltryptamine,
AMT, also called alpha‐
methyltryptamine, or
8
QFT Variable
(Testing
Outcome)
LS01k
(existing
question
moved to
new module;
no wording
changes)
INHINTRO
(question
revisions
were tested;
revisions
were
eliminated
from QFT)
Phase
Introduced
QFT Instrument Wording
Foxy, also called 5‐MeO‐DIPT?
Initial Test Wording
Notes/Testing Results
Phase 1
Have you ever, even once, used
Salvia divinorum?
No changes between initial testing and Question moved from
final QFT wording.
Special Drugs module to
a more appropriate
location in
Hallucinogens.
Phase 1
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
These next questions are about
liquids, sprays, and gases that people
sniff or inhale to get high or to make
them feel good.
Please do not include times when you
inhaled a substance accidentally —
such as when painting, cleaning an
oven, or filling a car with gasoline. The
questions use the word “inhalant” to
include any substances that you
sniffed or inhaled for kicks or to get
high
Press [ENTER] to continue.
A streamlined version of
the inhalants
introduction was tested
in the cognitive
interviewing. Words that
were not considered
essential were removed
to minimize burden on
respondents.
2012 Question Wording
Foxy, also called 5‐MeO‐DIPT?
Have you ever, even once, used
Salvia divinorum?
These next questions are about
liquids, sprays, and gases that
people sniff or inhale to get high or
to make them feel good.
We are not interested in times when
you inhaled a substance accidentally
— such as when painting, cleaning
an oven, or filling a car with
gasoline. The questions use the
word ‘inhalant’ to include all the
things listed below, as well as any
other substances that people sniff or
inhale for kicks or to get high. Take
a moment to look at the substances
listed below so you know what kinds
of liquids, sprays, and gases these
questions are about.
Amyl nitrite, ‘poppers,’ locker room
odorizers, or ‘rush’
Correction fluid, degreaser, or
cleaning fluid
Gasoline or lighter fluid
Glue, shoe polish, or toluene
Halothane, ether, or other
anesthetics
Lacquer thinner, or other paint
9
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Felt‐tip pens, felt‐tip markers, or
magic markers
Spray paints
Computer keyboard cleaner, also
known as air duster
Other aerosol sprays
Press [ENTER] to continue.
Have you ever, even once, inhaled
felt‐tip pens, felt‐tip markers, or
magic markers for kicks or to get
high?
IN01h1 (new
question,
tested, no
problems)
Phase 1
IN01ii (new
question
tested,
problems
found,
revised for
QFT)
Phase 1
Have you ever, even once, inhaled
computer keyboard cleaner, also
known as air duster, for kicks or to
get high?
ME01
(SD17a)
(existing
question
moved to
new module,
Phase 1
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
Initial Test Wording
Notes/Testing Results
No changes between initial testing and This question was added
final QFT wording.
because of references to
felt‐tip pens, felt‐tip
markers, and magic
markers in the write‐in
data in inhalants in the
annual survey.
In the past 12 months, have you
This question was added
because of references to
inhaled canned air for kicks or to get
computer keyboard
high?
cleaner and canned air in
the write‐in data in
inhalants in the annual
survey. In the cognitive
interviewing, participants
often did not know what
“canned air” was. New
terms were selected
based on cognitive
interviewing results.
No changes between initial testing and A new module was
created for
final QFT wording.
methamphetamine in
the core drugs modules.
Items measuring
methamphetamine use
2012 Question Wording
solvents
Lighter gases, such as butane or
propane
Nitrous oxide or ‘whippits’
Spray paints
Other aerosol sprays
Press [ENTER] to continue.
Not included in 2012
Not included in 2012
Methamphetamine, also known as
crank, ice, crystal meth, speed,
glass, and many other names, is a
stimulant that usually comes in
crystal or powder forms. It can be
smoked, “snorted,” swallowed or
10
QFT Variable
(Testing
Outcome)
no changes)
Phase
Introduced
ME02
(question
moved to
new module,
no changes)
MELAST3
(question
moved to
new module,
no changes)
MEFRAME3
(question
moved to
new module,
no changes)
Phase 2
MERAVE
(question
moved to
new module,
no changes)
MEMONAVE
(question
moved to
new module,
minor
changes)
MEWKAVE
QFT Instrument Wording
injected.
Have you ever, even once, used
methamphetamine?
How old were you the first time you
used methamphetamine?
Initial Test Wording
Notes/Testing Results
were moved from the
Stimulants module to
this new module.
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
2012 Question Wording
injected. Have you ever, even once,
used Methamphetamine?
SDME01 How old were you the first
time you used Methamphetamine?
Phase 2
How long has it been since you last
used methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
SD17b How long has it been since
you last used Methamphetamine?
Phase 2
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you’ve
used it?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On how many days in the past 12
months did you use
methamphetamine?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Phase 2
On average, how many days did you
use methamphetamine each month
during the past 12 months?
No changes between initial testing and A new module was
final QFT wording.
created for
methamphetamine in
the core drugs modules.
Now think about the past 12
months, from [DATEFILL] through
today. We want to know how many
days you’ve used
Methamphetamine during the past
12 months.
What would be the easiest way for
you to tell us how many days you
used Methamphetamine?
On how many days in the past 12
months did you use
Methamphetamine?
On average, how many days each
month during the past 12 months
did you use Methamphetamine?
Phase 2
On average, how many days did you
No changes between initial testing and A new module was
On average, how many days each
11
QFT Variable
(Testing
Outcome)
(question
moved to
new module,
no changes)
ME06 (new
question,
tested, no
changes)
INTROPR
(new
question
tested, no
changes)
PR01‐PR03
PR08
PR10
ST01‐ST04
ST05
ST06‐ST07
(question
tested,
problems
identified,
question
revised)
Phase
Introduced
Phase 2
QFT Instrument Wording
use methamphetamine each week
during the past 12 months?
[IF MELAST3 =1 OR MERECDK = 1
OR MERECRE = 1] Think specifically
about the past 30 days, from
[DATEFILL] up to and including
today. During the past 30 days, on
how many days did you use
methamphetamine?
Phase 1
These next questions are about any
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as
aspirin, Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
PR01
NOTE: Example text from PR01 is
ST01, ST02, displayed below. Equivalent text was
ST03, ST04, used for all listed variables from pain
relievers, stimulants, sedatives, and
ST06, &
tranquilizers modules.
ST07 –
Usability
Please look at the names and
pictures of the pain relievers shown
PR02,
below. Please note that some forms
PR03,
of these pain relievers may look
PR08,
different from the pictures, but you
PR10, &
should include any form that you
ST05 –
have used.
Phase 1
PROGRAMMER: DISPLAY PILLS HERE
TR01 &
FOR VICODIN, LORTAB, LORCET,
SV01 –
AND HYDROCODONE.
Phase 2
In the past 12 months, which, if any,
Initial Test Wording
final QFT wording.
Notes/Testing Results
created for
methamphetamine in
the core drugs modules.
No changes between initial testing and As part of creating a new
final QFT wording.
methamphetamine
module, a new 30‐day
methamphetamine use
variable was created.
2012 Question Wording
week during the past 12 months did
you use Methamphetamine?
Not included in 2012
These next questions are about the
use of prescription pain relievers.
Please do not include “over‐the‐
counter” pain relievers such as aspirin,
Tylenol, Advil, or Aleve.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription pain
relievers module.
PR01A. Please look at the pictures of
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in
either of these ways?
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
PR01A1. [IF PR01A = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
Not included in 2012
Initial versions of the
prescription drugs
screener questions were
written to capture
misuse only. Subsequent
to usability testing, the
question was revised to
capture first all types of
use then follow up
specific substances
endorsed with questions
about misuse.
Not included in 2012
12
QFT Variable
(Testing
Outcome)
PR04
PR06‐PR07
TR04 (new
question,
problems
found,
question
revised.)
Phase
Introduced
PR04,
PR06, &
PR07 –
Phase 1
TR04 –
Phase 2
QFT Instrument Wording
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Initial Test Wording
did you use in the past 12
months in either of these
ways?
Notes/Testing Results
2012 Question Wording
Without a prescription of
your own, or
Just for the effect it has
on you – not for its
intended medical use
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
Please look at the names and pictures
NOTE: Example text from PR04 is
displayed below. Equivalent text was of the pain relievers shown below.
used for all listed variables from pain
relievers, stimulants, sedatives, and PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER, ULTRACET,
tranquilizers modules.
RYZOLT, AND TRAMADOL.
Please look at the names and
pictures of the pain relievers shown In the past 12 months, which, if any,
below. Remember, some forms of
of these pain relievers have you used?
these pain relievers may look
different from the pictures, but you To select more than one drug from the
list, press the space bar between each
should include any form that you
number you have typed. When you
have used.
have finished, press [ENTER].
PROGRAMMER: DISPLAY PILLS HERE
FOR ULTRAM, ULTRAM ER,
ULTRACET, RYZOLT, AND
TRAMADOL.
In the past 12 months, which, if any,
of these pain relievers have you
Original test of wording
Not included in 2012
yielded results that
participants were asking
whether certain
prescription drugs should
be included even though
they looked different
from what was in the
pictures. Reminder
language was added to
every 2‐4 screens to
remind respondents that
the pictures may look
different from the form
pictured but that they
should still be included.
13
QFT Variable
(Testing
Outcome)
Phase
Introduced
PR05
(question
tested in
cognitive
interviewing,
problem
found, minor
changes
made)
Phase 1
PR11
TR06
ST06
SV06 (Change
PR11 &
ST06‐
Phase 1
QFT Instrument Wording
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
Please look at the names and
pictures of the pain relievers shown
below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you
used?
To select more than one drug from
the list, press the space bar between
each number you have typed.
When you have finished, press
[ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine 3 or 4
2
Codeine pills
95
I have not used any of
these pain relievers in the past 12
months
DK/REF
In the past 12 months, have you
used any other prescription pain
reliever?
Initial Test Wording
Please look at the names and pictures
of the pain relievers shown below.
PROGRAMMER: DISPLAY PILLS HERE
FOR TYLENOL WITH CODEINE AND
CODEINE.
In the past 12 months, which, if any,
of these pain relievers have you used?
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
SHOW 12‐MONTH CALENDAR ON
SCREEN.
1
Tylenol with codeine
2
Codeine pills
95
I have not used any of these
pain relievers in the past 12 months
DK/REF
Notes/Testing Results
2012 Question Wording
Initially, “Tylenol with
Not included in 2012
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
No changes between initial testing and The prescription drugs
final QFT wording.
modules were revised to
ask first about all types
of use. Respondents
Not included in 2012
14
QFT Variable
(Testing
Outcome)
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
TR06 &
Remember, do not include “over‐
SV06‐
the‐counter” pain relievers such as
Phase 2
aspirin, Tylenol, Advil, or Aleve.
PR12
TR07
ST07
SV07 (Change
tested, no
problems
found,
implemented
in QFT.)
PR12 &
ST07—
Phase 1
TR07 &
SV07—
Phase 2
INTROTR
INTROSV
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 2
Initial Test Wording
Notes/Testing Results
received follow up
questions about misuse
only of specific drugs
that were used. This
particular item was
administered when no
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
No changes between initial testing and The prescription drugs
[IF PR12MON = 2] Have you ever,
final QFT wording.
modules were revised to
even once, used any prescription
ask first about all types
pain reliever?
of use. Respondents
received follow up
Remember, do not include “over‐
questions about misuse
the‐counter” pain relievers such as
only of specific drugs
aspirin, Tylenol, Advil, or Aleve.
that were used. This
particular question was
asked when no 12‐month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed.
These questions were
NOTE: Example text from INTROTR is These next questions are about the
added to introduce the
use of prescription tranquilizers.
displayed below. Semi‐equivalent
Tranquilizers are usually prescribed to new prescription
text was used for INTROSV.
relax people, to calm people down, to tranquilizer and sedative
screener modules. Minor
These next questions are about any relieve anxiety, or to relax muscle
wording revisions were
spasms. Some people call
use of prescription tranquilizers.
made after testing.
Tranquilizers are usually prescribed tranquilizers “nerve pills.”
to relax people, to calm people
down, to relieve anxiety, or to relax Press [ENTER] to continue.
muscle spasms. Some people call
tranquilizers “nerve pills.”
2012 Question Wording
Not included in 2012
These next questions ask about the
use of tranquilizers. Tranquilizers
are usually prescribed to relax
people, to calm people down, to
relieve anxiety, or to relax muscle
spasms. Some people call
tranquilizers ‘nerve pills.’
Ask your interviewer to show you
Card B.
15
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTROST
(Change
tested, minor
problems
found,
implemented
in QFT.)
Phase 1
PRL01
TRL01
STL01
SVL01
(Change
tested, no
problems
found,
implemented
in QFT.)
PRL01 &
STL01—
Phase 1
TRL01 &
SVL01—
Phase 2
PRY01—
Usability
QFT Instrument Wording
Press [ENTER] to continue.
These next questions are about any
use of prescription stimulants.
People sometimes take these drugs
for attention deficit disorders, to
lose weight, or to stay awake.
Please do not include “over‐the‐
counter” stimulants such as
Dexatrim, No‐Doz, Hydroxycut, or 5‐
Hour Energy.
Press [ENTER] to continue.
Initial Test Wording
Notes/Testing Results
2012 Question Wording
These next questions are about the
use of prescription stimulants. People
sometimes take these drugs for
attention deficit disorders, to lose
weight, or to stay awake. Please do
not include “over‐the‐counter”
stimulants such as Dexatrim or No‐
Doz.
Press [ENTER] to continue.
This question was added
to introduce the new
prescription stimulants
screener module. Minor
wording revisions were
made after testing to
include additional
examples.
These next questions ask about the
use of drugs such as amphetamines
that are known as stimulants,
‘uppers,’ or ‘speed.’ People
sometimes take these drugs to lose
weight, to stay awake, or for
attention deficit disorders. We are
not interested in the use of “over‐
the‐counter” stimulants such as
Dexatrim or No‐Doz that can be
bought in drug stores or grocery
stores without a doctor’s
prescription.
Ask your interviewer to show you
Card C.
NOTE: Example text from PR05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
Have you ever, even once, used any
other prescription pain reliever,
besides the ones shown on Card A,
when it was not prescribed for you
or that you took only for the
experience or feeling it caused?
No changes between initial testing and The prescription drugs
NOTE: Example text from PRL01 is
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[IF PR12=1] Have you ever, even
once, used any prescription pain
that were used. This
reliever in any way a doctor did not
particular question was
direct you to use it?
asked when 12 month
use of prescription drugs
within a given category
(pain reliever,
tranquilizer, stimulant,
sedative) was endorsed
but no misuse was ever
endorsed when follow up
questions were asked
about specific drugs.
NOTE: Example text from PRY01 is
PR01C. Please look at the pictures of The revised prescription [IF PR04 = 1] Which of the pain
16
QFT Variable
(Testing
Outcome)
PRY04
PRY24
PRY32—
PRY34
PRY37
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
Phase
Introduced
QFT Instrument Wording
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR01=1] In the past 12 months,
did you use Vicodin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
the pain relievers shown
below. In the past 12
months, have you used any
of these pain relievers in any
of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
1
Yes
2
No
DK/REF
PR01C1. [IF PR01C = 1] Please look at
the pictures of the pain
relievers shown below.
Which of these pain relievers
did you use in the past 12
months in any of these ways?
Without a prescription of
your own,
In greater amounts,
more often, or longer
than you were told to
take it, or
Just for the effect it has
on you – not for its
intended medical use
Notes/Testing Results
drugs questions ask
about misuse of drugs
endorsed for any type of
12 month use in the
screener. The initial
wording of the third
component of the
definition of misuse,
“Just for the effect it has
on you – not for its
intended medical use”
did not help test
participants identify
misuse so it was revised
to the final QFT wording.
Also, the wording in the
2012 survey only
collected estimates of
lifetime use of individual
prescriptions, while the
new questions collect
past year use of
individual drugs.
2012 Question Wording
relievers shown below the red line
on Card A have you used when they
were not prescribed for you or that
you took only for the experience or
feeling they caused?
To select more than one drug from
the list, press the space bar between
each number you type. When you
have finished, press [ENTER].
4
Codeine
5
Demerol
6
Dilaudid
7
Fioricet
8
Fiorinal
9
Hydrocodone
10
Methadone
11
Morphine
12
OxyContin
13
Phenaphen with Codeine
14
Propoxyphene
15
SK‐65
16
Stadol
17
Talacen
18
Talwin
19
Talwin NX
20
Tramadol
21
Ultram
DK/REF
17
QFT Variable
(Testing
Outcome)
PRY05‐PRY17
PRY19
PRY20—
PRY23
PRY25—
PRY33
PRY35—
PRY36
PRY38—
PPRY39
TRY01—
TRY18
STY01—
STY23
SVY01—
SVY16
(Change
tested,
wording
Phase
Introduced
Phase 1
QFT Instrument Wording
[IF PR02=1] In the past 12 months,
did you use OxyContin in any way a
doctor did not direct you to use it?
If you want to see these ways again,
press F2.
Without a prescription of
your own.
In greater amounts, more
often, or longer than you
were told to take it
In any other way a doctor
did not direct you to use it.
Initial Test Wording
To select more than one drug
from the list, press the space
bar between each number
you have typed. When you
have finished, press [ENTER].
1
Vicodin
2
Lortab
3
Lorcet
4
Hydrocodone
DK/REF
NOTE: Example text from PRY05 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PR02=1] In the past 12 months, did
you use OxyContin in any way a doctor
did not direct you to use it?
Notes/Testing Results
Cognitive testing
identified repetitive
nature of prescription
drugs module questions
with long definitions of
misuse and raised
concerns about
respondent fatigue. As a
result, the definition was
displayed as optional text
available by pressing F2
key on some screens.
2012 Question Wording
How long has it been since you last
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
18
QFT Variable
(Testing
Outcome)
improvement
identified,
implemented
in QFT.)
PRY18
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY28
(Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY01a—
PRY39a
TRY01a—
TRY18a
STY01a—
STY23a
SVY01a—
SVY16a
(Change
tested,
wording
improvement
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Please look at the pain relievers
shown in Box 1 above the red line
on Card A.
Have you ever, even once, used
Darvocet, Darvon, or Tylenol with
codeine that was not prescribed for
you or that you took only for the
experience or feeling it caused?
Not included in 2012
Phase 1
[IF PR05=1] In the past 12 months,
did you use Tylenol with codeine 3
or 4 in any way a doctor did not
direct you to use it?
[IF PR05=1] In the past 12 months, did
you use Tylenol with codeine in any
way a doctor did not direct you to use
it?
Initially, “Tylenol with
codeine” was confusing
to some respondents and
respondents asked if
“Tylenol 3” or “Tylenol 4”
should be included. “3 or
4” was added to the
question to further
specify.
Phase 1
[IF PR07=4] In the past 12 months,
did you use fentanyl in any way a
doctor did not direct you to use it?
[IF PR07=4] In the past 12 months, did
you use fentanyl or “China Girl” in any
way a doctor did not direct you to use
it?
The street name for
fentanyl, “China Girl”
was not known to
cognitive interview
participants so it was
dropped from the QFT.
PRY01a‐
PRY39a &
STY01a‐
STY23a‐
Phase 1
TRY01a‐
TRY18a &
SVY01a‐
SVY16a‐
Phase 2
[IF PRY01=1] How old were you when
NOTE: Example text from PR01a is
displayed below. Equivalent text was you first used Vicodin in a way a
used for all listed variables from pain doctor did not direct you to use it?
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRFIRSTFLAG=1] Please think
about the first time you ever used
Vicodin in a way a doctor did not
direct you to use it.
[IF PRY01=1] How old were you
when you first used Vicodin in a way
a doctor did not direct you to use
it?
An introduction was
added to transition from
asking about whether
groups of drugs were
misused to asking about
the specifics of that
misuse.
NOTE: Example text from PR06 is
displayed below. Equivalent text was
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
How old were you the first time you
used any prescription pain reliever
that was not prescribed for you or
that you took only for the
experience or feeling it caused?
19
QFT Variable
(Testing
Outcome)
identified,
implemented
in QFT.)
PRY01b—
PRY39b
TRY01b—
TRY02b
STY01b
SVY01b—
SVY02b
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01c—
PRY02c
TRY01c—
TRY02c
STY01c
SVY01c—
SVY02c
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRY01b—
PRY39b &
STY01b‐
Phase 1
TRY01b—
TRY02b &
SVY01b‐
Phase 2
PRY01c—
PRY02c &
STY01c—
Phase 1
TRY01c—
TRY02c &
SVY01c—
SVY02c—
Phase 2
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Not included in 2012
NOTE: Example text from PRY01b is No changes between initial testing and The prescription drugs
modules were revised to
displayed below. Equivalent text was final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
only of specific drugs
[AGE1STPR1 = CURNTAGE AND
DATE OF INTERVIEW < DOB OR IF
that were used. This
AGE1STPR1 = CURNTAGE ‐ 1 AND
particular question was
DATE OF INTERVIEW ≥ DOB] Did you
asked when 12 month
first use Vicodin in a way a doctor
use of a prescription drug
did not direct you to use it in
was endorsed with first
[CURRENT YEAR ‐ 1] or [CURRENT
use within at
respondent’s current
YEAR]?
age. No change was
made to the item as a
result of cognitive
testing.
Not included in 2012
No changes between initial testing
The prescription drugs
NOTE: Example text from PRY01c is
modules were revised to
displayed below. Equivalent text was and final QFT wording.
ask first about all types
used for all listed variables from pain
of use. Respondents
relievers, stimulants, sedatives, and
received follow up
tranquilizers modules.
questions about misuse
[IF AGE1STPR2 = CURNTAGE ‐ 1
only of specific drugs
AND DATE OF INTERVIEW < DOB]
that were used. This
Did you first use Lortab in a way a
particular question was
doctor did not direct you to use it in
asked when 12 month
[CURRENT YEAR ‐ 2] or [CURRENT
use of a prescription drug
YEAR ‐ 1]?
was endorsed at
respondent’s age one
year younger than their
current age. No change
was made to the item as
a result of cognitive
20
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRY01d—
PRY02d
TRY01d—
TRY02d
STY01d
SVY01d—
SVY02d
(Change
tested, no
problems
found,
implemented
in QFT.)
PRY01d‐
PRY02d &
STY01d—
Phase 1
TRY01d‐
TRY02d &
SVY01d‐
SVY02d—
Phase 2
PRM02DKRE
TRM02DKRE
STM02DKRE
SVM02DKRE
(Change
tested, no
problems
found,
implemented
in QFT.)
PRM02DKR
E &
STM02DKR
E— Phase
1
TRM02DKR
E &
SVM02DKR
E—Phase 2
QFT Instrument Wording
Initial Test Wording
No changes between initial testing
NOTE: Example text from PRY01d is
displayed below. Equivalent text was and final QFT wording.
used for all listed variables from pain
relievers, stimulants, sedatives, and
tranquilizers modules.
[IF PRYFU1 NE 0 AND PRJANFLAG1 =
0 AND PRDECFLAG1=0 AND
PR1MTHFLAG1=0]
[IF PRYFU1 = CURRENT YEAR OR
CURRENT YEAR ‐1] Earlier, you
reported that you first used Vicodin
in a way a doctor did not direct you
to use it when you were
[AGE1STPR1] years old. Based on
your date of birth, you turned
[AGE1STPR1] in [FILL WITH
MONTH/YEAR FOR AGE1STPR1
BASED ON DOB].
[IF PRYFU1 NE 0] In what month in
[PRYFU1] did you first use Vicodin in
a way a doctor did not direct you to
use it?
No changes between initial testing and
NOTE: Example text from
final QFT wording.
PRM02DKRE is displayed below.
Equivalent text was used for
TRM02DKRE, STM02DKRE, &
SVM02DKRE.
[IF PRM02 = DK/REF] What is your
best estimate of the number of days
you used [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] during the past 30
days?
Notes/Testing Results
testing.
This question asked
recent initiates of
prescription drug misuse
about the month of first
use of in order to meet a
redesign goal of
capturing more precise
data on first use among
recent initiates of drugs.
No change was made to
the item as a result of
cognitive testing.
No changes were made
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse about
individual drugs are not
collected. This question
was introduced to
measure frequency of
misuse of all misused
substances.
2012 Question Wording
Not included in 2012
Not included in 2012
21
QFT Variable
(Testing
Outcome)
PRM03
TRM03
STM03
SVM03
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
PRM03 &
STM03—
Phase 1
TRM03 &
SVM03—
Phase 2
PRY40
TRY19
STY24
SVY19
PRY40 &
STY24—
Phase 1
TRY19 &
SVY19—
Phase 2
PRY41
STY25
TRY20
SVY18
Change
tested,
wording
improvement
identified,
implemented
in QFT.)
PRY41 &
STY25—
Phase 1
TRY20 &
SVY18—
Phase 2
QFT Instrument Wording
NOTE: Example text from PRM03 is
displayed below. Equivalent text was
used for TRM03, STM03, & SVM03.
[IF ALC30USE = 1 and PRMISUSE30 =
1)] During the past 30 days did you
use [PRNAMEFILL] in any way a
doctor did not direct you to use
[PRNUMFILL] while you were
drinking alcohol or within a couple
of hours of drinking?
NOTE: Example text from PRY40 is
displayed below. Equivalent text was
used for TRY19, STY24, & SVY19.
[IF PR11=1] In the past 12 months,
did you use any [IF PR11=1 AND
PRYRCOUNT > 1 FILL “other”]
prescription pain reliever in a way a
doctor did not direct you to use it?
Initial Test Wording
Notes/Testing Results
No changes between initial testing and No changes were made
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of misuse with alcohol
are collected in the
Consumption of Alcohol
module. Here it is asked
in the relevant
prescription drug
module.
2012 Question Wording
Think again about this last time you
drank any alcoholic beverages, when
you had [CA01 FILL] [drink/drinks].
Did you also use [DRUGFILL] while
you were drinking or within a couple
of hours of drinking?
No changes between initial testing and No changes were made
Not included in 2012
final QFT wording.
to the item as a result of
the cognitive testing. In
the 2012 survey, details
of past year misuse
about individual drugs
are not collected. This
question was introduced
to measure details of
misuse of all misused
substances.
[IF PRMISUSE12=1] Which of these
This question was added Not included in 2012
NOTE: Example text from PRY41 is
to capture more detail
displayed below. Equivalent text was statements describe your use of
about which ways the
[PRNAMEFILL] at any time in the past
used for STY25, TRY20, & SVY18.
respondents misused
12 months? To select more than one
statement, press the space bar
specific drugs. The “at
least once” wording was
[IF PRMISUSE12=1] Which of these between the numbers.
confusing to some
statements describe your use of
[PRNAMEFILL] at any time in the
1
At least once, I used
respondents in the first
past 12 months?
[PRNAMEFILL] without a
round of cognitive
prescription of my own.
interviewing, so it was
To select more than one statement, 2
At least once, I used
removed for subsequent
press the space bar between the
[PRNAMEFILL] in greater
rounds of cognitive
numbers. When you have finished,
amounts, more often, or for
interviewing and for the
press [ENTER].
longer than it was prescribed. QFT. Additional methods
3
At least once, I used
of misuse were also
22
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
I used [PRNAMEFILL]
without a prescription of
my own.
2
I used [PRNAMEFILL] in
greater amounts than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
3
I used [PRNAMEFILL] more
often than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
4
I used [PRNAMEFILL] for
longer than [IF
PRNAMEFILL= “codeine
pills” or PRNAMEFILL=
“prescription pain
relievers” THEN FILL WITH
“they were”; ELSE FILL
WITH “it was”] prescribed.
5
I used [PRNAMEFILL] in
some other way a doctor
did not direct me to use
[PRNUMFILL]
NOTE: Example text from PRY42b is
displayed below. Equivalent text was
used for STY26b, TRY42B, & SVY19B.
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
1
PRY42B
STY26B
TRY42B
SVY19B
(Change
tested,
PRY42B &
STY26B—
Phase 1
TRY42B &
SVY19B—
Phase 2
Initial Test Wording
[PRNAMEFILL] in some other
way a doctor did not direct
me to use [PRNUMFILL]
[IF PRMISUSE12=1 AND
PRMISCOUNT=1 AND
PRDKREFFLAG=2] Now think about
the last time you used [PRLASTFILL2]
in any way a doctor did not direct you
to use it.
Notes/Testing Results
identified and added to
the response options.
Response categories
were streamlined and
reduced as a result of
cognitive interviewing.
2012 Question Wording
Not included in 2012
23
QFT Variable
(Testing
Outcome)
wording
improvement
identified,
implemented
in QFT.)
PRY42C
STY26C
TRY21C
SVY19C
Phase
Introduced
PRY42C &
STY26C—
Phase 1
QFT Instrument Wording
the last time you used [PRLASTFILL2]
in any way a doctor did not direct
you to use [FILL WITH “them” IF
RPRY42A=19. ELSE FILL WITH “it”].
[IF PRMISUSE12 = 1] How did you
get the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more
than one doctor
3
I stole the [PRLASTFILL]
from a doctor’s office,
clinic, hospital, or
pharmacy
4
I got the [PRLASTFILL] from
a friend or relative for free
5
I bought the [PRLASTFILL]
from a friend or relative
6
I took the [PRLASTFILL]
from a friend or relative
without asking
7
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
8
I got the [PRLASTFILL] in
some other way
DK/REF
NOTE: Example text from PRY42c is
displayed below. Equivalent text was
used for TRY21C & SVY19C.
Initial Test Wording
[IF PRMISUSE12 = 1] How did you get
the [PRLASTFILL]? If you got the
[PRLASTFILL] in more than one way,
please choose one of these ways as
your best answer.
1
I got a prescription for the
[PRLASTFILL] from just one
doctor
2
I got prescriptions for the
[PRLASTFILL] from more than
one doctor
3
I wrote a fake prescription for
the [PRLASTFILL]
4
I stole the [PRLASTFILL] from
a doctor’s office, clinic,
hospital, or pharmacy
5
I got the [PRLASTFILL] from a
friend or relative for free
6
I bought the [PRLASTFILL]
from a friend or relative
7
I took the [PRLASTFILL] from a
friend or relative without
asking
8
I bought the [PRLASTFILL]
from a drug dealer or other
stranger
9
I bought the [PRLASTFILL] on
the Internet
10
I got the [PRLASTFILL] in some
other way
DK/REF
Notes/Testing Results
No changes between initial testing and Wording and fills in this
final QFT wording.
question were adapted
to redesign conventions.
No changes were made
2012 Question Wording
Not included in 2012
24
QFT Variable
(Testing
Outcome)
(Change
tested, no
problems
found,
implemented
in QFT.)
(PRHOSPYR1
&
STHOSPYR1)
(Change
tested,
problems
found,
dropped from
QFT)
(PRHOSPYR2
&
STHOSPYR2)
(Change
tested,
Phase
Introduced
QFT Instrument Wording
TRY21C &
SVY19C—
[IF PRY42B=5 AND CALCAGE >=18]
Phase 2
You reported that you got the
[PRLASTFILL] from a friend or
relative for free. How did your
friend or relative get the
[PRLASTFILL]?
Phase 1
Items dropped from questionnaire.
Phase 1
Item dropped from questionnaire.
Initial Test Wording
Notes/Testing Results
as a result of the
cognitive interviewing.
2012 Question Wording
NOTE: Example text from PRHOSPYR1
is displayed below. Equivalent text was
used for STHOSPYR1.
[IF PR12MON=1 AND (PR11=1 OR
PRYRCOUNT > 1)] The computer
recorded that, in the past 12 months,
you used [PRHOSPFILL].
In the past 12 months, did you use [IF
PR11=1 AND PRYRCOUNT=1, THEN
FILL WITH PRHOSPFILL. IF PRYRCOUNT
> 1 THEN FILL WITH “any of these
prescription pain relievers”] only
when you were in the hospital? That
would include staying in the hospital
as an inpatient or being treated in an
emergency room.
[IF PR12MON=1 AND PR11 NE 1 AND
PRYRCOUNT = 1] In the past 12
months, did you use [PRHOSPFILL]
only when you were in the hospital?
That would include staying in the
hospital as an inpatient or being
treated in an emergency room.
NOTE: Example text from PRHOSPYR2
is displayed below. Equivalent text was
used for STHOSYR2.
[IF PRHOSPYR1=1 AND PRYRCOUNT >
A question about
legitimate use of
prescription drugs only
while in the hospital was
added to the instrument
during the cognitive
interviewing in order to
account for this very
specific kind of use that
respondents might not
be sure how to count.
The question confused
several respondents so it
was eliminated.
Not included in 2012
This follow up to the
previous question
assessed which
prescription drugs were
used only in a hospital
Not included in 2012
25
QFT Variable
(Testing
Phase
Outcome)
Introduced
problems
found,
dropped from
QFT)
PRYMOTIV
Phase 2
QFT Instrument Wording
Initial Test Wording
1] Which of these pain relievers did
you use only when you were in the
hospital in the past 12 months?
(LIST DRUGS ANSWERED AS YES IN
PR01‐PR11. NUMBER RESPONSE
OPTIONS SEQUENTIALLY STARTING AT
1, BUT MAINTAIN UNIQUE CODES FOR
EACH DRUG. IF PR11=1, DISPLAY
"Another prescription pain reliever"
AS THE LAST CATEGORY. )
To select more than one drug from the
list, press the space bar between each
number you have typed. When you
have finished, press [ENTER].
NOTE: Example text from PRYMOTIV [IF PRMISCOUNT > 1 OR
is displayed below.
(PRMISCOUNT = 1 AND PRY40 NE 1
AND PRDKREFFLAG=1)]
[IF PRMISCOUNT > 1 OR
Now think about the last time you
(PRMISCOUNT = 1 AND PRY40 NE 1
used [PRLASTFILL2] in any way a
AND PRDKREFFLAG=1)]
doctor did not direct you to use [FILL
Now think about the last time you
WITH ‘them” IF RPRY42A=19. ELSE
used [PRLASTFILL2] in any way a
FILL WITH “it”].
doctor did not direct you to use [FILL
WITH ‘them” IF RPRY42A=19. ELSE
What were the reasons you used
[PRLASTFILL2] that time? To select
FILL WITH “it”].
more than one reason from the list,
press the space bar between each
What were the reasons you used
number you have typed. When you
[PRLASTFILL2] that time? To select
more than one reason from the list, have finished, press [ENTER].
press the space bar between each
number you have typed. When you
1
To relieve physical pain
have finished, press [ENTER].
2
To relax or relieve tension
3
To experiment or to see what
1
To relieve physical pain
[IF RPRY42A=19 THEN
2
To relax or relieve tension
“they’re” ELSE “it’s”] like
Notes/Testing Results
setting. It was dropped
along with the lead‐in
question due to
confusion.
This question was added
to gather data about
motivation for misuse of
Pain Relievers.
2012 Question Wording
Not included in 2012
26
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
4
To feel good or get high
To experiment or to see
what [IF RPRY42A=19 THEN 5
To help with my sleep
“they’re” ELSE “it’s”] like
6
To increase or decrease the
4
To feel good or get high
effect(s) of some other drug
5
To help with my sleep
Because I am “hooked” or I
7
6
To help me with my
have to have [IF RPRY42A=19
feelings or emotions
THEN “them” ELSE “it”]
7
To increase or decrease the
effect(s) of some other drug 8
I used [IF RPRY42A=19 THEN
8
Because I am “hooked” or I
“them” ELSE “it”] for some
have to have [IF
other reason
RPRY42A=19 THEN “them”
DK/REF
ELSE “it”]
9
I used [IF RPRY42A=19
THEN “them” ELSE “it”] for
some other reason
DK/REF
[IF SVMISCOUNT > 1 OR
[IF SVMISCOUNT > 1 OR
(SVMISCOUNT=1 AND SVY17 NE 1
(SVMISCOUNT=1 AND SVY17 NE 1
AND SVDKREFFLAG=1)] Now think
AND SVDKREFFLAG=1)] Now think
about the last time you used
about the last time you used
[SVLASTFILL2] in any way a doctor
[SVLASTFILL2] in any way a doctor did
did not direct you to use it.
not direct you to use it.
What were the reasons you used
What were the reasons you used
[SVLASTFILL2] that time? To select
[SVLASTFILL2] that time? To select
more than one reason from the list, more than one reason from the list,
press the space bar between each
press the space bar between each
number you have typed. When you number you have typed. When you
have finished, press [ENTER].
have finished, press [ENTER].
1
To relax or relieve tension
1. To relax or relieve tension
2. To experiment or to see
2
To experiment or to see what
what it’s like
it’s like
3. To feel good or get high
3
To feel good or get high
4. To help with my sleep
4
To help with my sleep
5. To help me with my
5
To increase or decrease the
feelings or emotions
Notes/Testing Results
2012 Question Wording
3
TRMOTIV
SVMOTIV
Phase 1
This question was added
to gather data about
motivation for misuse of
Tranquilizers and
Sedatives.
Not included in 2012
27
QFT Variable
(Testing
Outcome)
STYMOTIV
Phase
Introduced
Phase 2
QFT Instrument Wording
6. To increase or decrease the
effect(s) of some other
drug
7. Because I am “hooked” or I
have to have it
8. I used it for some other
reason
DK/REF
Initial Test Wording
effect(s) of some other drug
6
Because I am “hooked” or I
have to have it
7
I used it for some other
reason
DK/REF
Notes/Testing Results
[IF STMISCOUNT > 1 OR
(STMISCOUNT = 1 AND STY26NE 1
AND STDKREFFLAG=1)]Now think
about the last time you used
[STLASTFILL2] in any way a doctor
did not direct you to use [FILL WITH
‘them” IF RSTY42A=5. ELSE FILL
WITH “it”]
What were the reasons you used
[STLASTFILL2] that time? To select
more than one reason from the list,
press the space bar between each
number you have typed. When you
have finished, press [ENTER].
1. To help me lose weight
2. To help me
concentrate
3. To help me be alert or
stay awake
4. To help me study
5. To experiment or to
see what [IF
RSTY42A=5 THEN
“they’re” ELSE “it’s”]
like
No changes between initial testing and This question was added
to gather data about
final QFT wording.
motivation for misuse of
Stimulants.
2012 Question Wording
Not included in 2012
28
QFT Variable
(Testing
Outcome)
Phase
Introduced
PRMOTOT
TRMOTOT
STMOTOT
SVMOTOT
Phase 2
PRYMOT1
Phase 2
QFT Instrument Wording
6. To feel good or get
high
7. To increase or
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. I used [IF RSTY42A=5
THEN “them” ELSE “it”]
for some other reason
DK/REF
NOTE: Example text from PRMOTOT
is displayed below. Equivalent text
was used for TRMOTOT, STMOTOT,
& SVMOTOT.
[IF PRYMOTIV=9] Please type in the
other reason you used
[PRLASTFILL2] that time. When you
have finished typing your answer,
press [ENTER] to go to the next
question.
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN PRYMOTIV] Which was
the main reason you used
[PRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1 To relieve physical pain
2 To relax or relieve tension
3 To experiment or to see
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF PRYMOTIV=8] Please type in the
reason you used [PRLASTFILL2] that
time. When you have finished typing
your answer, press [ENTER] to go to
the next question.
This question was added
to gather other, specify
data about motivation
for misuse of
prescription drugs.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN PRYMOTIV] Which was the
main reason you used [PRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
PRYMOTIV
1
To relieve physical pain
2
To relax or relieve tension
3
To experiment or to see what
This question was added
to gather data about the
primary motivation for
misuse of pain relievers.
Edits to logic were made
as a result of cognitive
testing.
Not included in 2012
29
QFT Variable
(Testing
Outcome)
Phase
Introduced
TRYMOT1
SVYMOT1
Phase 2
STYMOT1
Phase 2
QFT Instrument Wording
what it’s like
4 To feel good or get high
5 To help me with my
feelings or emotions
6 It helps with my sleep
7 To increase or decrease the
effect(s) of some other
drug
8 Because I am “hooked” or I
have to have it
9 [IF PRYMOTIV=9] The other
reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
CHOSEN IN TRYMOTIV] Which was
the main reason you used
[TRLASTFILL2] that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to
see what it’s like
3. To feel good or get
high
4. To help me with my
feelings or emotions
5. It helps with my sleep
6. To increase or
decrease the effect(s)
of some other drug
7. Because I am “hooked”
or I have to have it
8. [IF PRYMOTIV=9] The
other reason I reported
[IF MORE THAN ONE RESPONSE 1‐9
4
5
6
7
Initial Test Wording
it’s like
To feel good or get high
It helps with my sleep
To increase or decrease the
effect(s) of some other drug
Because I am “hooked” or I
have to have it
Notes/Testing Results
2012 Question Wording
DK/REF
[IF MORE THAN ONE RESPONSE 1‐7
CHOSEN IN TRYMOTIV] Which was the
main reason you used [TRLASTFILL2]
that time?
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE CHOSEN IN
TRYMOTIV
1. To relax or relieve
tension
2. To experiment or to see
what it’s like
3. To feel good or get high
4. To help with my sleep
5. To increase or decrease
the effect(s) of some
other drug
6. Because I am “hooked”
or I have to have it
7. I used it for some other
reason
DK/REF
This question was added
to gather data about the
primary motivation for
misuse of tranquilizers
and stimulants. Edits to
logic were made as a
result of cognitive
testing.
Not included in 2012
[IF MORE THAN ONE RESPONSE 1‐9
This question was added
Not included in 2012
30
QFT Variable
(Testing
Outcome)
STY25A
(Change
tested, minor
wording
problems
found,
revisions
Phase
Introduced
Phase 1
QFT Instrument Wording
Initial Test Wording
CHOSEN IN STYMOTIV]
CHOSEN IN STYMOTIV] Which
Which was the main reason
was the main reason you
you used [STLASTFILL2] that
used [STLASTFILL2] that time?
time?
PROGRAMMER: FILL AS RESPONSE
PROGRAMMER: FILL AS RESPONSE
OPTIONS ONLY THOSE
OPTIONS ONLY THOSE
CHOSEN IN STYMOTIV
1.
To help me lose weight
CHOSEN IN STYMOTIV
1. To help me lose weight 2.
To help me concentrate
2. To help me
3.
To help me be alert or stay
concentrate
awake
3. To help me be alert or
4.
To help me study
stay awake
5.
To experiment or to see what
4. To help me study
[IF RSTY42A=5 THEN “they’re”
5. To experiment or to
see what [IF
ELSE “it’s”]like
RSTY42A=5 THEN
6.
To feel good or get high
“they’re” ELSE
7.
To increase or decrease the
“it’s”]like
effect(s) of some other drug
6. To feel good or get
8.
Because I am “hooked” or I
high
have to have [IF RSTY42A=5
7. To increase or
THEN “them” ELSE “it”]
decrease the effect(s)
of some other drug
8. Because I am “hooked”
or I have to have [IF
RSTY42A=5 THEN
“them” ELSE “it”]
9. [IF STMOTIV=9] The
other reason I reported
[IF STMISUSE12=1] At any time in
[IF STMISUSE12=1] At any time in the
the past 12 months, did you ever
past 12 months, did you ever use a
use a needle to inject
needle to inject [STNAMEFILL] in any
[STNAMEFILL]?
way a doctor did not direct you to use
[STNUMFILL]?
Notes/Testing Results
to gather data about the
primary motivation for
misuse of sedatives. Edits
to logic were made as a
result of cognitive
testing.
Question moved from
Special Drugs to
Stimulants main module.
As a result of cognitive
testing, the wording of
the question was
streamlined.
2012 Question Wording
Not included in 2012
31
QFT Variable
(Testing
Phase
Outcome)
Introduced
QFT Instrument Wording
implemented
in QFT.)
SD01 (Change Phase 2
The last questions were about
tested, no
prescription drugs. The next
problems
question is about non‐prescription
found,
cough or cold medicines, also known
implemented
as “over‐the‐counter” medicines.
in QFT.)
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
SD02(Change Phase 2
[IF SD01 = 1] How long has it been
tested, no
since you last took one of these
problems
cough or cold medicines to get high?
found,
implemented
in QFT.)
SD05/SD15
Phase 2
Have you ever, even once, used a
(Change
needle to inject any other drug that
tested, no
was not prescribed for you?
problems
found,
implemented
in QFT.)
SD10a
Phase 2
Have you ever, even once, used a
(Change
needle to inject
tested, no
methamphetamine?
problems
found,
implemented
in QFT.)
MJMM
(Change
tested, no
problems
found,
implemented
Phase 2
Initial Test Wording
Notes/Testing Results
No changes between initial testing and Wording introducing the
final QFT wording.
over the counter
medicine questions was
tailored to redesign
conventions and
question order.
No changes between initial testing and Collected recency data
final QFT wording.
for misuse of cough and
cold medicines.
No changes between initial testing and In order to maintain
final QFT wording.
conventions of revised
misuse wording, deleted
the words, “only for the
experience or feeling it
caused.”
No changes between initial testing and To maintain consistency
final QFT wording.
across modules, and to
limit follow up questions
to data collected in the
stand alone
methamphetamine
module, removed
reference to Desoxyn
and Methedrine.
Questions about medical
[IF (MJLAST3 = 1 ‐ 2 OR MJRECDK = 1 [IF MJ01=1 OR MJREF=1] Was any of
‐ 2 OR MJRECRE = 1 ‐ 2) OR BL03=2 ] your marijuana use recommended by use of marijuana were
a doctor?
added to the redesign
Earlier, you reported using
instrument. After
marijuana in the past year. Was any
cognitive interviewing,
of your marijuana use in the past 12
the wording was tailored
months recommended by a doctor?
2012 Question Wording
The last two questions were about
prescription drugs. The next
question is about non‐prescription
cough or cold medicines, also known
as over the counter medicines.
Have you ever, even once, taken a
non‐prescription cough or cold
medicine just to get high?
Not included in 2012
Have you ever, even once, used a
needle to inject any other drug that
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Have you ever, even once, used a
needle to inject Methamphetamine,
Desoxyn, or Methedrine when it
was not prescribed for you or that
you took only for the experience or
feeling it caused?
Not included in 2012
32
QFT Variable
(Testing
Outcome)
in QFT.)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
MJMM01
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
[IF MJMM=1] Was all of your
[IF MJMM=1] Was all of your
marijuana use in the past 12 months marijuana use recommended by a
recommended by a doctor?
doctor?
DRPR
DRTR
DRST
DRSV
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
NOTE: Example text from DRPR is
displayed below. Equivalent text was
used for DRTR, DRST, & DRSV.
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next
questions. Remember, we are only
interested in prescription pain
relievers that you used in any way a
doctor did not direct you to. Earlier
the computer recorded that in the
past 12 months you used [IF
PRMISCOUNT=1 FILL PRFILL2][IF
PRMISCOUNT>=2 FILL WITH “the
pain relievers listed below” ] in a
way a doctor did not direct you to
use [PRNUMFILL].
[IF PRMISCOUNT>=2 FILL WITH
DRUG NAMES FROM PRY01‐PRY39
BELOW. USE MULTIPLE COLUMNS
AS NEEDED. IF PRY40 = 1, ADD
"Some other prescription pain
reliever".]
The next questions refer to [IF
[IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested in
your use of prescription pain relievers
that were not prescribed for you or
that you used only for the experience
or feeling they caused.
Press [ENTER] to continue.
Notes/Testing Results
to 12 month users of
marijuana only.
Questions about medical
use of marijuana were
added to the redesign
instrument. After
cognitive interviewing,
the wording was tailored
to 12 month users of
marijuana only.
Wording in the
Substance Dependence
and Abuse section was
updated to reflect
revised wording
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
2012 Question Wording
Not included in 2012
IF PAI12MON = 1] Think about your
use of prescription pain relievers
during the past 12 months as you
answer these next questions.
Remember, we are only interested
in your use of prescription pain
relievers that were not prescribed
for you or that you used only for the
experience or feeling they caused.
Press [ENTER] to
continue.
33
QFT Variable
(Testing
Outcome)
Phase
Introduced
DRST05
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
Height
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase 1
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
PRY40 NE 1 AND PRMISCOUNT=1
FILL PRFILL2 as a prescription pain
reliever; IF PRY40 = 1 AND
PRMISCOUNT=1 FILL WITH “this
other prescription pain reliever”; IF
PRMISCOUNT>=2 FILL WITH “these
as prescription pain relievers”].
Press [ENTER] to continue.
[IF DRST04 = 1] Were you able to
No changes between initial testing and Wording in the Drug
keep to the limits you set, or did you final QFT wording.
Dependence and Abuse
section was updated to
often use prescription stimulants
reflect revised wording
more than you intended to?
surrounding prescription
drug misuse. Changes
made after cognitive
interviewing involved
logistical/spec issues
only.
HLTH04 This question asks about
HLTHNEW01
About how tall are
Height and weight
your height.
you, without shoes? First, please type questions were added as
part of the redesign in
in the number of feet, then press
[ENTER].
order to capture more
To answer in feet and inches, press
details on physical health
1. To answer in meters and
To answer using meters and
of respondents. Several
centimeters, press 2. To answer in
centimeters, press F3.
versions of these
inches only, press 3. To answer in
questions were tested in
centimeters only, press 4. Then
press [ENTER] to continue.
______feet [RANGE: 2‐8]
order to make it easier
DK/REF
for respondents to
1
I would rather answer in
answer in either metric
feet and inches
HLTHNEW01a (IF HLTHNEW01 NE
or US measurements.
2
I would rather answer in
DK/RF) Next, please type in the
Edits were also made so
meters and centimeters
number of inches and then press
that respondents could
3
I would rather answer only [ENTER].
answer only in inches or
centimeters.
in inches
4
I would rather answer only ______inches [RANGE: 0‐11]
in centimeters
DK/REF
DK/REF
2012 Question Wording
[IF DRPR04 = 1] Were you able to
keep to the limits you set, or did you
often use prescription pain relievers
more than you intended to?
Not included in 2012
34
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
HLTH05 [IF HLTH04=1] About how
tall are you, without shoes? First,
please type in the number of feet,
then press [ENTER].
______feet [RANGE: 0‐8]
DK/REF
HLTH06 [IF HLTH04 = 1 OR 3 AND
HLTH05 NE DK/RF] Please type in
the number of inches and then press
[ENTER].
______inch(es) [RANGE: 0‐110]
DK/REF
HLTH07 [IF HLTH04=2] About how
tall are you, without shoes? First,
please type in the number of
meters, then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTH08 [IF HLTH04 = 2 OR 4AND
HLTH07 NE DK/REF] Please type in
the number of centimeters and then
press [ENTER].
______centimeter(s) [RANGE:0‐300]
DK/REF
Initial Test Wording
PROGRAMMER: PLEASE DISPLAY
HLTHNEW01 AND HLTHNEW01a ON
THE SAME SCREEN.
HLTHNEW01b (IF HLTHNEW01=DK)
You may also report your height using
meters and centimeters. About how
tall are you, without shoes? First,
please type in the number of meters,
then press [ENTER].
_______ meters [RANGE: 0‐3]
DK/REF
HLTHNEW01c [IF HLTHNEW01b NE
BLANK] Next, please type in the
number of centimeters and then press
[ENTER].
______centimeters [RANGE:0‐275]
DK/REF
PROGRAMMER: PLEASE DISPLAY
THESE HLTHNEW01b AND
HLTHNEW01c ON THE SAME SCREEN.
Notes/Testing Results
2012 Question Wording
35
QFT Variable
(Testing
Outcome)
Weight
questions
(Change
tested,
problems
found,
revisions
implemented
in QFT.)
Phase
Introduced
Phase 1
QFT Instrument Wording
HLTH09 The next question asks
about your weight.
To answer in pounds, press 1. To
answer in kilograms, press 2. Then
press [ENTER] to continue.
1
I would rather answer in
pounds
2
I would rather answer in
kilograms
DK/REF
HLTH10 [IF HLTH09=1 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of pounds and then press [ENTER].
________pounds [RANGE: 50‐550]
DK/REF
HLTH12 [IF HLTH11=2 AND HLTH02
NE 1] About how much do you
weigh? Please type in the number
of kilograms and then press
[ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTH13 [IF HLTH02=1 AND
HLTH09=1] About how much did
you weigh before you got pregnant?
Please type in the number of pounds
and then press [ENTER].
Initial Test Wording
HLTHNEW02
(IF HLTH01 NE 1)
About how much do you weigh?
Please type in the number of pounds
and then press [ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02A (IF HLTHNEW02=DK)
You may also report your weight using
kilograms. About how much do you
weigh? Please type in the number of
kilograms and then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
HLTHNEW02p (IF HLTH01 = 1) About
how much did you weigh before you
got pregnant? Please type in the
number of pounds and then press
[ENTER].
To answer using kilograms, press F3.
________pounds [RANGE: 50‐550]
DK/REF
HLTHNEW02pp (IF HLTHNEW02p
=DK) You may also report your weight
using kilograms. About how much did
you weigh before you got pregnant?
Please type in the number of
kilograms and then press [ENTER].
Notes/Testing Results
2012 Question Wording
Height and weight
Not included in 2012
questions were added as
part of the redesign in
order to capture more
details on physical health
of respondents. Several
versions of these
questions were tested in
order to make it easier
for respondents to
answer in either metric
or US measurements.
36
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTHNEW03
(HLTH19)
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 1
HLTH22
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QFT Instrument Wording
________pounds [RANGE: 50‐550]
DK/REF
HLTH14 [IF HLTH02=1 AND HLTH09
=2] About how much did you weigh
before you got pregnant? Please
type in the number of kilograms and
then press [ENTER].
_______ kilograms [RANGE: 22‐275]
DK/REF
During the past 12 months, how
many times have you visited a
doctor, nurse, physician assistant or
nurse practitioner about your own
health at a doctor’s office, a clinic,
or some other place?
[((IF HLTH16>=1 AND (HLTH17=1 OR
HLTH19>=1)) OR (HLTH17=1 AND
(HLTH16>=1 OR HLTH19>=1 OR
HLTH19DK>=1)) OR ((HLTH19>=1 OR
HLTH19DK>=1) AND (HLTH16>=1 OR
HLTH17=1))) AND (ALLAST3=1 OR 2
OR ALRECDK=1 OR 2 OR ALRECRE=1
OR 2)] Please think about all of the
talks you have had with a doctor or
other health care professional
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussions you may
Initial Test Wording
Notes/Testing Results
2012 Question Wording
_______ kilograms [RANGE: 22‐275]
DK/REF
DK/REF
During the past 12 months, how many
times have you visited a doctor, nurse,
physician assistant or other health
care professional about your own
health at a doctor’s office, a clinic, or
some other place?
Do not include times you were
hospitalized overnight, visits to
hospital emergency rooms, home
visits by a health care professional,
dental visits, or telephone calls.
No changes between initial testing and
final QFT wording.
Questions about health
Not included in 2012
provider visits were
added as part of the
redesign. Instructions
about how to count visits
were deemed
unnecessary or confusing
and removed.
Not included in 2012
Questions were tested
about discussing alcohol
& drug use with a doctor.
No changed were made
as a result of testing.
37
QFT Variable
(Testing
Outcome)
Phase
Introduced
HLTH26othr Phase 2
(Change
tested, no
problems
found,
implemented
in QFT.)
HLTHNEW11 Phase 1
(HLTH29)
(Change
tested, no
problems
found,
QFT Instrument Wording
Initial Test Wording
have had in person with a doctor or
other health professional about your
alcohol use.
[((IF HLTH16>=1 AND HLTH17 NE 1
AND HLTH19<1 AND HLTH19DK<1)
OR (HLTH17 =1 AND HLTH16 = 0
AND HLTH19<1 AND HLTH19DK<1)
OR ((HLTH19 >=1 OR HLTH19DK>=1)
AND HLTH16 = 0 AND HLTH17 NE 1))
AND (ALLAST3=1 OR 2 OR
ALRECDK=1 OR 2 OR ALRECRE=1 OR
2)] Please think about [VISITFILL]
during the past 12 months when you
answer this question. Choose the
statement or statements below that
describe any discussion you may
have had in person with a doctor or
other health professional about your
alcohol use.
To select more than one statement,
press the space bar between each
number you type. When you have
finished, press [ENTER].
No change.
(IF HLTH26=30) Please tell me which
other kind of cancer you had.
[IF HLTH25=9 AND HLTH28 AND
HLTH28a AND HLTH28b AND
HLTH28c AND HLTH28c AND
HLTH28d AND HLTH28e AND
HLTH28f AND HLTH28g AND
HLTH28h AND HLTH28i AND
(IF HLTHNEW08=9) Did you have
cancer during the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
Questions about cancer
were tested. Only
logistical/spec changes
were made as a result of
the testing.
Not included in 2012
38
QFT Variable
(Testing
Outcome)
implemented
in QFT.)
Phase
Introduced
HLTHNEW12a Phase 1
(HLTH31)
(Change
tested, no
problems
found,
implemented
in QFT.)
Disability
NA
Items (QD55‐
QD61) Not
tested
QFT Instrument Wording
HLTH28j AND HLTH28k AND
HLTH28l AND HLTH28m AND
HLTH28n AND HLTH28o AND
HLTH28p AND HLTH28q AND
HLTH28r AND HLTH28s AND
HLTH28t AND HLTH28u AND
HLTH28v AND HLTH28w AND
HLTH28x AND HLTH28y AND
HLTH28z AND HLTH28aa AND
HLTH28bb AND HLTH28cc NE
CALCAGE] Did you have cancer
during the past 12 months?
[IF HLTH25=1 AND HLTH30 NE
CALCAGE] Did you have any kind of
heart condition or heart disease in
the past 12 months?
QD55 How well do you speak
English?
QD56 Are you deaf or do you
have serious difficulty hearing?
QD57 Are you blind or do you
have serious difficulty seeing, even
when wearing glasses?
QD58 Because of a physical,
mental or emotional condition, do
you have serious difficulty
concentrating, remembering, or
making decisions?
QD59 Do you have serious
Initial Test Wording
(IF HLTHNEW08=1) Did you have any
kind of heart condition or heart
disease in the past 12 months?
Notes/Testing Results
2012 Question Wording
Questions about health
conditions were added.
Only logistical/spec
changes were made as a
result of the cognitive
testing.
Not included in 2012
These items were added
in response to feedback
received about the
survey.
Not included in 2012
39
QFT Variable
(Testing
Outcome)
Phase
Introduced
QD17 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
difficulty walking or climbing stairs?
QD60 Do you have difficulty
dressing or bathing?
QD61 [IF CURNTAGE >14]
Because of a physical, mental or
emotional condition, do you have
difficulty doing errands alone such
as visiting a doctors’ office or
shopping?
No changes between initial testing and This question was moved
The next questions are about
final QFT wording.
to ACASI. No changes
school. Are you now attending or
were made as a result of
are you currently enrolled in school?
testing.
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
QD26 (moved Phase 2
to ACASI, no
problems
found,
implemented
in QFT)
No changes between initial testing and This question was moved
[IF CURNTAGE = 15 OR OLDER] The
final QFT wording.
to ACASI.
next questions are about working.
Explanatory/example
Did you work at a job or business at
text was added as
any time last week? By last week, I
optional text available
mean the week beginning on
with pressing the F2 key.
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
No changes were made
as a result of testing.
Press F2 to hear more information
about what types of work to include.
Military
QD10d For this question, please
NA
These items were added
2012 Question Wording
The next questions are about
school. Are you now attending or
are you currently enrolled in school?
By “school,” we mean an
elementary school, a junior high or
middle school, a high school, or a
college or university. Please include
home schooling as well.
1
YES
2
NO
DK/REF
[IF CURNTAGE = 15 OR OLDER] The
next questions are about working.
Did you work at a job or business at
any time last week? By last week, I
mean the week beginning on
Sunday, [STARTDATE] and ending on
Saturday, [ENDDATE].
1
YES
2
NO
DK/REF
Not included in 2012
40
QFT Variable
(Testing
Outcome)
Family Items
(QD10d &
QD10e) Not
tested
Phase
Introduced
PROXYINT
NA
NOPROX
(HINSINT)
(revisions
Phase 2
QFT Instrument Wording
include all persons in your
immediate family, whether or not
they live with you. Is anyone in your
immediate family currently serving
in the United States military? Press
F2 to see and hear definitions of
“immediate family” and “military.”
QD10e Which member or members
of your immediate family are
currently in the United States
military? Press F2 to see and hear
definitions of “immediate family.”
1 My spouse
2 Unmarried partner
3 My mother
4 My father
5 My son or sons
6 My daughter or
daughters
7 My brother or brothers
8 My sister or sisters
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you [IF FAMILY MEMBERS IN
ROSTER FILL “and your family”]
receive. This information will help in
planning health care services and
finding ways to lower costs of care.
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR
HASJOIN= 2 OR DK/REF] I’m going to
Initial Test Wording
Notes/Testing Results
in response to feedback
received about the
survey.
2012 Question Wording
The next questions are about your
health insurance coverage and the
kinds and amounts of income that
you receive. (This information will
help in planning health care services
and finding ways to lower costs of
care.)
New language was added Not included in 2012
[IF QP01=2 OR QP03 = 2 OR DK/REF
OR QP04 = 2 OR DK/REF OR HASJOIN = to assist the interviewer
with transitioning back to
2 OR DK/REF] I’m going to give the
Revised wording after
Phase 2 testing to
encourage nominations
of proxies when
necessary.
41
QFT Variable
(Testing
Outcome)
tested,
problems
found,
changes
implemented
in QFT)
QHI03
(Wording
revised for
accuracy
after testing,
implemented
in QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
give the computer back to you so
that you can complete the last part
of the interview on your own.
[IF (QP03 = 2 OR DK/REF) OR (QP04
= 2 OR DK/REF) OR (HASJOIN = 2 OR
DK/REF) ADD: Since your [QP02 FILL]
is not available, please answer these
next questions the best you can.]
[IF NEWPROX = 2 AND HASJOIN=1]
I’m going to give the computer to
you so that you can complete the
last part of the interview on your
own.
Please put on the headphones.
When you are ready, let me know.
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
There are certain programs that
cover active duty and retired career
military personnel and their
dependents and survivors and also
disabled veterans and their
dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to see and hear more
information about these programs.
CHAMPUS stands for Civilian
Health and Medical Program of
Initial Test Wording
computer back to you so that you can
complete the last part of the interview
on your own. When you are ready, let
me know.
[IF QP03 = 2 OR DK/REF OR QP04 = 2
OR DK/REF OR HASJOIN = 2 OR DK/REF
ADD THIS TEXT PRIOR TO THE
QUESTION: Since your [QP02 FILL] is
not available, please answer these
next questions the best you can.]
MOVE COMPUTER SO RESPONDENT
CAN USE IT. PRESS [ENTER] TO
CONTINUE.
Notes/Testing Results
ACASI for the final
questions. Language was
tailored to situations
where a proxy might be
required to answer the
income series. As a
result of testing,
revisions were made to
make the proxy
transition happen more
smoothly.
2012 Question Wording
There are certain programs that cover
active duty and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
Press F2 to hear more information
about these programs.
CHAMPUS stands for Comprehensive
Health and Medical Plan for the
Wording was revised for
ACASI administration,
and later to accurately
note program names.
[SAMPLE MEMBER A] currently
covered by TRICARE, or CHAMPUS,
CHAMPVA, the VA, or military health
care?
(These programs cover active duty
and retired career military
personnel and their dependents and
survivors and also disabled veterans
and their dependents and survivors.)
1
YES
2
NO
DK/REF
INTERVIEWER NOTE:
42
QFT Variable
(Testing
Outcome)
INTROINC
(revisions
tested, minor
problems
found,
changes
made for
QFT)
Phase
Introduced
Phase 2
QFT Instrument Wording
the Uniformed Services. It
provides health care in private
facilities for dependents of
military personnel on active duty
or retired for reasons other than
disability. In some areas, this
may be known as TRICARE.
Initial Test Wording
Uniformed Services. It provides health
care in private facilities for
dependents of military personnel on
active duty or retired for reasons
other than disability. In some areas,
this may be known as TRICARE.
CHAMPVA stands for Comprehensive
Health and Medical Plan of the
CHAMPVA stands for Civilian
Veterans Administration. It provides
Health and Medical Program of
health care for the spouse,
the Department of Veterans
dependents, or survivors of a veteran
Affairs. It provides health care
who has a total, permanent service‐
for the spouse, dependents, or
survivors of a veteran who has a connected disability.
total, permanent service‐
The VA provides medical assistance to
connected disability.
veterans of the Armed Forces,
particularly those with service‐
The VA provides medical
connected ailments.
assistance to veterans of the
Armed Forces, particularly those
with service‐connected ailments. Military health care refers to health
care available to active duty personnel
and their dependents.
Military health care refers to
health care available to active
duty personnel and their
dependents.
[IF NO FAMILY MEMBERS IN ROSTER]
[IF NO FAMILY MEMBERS IN
These next questions are about the
ROSTER] These next questions are
kinds and amounts of income that you
about the kinds and amounts of
receive.
income that you receive.
[IF ONE FAMILY MEMBER IN ROSTER [IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] These next
AND HASJOIN NE 1 ] These next
questions are about the kinds and
questions are about the kinds and
amounts of income that you and
amounts of income that you and your
your [FAMILY RELATIONSHIP FILL]
[FAMILY RELATIONSHIP FILL] receive.
receive.
Notes/Testing Results
2012 Question Wording
CHAMPUS stands for
Comprehensive Health and Medical
Plan for the Uniformed Services. It
provides health care in private
facilities for dependents of military
personnel on active duty or retired
for reasons other than disability. In
some areas, this may be known as
TRICARE.
CHAMPVA stands for
Comprehensive Health and Medical
Plan of the Veterans Administration.
It provides health care for the
spouse, dependents, or survivors of
a veteran who has a total,
permanent service‐connected
disability.
Military health care refers to health
care available to active duty
personnel and their dependents; in
addition, the VA provides medical
assistance to veterans of the Armed
Forces, particularly those with
service‐connected ailments.
Wording was revised for [IF NO FAMILY MEMBERS IN
ACASI administration. As ROSTER] These next questions are
a result of the cognitive
about the kinds and amounts of
income that you receive.
testing, changes were
made to make the
[IF ONE FAMILY MEMBER IN ROSTER
process go more
AND HASJOIN NE 1 ] These next
smoothly.
questions are about the kinds and
amounts of income that you and
your [FAMILY RELATIONSHIP FILL]
receive.
43
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you and your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, you and [IF
QD01=5 FILL his, QD01 = 9 FILL
her][FAMILY RELATIONSHIP FILLS]
living here – receive.
[PROGRAMMER NOTE: THE PROXY
SHOULD NOT APPEAR IN [FAMILY
RELATIONSHIP FILLS]. ALSO, USE
‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF HASJOIN NE 1] These questions
refer to the calendar year [CURRENT
YEAR ‐ 1] rather than to the past 12
Initial Test Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
you receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] These
next questions are about the kinds
and amounts of income that you, your
[FAMILY RELATIONSHIP FILLS] living
here receive.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN=1] These
next questions are about the kinds
and amounts of income that [SAMPLE
MEMBER] and [IF QD01=5 FILL his,
QD01 = 9 FILL her] family – that is,
your [SAMPLE MEMBER POSS]
[FAMILY RELATIONSHIP FILLS] living
here – receive.
These questions refer to the calendar
year [CURRENT YEAR ‐ 1] rather than
to the past 12 months that were
referred to in some earlier questions.
The calendar year [CURRENT YEAR ‐ 1]
would be from January 1st, [CURRENT
YEAR ‐ 1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
Notes/Testing Results
2012 Question Wording
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] These next
questions are about the kinds and
amounts of income that [SAMPLE
MEMBER] and you receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] These next questions are
about the kinds and amounts of
income that you, your [FAMILY
RELATIONSHIP FILLS] living here
receive.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND
HASJOIN=1] These next questions
are about the kinds and amounts of
income that [SAMPLE MEMBER] and
[IF QD01=5 FILL his, QD01 = 9 FILL
her] family – that is, your [SAMPLE
MEMBER POSS] [FAMILY
RELATIONSHIP FILLS] living here –
receive.
These questions refer to the
calendar year [CURRENT YEAR ‐ 1]
rather than to the past 12 months
that were referred to in some earlier
questions. The calendar year
[CURRENT YEAR ‐ 1] would be from
January 1st, [CURRENT YEAR ‐ 1],
through December 31st, [CURRENT
YEAR ‐ 1].
PRESS [ENTER] TO CONTINUE
44
QFT Variable
(Testing
Outcome)
QI12AN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
months that were referred to in
some earlier questions. The calendar
year [CURRENT YEAR ‐ 1] would be
from January 1st, [CURRENT YEAR ‐
1], through December 31st,
[CURRENT YEAR ‐ 1].
Press [ENTER] to continue
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Any other kind of non‐
monetary welfare or public
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family member
living here receive any type of welfare
or public assistance?
# OF MONTHS RECEIVED ASSISTANCE:
[RANGE: 1 ‐ 12]
DK/REF
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
45
QFT Variable
(Testing
Outcome)
QI12BN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
assistance
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Please include:
Cash assistance from a
state or county welfare program
such as [TANFFILL]
Initial Test Wording
Notes/Testing Results
2012 Question Wording
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did you
or your [FAMILY RELATIONSHIP FILL]
receive any type of welfare or public
assistance, not including food stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive any
type of welfare or public assistance,
not including food stamps?
[IF AT LEAST TWO FAMILY MEMBER IN
ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
Wording was revised for
ACASI administration. As
a result of the cognitive
testing, changes were
made to make the
process go more
smoothly.
[IF (QI08N=1 OR QI10N=1) AND
QI07N=(1, DK OR REF)]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1] For how many
months in [CURRENT YEAR ‐ 1] did
you or your [FAMILY RELATIONSHIP
FILL] receive any type of welfare or
public assistance, not including food
stamps?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] For how many
months in [CURRENT YEAR ‐ 1] did
[SAMPLE MEMBER] or you receive
any type of welfare or public
assistance, not including food
stamps?
[IF AT LEAST TWO FAMILY MEMBER
IN ROSTER] For how many months in
[CURRENT YEAR ‐ 1] did [SAMPLE
MEMBER] or any other family
member living here receive any type
of welfare or public assistance, not
including food stamps?
# OF MONTHS RECEIVED
ASSISTANCE: [RANGE: 1 ‐
12]
DK/REF
46
QFT Variable
(Testing
Outcome)
Phase
Introduced
INTRTINN
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
QI21B
(Change
tested, no
problems
Phase 2
QFT Instrument Wording
Any other kind of non‐
monetary welfare or public
assistance
Below is a list of some other sources
of income. When I ask you the next
questions, please consider these as
well as the other sources asked
about in earlier questions.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
Press [Enter] to continue.
[IF QI20N = 1 OR QI20NREF = 1] Of
these income groups, which
category best represents [SAMPLE
MEMBER POSS] total personal
Initial Test Wording
Notes/Testing Results
2012 Question Wording
Below is a list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we just
talked about.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
Wording was revised for
ACASI administration.
No changes were made
as a result of cognitive
testing.
HAND R SHOWCARD 16a. Here is a
list of some other sources of
income. When you answer the next
questions, please consider these as
well as the other sources that we
just talked about.
INTERVIEWER: PLEASE READ THIS
ALOUD TO THE RESPONDENT AS
HE/SHE FOLLOWS ALONG ON THE
SHOWCARD.
Veteran’s Administration payments
Other disability, retirement or
survivor pension
Unemployment or worker’s
compensation
Interest income
Dividends from stocks or mutual
funds
Income from rental properties,
royalties, estates or trusts
Alimony
Child support
PRESS [ENTER] TO CONTINUE.
[IF QI20N = 1] Of these income groups,
which category best represents
[SAMPLE MEMBER POSS] total
personal income during [CURRENT
Income categories were
revised to adjust for
inflation and the
question was revised for
[IF QI20N = 1] ENTER NUMBER THAT
BEST REPRESENTS (R’S/SAMPLE
MEMBER’S) TOTAL PERSONAL
INCOME DURING [CURRENT YEAR ‐
47
QFT Variable
(Testing
Outcome)
found,
implemented
in QFT.)
INTROFI1
(Change
tested, minor
problems
found,
changes
made for
QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
income during [CURRENT YEAR ‐ 1]?
1
$20,000 ‐ $24,999
2
$25,000 ‐ $29,999
3
$30,000 ‐ $34,999
4
$35,000 ‐ $39,999
5
$40,000 ‐ $44,999
6
$45,000 ‐ $49,999
7
$50,000 ‐ $74,999
8
$75,000 ‐ $99,999
9
$100,000 ‐ $149,999
10
$150,000 or more
DK/REF
Initial Test Wording
YEAR ‐ 1]?
21
$20,000 ‐ $24,999
22
$25,000 ‐ $29,999
$30,000 ‐ $34,999
23
24
$35,000 ‐ $39,999
25
$40,000 ‐ $44,999
26
$45,000 ‐ $49,999
27
$50,000 ‐ $74,999
28
$75,000 ‐ $99,999
29
$100,000 ‐ $149,999
30
$150,000 or more
DK/REF
[IF MORE THAN ONE FAMILY MEMBER
[IF MORE THAN ONE FAMILY
IN ROSTER AND IF QI21B NE 29]
MEMBER IN ROSTER AND IF QI21B
NE 30]
Next, we would like to know about the
total family income from all sources
Next, we would like to know about
during [CURRENT YEAR ‐ 1] before
the total family income from all
sources during [CURRENT YEAR ‐ 1] taxes and other deductions.
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN NE 1 ] We would like
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
you to combine everyone’s income – that is, yours and that of your [FAMILY
that is, yours and that of your
RELATIONSHIP FILL].
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
[IF ONE FAMILY MEMBER IN ROSTER AND HASJOIN=1] We would like you to
AND HASJOIN=1] We would like you combine everyone’s income – that is,
to combine everyone’s income –
[SAMPLE MEMBER POSS] and yours.
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] We
[IF AT LEAST TWO FAMILY
would like you to combine everyone’s
MEMBERS IN ROSTER AND HASJOIN income – that is, yours and that of
Notes/Testing Results
ACASI administration. No
changes were made as a
result of cognitive
testing.
2012 Question Wording
1].
21
22
23
24
25
26
27
28
29
$20,000 ‐ $24,999
$25,000 ‐ $29,999
$30,000 ‐ $34,999
$35,000 ‐ $39,999
$40,000 ‐ $44,999
$45,000 ‐ $49,999
$50,000 ‐ $74,999
$75,000 ‐ $99,999
$100,000 OR MORE
DK/REF
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration.
Edits were made to
family relationship fills to
result in a more natural
sentence structure.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
48
QFT Variable
(Testing
Outcome)
QI23A
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is, yours,
[SAMPLE MEMBER POSS] and that of
[IF QD01 = 5 FILL his, QD = 9 FILL
her] [FAMILY RELATIONSHIP FILLS]
living here. [PROGRAMMER NOTE:
THE PROXY SHOULD NOT APPEAR IN
[FAMILY RELATIONSHIP FILLS]. ALSO,
USE ‘other’ AS A MODIFIER TO THE
FAMILY RELATIONSHIP FILL WHEN
THE RELATIONSHIP TYPE IS EQUAL
TO PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
Initial Test Wording
your [FAMILY RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN = 1] We
would like you to combine everyone’s
income — that is, [SAMPLE MEMBER
POSS] and that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here.
Please include all of the sources of
income that we just talked about.
Notes/Testing Results
2012 Question Wording
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
[IF QI22=2 OR QI22REF=2]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, yours and that of your [FAMILY
RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
testing.
Next, we would like to know about
the total family income from all
sources during [CURRENT YEAR ‐ 1]
before taxes and other deductions.
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] We would like
you to combine everyone’s income –
that is, yours and that of your
[FAMILY RELATIONSHIP FILL].
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] We would like you
to combine everyone’s income –
49
QFT Variable
(Testing
Outcome)
QI23B
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase
Introduced
Phase 2
QFT Instrument Wording
YEAR – 1] – that is, your [SAMPLE
MEMBER POSS] and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 = 9
FILL her] [FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these
income groups, which category best
represents your total combined
Initial Test Wording
that is, your [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
Notes/Testing Results
[IF (QI22=1 OR QI20N = 1 OR
QI22REF=1) AND Q121B NE 30]
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN NE 1 ] Of these income
groups, which category best
represents your total combined family
Income categories were
revised to adjust for
inflation and the
question was revised for
ACASI administration. No
changes were made as a
result of cognitive
2012 Question Wording
that is, [SAMPLE MEMBER POSS]
and yours.
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] We would like you to combine
everyone’s income – that is, yours
and that of your [FAMILY
RELATIONSHIP FILLS].
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
= 1] We would like you to combine
everyone’s income — that is,
[SAMPLE MEMBER POSS] and that of
SAMPLE MEMBER POSS][FAMILY
RELATIONSHIP FILLS] living here.
Please include all of the sources of
income that we just talked about.
Collapsed from multiple 2012
variables due to change in modes.
50
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
family income during [CURRENT
YEAR – 1] – that is, yours and that of
your [FAMILY RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined
family income during [CURRENT
YEAR – 1] – that is, [SAMPLE
MEMBER POSS] and yours?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
NE 1] Of these income groups,
which category best represents your
total combined family income
during [CURRENT YEAR – 1] – that is,
yours and that of your [FAMILY
RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY
MEMBERS IN ROSTER AND HASJOIN
=1] Of these income groups, which
category best represents your total
combined family income during
[CURRENT YEAR ‐ 1]‐‐ that is, yours,
[SAMPLE MEMBER POSS], and that
of [IF QD01 = 5 FILL his, QD01 =9
FILL her][FAMILY RELATIONSHIP
FILLS] living here? [PROGRAMMER
NOTE: THE PROXY SHOULD NOT
APPEAR IN [FAMILY RELATIONSHIP
FILLS]. ALSO, USE ‘other’ AS A
MODIFIER TO THE FAMILY
RELATIONSHIP FILL WHEN THE
RELATIONSHIP TYPE IS EQUAL TO
PROXY RELATIONSHIP TYPE AND
Initial Test Wording
Notes/Testing Results
income during [CURRENT YEAR – 1] –
testing.
that is, yours and that of your [FAMILY
RELATIONSHIP FILL]?
[IF ONE FAMILY MEMBER IN ROSTER
AND HASJOIN=1] Of these income
groups, which category best
represents your total combined family
income during [CURRENT YEAR – 1] –
that is, [SAMPLE MEMBER POSS] and
yours?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN NE 1] Of
these income groups, which category
best represents your total combined
family income during [CURRENT YEAR
– 1] – that is, yours and that of your
[FAMILY RELATIONSHIP FILLS]?
[IF AT LEAST TWO FAMILY MEMBERS
IN ROSTER AND HASJOIN =1] Of these
income groups, which category best
represents your total combined family
income during [CURRENT YEAR ‐ 1]‐‐
that is, [SAMPLE MEMBER POSS] and
that of SAMPLE MEMBER
POSS][FAMILY RELATIONSHIP FILLS]
living here?
2012 Question Wording
51
QFT Variable
(Testing
Outcome)
Phase
Introduced
CELL1
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
CELL2
(Change
tested, no
problems
found,
implemented
in QFT.)
Phase 2
(MJE01 –
MJE70)
(entire
existing
module
dropped)
LU22 – LU26
(existing
question
dropped)
None
None
QFT Instrument Wording
Initial Test Wording
ONE OF THESE RELATIONSHIP TYPES
IS STILL IN THE LIST.]
Is there at least one telephone at
Is there at least one telephone at this
this address that is not a cell phone? address that is not a cell phone?
Do you or anyone at this address
have a working cell phone?
Do you or anyone at this address have
a working cell phone?
Notes/Testing Results
2012 Question Wording
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
The last question has to do with
telephones in your household. How
many different telephone numbers
do you have in this household?
Please don’t include cellular phones
in your answer. Also, don’t count
business numbers or extensions
with the same number.
INTERVIEWER NOTE:
Do not include phone lines that are
used only for fax machines and/or
Internet access.
Not included in 2012
This question was added
to allow us to use NSDUH
to assess telephone
survey bias of substance
use and mental health
estimates, and to
address the changing
environment regarding
land line vs. cell phones
2012 instrument items or modules removed from consideration for the QFT
Module dropped from QFT.
Module dropped from QFT.
These questions were
outdated in terms of the
current marijuana
market. They were
removed to make room
for other important
topics.
Item dropped from QFT.
Item dropped from QFT.
Dropped “which came
first” questions from the
Prior Substance Use
module to make room for
Market Information for Marijuana
Module
LU22
Earlier, the computer
recorded that you started using
alcohol and cigarettes when you
were [AFUFILL] years old. Which
52
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
SEN04/YE04
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
CA12‐CA14d
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Notes/Testing Results
additions to the
Prescription Drug
modules.
This question had little
analytic value and was
dropped to make room
for more important
topics.
The Alcohol module now
measures gender specific
binge drinking
prevalence, so the ‘4 or
more drinks’ questions
were deleted.
2012 Question Wording
did you use first?
LU23 Earlier, the computer
recorded that you started using
marijuana and cigarettes when you
were [AFUFILL] years old. Which
did you use first?
LU24 Earlier, the computer
recorded that you started using
alcohol and marijuana when you
were [AFUFILL] years old. Which
did you use first?
LU25 Earlier, the computer
recorded that you started using
alcohol, cigarettes and marijuana
when you were [AFUFILL] years old.
Which did you use first?
LU26 After first using [FILL
LU25], which of these did you use
next?
How many times have you moved in
the past 5 years?
CA12 Have you ever had 4 or more
drinks on the same occasion?
CA13 During the past 30 days, that
is, since [DATEFILL], on how many
days did you have 4 or more drinks
on the same occasion?
CA14 How old were you the first
53
QFT Variable
(Testing
Outcome)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
Industry and
Occupation
Questions –
INOC01 –
INOC06
(existing
question
dropped)
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these questions
due to low levels of
analytic utility and
difficulties administering
these questions in ACASI
Household
Roster –
None
Item dropped from QFT.
Item dropped from QFT.
Dropped these items due
to low analytic utility
2012 Question Wording
time you had 4 or more drinks on
the same occasion?
INOC01
[IF QD26 =
1 OR QD27 = 1] In what kind of
business or industry do you work?
That is, what product is made or
what service is offered?
INOC02 [IF QD26=1 OR QD27=1
AND INOC01 NE DK/REF] HAND R
SHOWCARD 7. Which of these
categories best describes the
business or industry in which you
work?
INOC02M What do they make?
INOC02T What do they sell?
INOC03Please describe the business
or industry in which you work.
INOC04 What kind of work do you
do? That is, what is your
occupation?
INOC05 What are your most
important activities or duties in that
job?
INOC06 Which of these categories
best describes the business in which
you work?
Is (s)he your biological, step‐,
adoptive, or foster [FILL
54
QFT Variable
(Testing
Outcome)
FTHRTYPE,
MTHRTYPE,
SONTYPE,
DAUTYPE
(existing
question
dropped)
Phase
Introduced
QFT Instrument Wording
Initial Test Wording
Notes/Testing Results
2012 Question Wording
RELATIONSHIP]?
55
2015 NSDUH, Supporting Statement
Attachment Z – Contact Materials Study Report
2009 NATIONAL SURVEY ON
DRUG USE AND HEALTH
REPORT ON THE CONTACT
MATERIALS FOCUS GROUP
FINDINGS AND
RECOMMENDATIONS
Contract No. 283-2004-00022
RTI Project No. 0209009
Authors:
Project Director:
Doug Currivan
Emilia Peytcheva
Sonia Rodriguez
Stephanie Stolzenberg
Jim Leiman (Morpace, Inc.)
Arturo Obscura (Morpace, Inc.)
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
November 2009
2009 NATIONAL SURVEY ON
DRUG USE AND HEALTH
REPORT ON THE CONTACT
MATERIALS FOCUS GROUP
FINDINGS AND
RECOMMENDATIONS
Contract No. 283-2004-00022
RTI Project No. 0209009
Authors:
Project Director:
Doug Currivan
Emilia Peytcheva
Sonia Rodriguez
Stephanie Stolzenberg
Jim Leiman (Morpace, Inc.)
Arturo Obscura (Morpace, Inc.)
Thomas G. Virag
Prepared for:
Substance Abuse and Mental Health Services Administration
Rockville, Maryland 20857
Prepared by:
RTI International
Research Triangle Park, North Carolina 27709
November 2009
Acknowledgments
This report would not be possible without the guidance and input of staff from the Center for
Behavioral Health Statistics and Quality. In particular, Joel Kennet provided useful comments.
At RTI International (a trade name of Research Triangle Institute), Roxanne Snaauw and Richard
Straw provided report production assistance.
Table of Contents
Chapter
Page
1.
Introduction ......................................................................................................................... 1
2.
Design and Methods ........................................................................................................... 3
2.1
Purpose and Goals of Focus Groups ....................................................................... 3
2.2
Focus Group Protocol and Procedures .................................................................... 4
2.3
Recruitment of Participants..................................................................................... 5
2.4
Characteristics of Participants................................................................................. 6
3.
Results ................................................................................................................................. 9
3.1
Lead Letter Envelope .............................................................................................. 9
3.1.1 Considerations in Deciding Whether to Open a Piece of Mail ................... 9
3.1.2 Reactions to the Two NSDUH Envelopes ................................................ 10
3.2
Lead Letter Text .................................................................................................... 12
3.3
Lead Letter Graphics............................................................................................. 15
3.4
Question and Answer Brochure ............................................................................ 17
3.5
Special Concerns of Spanish-Speaking Participants............................................. 19
3.6
Additional Suggestions ......................................................................................... 20
4.
Summary and Recommendations ..................................................................................... 21
4.1
Lead Letter Envelopes .......................................................................................... 21
4.2
Lead Letter Text .................................................................................................... 22
4.3
Lead Letter Graphics............................................................................................. 22
4.4
Q&A Brochure ...................................................................................................... 23
5.
References ......................................................................................................................... 25
iii
iv
List of Appendices
Appendix
Page
A
Focus Group Moderator’s Guide .................................................................................... A-1
B
Participant Informed Consent Form................................................................................ B-1
C
Incentive Receipt ............................................................................................................ C-1
D
Sample Focus Group Recruitment Advertisement ......................................................... D-1
E
Focus Group Recruitment and Screening Protocol ..........................................................E-1
F
Contact Materials – Lead Letter Envelope (Regular Size) .............................................. F-1
G
Contact Materials – Lead Letter Envelope (9 x 12)........................................................ G-1
H
Contact Materials – Lead Letter Version 1a (Text Only) ............................................... H-1
I
Contact Materials – Lead Letter Version 2a (Text Only) ................................................. I-1
J
Contact Materials – Lead Letter Version 3a (Text Only) ................................................ J-1
K
Contact Materials – Lead Letter Version 1b (Graphics Only) ........................................ K-1
L
Contact Materials – Lead Letter Version 2b (Graphics Only) .........................................L-1
M
Contact Materials – Lead Letter Version 3b (Graphics Only) ....................................... M-1
N
Contact Materials – Q&A Brochure Version 1............................................................... N-1
O
Contact Materials – Q&A Brochure Version 2............................................................... O-1
v
vi
List of Tables
Table
Page
2.1
Characteristics of English Focus Group Participants.......................................................... 7
2.2
Characteristics of Spanish Focus Group Participants ......................................................... 7
3.1
Preference Counts for Each Type of Contact Material ..................................................... 11
vii
viii
1. Introduction
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance
Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S.
civilian noninstitutionalized population aged 12 or older. The conduct of NSDUH is paramount
in meeting a critical objective of SAMHSA's mission to maintain current data on the prevalence
of substance use in the United States.
Like other surveys, the potential for nonresponse bias is an ongoing concern.
Nonresponse bias is a function of the nonresponse rate and differences between respondents and
nonrespondents on key survey variables. On NSDUH, the key survey items are substance use
prevalence rates. Historically, interview response rates have been lower among some
demographic subgroups, such as those 50 or older (50+) (Murphy, Eyerman, & Kennet, 2004),
and to a lesser extent those living in urban areas and males (Table B.5 in the 2006 Summary of
Findings). Recent NSDUH data indicate sample members who are more likely to be
underrepresented in the interview data:
•
Men are somewhat less likely than women to participate.
•
Adults over 50 years old, and especially those 65 or older, are somewhat less likely
than adults under 50 to participate.
•
Respondents who are categorized in the Other race subgroup and White respondents
who are not Hispanic are somewhat less likely to participate than respondents who
are Hispanic or non-Hispanic Black.
Lower response rates among subgroups may increase bias in overall estimates of prevalence
rates. Increasing response rates across all groups would help minimize bias.
One way to ensure high participation across subgroups in the population would be to
improve any aspects of the survey protocol that could directly influence the decision to
participate. In the spring of 2005, RTI carried out 12 focus groups with potential respondents to
examine the issue of nonresponse among persons 50 and over (Murphy, Schwerin, Hewitt, &
Safir, 2005). This study was designed to explore how persons age 50 and over perceived the
NSDUH interview request. Subjects were shown a video of an NSDUH interviewer contacting a
sample member, enlisting cooperation to complete the household screening, and then enlisting
cooperation for a selected household member to conduct the interview. Participants were also
provided with copies of the materials given to respondents, including the lead letter, question and
answer (Q&A) brochure, and other materials. Focus group members were then asked questions
about the screening and interview process, as well as the contact materials being used during
these processes. One set of recommendations from this study focused on examining how well the
NSDUH contact materials:
•
establish the legitimacy of the sponsoring and research organizations,
•
clearly convey the survey objectives and importance of participation, and
•
describe the selection process and importance of the selected individual's
participation.
1
Conveying this information to sample members is clearly dependent on the screening
respondent actually seeing the materials. The results of a cursory analysis of whether the
screening respondent recalled seeing the lead letter in the 2001 NSDUH 1 indicated that
approximately 48 percent of respondents aged 50+, 48 percent of respondents aged 35 to 49,
45 percent of respondents aged 26 to 34, and 35 percent of respondents aged 18 to 25 recalled
seeing the lead letter.
The investigation described in this report was designed to utilize feedback from focus
groups to identify ways to improve the NSDUH contact materials to maximize cooperation
among sample members. To this end, SAMHSA and RTI undertook two specific tasks. First,
alternative versions of the lead letter envelopes, lead letters, and question and answer (Q&A)
brochure were developed based on survey design principles usually associated with the decision
to participate (Groves, Cialdini, & Couper, 1992).The final versions of the lead letters and Q&A
brochures were also translated into Spanish. 2 Second, focus groups with participants from
different parts of the United States were conducted to evaluate how members of the target
population would react to the different versions of the contact materials. This report describes the
procedures followed and results obtained from 17 focus groups involving both English and
Spanish-speaking adults across the United States. In addition to describing the key themes
emerging from these focus group discussions, this report presents general recommendations for
revising the contact materials based on the findings.
1
NSDUH was titled the "National Household Survey on Drug Abuse" (NHSDA) prior to 2002.
For the focus groups, the return address or other text on the lead letter envelopes was not translated into
Spanish. NSDUH lead letters that include the Spanish translation of the letter on the reverse side are currently
mailed in the same envelopes as English-only letters.
2
2
2. Design and Methods
2.1
Purpose and Goals of Focus Groups
Given that the purpose of the overall investigation was to improve the design of the
NSDUH contact materials in ways likely to maximize cooperation rates, a key objective of this
study was to evaluate alternative versions of the materials through focus group discussions. The
contact materials developed for the focus groups included multiple versions of the lead letter
envelope, the lead letter text without graphics, the lead letter graphics without text, and the Q&A
brochure. The revised contact materials resulting from this study will likely be used for the
upcoming NSDUH redesign, which is currently slated to begin in 2013.
The feedback provided by focus groups will assist in identifying which of the lead letter
envelope, lead letter, and Q&A brochure designs might maximize positive responses to requests
to complete the screener and interview among households selected for NSDUH. As Fowler
(1995) noted, focus groups can quickly broaden researchers' perspective on how people think
about the issues under study. As such, the focus groups conducted for the NSDUH contact
materials can help in two ways:
1. To examine assumptions about the likely reactions members of the target population
would have toward the materials and participation in the survey and
2. To evaluate assumptions about how members of the target population understand the
key concepts and specific terms used to describe the survey.
The focus group sessions primarily provided participants' responses to a set of structured
questions and probes, but also allowed participants to add spontaneous comments on the
materials. The qualitative information derived from the focus groups included both overall
reactions to the contact materials as well as specific feedback on individual elements. We also
planned to count participants' preferences for specific versions of each type of contact material,
to provide some quantitative context for the reactions and feedback.
Another important goal of the focus groups was to involve members of the target
population who represented different regions of the country and different languages spoken. For
this reason, a total of 17 focus groups were conducted across five metropolitan areas—Chicago,
Dallas-Fort Worth, Los Angeles, Raleigh-Durham, and Washington, DC. To ensure
representation of U.S. residents who speak primarily English and those who speak primarily
Spanish, 11 of the focus groups were conducted in English and 6 in Spanish.
A final objective in conducting the focus groups was to ensure the groups were conducted
by experienced professionals who did not have a direct stake in the specific study materials or
protocol. This facilitated a more objective treatment of the issues and ensured that detailed
knowledge of the NSDUH did not unduly influence the direction or content of the focus group
discussions. RTI hired professional focus group moderators from Morpace, Inc., to conduct both
the English and Spanish focus groups. In addition to having extensive experience in conducting
focus groups, the Morpace moderators also had experience and training in survey research
methods.
3
2.2
Focus Group Protocol and Procedures
The focus group protocol guide was developed by RTI and SAMHSA, translated into
Spanish by RTI, and distributed to the Morpace moderators to study. The protocol is included
with this report as Appendix A. The protocol consisted of six major sections:
1. informed consent,
2. introduction,
3. lead letter envelope discussion,
4. lead letter discussion,
5. Q&A brochure discussion, and
6. concluding remarks.
The informed consent part of the protocol took about five minutes and provided an
overview of the study and specifics about the participants' rights as study volunteers. The
participant informed consent form is included with this report as Appendix B. The introduction
also lasted about 5 minutes and was intended to set up the discussion rules and acquaint the
participants in each group. The discussion of the lead letter envelopes was designed to take about
15 minutes and involved discussions of (1) how respondents screen their mail and (2) how they
perceive alternative versions of the envelope. About 45 minutes was allotted for the discussion of
the lead letter and intended to cover both alternative versions of the letter text and graphics. The
section on the Q&A brochure was planned for about 30 minutes and was aimed at comparing the
current brochure (version 1) with a redesigned version (version 2). The discussion of the Q&A
brochure involved participants' preferences in terms of the content, visuals, and design. At the
completion of the discussion of each type of contact material, the moderator encouraged
participants to indicate which version they preferred. The concluding section of the protocol was
intended to give both participants and observers a final opportunity to ask questions or make
comments. At the end of each focus group session, each participant received $75 in cash and
signed a receipt for the incentive. The incentive receipt form is included with this report as
Appendix C.
Both the English and Spanish focus groups followed the same protocol. Up to 10
participants comprised each focus group, with most sessions including 7 to 9 people. Upon
entering the room, participants were presented with two copies of the informed consent form.
The moderator briefly explained the consent form and asked participants to read over it, sign
both copies, return one to the moderator, and keep the other for their records. Next, the
moderator introduced some ground rules related to cell phone use, taking breaks, and side
conversations among participants. He also encouraged participants to share their opinions,
especially when they differed from the views expressed by other participants. The moderator
explained that each session was video and audio recorded and briefly introduced himself. He
then went around the room and asked each participant to briefly introduce himself or herself.
After the introduction, the moderator began following the protocol for the discussion of
each type of contact material. The moderator guided participants through the protocol by asking
questions about the material and following up with probes to address specific points that were
not mentioned by participants. As appropriate, the moderator expanded the discussion to build on
4
participants' comments and asked others to share their views. A key strategy used by the
moderators was to direct questions to participants who had not yet contributed to the current
discussion. This ensured that most participants were heard throughout the session, so that the
impressions gleaned from the groups were not based solely on a few participants who were most
forthcoming.
Following the discussion of each type of contact material, the moderator asked
participants to mark the version of the material they preferred with the number one. Not all
participants chose to select a preferred version for each type of material. Although some
participants expressed mixed feelings about specific features of different versions, in most
groups a majority of participants did indicate a preference for each type of contact material.
2.3
Recruitment of Participants
RTI recruiters posted advertisements on craigslist.com for the Raleigh-Durham, North
Carolina, Washington, DC, Dallas, Texas, Chicago, Illinois, and Los Angeles, California
metropolitan areas. These advertisements explicitly noted that in order to be eligible for
participation, the respondent must be age 18 or older, not currently serving on active duty in the
military, not currently employed by RTI International, and not a family member of an RTI
employee. The text for the recruitment advertisement for English focus groups can be found in
Appendix D.
A key consideration for our recruitment efforts was to reach Spanish-only, or mostly
Spanish, speakers who would not be included in the pool of potential participants in the English
focus groups. Our recent experiences on recruiting this population for focus groups and cognitive
interviews indicated that working with local Hispanic/Latino community organizations was an
effective way to tap into this subpopulation. RTI bilingual recruiters partnered with local
community organizations to legitimize the research in these target communities by assisting with
recruiting and providing a facility at their site for hosting the focus groups. Advertisements were
posted in highly concentrated Hispanic areas, such as Hispanic shopping centers, community
centers, and La Raza, a Spanish-speaking newspaper for the Chicago area. Word of mouth was
the most valuable way of getting calls from prospective participants. The local community
centers assisting with these efforts were El Pueblo in Raleigh, North Carolina, and Casa Central
in Chicago, Illinois.
When a prospective respondent called RTI in response to a recruitment ad, he or she was
screened using the recruitment scripts found in Appendix E. Specifically, all respondents were
screened for age, race, level of education, total household income, distance from the focus group
site, geographic area (urban/suburban/rural), whether or not they spoke English/Spanish as their
native (primary) language, whether they could read English/Spanish, and how well they could
read English/Spanish (very well, well, or not well). Eligible respondents were informed that the
focus group would be audio and video recorded and then provided with the date and time the
group would be conducted. In order to recruit the desired number of participants and encourage a
sufficient number of them to show up to each focus group, an incentive of $75 was offered.
5
2.4
Characteristics of Participants
RTI used the screener in Appendix E to recruit participants and place eligible participants
in the appropriate session for their age. Recruiters were asked to strive for the greatest possible
variation in age, gender, and other demographic characteristics, to ensure a heterogeneous set of
viewpoints in each group. The goal was to recruit 10 participants for each group to ensure at least
7 or 8 people would attend each focus group session.
The first round of recruitment began on September, 23, 2009, and concluded on October
16, 2009. This initial round recruited subjects for focus groups conducted on October 12-13 in
Durham, North Carolina, October 20-21 in Raleigh, North Carolina and October 21 in
Washington, DC. Round two of recruitment included focus groups held on November 2-3 in
Irvine, California, November 4-5 in Addison, Texas, and November 4-5 in Chicago, Illinois.
This recruitment phase ran from October 19, 2009, through October 30, 2009. Tables 2.1 and 2.2
present the final demographic composition of all participants across the 17 focus groups,
including age, gender, race, education, income, and urbanicity for each participant.
6
Table 2.1
Characteristics of English Focus Group Participants
Gender
7
Location and
Age Group
Total
Raleigh, NC
18–29
30–49
50+
Washington, DC
18–29*
50+
Addison, TX
18–29
30–49
50+
Irvine, CA
18–29
30–49
50+
Race
White
(Non-Hisp) Black
56
24
Education
Edu
Edu
≤ HS/GED > HS/GED
11
84
Other
Race
16
Income
Inc
Inc
≤ $100,000 > $100,000
81
14
Geographic Area
Number
96
M
33
F
63
Urban
35
Suburban
57
Rural
3
8
8
10
3
3
4
5
5
6
5
3
7
3
4
3
1
-
1
1
1
7
7
9
8
6
8
2
2
3
3
3
4
3
7
1
2
-
9*
10
4
5
5*
5
4
6
2*
4
3
-
1
2
7
8
6
8
2
2
3
4
5
6
-
7
10
7
2
2
3
5
8
3
2
6
6
2
4
-
3
1
1
1
3
6
9
4
6
9
7
1
1
-
2
4
3
5
6
4
-
9
10
8
1
4
2
9
6
6
4
6
7
1
1
-
4
3
1
-
9
10
8
7
8
8
2
2
-
1
4
5
8
6
3
-
* One no show was replaced by an eligible participant onsite. Because this participant was not completely screened prior to arrival, not all demographic
characteristics could be recorded for inclusion in this table.
Table 2.2
Characteristics of Spanish Focus Group Participants
Gender
Location and
Age Group
Total
Raleigh, NC
18-29
30-49
50+
Chicago, IL
18-29
30-49
50+
Number
49
M
20
Country of Origin
Education
Income
Puerto Central
South
Edu
Edu
Inc
Inc
F Mexico Rico America America ≤ HS/GED > HS/GED ≤ $100,000 > $100,000
29
24
8
3
14
28
21
49
0
Geographic Area
Urban
46
Suburban
2
Rural
1
5
8
7
2
3
4
3
5
3
2
3
1
1
1
1
-
2
4
5
4
5
3
1
3
4
5
8
7
-
5
8
6
-
1
9
10
10
3
2
6
6
8
4
6
8
4
1
2
3
2
-
3
6
5
5
3
5
5
9
10
10
-
8
10
9
1
1
-
8
3. Results
3.1
Lead Letter Envelope
3.1.1
Considerations in Deciding Whether to Open a Piece of Mail
Before presenting the lead letter envelopes, the moderators asked focus group participants
to think about the mail they receive each day and identify factors related to their likelihood of
opening a specific piece of mail. This discussion was intended to provide background
information on how people generally perceive the different types of mailings they receive and
how this might affect the likelihood of opening the NSDUH lead letter envelope. The moderators
probed participants by asking the following questions:
What kinds of things do you consider in deciding whether to open a piece of mail?
What kinds of things do you look for on an envelope to determine whether you will
open it?
What kinds of envelopes do you tend to throw out or recycle without opening?
Most participants indicated that their households receive a great amount of mail and that
they spend time sorting and separating the important pieces from what they considered to be
"garbage" or "junk mail." Some participants indicated that they open all the mail they receive,
often looking for interesting promotions or coupons, but others indicated that they regularly
throw out at least some mail unopened. In some of their households, participants noted that there
is one individual who does most of the mail sorting. In these cases, the rest of the household
members would only receive pieces of mail that the sorter considers to be important mail.
Focus group participants felt the decision to keep a piece of mail and open it is usually
based on what they know about the sender and the characteristics of the envelope. Participants
indicated they usually try to determine whether there is some kind of "connection" between
themselves and the senders of the mail. They typically assess this by examining both the return
address and to whom the mail is addressed. For example, they would be more likely to open a
piece of mail from a company with whom they already have a commercial relationship.
Participants also stated they would be less likely to open mail addressed simply to "Resident,"
and some indicated they routinely throw out mail addressed in this way. Some participants in
Spanish groups who live in apartment buildings mentioned that they never open mail addressed
only to "Resident" because they think this mail was sent to the landlord. This can be a problem in
cases where the mail of more than one family is received in the same mailbox and people do not
want to open their neighbor's mail by mistake.
For most people, simply adding some version of "Resident of ___________ County" to
the address would not significantly diminish their inclination to ignore mail addressed in this
way. Some participants in Spanish groups and some participants aged 50+ in English groups
thought that personalizing the mailing address in this way would make them more likely to open
the mail. Those who preferred adding "Resident of ___________ County" mentioned their
likelihood of opening the envelope would increase because addressing the envelope this way
9
would indicate to them that the mail contains something important for them and their local
community.
In terms of the physical characteristics of the envelopes, participants stated they usually
associate pieces of mail that use bright colors and glossy paper, have messages written on them,
and use bulk mail postage with marketing and promotions. For this reason, they are more likely
to dispose of this mail unopened. Some participants mentioned there are some law firms and
companies that try to make their promotions look more "official" or "important" in order to trick
people into opening them. These participants noted that they identify such efforts with the use of
plain white envelopes with minimal information on the outside about the sender.
Overall, focus group participants indicated the mail they are most likely to open would
have the following characteristics:
3.1.2
mail addressed to a specific person in the household,
mail that includes the name of a company with which the recipient is familiar, and
mail that uses mostly white envelopes with familiar and official-looking logos.
Reactions to the Two NSDUH Envelopes
Following the discussion of what factors influence the decision to open a piece of mail,
the moderators distributed two versions of the lead letter envelope. The first version was a white
standard number 10 size window envelope, printed as shown in Appendix F. The second
envelope was a white 9 x 12 inch catalog envelope, with the return address and address window
arranged in portrait orientation. The image for this larger envelope is exhibited in Appendix G.
Both envelopes included the same agency logo and return address.
Initial Reactions and Likelihood of Opening the Two Versions of the Envelopes
Table 3.1 presents focus group participants' preferences with respect to each type of
contact material. In the tabulation of participants preferences for the lead letter envelopes,
slightly more than half of the English group participants preferred the larger envelope. Two
thirds of the Spanish group participants preferred the larger envelope.
Overall, initial reactions to the envelopes were very positive. Based on their appearance,
most participants indicated they would likely open either the standard size or the larger size
version of the envelope if they received it in their mail. A key factor stated almost universally by
participants in the English language groups was that they would open the envelopes because of
the Department of Health and Human Services (DHHS) return address. People emphasized that
the envelope should look as "business-like" and "official" as possible, and this is generally
accomplished by using the DHHS logo and title. For this reason, most participants stated they
would open the envelope, regardless of the size and the use of the generic addressee of
"Resident." Because the mailing would be coming from the DHHS, many participants suggested
that there could be important information concerning recent developments in health care. The
Rockville, Maryland, return address did not seem to bother participants in any of the cities.
Inclusion of the RTI project number also did not seem to bother anyone, but it was mentioned by
a few that this information did suggest that the mailing might be in reference to a survey.
10
Table 3.1
Preference Counts for Each Type of Contact Material
Durham,
NC
11
Lead letter envelopes
Lead letter envelope, regular size
Lead letter envelope, 9x12
No preference expressed
TOTAL
Lead letter text only
Lead letter, Version 1a
Lead letter, Version 2a
Lead letter, Version 3a
No preference expressed
TOTAL
Lead letter graphics only
Lead letters, Version 1b
Lead letters, Version 2b
Lead letters, Version 3b
No preference expressed
TOTAL
Q& A brochure
Q & A brochure, Version 1
(current)
Q & A brochure, Version 2 (new)
No preference expressed
TOTAL
English Groups
Washington,
Irvine,
DC
CA
English Totals
Addison,
TX
Count
%
Spanish Groups
Raleigh, Chicago,
NC
IL
Spanish Totals
Count
%
9
17
0
26
9
10
0
19
8
14
5
27
13
10
1
24
39
51
6
96
40.6%
53.1%
6.3%
100.0%
4
15
3
22
6
18
3
27
10
33
6
49
20.4%
67.3%
12.2%
100.0%
9
7
8
2
26
12
3
3
1
19
18
1
1
7
27
11
3
8
2
24
50
14
20
12
96
52.1%
14.6%
20.8%
12.5%
100.0%
5
7
8
0
20
12
4
13
0
29
17
11
21
0
49
34.7%
22.4%
42.9%
0.0%
100.0%
12
8
4
2
26
2
8
8
1
19
17
7
2
1
27
13
6
4
1
24
44
29
18
5
96
45.8%
30.2%
18.8%
5.2%
100.0%
10
9
1
0
20
5*
16
9*
0
30
15
25
10
0
50*
30.0%
50.0%
20.0%
0.0%
100.0%
6
3
5
10
24
25.0%
1
7
8
16.3%
10
10
26
16
0
19
17
5
27
6
8
24
49
23
96
51.0%
24.0%
100.0%
19
0
20
22
0
29
41
0
49
83.7%
0.0%
100.0%
* One Spanish group participant selected both versions 1b and 3b.
In the Spanish groups, most participants were not familiar with the DHHS, but did
indicate that they would still open the envelopes because they look "official" and different from
marketing materials. The white envelope and the logo with an eagle in it clearly communicated
to these participants that the mail came from the U.S. government. Similar to the English
language groups, the Rockville, Maryland, return address reinforced the idea that the mail is
legitimate because people recognized that many government offices are located in that part of the
country.
Specific Reactions to the Different Sizes of the Envelope
Reactions from participants suggested that the larger envelope would certainly garner
more initial attention than the standard size envelope. Across the groups, people suggested that
the larger envelopes are often used to send legal documents, so they look more "official." Among
the Spanish group participants the larger envelope communicated that the documents inside are
important and cannot be folded. Some of the 18 to 29 year old participants in the Durham
English group preferred the smaller envelope because they felt it looked "more business-like."
Also most participants in the 50+ Spanish group in Chicago felt there was no need to use a large
envelope, perhaps because it would be a waste of resources.
In some of the English-language groups, after discussing the letters the moderator
returned to the question of which of the two envelopes should be used. Once participants realized
that both envelopes would contain only a single sheet of paper, some reversed their earlier
preference for the larger envelope suggesting that it would be a waste of paper and postage.
Some of these people were motivated by a concern for the environment. Others were motivated
by a concern with government spending.
Use of the "Official Business" Endorsement on the Envelopes
The text on the envelopes that reads "OFFICIAL BUSINESS. PENALTY FOR
PRIVATE USE $300" tended to enforce the official nature of the mailing for focus group
participants. Despite this view, the great majority of participants did not understand exactly what
this statement actually means. In the Spanish groups, some people suggested that this statement
means that the envelope must only be opened by the person to whom it is addressed and no one
else. This perception could be problematic, given that the mail is addressed to "Resident" and in
some cases the household receives mail in a depository shared with other households. There
were also a few Spanish group members who felt it was intimidating to have the penalty note on
the envelope.
3.2
Lead Letter Text
Three versions of the lead letter with text only and no graphics were presented to
participants and labeled as versions 1a, 2a, and 3a. These versions varied in how or whether
various aspects of the survey request and protocol were mentioned and the emphasis given to
each element. For example, version 3a provided the study name in the first paragraph, while
versions 1a and 2a did not. Versions 1a and 2a provided the URL for the NSDUH Web site.
Version 3a of the lead letter text was based on a letter used by the National Survey of Family
Growth and only included a signature from SAMHSA staff (with the RTI signature omitted).
This version thereby attempted to develop a more "personal" approach and appeal to recipients.
12
The three versions of the letters with text only are presented in numerical order as Appendices H,
I, and J.
Initial Reactions to the Three Alternative Versions of the Letter Text
As Table 3.1 indicates, focus group participants varied somewhat in their preferences for
each version of the lead letter text. Slightly more than half of the English group participants
preferred version 1a, but Spanish group participant preferences were even more mixed.
A plurality of about 43 percent of Spanish group participants preferred version 3a and another
35 percent preferred version 1a. Version 2a was the least popular among both English and
Spanish group participants. Versions 1a and 3a were viewed by participants as being better
organized, shorter, and more direct than version 2a. Participants also felt both of these versions
contained most of the information participants wanted to know. In addition, the way the text was
distributed on the page made versions 1a and 3a easier to read, understand, and communicate to
other members of the household.
One of the main differences between version 1a and version 3a was that the latter
mentions the specific topic of the survey (drug use, alcohol, and tobacco). Many participants felt
that being specific about the topic would increase their interest in the survey and would make
them more likely to participate. In addition, knowing the topic would also prevent surprises at
the time of the interview when the questions start asking about sensitive subjects. Spanish group
participants, who preferred version 3a, suggested that mentioning in the letter (as in the
brochure) that both users and nonusers of drugs and alcohol needed to participate would be a
good idea. However, there were strong differences among the Spanish group participants in
terms of reading skills and ability to understand the text of the letters. Only the more educated
participants with a college education were able to talk about differences that were too subtle for
the rest. This may be one reason why the two shorter versions of the letter were preferred.
Version 1a was particularly strong among the younger participants who believed that the
text in the letter was shorter, more professional, and straight to the point. Version 3a was
particularly popular among the older participants in the English-language groups, and nearly half
of the Spanish group participants. One of the reasons cited was the more personal tone of the
letter. However, many of the younger and middle aged participants in the English-language
groups really disliked this tone particularly mentioning the introduction, "My agency…"
Other Specific Features of the Letter Text
One feature used only in version 2a viewed favorably by participants was how the
"Members of the Household" and "Resident of the [city, county, or state]" made the letter more
seem more personal. These participants were particularly thinking about situations when more
than one person in the household would read the letter.
Participants were asked to compare how the three letters handled the issue of survey
confidentiality. A large number of the participants in the English-language groups preferred the
single sentence used in version 3a as it was short and to the point. Some participants pointed out
that the mention of the random selection of the address, not the particular person (version 2a)
further enhanced confidentiality. Furthermore, they liked the fact that the sentence was its own
13
paragraph that made the topic "stand out." There was no discussion about the small font text at
the bottom of the letter that further addressed confidentiality.
Participants from the Spanish groups did not notice big differences across letters in terms
of information about confidentiality. To them, the statements in the three versions were sufficient
to make the reader comfortable about participating in the survey. Stating that the confidentiality
of the responses will be protected by federal law was one of the key elements for the Spanish
group participants. Only a few noticed the small font text at the bottom of the page addressing
confidentiality. Once it was pointed out, however, most agreed that it was a positive element and
should be included in the letter.
There were mixed responses to the handling of signatures. Some suggested that two
signatures made the letter appear more official and important; others said they were not very
interested in how the government was conducting the survey and a single signature would
suffice. However, if a single signature would be used, it should be the one from DHHS rather
than RTI.
There were mixed reactions to the concept in version 2a of using other organizations to
endorse the survey. The 30 to 49 year old English participants in Durham and Dallas said an
endorsement by the American Medical Association (AMA) might help give some credibility to
the survey but an endorsement by the American Association of Retired Persons (AARP) would
not carry much weight. Adults in the 18 to 29 year old group in Durham did not seem to think
endorsements would add much. The youngest group of adults in Washington, DC, suggested that
maybe an endorsement by the Centers for Disease Control and Prevention (CDC) might enhance
the appeal. In California, there was little support for the concept of using endorsements. There, it
was emphasized that the merits of the study itself should be enough to convince people to
participate. In Dallas, it was suggested that the use of endorsements from other organizations
carried some risk because some organizations could be perceived as detractors and this might
cause less favorable disposition towards survey participation. Some of the frequently mentioned
organizations were the American Health Association, the Red Cross, and the March of Dimes.
Organization like AMA and AARP had a low level of awareness among the Spanish
group participants, and their endorsement would not have a strong effect on people's willingness
to participate in the survey. At the same time, it was suggested that mentioning the endorsement
of a health institution would be more beneficial than endorsements from other types of
institutions. Further, mentioning that the survey was required by the U.S. Congress reinforced
that the survey was official, serious, and relevant, and some of the Spanish group participants
said it would make them feel more obligated to participate. Most of the participants said that
mentioning of the specific government code enabling the survey in version 2a was not needed,
and if anything, might be intimidating.
Nearly everyone in all the English-language groups said they would use the Internet to
access the RTI Web site indicated in the first two versions of the letters. Few said they would
call the toll-free number, but only after having visited the Web site first, and only if they had
additional questions about the survey. The lack of a Web site address in version 3a was often
cited by participants as a reason why they did not choose that version of the letter.
14
Spanish group participants also believed that having a Web site address was a very
positive element in the text. Most said that they would immediately go to the Web to find more
information and make sure that the survey was legitimate. However, for Spanish group
participants, having a telephone number was also important because there were many
participants who did not have access to the Internet, or they did not feel comfortable using
computers.
In general, participants were not familiar with RTI. The three versions of the letter did a
good job explaining the role of RTI—most participants understood that it was the institution that
would conduct the interviews, and they thought it was good to mention that RTI is a nonprofit
organization. However, there was little interest in including more details about RTI.
The bolding and highlighting of the incentive sentence caught the attention of the readers.
It did not seem to have a negative connotation and participants admitted that this line would
increase their interest in the letter, and because of it, they would be more likely to pay attention
and read the complete text more carefully. The fact that this information was not bolded in
version 3 was often cited as a reason for not choosing that version of the letter.
After reading the texts, most participants had a good understanding of the ideas
communicated in the letters. The language was appropriate and most of the information people
wanted to know was included. However, some participants believed the letter should include
more specific information about the interviewer's visit—specifically, when the interviewer will
call on their house, and who in their household they would want to talk to. Spanish group
participants from the two younger groups (18 to 29 and 30 to 49 years old) in Chicago expressed
interest in being able to determine whether they would be eligible to participate in the survey by
including eligibility age range in the letter.
3.3
Lead Letter Graphics
In addition to text, the graphics for the lead letters also included three alternative
versions, labeled 1b, 2b, and 3b. These versions varied in the size, format, and content of the
graphics used to "package" the letters. For example, the image on the example identification
badge varied from a dark gray silhouette (versions 1b and 3b) to an actual photo of a person,
printed in color (version 2b). Participants were asked to comment on the graphical layout of the
letter independent of its content (this was accomplished by providing letters with graphics, but
no text). The three versions of the letter graphics are exhibited in numerical order as Appendices
K, L, and M.
Initial Reactions to the Three Alternative Versions of the Letter Graphics
As Table 3.1 shows, focus group participants indicated mixed preferences for which
version of the lead letter graphics they preferred. None of the versions garnered majority
approval in either the English or Spanish groups, but half of participants in the Spanish groups
preferred version 2b. In contrast, about 45 percent of English group participants preferred
version 1b.
Participants immediately noticed the differences in the headers and sizes of the logos.
They believed the logo was important because it reinforced the official nature of the letter.
15
Adding Rockville, Maryland, and a ZIP code in the header was also a positive element and
helped to make the letters appear more "business-like" and official. Some participants liked the
larger DHHS logo in version 1b as it allowed them to read it and that was why they preferred
that version over the others.
Other Specific Features of the Letter Graphics
Across the groups there was a difference in opinion regarding the line in the header of
version 1a, "An Important Request from The U.S. Department of Health & Human Services."
Some participants liked how it emphasized the importance of the survey, while others suggested
that it made the letter look "less business-like." In the Irvine groups, it was suggested that the
importance of the survey should be obvious without having to explicitly state it.
Overall, greater support for the use of this tagline in version 1a was evidenced in the 50+
age groups than in the younger-aged groups. Among the 50+ year olds in the Addison groups the
appeal of this tag line was consistent with their preference for the text in letter version 3a, which
begins with, "My agency, …, needs your help." In Irvine, however, the older adults did not like
the header on version 1b saying that it looked less professional than the other versions. They also
suggested that it was best if the recruitment materials did not make DHHS sound "too desperate."
Some Addison group participants similarly indicated that the materials should not appear to be
using "hard sell" approaches.
Among the Spanish group participants, the tagline did not have any negative effect, but it
was not considered a significant positive element. However, it is important to mention that a
couple of younger participants (who could also speak some English) thought that the word
"solicitud" was usually associated with an application they had to fill out and suggested using the
word "petición" instead.
There was a consistent response to the image of the identification badge using an actual
photograph, as in version 2b. Nearly all participants said their first impression was that this was
the individual who would "show up" at their home. When informed otherwise, nearly everyone
said if the badge had some indication that it was being used for illustrative purposes only, like
having "sample" written across it, they would still prefer to see an actual face on the badge.
Some respondents also thought it was too much to show Ilona Johnson's signature on the
identification badge and that her signature on the letters (in versions 1b and 2b) was sufficient.
Participant feedback indicated that the way the identification badge is identified must be clear
and somehow visual for people who do not read English. There were a couple of instances where
Spanish group respondents thought that the person in the photo was Ilona Johnson because they
read the name in text on the badge, but they did not understand the words around it.
After the discussion about the badge, most participants agreed that having the hand
written name of the interviewer that would visit their home was a good idea. However, some
Spanish group participants were skeptical about the ability of RTI to actually send the person
who signed the letter.
16
3.4
Question and Answer Brochure
Focus group participants were shown two versions of the Q&A brochure. Version 1 was
the current Q&A brochure, updated to reflect planned for the NSDUH redesign. Version 2 was
an updated version which used a variety of background colors and photographs. The brochures
included some similar questions and answers, but also had significant differences in both content
and format. The two versions of the Q&A brochures are displayed in numerical order as
Appendices N and O.
Initial Reactions to the Two Alternative Versions of the Q&A Brochure
Overall, version 2 was preferred by participants over version 1. These preferences
differed somewhat between the English and Spanish group participants. Over 80 percent of
participants in the Spanish groups preferred version 2, and 51 percent of English group
participants preferred version 2. Participants in the English groups (24 percent) were also much
more likely to decline to indicate a preference between the two versions compared with those in
the Spanish groups (0 percent).
Most participants indicated that version 2 was more appealing and something that most
people would find more inviting to read. Older participants remarked that a slightly larger type
font made this version easier to read. Participants generally felt that the photos included in
version 2 of the brochure do a good job communicating that different types of people of different
ages, occupations, and walks of life are participating in the survey. This feature led people to
note that version 2 appeared more friendly and personal. Participants also liked the colors used
for this brochure.
The only image that people had some trouble with was the map. Participants were not
sure why this image was included and what the different shades of blue and colors signified.
Some people believed that the map was communicating the fact that NSDUH is a national
survey, but no one associated this image with the selection of sample units.
Version 1 of the brochure was preferred by a minority of the participants. A negative
comment among 18 to 29 year olds suggested that this version looked like "something from the
80s." A common complaint among all participants was that version 1 contained too much
information. People suggested this made the brochure somewhat overwhelming and, therefore,
less compelling for them to read. At the same time, a minority of participants in each group
indicated they liked the greater content in version 1. Some of these participants suggested that
this version of the brochure might be more effective for describing the survey to another
household member. These people suggested that dividing the information into more specific
topics in version 1 was a better approach than the way the topics were organized than in
version 2.
One section in version 1 of the brochure that was identified as being particularly effective
by some participants was the section with the title "What If I Do Not Smoke, Drink, or Use
Illegal Drugs?" One section that was considered unimportant by most people was "How Does the
Government Conduct the Study?" Words that summarize a number of participants' reactions that
SAMHSA uses a competitive bid process to select a vendor included "I don't care." In each of
the Irvine groups, it was pointed out that the way in which version 1 addressed the question of
17
how the survey data would be used is very good. It was emphasized, particularly in the 50+ age
group, that it is important to tell people how this information will be used.
Overall, participants felt that the information in the brochures is more understandable and
complete, especially compared to the more limited information presented in the letter. The
technical and administrative details of the survey seemed less relevant to participants than
knowing the purpose and utility of the survey. The brochures also do a much better job
explaining that not only drug and alcohol users need to participate in the survey. Some
participants said this will make people more willing to participate, although they also
acknowledged that this might discourage participation among those who have considerable
substance use to report.
Other Specific Features of the Q&A Brochures
When asked if any of the information in either version of the brochure was "confusing,"
participants in the English groups generally indicated they did not have difficulty with most of
the phrases used to describe the survey process, such as "randomly selected" and "chosen at
random through scientific methods." One concept that some people did have difficulty with was
the juxtaposition of "random" with "scientific" in the phrase "…chosen at random through
scientific methods." It was suggested by some participants that "random" and "scientific" is a
contradiction. In the Spanish groups, explanations of how participants are selected randomly and
scientifically were not understood by participants with lower education levels. At the same time,
these participants did not express much concern about why they would be asked to participate.
Although much the same information is contained in both versions of the brochures,
participants seemed to have an easier time identifying the listing of relevant Web sites in version
1. Because the Web site is likely to be used by people to validate information about NSDUH, use
of the format for the Web addresses in version 1 of the brochure should be considered.
The mention of computers in the brochure was only mentioned as a concern in the
Durham group of 18 to 29 year olds. This concern was not expressed for them personally, but
instead they suggested some older adults might be concerned about having to use a computer and
therefore be less interested in participating. None of the older adults in the English or Spanish
language groups expressed any such concern.
In the Spanish groups some participants mentioned that not using a computer would
make them doubt the legitimacy of the survey because today everybody uses computers for
everything. On the other hand, older participants in the Spanish groups who did not have much
experience working with computers appreciated the information included in the brochures
explaining that knowledge of computers is not necessary. These participants did not quite
understand whether they would actually have to use the computer themselves or whether the
interviewer would enter their responses for them.
Overall, participants felt that the brochures do a better job than the letters in explaining
the role of RTI in the study. People noted that the brochures provide more detailed information
and history about both NSDUH and the institutions involved. Some participants suggested that
this information would cause them to be more favorably disposed toward participating in the
survey.
18
Participants generally thought it is important to include the logos of SAMHSA and RTI
in the way they are presented in version 2 of the brochure. Even though sample members may or
may not be aware of these institutions, people indicated the logos would make them more
comfortable by reinforcing the importance and legitimacy of the survey.
The 50+ participants in the English groups seemed sincerely motivated to participate in
such a survey if it helped the government's health planning and related public policy initiatives.
Messages focused in terms of how survey participation would support a worthy endeavor seem
likely to be received favorably by older adults.
3.5
Special Concerns of Spanish-Speaking Participants
Overall, Spanish-speaking participants shared many of the same impressions of the
materials as the English group. There were only some special concerns with the Spanish version
of the materials. The most important concern was that the reading level of the letters was only
appropriate for Spanish-speakers with at least a high school education or greater. During the
focus groups, strong differences across participants were noticeable in terms of reading skills and
ability to understand the letters text. There was one participant in Chicago who said she did not
understand any of the letters, and others just repeated part of other people's comments. Only the
more educated participants with a college education were able to talk about differences that
seemed too subtle for the rest.
Like English group participants, the text on the envelopes that reads "OFFICIAL
BUSINESS. PENALTY FOR PRIVATE USE $300" was frequently misunderstood by most
Spanish speakers. In addition, many Spanish group participants indicated that they also felt
uncomfortable opening an envelope with this note. Some felt intimidated by it. Participants felt it
meant that the envelope must only be opened by the person that it is addressed to, and no one
else. This was considered to be confusing, particularly if the mail is addressed to "Resident" and
the household receives its mail in a depository with other households. Many participants thought
they would have to pay a fine of $300 if they opened the envelope.
The use of acronyms in Spanish is not as common as in English, and it has always shown
to be problematic for Spanish group participants who are unfamiliar with U.S. government
agencies, associations or U.S. code. Many Spanish-speaking participants pointed out their unease
with acronyms, and recommended avoiding them all together.
Explanations of how participants are selected randomly and scientifically were not really
understood by those participants with lower education levels. And the Spanish term "al azar" (at
random) was confusing and many did not understand what that meant.
Spanish group participants said their first impression of the picture on the identification
badge was that it was either the individual who would "show up" at their home or the project
director. When informed otherwise, some participants said they would feel distrustful if the field
interviewer who showed up at their door was not the same person shown in the picture.
Letters addressed to "Resident" made people in the Spanish groups think that it was not
necessarily sent to them. Many thought the letter was addressed to landlords, such as in cases
where people live in rented homes.
19
3.6
Additional Suggestions
In addition to the discussion of the features of the envelopes, letters, and brochures
already detailed, several further suggestions were identified by focus group participants:
•
Participants suggested the use of a regular stamp to make the envelope look more
official and distinguish it from junk mail. This recommendation is in sync with the
Tailored Design Method for survey mailings, proposed by Dillman (2000).
•
Many participants suggested putting the incentive amount on the envelope or
somehow suggesting that the recipients can get paid for participation in an official
government survey. Such a message would make sample persons more likely to open
the envelope and read the letter.
•
Generally, participants did not like the fact that the envelope might be addressed to
"Resident." They preferred "Resident of ____________ County" as this placed them
in a group and suggested the letter contained something that applied to them. Several
suggestions were made to better address this issue. Among the proposed addressees
were "Head of household at [ADDRESS]", "Randomly selected resident at
[ADDRESS]", and "Survey to resident at [ADDRESS]."
•
In most groups, there was some misunderstanding about the statements in letter
versions 2a and 3a regarding the interviewer visit. On first read, many participants
believed the letters stated they would be given a $40 incentive for answering a few
questions (in version 2a) or a 5-minute interview (in version 3a). These statements
caused some confusion about the survey protocol. During the discussions, participants
eventually realized the letters were referring to the screening process that would
determine their eligibility to participate in the survey interview.
•
Several participants suggested mailing the brochure with the initial letter to make the
mailing look more official overall.
20
4. Summary and Recommendations
The 17 focus groups conducted in various regions of the United States confirmed the
importance of much of the content and formatting elements used in the contact materials, but
also highlighted some key issues that might not be fully addressed in each type of contact
materials. This section synthesizes the focus group reactions into specific recommendations for
improving each type of contact materials. Recommendations accepted by SAMHSA will then be
used in creating a new version of each type of materials for the NSDUH redesign scheduled for
2013.
4.1
Lead Letter Envelopes
A majority of focus group participants stated a preference for the larger 9x12 envelope.
This majority was greater in the Spanish focus groups. At the same time, participants in the
English groups did express concern about using large envelope for only a letter and no additional
study materials. In addition, the overwhelming majority of participants indicated that they would
open either envelope, primarily because the DHHS logo led them to believe that the mailing was
important. The main benefit of the larger envelope appears to be that it will attract greater
attention, at least in some households. One benefit for NSDUH interviewers is that they would
not have to fold the letters to insert them into the envelopes, which would increase the likelihood
that the address information is visible in the address window.
Based on these results, the recommendation would be to carefully consider the costs and
benefits of using 9x12 envelopes versus the standard number 10 size. Using these larger
envelopes would require additional material costs of about $4,800 and additional postage costs of
approximately $116,000 annually, assuming 200,000 lead letters are mailed each year. Although
focus group participants generally preferred the larger envelope, this preference was tempered by
knowing that only a single letter would be included in the mailing. Given that the standard size
envelope did not generate negative reactions, this size envelope with logo and return address
presented in the focus groups may be equally effective as the larger envelope.
Including the endorsement "OFFICIAL BUSINESS. PENALTY FOR PRIVATE USE
$300" appears to have advantages and drawbacks. Focus group participants felt that this
statement made the envelopes look more official, but almost no participants actually understood
what this statement means. The phrase "PENALTY FOR PRIVATE USE $300" was particularly
confusing to many participants. These reactions suggest that it may be advisable to keep the
phrase "OFFICIAL BUSINESS," but drop the second part of the statement if possible.
U.S. Postal Service guidelines indicate that this full statement would have to be included on the
envelope. The current envelope used for mailing the lead letters and frequently asked questions
for the National Immunization Survey, a study sponsored by DHHS, includes only the
"OFFICIAL BUSINESS" part of this statement. The potential for using only the first part of this
statement should be investigated further, so that a final decision can be made on whether to
include this statement. Given that most participants indicated they would open the envelope
because of the DHHS logo, including this statement does not seem critical to the effectiveness of
the lead letter envelope.
21
4.2
Lead Letter Text
A majority of participants in the English groups preferred version 1a of the lead letter
text, but a plurality of Spanish group participants preferred version 3a. In most groups,
participants made compelling arguments for either version. Version 2a was generally viewed as
too lengthy and complicated, and participants therefore felt they could recommend few parts of
the text in this letter as preferable.
Based on these results, the primary recommendation would be to create a hybrid of the
text in version 1a and 3a that would combine the preferred text of each letter and avoid any text
considered to be problematic. For example, many participants indicated version 1a was wellorganized and covered the most important information about the study. Participants also thought
it was useful that version 3a specifically mentions the topic of the survey and indicates (as in the
Q&A brochure) both users and nonusers of drugs and other substances are needed to participate.
These reactions can be used to update version 1a to include some of the content and phrasing of
version 3a to produce a stronger letter overall.
One element of the version 2a letter text that could be used in the new letter was
addressing the letter to "Resident of ____________ County." Some participants did not feel this
would significantly increase their likelihood of opening the letter, but many did feel this would
indicate to them that the mailing is important for them and their local community. For this
reason, it might be worthwhile to investigate the costs and logistics of adding the county, parish,
or district for each addressee.
On the issue of two signatures (included in versions 1a and 2a) versus a single signature
(used in version 3a), participants did not indicate a strong preference. Most felt including both
signatures was the better approach, so recipients would more clearly understand both SAMHSA's
and RTI's role in conducting the study. Given that there were really no negative reactions to
including both signatures, it seems like the letter should continue to provide both the SAMHSA
and RTI signatures.
4.3
Lead Letter Graphics
Focus group participants offered rather mixed preferences for which version of the lead
letter graphics they preferred. None of the versions garnered majority approval in either the
English or Spanish groups, but half of the participants in the Spanish groups preferred version
2b. In contrast, about 45 percent of English group participants preferred version 1b.
Specific elements of the lead letter graphics seemed to heavily influence participant
preferences. The larger DHHS logo on version 1b was often cited as preferable to the smaller
version displayed on versions 2b and 3b. Another key element was the use of a gray silhouette
versus an actual picture in the image of the field interviewer's identification badge. Overall,
participants preferred the actual picture on the identification badge, even when it was pointed out
to them that the picture could not be tailored to show the actual field interviewer assigned to each
selected household. One qualification on this point is that participants agreed that a watermark or
other graphical feature should be used to indicate the identification badge is only a sample. Many
participants were initially unclear that the picture was just a sample, and therefore they would
have expected the person in the picture to be the actual field interviewer assigned the recipients'
22
household. One aspect of the graphics in version 1b that was not received favorably by
participants was including the phrase "An Important Request from The U.S. Department of
Health & Human Services" in the header. Most participants viewed this as superfluous.
These reactions suggest that the lead letter graphics should incorporate various elements
used across the three versions, including:
•
the larger DHHS logo used in version 1b;
•
the sample picture on the identification badge used in version 2b, with a watermark
indicating that the badge is just a sample; and
•
the line for the interviewers' name under the identification badge used in all three
versions.
Participants did not express strong preferences for how the return address was presented in the
header. This feature of the header seems unlikely to have a significant impact on recipients'
reaction to the letter.
4.4
Q&A Brochure
Although a majority of focus group participants preferred version 2 of the Q&A
brochure, preferences did differ significantly between the English and Spanish group
participants. Whereas over 80 percent of participants in the Spanish groups preferred version 2,
51 percent of English group participants preferred version 2. Participants in the English groups
were also much more likely to decline to indicate a preference between the two versions
compared with those in the Spanish groups.
The primary appeals of version 2 of the brochure appeared to be the use of colors and
pictures, as well as the layout and amount of text presented. Participants who perceived version 1
as providing more detail overall than version 2 viewed this alternatively as either a positive or
negative feature. Some felt the additional details were informative and useful, but others thought
these details were overwhelming and would discourage people from reading the brochure.
Feedback on the Q&A brochures indicates development of version 2 should continue, but
useful elements from version 1 should be incorporated into the brochure. For example, some
participants suggested the way the topics were organized in version 1 was a better than in version
2. In addition, the brochure should retain sections viewed as particularly useful by participants
and consider reducing or dropping sections viewed as less important. Participants felt the section
"What If I Do Not Smoke, Drink, or Use Illegal Drugs?" in version 1 of the brochure was
particularly effective. One section that was identified as less important by most people was "How
Does the Government Conduct the Study?" Creating a new brochure should address these
concerns.
Preliminary cost estimates gathered during the design phase for the contact materials
indicated that version 2 of the Q&A brochure should not cost significantly more than version 1 to
print. A final cost estimate can be determined once the brochure design is finalized.
23
24
5. References
Dillman, D. (2000). Mail and internet surveys: The tailored design method. New York, NY: John
Wiley and Sons
Fowler, F. J. (1995). Improving survey questions: Design and evaluation. Thousand Oaks, CA:
Sage.
Groves, R. M., Cialdini, R. B., & Couper, M. P. (1992). Understanding the decision to
participate in a survey. Public Opinion Quarterly, 56, 475-495.
Murphy, J., Eyerman, J., & Kennet, J. (2004). Nonresponse among persons age 50 and older in
the National Survey on Drug Use and Health. In S. B. Cohen & J. M. Lepkowski (Eds.), Eighth
Conference on Health Survey Research Methods: Conference proceedings (pp. 73-78, HHS
Publication No. PHS 04-1013). Hyattsville, MD: U.S. Department of Health and Human
Services, Public Health Service, Centers for Disease Control and Prevention, National Center for
Health Statistics.
Murphy, J., Schwerin, M., Hewitt, D., & Safir, A. (2005) Nonresponse among respondents aged
50 and older: Potential respondents focus group report (prepared for the Substance Abuse and
Mental Health Services Administration). Research Triangle Park, NC: RTI International.
25
26
Appendix A:
Focus Group Moderator’s Guide
MODERATOR: THIS IS A GUIDE TO HELP YOU LEAD THE DISCUSSION. YOU CAN
MODIFY AND ADAPT THESE QUESTIONS TO MEET SITUATIONAL NEEDS DURING
EACH SESSION.
SECTION I: INFORMED CONSENT (5 minutes)
MODERATOR: PARTICIPANTS SHOULD BE SITTING AROUND THE TABLE WITH THE
SEAT AT THE HEAD OF THE TABLE RESERVED FOR THE MODERATOR. THE FIRST
TASK WILL BE TO REVIEW THE INFORMED CONSENT FORM AND HAVE PARTICIPANTS
SIGN THEM.
INTRODUCTION OF SELF: Hello, and thank you for attending this group discussion. My name
is [Jim Leiman OR Arturo Obscura] with Morpace, a research company. We are working with
RTI International, a not-for-profit research company, to improve the materials used for a large
national survey.
Before we start, I would like you to read over the consent form in front of you. There are two
copies. This consent form provides an overview of this research study and information about
your rights as a study volunteer. Once you are done reading the form, please sign both copies.
RTI will keep one and you will keep one. If you have any questions about the consent form,
please do not hesitate to ask me.
ALLOW PARTICIPANTS TIME TO REVIEW THE INFORMED CONSENT AND ANSWER ANY
QUESTIONS THEY HAVE. THE RTI NOTETAKER WILL CHECK THE SIGNATURE AND
DATE ON THE COPIES THAT RTI WILL RETAIN.
AFTER COLLECTING THE SIGNED CONSENT FORMS, ASK PARTICIPANTS TO WRITE
THEIR FIRST NAMES (OR THE NAME THEY PREFER TO BE CALLED) ON BOTH SIDES OF
A “NAME TENT” AND PLACE IT SO IT CAN BE SEEN FROM THE FRONT OF THE ROOM.
A-1
SECTION II: INTRODUCTION (5 minutes)
This group discussion is in support of an ongoing national survey that examines health and
health related behaviors. We want to improve the materials that we provide to people such as
yourself to inform them about the survey. I will be leading today’s discussion ,sharing some
materials with you, and asking you some questions to guide the discussion.
We just have a few ground rules for our discussion:
Most importantly, there are no right or wrong answers to the questions I will be asking.
Everyone’s input is equally important and helpful. We are interested in all your ideas,
comments, and suggestions. It is OK to disagree with what someone says, but we ask
that you do so respectfully.
We are video recording the session and also have a note taker so we don’t miss
anything that is said, but we will not link comments with anyone personally. We will only
use first names during this discussion, and we won’t ask questions that would identify
where you live or other personal information. The video recording will be destroyed when
we have completed the study.
Please avoid side conversations among yourselves. Only one person should speak at a
time. This serves two purposes. First, it lets the whole group hear the remarks someone
makes. Second, it ensures that the recording will be clear.
To get the best benefit from this group, we want to hear from everyone in the room. Like
any group, I imagine some of you like to talk while others may be quieter. So if I haven’t
heard from you, I may call on you. This allows us to hear from everyone a few times by
the end of the session. If you’d rather not answer a particular question, you can just tell
me that you would like to “pass.”
Please take a minute now to turn off your cell phones or pagers so we aren’t interrupted.
If you need to take a break or use the restroom, please leave the room quietly.
At the end of the session, we will give you each $75 cash as a token of our appreciation.
We will also ask you to sign a receipt to document that you have received this incentive.
Are there any questions or comments? [ANSWER ANY QUESTIONS AT THIS POINT]
Before we begin, let’s introduce ourselves. How about if we go in the order of the month you
were born in? Let’s start with January. Whose birthday is in January? When I call on you, please
just give us your first name.
PARTICIPANTS RAISE THEIR HANDS AND INTRODUCE THEMSELVES, GIVING THEIR
FIRST NAME ONLY. THE MODERATOR GOES THROUGH EACH MONTH UNTIL
EVERYONE IN THE ROOM (INCLUDING MODERATOR AND NOTETAKER) HAS
INTRODUCED THEMSELVES.
A-2
SECTION III: LEAD LETTER ENVELOPE (15 minutes)
Great. As I mentioned, we are discussing the materials that are used on a national survey, in
order to improve them. Households that are selected for this national survey usually learn about
the study by receiving a letter describing the survey. The letter also indicates that an interviewer
will visit the household to ask a few questions about the members of the household and then
may ask one or two members of the household to complete an interview.
I’d first like to ask you about the envelopes that would be used to send these letters to
households selected for the survey. Think about the mail you receive each day.
1. What kinds of things do you consider in deciding whether to open a piece of mail?
2. What kinds of things do you look for on an envelope to determine whether you will open it?
3. What kinds of envelopes do you tend to throw out or recycle without opening?
Here are two different examples of envelopes that could be used for the letters sent to
households selected for this survey. Please take one of each type of envelope.
[DISTRIBUTE STARTING AT BOTH ENDS OF THE TABLE AND GIVE PARTICIPANTS A
MOMENT TO LOOK AT THE ENVELOPES. THE STANDARD SIZE ENVELOPE ADDRESSED
TO “RESIDENT” WILL BE REFERRED TO AS VERSION 1 AND THE 9x12 ENVELOPE
ADDRESSED TO “RESIDENT” WILL BE REFERRED TO AS VERSION 2.]
Imagine that you received either one of these envelopes in your mail, addressed to “Resident”
at your home address, just like the example provided on these envelopes.
1. What are some of the first things you noticed about these envelopes? Are there any
similarities or differences between the two versions that you noticed? [PROBE
RESPONDENTS TO ELABORATE ON ANY DIFFERENCES THEY IDENTIFY BETWEEN
THE ENVELOPES]
2. Which of these envelopes would you be most likely to open? Why do you think so?
3. What do you think could be changed to make it less likely that you would throw them out?
ASK THE FOLLOWING QUESTIONS ONLY IF THESE POINTS WERE NOT ALREADY
MENTIONED IN THE DISCUSSION:
1. Notice there are two different envelope sizes. Would the size of the envelope have any
impact on how likely you would be to open it?
2. The envelopes have the statement “Official Business, Penalty For Private Use $300” printed
on them. Does making a mailing look more official in this way make you more likely to open
it? What impression does this statement give you about the purpose of the mailing and the
contents of the envelope?
3. Does it make any difference to you if the envelope was addressed to “COUNTY/DISTRICT
NAME Resident” instead of just “Resident?” How would this make a difference to you?
4. What do you think about the United States Department of Health and Human Services
(DHHS) logo, which is on both versions of the envelope? Do you think this logo would make
you more or less likely to open the envelope?
A-3
5. What do you think about the return address, which indicates the RTI project number and
provides a Rockville, Maryland address? Does this look like the kind of address you would
expect to see on a letter regarding a Federal government project?]
IF TIME PERMITS, PLEASE USE THE FOLLOWING SCRIPT TO ASK PARTICIPANTS TO
INDICATE THEIR PREFERENCE BETWEEN THE TWO ENVELOPES AND PASS THEM TO
YOU: Before we move on, could you indicate which envelope you prefer by writing the number
one (1) in the upper right corner of the envelope that you like better and passing it to me? Also,
if you would prefer to have the name of the county in which the resident lives included on the
envelope, please write the word “county” by the address on your preferred envelope. Thank
you.]
A-4
SECTION IV: LEAD LETTER (45 minutes)
Here are three different versions of the letter that households selected for the survey might
receive, usually before an interviewer visits the homes. Please take a set of three and pass
them along.
[DISTRIBUTE THE TEXT-ONLY VERSIONS OF THE LETTERS IN ORDER STARTING AT
BOTH ENDS OF THE TABLE AND GIVE PARTICIPANTS A MOMENT TO READ THEM. THE
UPDATED CURRENT LETTER WILL BE REFERRED TO AS VERSION 1a, THE
DILLMAN/CIALDINI-INFLUENCED LETTER WILL BE REFERRED TO AS VERSION 2a, AND
THE NSFG-INFLUENCED LETTER WILL BE REFERRED TO AS VERSION 3a. ]
We are interested first in your initial reactions to the content of these letters, so these versions
only have the text of each letter, without any graphics.
[ASK BEFORE PARTICIPANTS READ THE LETTERS] Just looking at the letters, do you think
you would be more likely to read one more than the other two, if you received it in the mail?
Why or why not?
Please take a few minutes to read the three letters all the way through. I will ask you some
questions about the letters shortly. [GIVE PARTICIPANTS TIME TO READ THE LETTERS]
A. First, I have some questions about the content of these letters:
1. What are some of the first things you noticed about the content of these three letters? Are
there any similarities or differences between the three versions that you noticed? [PROBE
RESPONDENTS TO ELABORATE ON ANY DIFFERENCES THEY IDENTIFY AMONG
THE CONTENT OF THE LETTERS]
2. Which letter would seem most likely to make you feel comfortable participating in this study?
Why do you think so?
3. Is there information that you would like to know about the survey, but is not included in any
of these letters?
4. Thinking about any of the three letters, do you think you would share the letter you received
or information in the letter with anyone in your household? What would make you more likely
to share the letter you received?
5. Thinking again about any of the three letters, do you think you would call the toll-free
number or visit the web site provided in the letter for more information about the study?
ASK THE FOLLOWING QUESTIONS ONLY IF THESE POINTS WERE NOT ALREADY
MENTIONED IN THE DISCUSSION:
1. Which letter seems to have the most professional approach? Why?
2. Which letter seems to have the most personal touch? Why?
3. Do you think any one of these letters is more informative to you than the other two? Please
explain.
4. Do any of the letters make it seem particularly important that you should participate in the
survey and make you willing to talk to an interviewer who visited your home?
5. Each of these letters describes RTI’s role in conducting this study for SAMHSA using
somewhat different language. For example:
The first letter states that “Research Triangle Institute (RTI) is the nonprofit
organization assigned to carry out this important study.”
A-5
The second letter states that “RTI International is the nonprofit organization that
is conducting interviews for this important study.”
The third letter states that “We (MEANING SAMHSA) have asked Research
Triangle Institute to do these interviews for us.”
Did you notice these differences? Which of these descriptions of RTI’s role seems most
clear to you? Are any of the words/phrases used to describe RTI’s role unclear to you?
6. The first two versions of these letters are signed by two people, one from the sponsoring
federal agency (SAMHSA) and one from the survey organization (RTI). The third version is
only signed by the person representing the sponsoring federal agency. What impression do
the signatures give you about the survey?
7. In addition to the two signatures, the letter also has a space for the survey interviewer
assigned to the household to hand-print her or his name. How do you think having the
interviewer’s name hand-printed on the letter would make you feel about the letter?
[REFER PARTICIPANTS TO THE VERSION OF THE LETTER WITH THE FI NAME
PRINTED BELOW THE ID BADGE]
8. The second version of the letter (in the second paragraph) mentions that the survey is
“required by Congress.” Does this statement make you feel that you would be required to do
the survey if selected? Would the fact that the survey is required by Congress influence your
decision to participate in the survey? Why?
9. The second version of the letter also indicates that different organizations may endorse this
survey. What association or organization would make you more likely to participate? [IF
NONE ARE OFFERED, OR TO REDIRECT THE DISCUSSION, SPECIFICALLY
MENTION: (1) The American Medical Association (AMA) and (2) AARP.] How would you
feel about endorsement by the AMA or the AARP? Would anyone have concerns about
either of these organizations sponsoring this survey that might make you less likely to
participate? [IF YES] Can you tell me what your concerns would be?
10. Letter 3 includes the name of the survey, the National Survey on Drug Use and Health. Do
you think knowing the name of the survey would make you more or less likely to participate
in the survey?
[DISTRIBUTE THE THREE VERSIONS OF THE LETTERS GRAPHICS ON
TRANSPARENCIES IN ORDER, STARTING AT BOTH ENDS OF THE TABLE. GIVE
PARTICIPANTS A MOMENT TO REVIEW THEM AND ENCOURAGE THEM TO PLACE THE
TRANSPARENCIES OVER EACH VERSION OF THE LETTER TEXT TO SEE HOW THEY
LOOK. THE UPDATED CURRENT LETTER WILL BE REFERRED TO AS VERSION 1b, THE
DILLMAN/CIALDINI-INFLUENCED LETTER WILL BE REFERRED TO AS VERSION 2b, AND
THE NSFG-INFLUENCED LETTER WILL BE REFERRED TO AS VERSION 3b. ]
B. Now I have a few questions about the graphics that are included on these versions of the
three letters:
1. What are some of the first things you noticed about the graphics on these three letters? Are
there any similarities or differences between the three versions that you noticed? [PROBE
RESPONDENTS TO ELABORATE ON ANY DIFFERENCES THEY IDENTIFY AMONG
THE GRAPHICS]
ASK THE FOLLOWING QUESTIONS ONLY IF THESE POINTS WERE NOT ALREADY
MENTIONED IN THE DISCUSSION:
A-6
1. All versions of the letter include the DHHS logo. Do you think this logo would make you
more or less likely to read the letter?
2. The first letter indicates at the top that this is “an important request” from DHHS. Did you
notice that? How do you think that feature might influence whether or not you would read the
letter?
3. What do you think about the silhouette of the interviewer in letters 1 and 3? What do you
think about the picture of the interviewer in letter 2? Which do you prefer?
IF TIME PERMITS, PLEASE USE THE FOLLOWING SCRIPT TO ASK PARTICIPANTS TO
INDICATE THEIR PREFERENCES FOR TEXT AND GRAPHICS AMONG THE THREE
LETTERS AND PASS THEM TO YOU:
Before we move on, could you indicate the version of the letter with text only (that is, without
the graphics) you prefer? Please write the number one (1) in the upper right corner of the
text-only letter that you like best and pass it to me.
Next, please indicate which version of the letter graphics (that is, the logos, pictures, and
letterhead features) you prefer by writing the number one (1) in the upper right corner of the
letter with the graphics that you like best and pass it to me.
A-7
SECTION V: Q&A BROCHURE (30 minutes)
Here are two versions of a brochure describing the study that will not be mailed with the letter,
but will only be provided to individuals at households when they are contacted in person by the
interviewer. Please take one of each version and pass them along.
[DISTRIBUTE THE TWO BROCHURES STARTING AT BOTH ENDS OF THE TABLE AND
GIVE PARTICIPANTS A BRIEF MOMENT TO LOOK OVER THEM. THE UPDATED
CURRENT BROCHURE WILL BE REFERRED TO AS VERSION 1 AND THE REDESIGNED
BROCHURE WILL BE REFERRED TO AS VERSION 2. ]
We are interested in your initial reactions as well as the content of each version of the brochure.
Please take a moment to look at both brochures.
1.
What are some of the first things you noticed about these brochures? Are there any
similarities or differences between the two versions that you noticed? [PROBE
RESPONDENTS TO ELABORATE ON ANY DIFFERENCES THEY IDENTIFY BETWEEN
THE TWO BROCHURES]
2. In your own words, what are the main points that these brochures are telling you about the
survey? Do the main points seem to be about the same, or somehow different, across the
two brochures?
3. For both versions of the brochure, do you think too much, enough, or not enough
information is provided about the survey?
4. Would you like to see more information added to either brochure? What kind of information
would you add that might help people in deciding whether to participate in the survey?
5. Do you think any information should be removed from either of the brochures? If so, which
information do you think should be removed?
6. Does any of the information on either version of the brochure seem confusing to you?
Please explain.
7. In addition to the information provided in the letter, do you think seeing either version of the
brochure would make you more or less likely to participate in the survey?
ASK THE FOLLOWING QUESTIONS ONLY IF THESE POINTS WERE NOT ALREADY
MENTIONED IN THE DISCUSSION:
1. The brochures use a few different terms to describe the survey process used in this study. In
your own words, what do each of the following terms mean to you? Also, please let me
know if you’re really not sure what any of these terms means. (The first one is …)
(1) randomly selected
(2) chosen at random through scientific methods
(3) accurately represent the many different types of people in the United States
2. How do you feel about the colors used in each brochure? Are there other color(s) that you
think would make the brochures more appealing to you?
3. Which pictures or graphics do you like best in each brochure? Are there any pictures or
graphics you do or do not like in either brochure?
4. Does the mention of using a computer to complete the interview or the picture of a laptop
computer on either version of the brochure worry you at all? (If so, why are you concerned
about the use of a computer?)
A-8
5. Do you think it’s important to include the DHHS logo, which is on both versions? Why or why
not?
6. The second version also includes the SAMHSA and RTI logos. Do you think it’s important to
include these logos on the brochure? Do you think these logos should be on the front or on
the back of the brochure, or both the front and back?
7. In the section “What Is the National Survey On Drug Use and Health?,” the second version
indicates that “SAMHSA is working with Research Triangle Institute (RTI), a nonprofit
research organization, to complete the interviews.” Also, in the section “How is the Study
Conducted,” the brochure indicates that “SAMHSA has selected Research Triangle Institute
(RTI), a nonprofit research organization, to complete the interviews.”
You may recall the three letters we looked at earlier all described RTI’s role somewhat
differently – “RTI is assigned to carry out this important study,” “RTI is conducting interviews
for this important study, ” and “We (SAMHSA) have asked Research Triangle Institute to do
these interviews for us.”
Do you think the language used in this brochure to describe RTI’s role is clearer, about as
clear, or less clear than the language you preferred from the three letters? Why?
IF TIME PERMITS, PLEASE USE THE FOLLOWING SCRIPT TO ASK PARTICIPANTS TO
INDICATE THEIR PREFERENCE BETWEEN THE TWO BROCHURES AND PASS THEM TO
YOU:
Before we move on, could you indicate which brochure you prefer by writing the number one (1)
in the upper right corner of the brochure that you like better and passing it to me?
SECTION VI: CONCLUSION (5 minutes)
Are there any final comments or any questions?
I want to thank you all for your participation. We will now distribute the $75 incentive we
promised you and ask you to sign a receipt form verifying that you received it.
THE RTI NOTETAKER WILL NOW TURN OFF THE VIDEO CAMERA.
THE RTI NOTETAKER WILL THEN DISTRIBUTE THE CASH GIFTS IN ENVELOPES TO
EACH PARTICIPANT AND COLLECT EACH PARTICIPANT’S SIGNATURE ON THE
INCENTIVE RECEIPT FORM.
A-9
A-10
Appendix B:
Participant Informed Consent Form
Focus Group Participant Informed Consent Form
National Survey on Drug Use and Health (NSDUH)
The Substance Abuse and Mental Health Services Administration (SAMHSA) has contracted with RTI
International to complete a large national survey conducted with about 70,000 people each year. In
association with this project, RTI has been asked to conduct focus groups like this one with adults to
evaluate the contact materials used on the survey.
Your participation in this focus group will involve answering questions and discussing issues about the
content and design of contact materials, such as the letter sent to households and informational
brochures. We will ask how we can improve these materials to increase survey participation. There will
be up to 10 people participating in the group. You are one of approximately 170 people participating in
a focus group on the improvement of these contact materials. There are no right or wrong answers; we
just want your opinions.
Your participation in this group discussion is voluntary and you may withdraw at any time. Your
participation and any comments you make will be kept confidential. Comments from all participants
will be combined in a summary report that will not identify any individual. We ask that you show
consideration for others and refrain from sharing information after leaving this focus group. This focus
group will last two hours.
Your personal information will not be connected to your remarks in any way. We will be recording this
focus group on audio and videotape, and transferring this recording to DVD, in order to more easily
prepare our summary report. The tapes and DVD will be destroyed after they have been used. You will
receive $75 as a token of appreciation for your participation.
There are no physical risks to you from participating in this focus group other than those associated
with everyday living. You do not have to answer any questions that you do not want to. There are no
direct benefits to you from participating in this focus group.
If you have any questions about this focus group, please ask your group moderator. If you have any
questions about your rights as a study participant, you can call RTI's Office of Research Protection at
1-866-214-2043 (a toll-free number). For any other questions about the study, you can call Doug
Currivan of RTI at 1-800-334-8571, extension 23334.
With my signature below, I confirm that this information has been explained to me and I give consent
for my participation in this focus group and its audio and video recording.
___________________________
Signature Participant
_________________________
Date
___________________________
Signature of Moderator
_________________________
Date
B-1
B-2
Appendix C:
Incentive Receipt
Receipt Form for Focus Groups on Survey Participation
National Survey on Drug Use and Health (NSDUH)
We thank you for participating in our focus group. Your help with this research study
sponsored by the Substance Abuse and Mental Health Services Administration
(SAMHSA) and conducted by RTI International will help us plan strategies for the
National Survey on Drug Use and Health (NSDUH). For your participation, we are
pleased to provide you with this cash incentive as an expression of our gratitude.
By signing below, you confirm that you received $75 in appreciation for your
participation in this project.
Please initial here: __________________________________________________
Date:_____________________________________________________________
Witnessed by: _____________________________________________________
C-1
C-2
Appendix D:
Sample Focus Group Recruitment Advertisement
Participate in a Discussion Group for a Research Study
RTI International, a not-for-profit research organization with offices in Research Triangle Park,
is looking for adults to take part in discussion groups scheduled for [INSERT DATES]. The
purpose of these discussion groups is to get input from [INSERT AREA] residents like you to
improve the materials used for a large national study. If you are eligible and participate in a
discussion group, you will be given $75 in cash as a token of our appreciation. Each discussion
group will take from about 1 ½ to 2 hours. We are looking for people who must be:
1) Age 18 and over
2) Not currently serving on active duty in the military
3) Not currently employed by RTI International
4) Not a family member of an RTI employee
To find out more about the study, call [TOLL FREE NUMBER] and ask for [NAME] or email
her at [EMAIL]. All information you provide RTI prior to or during the discussion group will
remain strictly confidential.
D-1
D-2
Appendix E:
Focus Group Recruitment and Screening Protocol
Date/Time of Call ___________________
Hello, this is [NAME] calling from RTI International. (Were you calling about the ad?)
Let me tell you a little about the study. The purpose of the study is to improve the contact
materials, such as letters and brochures, which are used in a large national survey. The Substance
Abuse and Mental Health Services Administration has hired us to conduct the survey and we are
working with them to improve the existing contact materials. We are trying to find people who
are interested in helping us by providing their opinions about the materials we want to test. If you
are interested and eligible, we would like you to come to a small discussion group that will take
about 1 ½ to 2 hours. At the end of the discussion, you will receive $75 in cash in appreciation
for your time. To make sure you are eligible for the study, I need to ask you some questions. This
will only take about 5 minutes. (Is this a good time?)
[RECORD CANDIDATE’S RESPONSES ON PAPER SCREENING ROSTER.]
1. [RECORD GENDER.] (IF NECESSARY, ASK: Are you male or female?)
1
FEMALE
2
MALE
2. How did you hear about the study?
1
FROM CRAIG’S LIST
2
FROM THE NEWSPAPER
3
FROM A FLYER
4
FROM A FRIEND OR FAMILY MEMBER
5
COMMUNITY ORGANIZATION: ____________
6
SOME OTHER METHOD ___________________
3. And how old are you?
[CHECKPOINT 1: IF CALLER IS UNDER 18, (S)HE IS INELIGIBLE. THANK R
AND TERMINATE CALL.]
[CHECKPOINT 2: IF THE QUOTA FOR THIS AGE GROUP IS NOT FILLED,
CONTINUE WITH Q4.]
[IF THE QUOTA FOR THE CALLER’S AGE GROUP IS FILLED, THANK AND
TERMINATE CALL.]
[CODE AGE RANGE BELOW]
1 18-29
2 30-49
3 50+
E-1
4. Are you of Hispanic, Latino, or Spanish origin?
1 YES → CONTINUE
2 NO → GO TO Q6
5. Are you Mexican, Puerto Rican, Cuban, Central or South American, or some other Hispanic,
Latino or Spanish origin?
1
Mexican
2
Puerto Rican
3
Cuban, Central or South American
4
Or some other Hispanic, Latino, or Spanish Origin
6. Are you White, Black or African American, American Indian or Alaskan Native, Asian,
Native Hawaiian or Other Pacific Islander, or Some other race?
[CAN SELECT MORE THAN 1 RESPONSE.]
1
2
3
4
5
6
White
Black or African American
American Indian or Alaskan Native
Asian
Native Hawaiian or other Pacific Islander
Other Race
7. What is the highest level of education you have completed?
1
Less than high school
2
High school graduate, including GED
3
Technical or vocational school (attended or graduated) or some college
4
College graduate, including postgraduate degree
8. What was your household’s total income, from all sources, during the past 12 months?
Was it …
1
Less than $50,000
2
More than $50,000 but less than $100,000
3
More than $100,000
9. In what city or town do you live?________________ [TO ASSURE PERSON IS WITHIN
DRIVING DISTANCE TO FACILITY]
E-2
10. Would you describe where you live as urban, suburban or rural?
1
URBAN
2
SUBURBAN
3
RURAL
11. What is your ZIP code? _________________ [TO GET CENSUS INCOME DATA IF
INCOME IS NOT PROVIDED]
[CHECKPOINT 3: IF SCREENER IS IN ENGLISH ASK Q12 AND Q13;
IF SCREENER IS IN SPANISH, GO TO CHECKPOINT 4]
12. Do you speak English as your native (primary) language?
1
YES
2
NO [THANK AND TERMINATE]
13. Can you read English?
1
YES [GO TO CHECKPOINT 4]
2
NO [THANK AND TERMINATE]
[CHECKPOINT 4: IF SCREENER IS IN ENGLISH, GO TO CHECKPOINT 5]
IF SCREENER IS IN SPANISH, CONTINUE WITH Q14-18]
14. Do you speak Spanish as your native (primary) language?
1
YES
2
NO [THANK AND TERMINATE]
15. Can you read Spanish?
1
YES
2
NO [THANK AND TERMINATE]
16. Do you speak English?
1
YES
2
NO [GO TO CHECKPOINT 5]
17. Would you say you speak English …?
1
VERY WELL
2
WELL, OR
3
NOT WELL
E-3
18. Would you say you read Spanish…?
1
VERY WELL
2
WELL
3
NOT WELL, OR
4
NOT AT ALL
[CHECKPOINT 5: REFER TO RECRUITMENT GRID. IF THE LIMIT IS REACHED FOR
THE CALLER’S AGE GROUP, THANK AND COLLECT PHONE NUMBER TO PUT ON
THE WAITING LIST]
[IF ELIGIBLE FOR ONE OF THE GROUPS, CONTINUE]:
You are eligible to participate in our focus group! Taking part in the focus group is voluntary. It
will be conducted at [our office/NAME OF FOCUS GROUP FACILITY] in [CITY]. Any
remarks from the discussion will remain confidential. As such, we will not share information you
give us with anyone other than project staff. We will audio and video record the focus group to
help us write a summary report about the findings, but the recordings will be destroyed once we
complete our report.
Would you like to take part in the focus group?
1
2
YES [SCHEDULE FOR GROUP, PROVIDE DATE, TIME, DIRECTIONS]
NO [THANK AND TERMINATE CALL]
E-4
Appendix F:
Contact Materials – Lead Letter Envelope (Regular Size)
U.S. DEPARTMENT OF HEALTH & HUMAN SERVICES
RTI Project 9009
6100 Executive Boulevard, Suite 902
Rockville, Maryland 20852-3907
OFFICIAL BUSINESS
PENALTY FOR PRIVATE USE $300
Standard size
F-1
F-2
Appendix G:
Contact Materials – Lead Letter Envelope (9 x 12)
U.S. DEPARTMENT OF HEALTH & HUMAN SERVICES
RTI Project 9009
6110 Executive Boulevard, Suite 902
Rockville, Maryland 20852-3907
OFFICIAL BUSINESS
PENALTY FOR PRIVATE USE $300
G-1
G-2
Appendix H:
Contact Materials – Lead Letter Version 1a (Text Only)
______________, 2009
Resident
1234 Main Street
Anywhere, XX 12345
Dear Resident:
The U.S. Department of Health and Human Services (DHHS) is conducting a national study on
health-related issues. Your address was randomly chosen to participate along with more than
200,000 others. Research Triangle Institute (RTI) is the nonprofit organization assigned to carry
out this important study. Soon, an RTI interviewer will be in your neighborhood to provide more
information.
The interviewer will be carrying an identification card like the example shown below. The
interviewer will ask you a few questions, and then may ask one or two members of your
household to complete an interview. It is possible that no one from your household will be
chosen to be interviewed. Every person who is chosen and completes the full interview will
receive $40 in cash.
All information collected is confidential and will be used only for statistical purposes. This is
assured by federal law. This letter is addressed to “Resident” because your address was selected,
and we do not know your name. Please share this information with any others in your
household.
Feel free to ask the interviewer any questions you have about this study. More information is
also available on the study website at: http://nsduhweb.rti.org or you may contact us at (800)
848-4079.
Your help is very important to this study’s success. Thank you for your cooperation.
Sincerely,
Joel Kennet, Ph.D.
National Study Director, DHHS
Ilona S. Johnson
National Field Director, RTI
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by the U.S. Congress as part of Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
Version 1a
H-1
H-2
Appendix I:
Contact Materials – Lead Letter Version 2a (Text Only)
______________, 2009
Household members at:
1234 Main Street
Anywhere, XX 12345
Dear [city/town/county] area resident:
You have received this letter because in the next few days, the interviewer named below will be
coming to your household to request your participation in an important national study on health
issues.
This annual national study is sponsored by The U.S. Department of Health and Human Services
(DHHS) and required by Congress (U.S. Code 42 USC290aa4). The study has also been
endorsed by the following organizations: ________________________. RTI International is the
nonprofit organization that is conducting interviews for this important study.
When the interviewer whose name appears below comes to your address, please ask to see their
identification card. It will look like the one that appears below. The interviewer will ask you a
few questions, and then may ask one or two members of your household to complete an
interview. It is possible that no one from your household will be chosen to be interviewed.
Every person who is chosen and completes the full interview will receive $40 in cash in
appreciation for their help.
This important national survey is the only one that provides much-needed national statistics on
health and health-related behaviors of people throughout the United States. The privacy of all
the information we collect is assured by federal law. Your answers are confidential, and can
only be used for statistical purposes.
Feel free to ask our interviewer any questions you have about this study. If you have internet
access you may also find out more at: http://nsduhweb.rti.org or you may contact us at (800)
848-4079.
Your help is very important to the success of this study and we thank you for your cooperation.
Please share this information with any others in your household.
Sincerely,
Joel Kennet, Ph.D.
National Study Director, DHHS
Ilona Johnson
National Field Director, RTI
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
Version 2a
I-1
I-2
Appendix J:
Contact Materials – Lead Letter Version 3a (Text Only)
______________, 2009
Resident
1234 Main Street
Anywhere, USA 12345
Dear Resident:
My agency, part of the U.S. Department of Health and Human Services (DHHS), needs your
help. We are doing an important study called the National Survey on Drug Use and Health. The
study asks questions about use of prescription and non-prescription drugs, alcohol and tobacco,
and other aspects of health and mental health. The information is used for planning and
providing health services in all 50 states and the District of Columbia.
We are asking a scientific sample of households to take part in a short interview. We have asked
Research Triangle Institute to do these interviews for us. In a few days, an interviewer will visit
your home to ask you a few initial screening questions. The visit will only take about 5 minutes,
and any adult who lives in the home can answer. After these initial questions, you or someone in
your household may be asked to complete our main interview. At that time our interviewer will
be able to answer any questions you might have.
The interviewer will be carrying an identification card like the example shown below. Every
person who is chosen and completes the main interview will receive $40 in cash.
By Federal law, the answers you give are confidential and will be used for research only.
If you have any questions before the interviewer visits, feel free to call the study information line at
(800) 848-4079. We look forward to speaking with you soon. I thank you for your help with this
important study. Please share this information with any others in your household.
Sincerely,
Joel Kennet, Ph.D.
National Study Director, DHHS
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
J-1
Version 3a
J-2
Appendix K:
Contact Materials – Lead Letter Version 1b (Graphics Only)
AN IMPORTANT REQUEST FROM
THE U.S. DEPARTMENT OF HEALTH & HUMAN SERVICES
Sincerely,
NATIONAL STUDY conducted for the
U.S. Department of Health & Human
Services
Field Interviewer Name
Joel Kennet, Ph.D.
National Study Director, DHHS
ID: 999999
Issue date: 1/1/2009
Expiration date: 12/31/2009
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by the U.S. Congress as part of Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
Version 1b
K-1
K-2
Appendix L:
Contact Materials – Lead Letter Version 2b (Graphics Only)
UNITED STATES DEPARTMENT OF HEALTH & HUMAN SERVICES
Sincerely,
Rockville, MD
NATIONAL STUDY conducted for the
U.S. Department of Health & Human
Services
Field Interviewer Name
ID: 999999
Issue date: 1/1/2009
Expiration date: 12/31/2009
Joel Kennet, Ph.D.
National Study Director, DHHS
Ilona Johnson
National Field Director, RTI
You will be contacted by: ________________________________________
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
Version 2b
L-1
L-2
Appendix M:
Contact Materials – Lead Letter Version 3b
(Graphics Only)
UNITED STATES DEPARTMENT OF HEALTH & HUMAN SERVICES
Rockville, MD 20857
Sincerely,
NATIONAL STUDY conducted for the
U.S. Department of Health & Human
Services
Field Interviewer Name
Joel Kennet, Ph.D.
National Study Director, DHHS
ID: 999999
Issue date: 1/1/2009
Expiration date: 12/31/2009
You will be contacted by: ___________________________________________
Confidentiality protected by the Confidential Information Protection and Statistical Efficiency Act of 2002 (PL 107-347)
Authorized by Section 505 of the Public Health Service Act (42 USC 290aa4)
Approved by Office of Management and Budget (OMB Approval No. 0930-0110)
NC10010052
M-1
Version 3b
M-2
Appendix N:
Contact Materials – Q&A Brochure Version 1
N-1
N-2
Appendix O:
Contact Materials – Q&A Brochure Version 2
O-1
O-2
| File Type | application/pdf |
| Author | gmchenry |
| File Modified | 2014-06-09 |
| File Created | 2014-06-09 |