Redesign of SUD Modules Report

National Survey on Drug Use and Health (NSDUH): Redesign of Substance Use Disorder Modules

Substance Abuse and Mental Health Services Administration
Center for Behavioral Health Statistics and Quality
Rockville, Maryland

September 2019

National Survey on Drug Use and Health (NSDUH): redesign of Substance Use Disorder Modules

For questions about this report, please e-mail [email protected].

Prepared for Substance Abuse and Mental Health Services Administration, Rockville, Maryland

Prepared by RTI International, Research Triangle Park, North Carolina

September 2019

Recommended Citation: Center for Behavioral Health Statistics and Quality. (2019). National Survey on Drug Use and Health (NSDUH): Redesign of Substance Use Disorder Modules. Rockville, MD: Substance Abuse and Mental Health Services Administration.

Table of Contents

Section Page

Executive Summary 1

1. Introduction 3

2. Historical Work 5

2.1 Literature Review 5

2.2 Development of DSM-5 Survey Items 8

2.3 2016 DSM-5 Cognitive Interviewing 8

3. SUD Redesign 11

3.1 Overview 11

3.1.1 Delay of DSM-5 Implementation 11

3.1.2 2017 Expert Panel 12

3.1.3 Statistical Analyses 13

3.1.4 2018 SUD Cognitive Interviewing 13

3.1.5 Report Organization 13

3.2 Overall SUD Assessment 13

3.3 SUD Module Entry 15

3.3.1 Frequency-of-Use Threshold 15

3.3.2 Use but Not Misuse of Prescription Drugs 22

3.3.3 Sedatives and Tranquilizers 22

3.4 SUD Criteria 22

3.4.1 Existing Items 22

3.4.2 New Items 55

3.5 Validation 65

3.5.1 Importance 65

3.5.2 Mode 65

3.5.3 Gold Standard 66

3.5.4 Age 66

3.5.5 Language 67

3.6 Other Topics 68

3.6.1 Emerging Drugs and Modes of Use 68

3.6.2 Prescription Drugs 68

3.6.3 Medical Marijuana 69

3.6.4 Tobacco 69

3.6.5 Inhalant Use Disorder 69

3.6.6 Remission 69

3.6.7 Behavioral Addictions 69

4. Summary 71

References 73

List of Contributors 75

Table of Contents (continued)

Appendix Page

A Module Entry Analyses Tables A-1

B Prescription Drug Use but Not Misuse Memo B-1

C 2020 Clinical Validation Study NSDUH Alcohol Use Disorder Module C-1

List of Tables

Table Page

2.1 Comparison of DSM-IV and DSM-5 Diagnostic Criteria with the 2017 NSDUH
SUD Assessment 6

3.1 Potential Misclassification for DSM-5 Alcohol and Marijuana Use Disorder in
NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs,
All Ages (12 or Older) 20

3.2 Summary if Effect of Module Entry Rule is More than 5 Days of Use for Alcohol
and Marijuana (DSM-IV) 21

3.3 Crosswalk of NSDUH, DSM-IV-TR, and DSM-5 Substance Use Disorder Criteria 23

3.4 Assessment by Other Surveys of DSM-5 SUD Criterion A3: Time Spent 25

3.5 Cognitive Interviewing Question Versions Assessing Criterion A3 for NSDUH 26

3.6 Assessment by Other Surveys of DSM-5 SUD Criterion A1: Larger Amounts 29

3.7 Cognitive Interviewing Question Versions Assessing Criterion A1 for NSDUH 29

3.8 Assessment by Other Surveys of DSM-5 SUD Criterion A10: Tolerance 31

3.9 Cognitive Interviewing Question Versions Assessing Criterion A10 for NSDUH 32

3.10 Assessment by Other Surveys of DSM-5 SUD Criterion A2: Persistent Desire, Unsuccessful Control 34

3.11 Cognitive Interviewing Question Versions Assessing Criterion A2 for NSDUH 34

3.12 Assessment by Other Surveys of DSM-5 SUD Criterion A11a: Withdrawal Criterion A11a 38

3.13 Cognitive Interviewing Question Versions Assessing Criterion A11a for NSDUH 40

3.14 Assessment by Other Surveys of DSM-5 SUD Criterion A9: Continued Use
Despite Health Problems 43

3.15 Cognitive Interviewing Question Versions Assessing Criterion A9 for NSDUH 44

3.16 Assessment by Other Surveys of DSM-5 SUD Criterion A7: Giving Up Activities 46

3.17 Cognitive Interviewing Question Versions Assessing Criterion A7 for NSDUH 47

3.18 Assessment by Other Surveys of DSM-5 SUD Criterion A5: Role Failure 49

3.19 Cognitive Interviewing Question Versions Assessing Criterion A5 for NSDUH 50

3.20 Assessment by Other Surveys of DSM-5 SUD Criterion A8: Hazardous Use 51

3.21 Cognitive Interviewing Question Versions Assessing Criterion A8 for NSDUH 52

3.22 Assessment by Other Surveys of DSM-5 SUD Criterion A6: Use Despite Social Problems 54

3.23 Cognitive Interviewing Question Versions Assessing Criterion A6 for NSDUH 54

List of Tables (continued)

Table Page

3.24 Assessment by Other Surveys of DSM-5 SUD Criterion A4: Craving 57

3.25 Cognitive Interviewing Question Versions Assessing Criterion A4 for NSDUH 58

3.26 Assessment by Other Surveys of DSM-5 Marijuana Withdrawal 60

3.27 Cognitive Interviewing Question Versions Assessing Criterion A11a for NSDUH 61

3.28 Assessment by Other Surveys of DSM-5 Withdrawal Criterion B 64

3.29 Cognitive Interviewing Question Versions Assessing Criterion A10b for NSDUH 65

3.30 Spanish-Language Respondents in the 2014 NSDUH, by Age and AUD, DUD,
and SUD (Drug or Alcohol Use Disorder) Status: Unweighted n, Weighted N
(in Thousands), Percentages, and Standard Errors 68

A.1 Potential Misclassification for DSM-5 Cocaine, Heroin, Hallucinogen, and Inhalant
Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older) A-3

A.2 Potential Misclassification for DSM-5 Prescription Pain Reliever, Tranquilizer, Stimulant, and Sedative Use Disorder in NSDUH, Based on Unweighted Frequency
of Use for the 2004-2014 NSDUHs, All Ages (12 or Older) A-5

B.1 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Stimulants If They Were Routed into the Stimulant Disorder Section,
by Age Group: NSDUH 2015-2016, Annual Averages B-4

B.2 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Sedatives/Tranquilizers If They Were Routed into the
Sedative/Tranquilizer Use Disorder Section, by Age Group: NSDUH 2015-2016,
Annual Averages B-5

B.3 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Pain Relievers If They Were Routed into the Opioid Use Disorder (OUD) Section, by Age Group: NSDUH 2015-2016, Annual Averages B-5

Executive Summary

In 2013, the American Psychiatric Association released the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013), the guidelines used to assess mental illnesses, including substance use disorders (SUDs). In response, the Substance Abuse and Mental Health Services Administration (SAMHSA) began an investigation into the impact of these changes on the National Survey on Drug Use and Health (NSDUH), the primary source for statistical information on illicit drug use, alcohol use, SUDs, and mental health issues.

The process of developing the revisions to the NSDUH SUD assessments was extensive. Initial review of the DSM-5 diagnostic changes and the literature about these changes resulted in the generation of an extensive report on the impact of these changes on the prevalence estimates of SUDs in the nation (Center for Behavioral Health Statistics and Quality, 2016b). This report along with copies of the existing NSDUH SUD assessments and new draft items for DSM-5 (e.g., craving and marijuana withdrawal items) were provided to a panel of external experts for written review in 2015. The experts were instructed to review the craving and marijuana withdrawal draft items. Experts critiqued the draft items and also provided feedback on existing NSDUH items indicating that improvement could be made for both the draft items and existing items. Cognitive interviewing on the draft items was then conducted in 2016. Results from the cognitive interviews suggested that further testing was needed on the new items. Results from cognitive interviewing on new items, 2015 expert feedback on some portions of the NSDUH questionnaire not related to the DSM-5 changes, and prior validation work on NSDUH (Jordan, Karg, Batts, Epstein, & Wiesen, 2008) led to delayed implementation of the DSM-5 items in favor of a more extensive redesign of the SUD module (Center for Behavioral Health Statistics and Quality, 2016a). In 2017, work on a more extensive redesign of the SUD module began, which included expert review and input on all items, cognitive testing, and a clinical validation study conducted in conjunction with the 2020 NSDUH.

In 2017, the full SUD module redesign process was initiated, beginning with an in-person expert panel review of the NSDUH SUD module, complete with the results of prior validation work and the 2016 cognitive interviewing results. Based on the 2017 Expert Panel Findings, the NSDUH SUD module was extensively revised, and these revisions were tested using cognitive interviewing in 2018-2019. This resulted in the revised module being finalized in 2019 for implementation in the 2020 NSDUH Clinical Validation Study.

This report documents the steps of the redesign from the initial response to the DSM-5 revision through the development of the final NSDUH SUD module for validation in 2020. It serves as a historical record of the considerations and decisions completed at each step of the SUD redesign, with references to related reports as appropriate.

1. Introduction

The National Survey on Drug Use and Health (NSDUH) is a premier source of nationally representative data on substance use and mental health in the United States. Sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA) and supervised by SAMHSA's Center for Behavioral Health Statistics and Quality, NSDUH provides policymakers, researchers, public health practitioners, and the public with national, state-level, and substate-level data on tobacco, alcohol, and illicit drug use (including nonmedical use of prescription drugs); substance use disorders (SUDs) and treatment; mental health issues and service use; and co-occurring SUDs and mental health issues. NSDUH data have been collected annually since 1971, and the survey currently collects substance use and mental health information from approximately 68,000 residents of households and noninstitutional group quarters (e.g., shelters, rooming houses, dormitories) and from civilians living on military bases. The survey includes interviews with adults (aged 18 or older) and adolescents (aged 12 to 17) in English or Spanish using audio computer-assisted self-interviewing for questions about illicit drug use, other sensitive behaviors, and mental health and interviewer-administered questions for less sensitive topics.

NSDUH's current assessment of SUDs is based on the diagnostic guidelines for substance dependence and substance abuse found in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV; American Psychiatric Association, 1994). DSM-IV provides a classification system for clinicians, insurance providers, researchers, and policymakers to use in matters related to diagnosing, researching, and treating mental illness. NSDUH defines SUD as meeting criteria for DSM-IV substance dependence or abuse for each of the following substances: alcohol; marijuana; cocaine (including crack); heroin; hallucinogens; inhalants; and prescription pain relievers, stimulants, sedatives, and tranquilizers.

In 2013, DSM-IV was replaced with the fifth edition of the manual (DSM-5; American Psychiatric Association, 2013). The DSM-5 revision contained changes in organization and changes to the diagnostic criteria for nearly every DSM-IV disorder, including those for SUDs (Center for Behavioral Health Statistics and Quality, 2016b). These changes prompted a revision process to redesign and update NSDUH to provide high-quality data on SUDs that reflect the DSM-5 criteria. This process included convening an expert panel to evaluate the current NSDUH SUD modules and identify areas for improvement based upon the best current knowledge in the field.

A key feature of NSDUH is that it provides policymakers and researchers with estimates of SUDs and other substance use and mental health issues over time. Because changes in the questionnaire can disrupt the ability to interpret changes over time, changes to NSDUH are made infrequently. When NSDUH redesigns are conducted, they provide an important opportunity to update the survey to reflect advancements in the field and current public health issues related to substance use and mental health. These redesigns involve considerable effort, including reviewing extant literature, obtaining input from substantive experts and stakeholders, consulting with survey methodologists, completing cognitive interviewing and other pilot testing of revised or new questions, and performing other tasks designed to strengthen the validity and reliability of the survey instrument. This report documents the activities undertaken in the process of redesigning the NSDUH SUD module. (Note that there is only one template SUD module, but this module sometimes varies by substance. This report uses the plural modules in reference to the different module versions). This report documents the steps involved in this process, including (1) reviewing the literature, (2) identifying potential items to add to the survey, (3) obtaining written expert review on the potential new items and revising them accordingly, (4) cognitively testing new items, (5) assembling and completing an in-person expert panel to identify next steps in the redesign process, and (6) developing and testing new and revised SUD modules for a clinical validation study, conducted in tandem with the 2020 NSDUH.

2. Historical Work

2.1 Literature Review

The first step undertaken in the substance use disorder (SUD) redesign was a review of the changes to the classification and diagnostic criteria for SUDs, based on the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013). It included a review of the existing literature on the current state for assessing SUDs, how changes in the diagnostic criteria would change the classification status for people with and without an SUD, and how these changes would affect prevalence estimates. This review culminated in a report documenting the anticipated effect of revisions to DSM on estimates of SUDs in the nation (Center for Behavioral Health Statistics and Quality, 2016b). Several changes to SUD diagnosis criteria are noted in this report and summarized in Table 2.1. The first major DSM-5 change was the changing of disorder class, a term used to group similar disorders in DSM. Under DSM-IV (fourth edition; American Psychiatric Association, 1994), SUDs were in a class by themselves. Under DSM‑5, the class was renamed substance-related and addictive disorders and was broadened to include gambling disorder, a type of behavioral addiction, which is not currently assessed in the National Survey on Drug Use and Health (NSDUH). Behavioral addictions are a growing area of research interest that may be considered for inclusion in future NSDUHs, depending on stakeholder needs (Grant, Potenza, Weinstein, & Gorelick, 2010; Robbins & Clark, 2015).

In addition to the change in the classification of SUDs, the diagnostic structure for SUD changed. DSM-5 combined two distinct DSM-IV disorders, substance abuse and substance dependence, into a single drug use disorder with mild, moderate, and severe subclassifications. This structural change applies to each type of substance (e.g., alcohol or hallucinogens). Under DSM-IV, a diagnosis of substance abuse required the presence of one or more of four diagnostic criteria within a 12-month period and no history of substance dependence for that type of substance. A diagnosis of substance dependence required the presence of three or more of six or seven criteria, depending on substance type, in a 12-month period. If a person met the diagnostic criteria for dependence and had one or more symptoms of abuse, then he or she was diagnosed as having substance dependence to reflect the conceptually higher severity of dependence over the abuse diagnosis. DSM-5 eliminated the distinction between abuse and dependence for several reasons. First, the DSM-IV structure permitted the creation of diagnostic orphans, where a person met two dependence symptoms and abuse symptoms did not meet the requirements for either diagnosis, despite having symptoms conceptually more severe than those required for an abuse diagnosis. DSM-5 eliminated this kind of anomaly by combining the abuse and dependence criteria under a single new SUD diagnosis, which requires 2 out of 11 criteria within a 12-month period. The severity designation is based on the number of criteria met (mild if two or three criteria are met, moderate if four or five criteria are met, and severe if six or more criteria are met).

Table 2.1 Comparison of DSM-IV and DSM-5 Diagnostic Criteria with the 2017 NSDUH SUD Assessment

Characteristic	DSM-IV	DSM-5	NSDUH
Disorder Class	Substance-related disorders, included only SUDs	Substance-related and addictive disorders class now includes SUDs and gambling disorder (formerly pathological gambling)	Same as DSM-IV
Disorder Types	Abuse and dependence hierarchical diagnostic rules meant that people ever meeting criteria for dependence did not receive a diagnosis of abuse for the same class of substance	Substance abuse and dependence have been eliminated in favor of a single diagnosis: SUD	Same as DSM-IV
Assessed Disorders	Substance abuse: One or more symptoms	SUD: Two out of 11 criteria clustering in a 12-month period are needed to meet disorder threshold	Substance abuse: One or more symptoms in the past year
	Recurrent substance-related legal problems	Dropped	DSM-IV criterion assessed
	Recurrent substance use in situations where it is physically hazardous	Same	Assessed
	Recurrent substance use resulting in a failure to fulfill major role obligations at work, school, or home	Same	Assessed
	Continued substance use despite having persistent or recurrent social or interpersonal problems caused or exacerbated by the effects of the substance	Same	Assessed
		Added: Craving or a strong desire or urge to use the substance	DSM-5 craving criterion not assessed
	Substance dependence: Three or more symptoms in the same 12-month period (or one symptom if dependence criteria have been met previously in the lifetime)		Substance dependence: Three or more symptoms in the past year
	Substance is taken in larger amounts or over a longer period than was intended	Same	Assessed
	There is a persistent desire or unsuccessful efforts to cut down or control substance use	Same	Assessed
	A great deal of time is spent in activities necessary to obtain the substance, use the substance, or recover from its effects	Same	Assessed
	Important social, occupational, or recreational activities are given up or reduced because of substance use	Same	Assessed
	Substance use is continued despite knowledge of having a persistent or recurrent physical or psychological problem that is likely to have been caused or exacerbated by substance use	Same	Assessed

Table 2.1 Comparison of DSM-IV and DSM-5 Diagnostic Criteria with the 2017 NSDUH SUD Assessment (continued)

Characteristic	DSM-IV	DSM-5	NSDUH
	Tolerance, as defined by (1) a need for markedly increased amounts of the substance to achieve intoxication or the desired effect, or (2) a markedly diminished effect with continued use of the same amount of the substance	Same	Assessed
	Withdrawal, as manifested by (1) the characteristic withdrawal syndrome for the substance (excludes marijuana, hallucinogens, and inhalants), or (2) the substance (or a similar substance) is taken to relieve or avoid withdrawal symptoms	Withdrawal, as manifested by (1) the characteristic withdrawal syndrome for the substance (excludes phencycli­dine, other hallucinogens, and inhalants), or (2) the substance (or a closely related substance) is taken to relieve or avoid withdrawal symptoms NOTE: This criterion is not considered met for those taking opioids, sedatives, hypnotics or anxiolytics, or stimulant medications solely under appropriate medical supervision.	Part 1 of the criterion is assessed; DSM-5 Marijuana withdrawal is not assessed
Severity	No severity criteria	Severity is assessed in terms of the number of symptoms that meet criteria: Mild: two to three symptoms Moderate: four to five symptoms Severe: six or more symptoms	No severity criteria assessed
Additional Specifications	With or without physiological dependence, early full remission, early partial remission, sustained full remission, sustained partial remission, on receiving agonist therapy, and in a controlled environment	Early or sustained remission and if the person is in a controlled environment where access to the substance is restricted	Not assessed

DSM-IV = Diagnostic and Statistical Manual of Mental Disorders, fourth edition (American Psychiatric Association, 1994); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013); SUD = substance use disorder.

In addition, there were three criteria changes under DSM-5. First, DSM-5 eliminated the abuse criterion related to recurrent substance-related legal problems (e.g., arrests for substance-related disorderly conduct) for all substances. Second, DSM-5 added a craving criterion (i.e., a "strong desire or urge to use the substance") for all substances. Third, a withdrawal syndrome for marijuana/cannabis (i.e., physical or psychological symptoms experienced shortly after stopping heavy and prolonged use of marijuana) was added under DSM-5.

The changes to SUD diagnostic requirements were anticipated to affect the population estimates of SUD in NSDUH. Specific details on the effects of these changes on each type of SUD (e.g., alcohol, marijuana, cocaine) are provided in the Impact of the DSM-IV to DSM-5 Changes on the National Survey on Drug Use and Health report (Center for Behavioral Health Statistics and Quality, 2016b). In general, prevalence estimates of SUD for each substance were anticipated to be higher under DSM-5 SUD criteria than DSM-IV abuse or dependence criteria; however, the magnitudes of the anticipated increases were not predicted.

2.2 Development of DSM-5 Survey Items

The next step in the SUD redesign was to develop new survey items to reflect the criteria changes to SUD diagnosis under DSM-5. New items were needed to assess the new DSM-5 craving criterion across all substances and the new DSM-5 withdrawal syndrome for marijuana and cannabis specifically. In addition, this piece of the redesign included the development of a new NSDUH item to assess withdrawal criterion 11B, which specifies that a person uses the substance or a closely related substance to avoid having withdrawal symptoms. This criterion was included in DSM-IV and was carried over to the DSM-5 criteria but previously had not been included in NSDUH.

Development of the items for the three new criteria involved a combination of input from substantive experts and survey methodologists. Once the new items were drafted, they were sent in 2015 to external experts for review and written feedback, along with the Impact of the DSM‑IV to DSM-5 Changes on the National Survey on Drug Use and Health report (Center for Behavioral Health Statistics and Quality, 2016b). The 2015 substantive experts were chosen based on prior work on development of the DSM-5 criteria and SUD assessment and their methodological experience in conducting national epidemiological surveys. Experts were selected to include a broad range of expertise in SUD assessment across substances, across adolescent and adult populations, and across English- and Spanish-speaking populations. Although they were asked to focus on the newly drafted items, some reviewers provided feedback on existing items as well (discussed further in Section 3.4.1, Existing Items). After the written expert feedback was received, the draft NSDUH items were further modified to incorporate the experts' recommendations. The resulting items were then tested through a cognitive interviewing process, which culminated with a final internal memo summarizing the revision process and cognitive interview findings (Center for Behavioral Health Statistics and Quality, 2016a).

2.3 2016 DSM-5 Cognitive Interviewing

The next step in the NSDUH SUD redesign process was testing the newly developed question items through a series of cognitive interviewing sessions. Cognitive interviewing is a structured interviewing process used to test the effectiveness of questionnaire wording and structure. People drawn from a population like those eventually receiving the questionnaire items are administered the specific survey questions of interest, and then they answer further questions about how they answered and understood the survey items. Three rounds of cognitive interviewing were conducted on the draft SUD items in 2015. The first round included English-speaking adults and adolescents with at least some substance use in the past year, whereas the last two rounds included English- and Spanish-speaking adults and adolescents with past year substance use. Between each round, the draft SUD items were reviewed and sometimes modified based on the feedback from the cognitive interviews in the prior round. The initial and revised draft items were tested with a total of 42 English-language participants and 12 Spanish-language participants over the three rounds. At the completion of the 2015 cognitive interviews, an internal report that documented the 2015 cognitive interviewing procedures and results was finalized in May 2016 (Center for Behavioral Health Statistics and Quality, 2016a).

The general findings indicated that, although the final version of the craving and marijuana withdrawal (criterion A11a) items performed well, there were concerns about the items developed to measure withdrawal criterion 11b for all substances (excluding inhalants and hallucinogens, which do not have withdrawal symptoms). Most participants found the A11b withdrawal items to be confusing. Therefore, the final cognitive interviewing report noted that, although craving and marijuana withdrawal criterion A11a may be ready for inclusion in NSDUH, the withdrawal criterion A11b items were not appropriate for addition to NSDUH in their current form. The final items tested are presented in Section 3.4.2.

3. SUD Redesign

3.1 Overview

This chapter documents the steps taken after completion of the 2016 cognitive interviewing and the results of these steps, which culminated in a more extensive redesign of the SUD modules for the 2020 National Survey on Drug Use and Health (NSDUH) Clinical Validation Study, with implementation of the changes in the 2025 NSDUH redesign. The overview provides a historical outline of the activities undertaken, with later sections providing detailed results of these activities.

3.1.1 Delay of DSM-5 Implementation

At the completion of these formative steps in the substance use disorder (SUD) redesign process, the implementation of the SUD redesign based on the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013) was postponed. The primary reasons for postponing the redesign of the SUD module were (1) feedback from expert reviewers, cognitive interview participants, and prior validation testing (Jordan, Karg, Batts, Epstein, & Wiesen, 2008), which suggested that existing NSDUH SUD items needed revision; (2) insertion of the new items into the SUD modules would reduce comparability of related estimates across years (i.e., break trends); and (3) insertion of the new items would occur too close to the partial trend break that occurred due to a 2015 survey revision. Therefore, instead of inserting the new DSM-5 items immediately into the NSDUH questionnaire, a substantive review and revision of all items in the SUD module was optimal. As part of this larger redesign, an additional in-person expert panel was convened to review the NSDUH SUD modules in their entirety, including existing and new items.

The implementation of substantive changes to NSDUH required several considerations because of the potential impact on data analysis and comparability across years. Modification to existing questions may affect the measurement of trends. Trend breaks can create issues for the many stakeholders relying on NSDUH for time series data. The pros and cons of survey changes need to be carefully weighed before a decision is made. Typically, expert panels are convened to identify evolving data needs on different topics of interest and to provide input on priority items, and then the expert recommendations are carefully weighed against available resources. Methodologists then investigate existing survey questions or create new ones to test and refine via cognitive, reliability, and validity testing. Altering the NSDUH SUD modules presents unique technical and procedural issues to consider. For example, SUD assessment typically requires a clinical measurement tool, but NSDUH is a population-based survey administered by nonclinicians. Therefore, any new question wording must be designed to capture the complexities and nuances of an evolving clinical diagnostic landscape, while minimizing participant burden and remaining accessible across a diverse range of clinical and nonclinical populations. Even subtle changes in question wording have the potential to affect the subjective interpretation of the question's meaning by individual respondents. Therefore, the DSM-5 implementation was delayed. Instead, further module revisions were conducted to be fielded in a clinical validation study conducted in tandem with the 2020 NSDUH, which allowed for a more thorough evaluation of all NSDUH SUD items, better positioned the redesign to be clinically validated, and addressed the concerns over disruption of data trends. The revised items, pending results of the 2020 Clinical Validation Study, were implemented as part of the 2025 NSDUH redesign.

3.1.2 2017 Expert Panel

In 2017, an in-person expert panel was held to build on the earlier 2015 expert review and cognitive interview results. For this full-module review, a formal SUD Assessment Expert Panel Meeting was held on May 8 and 9, 2017, in Rockville, Maryland. This meeting commenced the second major phase in evaluating the NSDUH SUD modules. The goals of this meeting were to examine the strengths and limitations of the current NSDUH SUD assessment in its entirety and to identify instrument changes that would better align the survey with DSM-5 SUD criteria and current stakeholder needs.

The 2017 expert panel convened in a conference room to facilitate the exchange of ideas over the course of the 1.5-day meeting. The expert panelists were asked a set of preplanned questions, debate was encouraged, feedback was summarized, and statements were confirmed. The first set of panel questions addressed whether NSDUH should alter the way it identifies respondents who enter the SUD modules. The second set of questions requested feedback from panel members about the existing questions in the NSDUH SUD modules and the new items developed to assess new or revised criteria based on DSM-5. The facilitator walked panel members through each set of questions organized by criterion, requested feedback about how well the questions captured the diagnostic criteria and whether respondents aged 12 or older could accurately answer the questions as they were written. The panel members were asked to recommend changes to improve the questions on both fronts. Responses were recorded, and when possible, the panel reached a consensus on each question. Also, the new DSM-5-based questions that were developed and cognitively tested in 2016 were presented for evaluation and feedback.

The third set of panel questions focused on soliciting general comments about the NSDUH SUD modules from panel members. Topics queried and discussed included the order and flow of the items, considerations for Spanish-speaking respondents, and whether any questions in the modules could be dropped or required modifications. The fourth set of panel questions focused on whether and how to conduct a clinical reappraisal study of the questions in the SUD modules. Panel members weighed pros and cons of validating at the disorder level or at the criterion level, considered gold standard issues, explored whether inter-rater reliability may be an alternative to assessing validity, discussed adult versus adolescent considerations, and considered how to validate a Spanish-language version of the SUD modules. The final topic of discussion involved how NSDUH can be used to serve its users' current and future data needs.

At various points during the meeting, expert panel comments were summarized and repeated to solicit additional feedback and encourage further consideration. The facilitators compiled action items for each topic and shared them with panel members during the meeting to encourage further discussion and ensure broad consensus on next steps. At the conclusion of the meeting, panel members were encouraged and invited to provide additional feedback on the revisions via various communication channels.

3.1.3 Statistical Analyses

Upon recommendation by the 2017 expert panel, several sets of analyses were conducted to evaluate any bias introduced in assessing alcohol and marijuana use disorder among those with 6 or more days of use in the past 12 months, which is the frequency-of-use threshold for the SUD module. Weighted and unweighted prevalence estimates were calculated for each SUD among lower frequency users. The results of these analyses are discussed in Section 3.3, and Appendix A contains supplementary analysis tables.

3.1.4 2018 SUD Cognitive Interviewing

Appendix C provides the final version of the 2020 Clinical Validation Study SUD items for alcohol.

3.1.5 Report Organization

The remaining sections of this chapter discuss the SUD redesign up to the point of implementation and clinical validation, which is in a separate report on the findings of the 2020 NSDUH Clinical Validation Study. First, the overall SUD assessment by experts is presented, followed by an examination of which respondents should receive the SUD modules. After these general sections, the findings from the expert panel critical evaluation and results from further cognitive testing of specific items conducted in 2017 are presented. Finally, a discussion of the validation of SUD modules and data needs provided by the 2017 expert panelists is presented.

Although prior work reported elsewhere is not reproduced in this report, salient details are presented from prior efforts to contextualize and enrich the feedback received during the expert panel. These details include results of the literature review, prior written feedback provided by external experts (particularly when additional concerns were noted by these reviewers that were not voiced during the in-person panel), prior validation study findings, information on other survey methods, and findings from earlier cognitive interviews conducted in 2016.

3.2 Overall SUD Assessment

The 2017 expert panelists were asked to share their overall impression of SUD assessment in NSDUH, focusing on NSDUH's target sample of adolescent and adult English and Spanish speakers who vary in their level of ability to comprehend the terms used in NSDUH questions to describe SUD symptoms.

The first discussion focused on the ordering of the individual criteria, including (1) whether there was a rationale for the order presented in the DSM and (2) whether there was a preferred sequence for NSDUH SUD questions. Panelists who served on the original DSM-5 revision committee indicated that there was no specific determination of the order of items in DSM-5. However, in DSM-5, criteria are generally grouped conceptually and retain their historical ordering, despite changes in the criteria over time (i.e., combining DSM-IV [fourth edition; American Psychiatric Association, 1994] substance abuse and substance dependence disorders into DSM-5 SUD). Additionally, literature reviews did not identify research on potential optimal ordering of the criteria. The possibility of randomizing the order of items assessing withdrawal symptoms was raised during the meeting. The benefit of randomization would be a reduction of order effects on the individual withdrawal symptom items. However, a drawback of item randomization is that some respondents might be abruptly presented with the most severe withdrawal symptoms first (e.g., hallucinations or seizures). This may result in respondents denying subsequent withdrawal symptoms without reading them because they believe that all withdrawal symptom items reflect behaviors as severe as those at the beginning of the list. The panelists agreed that this possibility far outweighed the benefits of randomization and suggested that items should be grouped conceptually and ordered from most to least severe (e.g., it was noted that withdrawal, which can be a severe criterion, is in the middle of an SUD module and reordering should be considered to move it closer to the end). The possibility of partial randomization (e.g., randomizing within the low-severity items) was not discussed. A similar discussion about randomizing the order of other symptom questions in the SUD modules resulted in similar conclusions. Several panel members suggested clustering or grouping symptoms conceptually from less to more severe.

The second discussion focused on SUD assessment among adults versus adolescents. For this discussion, the panelists were unified in their feedback that when written appropriately, the same items could be used to assess SUDs among adolescents and adults. However, as NSDUH items were written (experts reviewed the 2017 instrument), there were concerns over the reading level of the items and their complexity. That is, many of the current NSDUH items are compound items with multiple constructs contained within the same item.

Panelists were also asked about NSDUH's ability to assess SUDs with equal accuracy among those who have little formal knowledge of substance use compared with those who have extensive knowledge (e.g., those who have received treatment for SUDs or those with medical training or expertise). The panelists had no specific concerns about NSDUH's ability to produce valid assessments across populations with different levels of knowledge.

SUD assessment among Spanish-speaking populations was discussed in a follow-up phone conversation with an expert in SUD assessment among Spanish-speaking populations. The primary concern in assessing Spanish-speaking respondents is the low socioeconomic status of many people in the population and the lower reading level of the language in the interview among many in this population (4th to 5th grade range). Although most Spanish speaking youths will choose to take the instrument in English, Spanish-speaking adults will likely have limited English understanding and choose to take the assessment in Spanish. Feedback on the current Spanish-language version was that it was too complex and had too high a reading level for the general public. This feedback was consistent with the overall sentiment that the existing NSDUH assessment required too high a reading level and contained too many complex constructs. One specific challenge noted by the expert was that when testing items with Spanish-speaking populations, there was a decrease in validity when specific amounts and timings were specified in the question (i.e., "more" or "a lot" performed better than including a specific quantity, frequency, or timing in the question). It was noted that if specific timings are necessary, then establishing a timeline with a respondent and anchoring significant life events that fall within each respondent's timeline generally improved temporal recall but could significantly lengthen the interview process.

Cultural differences in SUD assessment were also discussed. It was noted that there is heterogeneity among Latino populations that have more to do with regional differences rather than differences between Spanish-speaking and English-speaking people. Therefore, if the goal is to improve assessment among Spanish speakers, the instrument would have to be tested regionally. However, this could lead to a measure that works well in one subgroup of Spanish speakers but not as well in another. Strategies for dealing with regional variations are provided in the Toolkit on Translating and Adapting Instruments (Chávez & Canino, 2005).

3.3 SUD Module Entry

3.3.1 Frequency-of-Use Threshold

The first research question evaluated by the 2017 expert panelists was which subset of NSDUH respondents should receive the SUD modules for each substance. Currently, NSDUH assesses alcohol use disorders and marijuana use disorders among respondents who have used those substances on 6 or more days in the past 12 months or whose days of use are unknown. For all other substances, respondents who report using the substance on at least 1 day during the past 12 months are assessed for the disorder related to that substance. Deciding where to set a threshold for entry into an SUD module involves two major tradeoffs.

The first trade-off is that increasing the threshold for entry into SUD modules could shorten the survey but would be at the cost of potentially missing cases. Decreasing the threshold could lengthen the survey but could also lead to fewer missing cases. However, lowering the threshold may also increase the risk of false positives or respondent irritation (e.g., a respondent who reported a single day of use might become irritated at being asked a long string of questions that do not apply to him or her, or the respondent could be offended at the inadvertent suggestion that he or she has a substance use problem or was untruthful in earlier reporting). Respondent irritation or confusion risks damaging the quality of all subsequent responses.

However, a second concern is that raising the threshold could affect the balance of false positive and false negative cases. Setting the threshold higher is likely to reduce the number of cases incorrectly identified as having an SUD (false positives), but at the likely cost of missing some cases that should be identified as having an SUD (false negatives). DSM-IV and DSM-5 criteria for SUDs refer to recurrent impairing patterns of use and continued use despite the experience of adverse effects related to that use. Therefore, for substances that screened for SUD assessment based on respondent endorsement of use on only 1 day of use (i.e., all substances except alcohol and marijuana), there were concerns about the validity of SUD diagnoses. Consideration could be given to whether respondents who only used on a few days in the past year but endorsed SUD criteria were actually reporting on a recurrent impairing pattern of use or whether they were misunderstanding the questions. There are several challenges in assessing this. In some circumstances, respondents could report use on only 1 day while still meeting the recurrent impairing pattern of use. For example, a single day of reported use that occurred early in the 12-month window could represent the end of a period of use that led to SUD symptoms (e.g., that day of use could be continued use after recurrent problems with the family that occurred before that day of use). In addition, although the core of the SUD diagnosis has not changed from DSM-IV to DSM-5, the latter introduced the craving criterion. Craving is the only criterion that does not require current or recent use because someone with a history of substance use can experience cravings even if they had not used in the past year.

An additional consideration is that current entry thresholds are based on frequency (e.g., 1 day of use) but not specifically on quantity or pattern of use (e.g., binge alcohol use on only a few days), which may be important to understanding SUD symptoms reported by low-frequency users. This may merit multiple pathways of entry into an SUD module (e.g., frequency of use or whether binge use is reported). However, not all substances are assessed for binge use (e.g., alcohol has a binge use assessment but inhalants do not), and setting inconsistent thresholds or different criteria for entry across substances could lead to difficulty comparing SUD estimates across specific substances.

Prior written expert review in 2015 suggested that, for future analyses, it may be useful to expand beyond tracking of presence or absence of an alcohol use disorder (AUD) or SUD diagnosis toward tracking of total number of DSM-5 criteria met. Without collecting SUD criteria from low-frequency users, any analysis of symptom counts in the population would be incomplete. The experts pointed out that tracking the number of endorsed criteria would help data users look at population data as a count of DSM-5 criteria. Furthermore, DSM-5 uses the number of SUD symptoms present as an indicator of severity. In addition, it was noted that population counts/rates would likely be lower (as a count) than those of clinical or incarcerated populations, which are more likely to have moderate or severe issues.

Finally, additional written expert review raised the question of whether module entry cutoffs for adolescent respondents should necessarily be the same as those for adults. The written review noted that although the current entry criteria of 6 or more days in the past 12 months for the assessment of alcohol use disorders and marijuana use disorders may be appropriate for adults, it may need to be lowered for adolescents. It was further noted that the Screening to Brief Intervention (S2BI) adolescent screening tool has found low frequency-of-use cutoffs of 2 days or more to correspond to SUD diagnoses for alcohol and marijuana among youths. It was suggested that a lower cutoff be used for adolescents, but that the same cutoff should be used for alcohol, marijuana, and other drugs for this age group. Like expert panel concerns around quantity or pattern of use, the written review suggested clarifying the meanings of "drank" or "used" in terms of volume; otherwise, the proposed lower threshold adolescent entry criteria may be perceived as too low.

Although no resolutions to many of the questions raised by experts were reached during the expert panel, the experts unanimously recommended that any decisions about SUD thresholds for SUD assessment would need to be made using data to examine the effect of setting different thresholds on entry points. In anticipation of this discussion, results from a variety of analyses were presented to examine the proportion of respondents who might be affected by these issues. The first set of analyses examined how many respondents at low frequencies of use met criteria for each SUD. Table 3.1 presents the potential misclassification (being reported as not meeting SUD when they do meet SUD) that would occur for selected alcohol and marijuana criteria if the threshold was raised. Appendix A provides the tables for all other substances. Although no specific cut points were determined at the expert panel meeting, these analyses were designed to guide future decision making about redesigning the SUD module. For example, about 1.2 percent of respondents reported drinking alcohol on 6 days in the past 12 months. Of these, 2.2 percent reported two or more DSM-5 AUD criteria. These respondents would be misclassified as not having SUD if the threshold were raised to 7 or more days of alcohol use. An estimated 7.8 percent of respondents who reported drinking on 6 days in the past 12 months reported one DSM-5 AUD criterion. These respondents could potentially be misclassified as not meeting the SUD diagnosis if they were to also endorse craving (a symptom not currently assessed in NSDUH). Therefore, if the threshold was raised to 7 or more days of alcohol use, up to 10 percent of respondents who drank on 6 days in the past 12 months (2.2 percent who would definitely be misclassified, plus the 7.8 percent who might be misclassified) could be misclassified as not having AUD (presuming that the symptoms were correctly endorsed). In context of the total sample, if 1.2 percent of the sample drank on 6 days in the past 12 months, and 10 percent of them could be misclassified if the craving criterion were added, that equates to approximately 800 people per year in the sample, representing approximately 227.8 thousand people in the population annually.

In addition to the misclassification analyses, the expert panel recommended conducting analyses to examine concerns about respondents who met criteria for each SUD despite low frequency use of the substance. Specifically, the panelists suggested evaluating the specific criteria reported by respondents with low frequencies of substance use in the past 12 months and considering whether endorsement of these criteria made sense despite the low frequency of use. Overall, the estimated proportion of the population using at a low frequency who were also identified as having SUD is very small. Thus, a small change in the SUD module entry criteria based on frequency of use will have only minor effects on SUD prevalence estimates. Prior validation work on the NSDUH SUD assessment demonstrated that rates of false positives were generally larger than rates of false negatives across substances, but samples were not large enough to look at these rates for low-frequency users of any substance other than alcohol and marijuana (Jordan, Karg, Batts, Epstein, & Wiesen, 2008). For alcohol and marijuana, estimates of false positives and false negatives among low-frequency users were substantially lower than among higher frequency users. Therefore, lowering the threshold for entry into the AUD and marijuana use disorder modules would only slightly reduce the rates of false positives and false negatives. These estimates assume no error in respondents' recall of frequency of use. A prior reliability study of NSDUH found that number of days used in the past year had only fair to moderate reliability (Substance Abuse and Mental Health Services Administration, 2010). Changing the entry criteria for SUD modules will not correct this misreporting but may adjust for it indirectly. If respondents are more likely to underreport their use (presuming they do not deny use entirely), then lower thresholds will still bring these individuals into the corresponding SUD module, whereas higher thresholds may incorrectly exclude these respondents.

Experts also expressed concerns about cross-substance use among low-frequency users. For example, if a respondent uses heroin on 1 or 2 days but also misuses prescription pain relievers, then the respondent who appears to have a low frequency of use could report symptoms related to opioids as a whole. To explore this possibility, analyses were completed examining the overlap of SUDs at low frequencies of use. These analyses are not presented in this report because of high levels of suppression (i.e., the data cannot be displayed because of imprecision of the estimates) and were produced for internal use only.

These analyses were presented in a 2018 internal memo titled "DSM-5 SUD Module Entry Analyses." The memo also included further considerations related to changing the frequency-of-use thresholds for entering the SUD module:

1. Symptom count. A new feature of DSM‑5 is a measure of SUD severity using a count of symptoms. Currently, estimates of the distribution of symptoms in the U.S. population are not available. Experts noted that this would be a valuable addition to the field and may also be important for surveillance efforts. Increasing the thresholds for entering the SUD module (for any substance) would preclude getting these data for all respondents and would therefore overlook subpopulations that may be at risk for developing an SUD. The overlooked subpopulations would be low-frequency users who may have some evidence of problematic use who could escalate or expand their problematic use. Reducing the SUD module thresholds may be an important addition for surveillance activities but must be weighed against other considerations, notably, survey length and respondent burden.

2. Survey length. Most analyses indicated that increasing the threshold for entry into the module would not produce large changes in the population estimates because low-frequency users endorsing criteria is rare. Therefore, increasing the threshold for entry could result in a shorter interview. However, for most substances (except alcohol and marijuana), this reduction would be minor and affect only up to 2.5 percent of the NSDUH sample annually. In contrast, consideration is being given to lowering the threshold for alcohol and marijuana to "any past year use." Using timing estimates from NSDUH participants who used 6 to 10 times in the past year, in combination with the number of participants who used 1 to 5 times, indicated that lowering the threshold for alcohol and marijuana would increase the average total length of NSDUH by an average of 0.28 minutes per respondent: 0.22 minutes for alcohol and 0.06 minutes for marijuana. However, the current NSDUH does not include withdrawal questions for marijuana; therefore, the additional timing may become slightly longer if withdrawal questions were added per the DSM-5 revision to marijuana use disorder. The alcohol withdrawal questions account for 0.02 minutes per NSDUH respondent, suggesting an increase of up to 0.08 minutes if marijuana withdrawal were added and the threshold were lowered for marijuana use disorder module entry. Specifically considering low-frequency alcohol and marijuana users, the mean NSDUH length among those who drank on 1 to 5 days in the past year and did not use marijuana is 61.0 minutes (median 57.8 minutes). The mean NSDUH length among those who drank on 1 to 5 days in the past year and used marijuana on 1 to 5 days in the past year is 63.4 minutes (median 60.8 minutes). Similarly, the mean NSDUH length among those who used marijuana on 1 to 5 days in the past year and did not use alcohol is 64.4 minutes (median 60.9 minutes). Adding the AUD module for those with low-frequency alcohol use would add an average of 2.1 minutes to the survey length for those respondents, and adding the marijuana use disorder module for those with low-frequency marijuana use would add an additional average of 1.4 to 1.6 minutes for those respondents.

3. Respondent irritation and fatigue. Asking low-frequency users about substance use problems may result in respondent irritation or fatigue since these questions generally have no relevance for them. However, prior evaluation of NSDUH has shown no indication of this problem (e.g., sudden drop-off rates or incomplete cases), and there is research value in knowing the number of criteria met by respondents across the entire frequency-of-use spectrum (even those who only used a few times in the past year). Raising the threshold for module entry may not be a good trade-off in this case.

4. Variable thresholds. If thresholds vary too much across substances assessed for SUD, then the comparability of prevalence rates for SUDs across substances could be affected. Moreover, changes in thresholds that deviate from other surveys may make it difficult to compare estimates across surveys. The National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) asks the past year SUD modules of everyone with at least one use of the substance in the past year. In contrast, the Structured Clinical Interview for DSM-IV (SCID-IV) asks about SUD among those who used more than five times in the past year.

5. Analytic flexibility. A drawback of increasing thresholds is a reduction in analytic flexibility. If thresholds were lowered to any use in the past year, the impact of changing those thresholds on population estimates can be evaluated statistically. For example, if the NSDUH threshold for use was any use in the past year, and comparability to SCID-IV was of interest, then analyses could be run among only those with more than 5 days of past year use, while not losing the information of respondents who used less. However, if the data are collected only among those who used on more than 5 days in the past year, the reverse cannot be done to facilitate comparisons with other surveys that assess SUDs among those with lower levels of use.

Each of these points needs to be weighed when determining a threshold for module entry. Overall, increasing the thresholds for most substances would have little impact on the population-level estimates because the number of people these groups represent is small. However, it may somewhat improve measurement validity because false positives tended to exceed false negatives for the SUD criteria. For drugs other than alcohol and marijuana, the numbers of survey respondents affected by threshold changes are also low, leading to small decreases in survey time if the thresholds were increased. However, doing so would come at the expense of analytic flexibility. Modifications to the alcohol and marijuana use disorder modules are the most likely to have substantial effect on NSDUH, given the larger numbers of people who use these substances at lower frequencies (see Table 3.2 for a summary). For alcohol, there would be little change in measurement validity if the threshold were reduced or increased because false positives and false negatives were similar among those with low frequency use. However, reducing the threshold for marijuana use would likely reduce validity because the ratio of false positives to false negatives was nearly 2 to 1 among low-frequency users. Reducing the SUD module threshold for marijuana would also extend survey length and potentially induce irritation among low-frequency users, most of whom do not have marijuana use disorder.

Based on the discussion from the expert panel meeting and analyses conducted for this memo, it was determined that all respondents who had used alcohol or any other drugs at least once in the past 12 months would be routed into the SUD module. In practice, this decision means lower thresholds for assessing AUD and marijuana use disorder and no change for any other substances. However, this decision also allows for analytic flexibility should different thresholds be determined in the future.

Table 3.1 Potential Misclassification for DSM-5 Alcohol and Marijuana Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older)

Substance Use	Total1 %	1 Criterion2 %	≥ 2 Criteria3 %	Potential Sample Misclassification (1 Criterion)4	Sample Misclassified (≥ 2 Criteria)5	Total Possible Misclassification6 %	Possible Sample Misclassification n (in Hundreds)7	Population Misclassification N (in Thousands)8
Alcohol
6 Days	1.2	7.8	2.2	0.0009	0.0002	0.0011	800	277.8
7 Days	0.7	9.8	4.7	0.0007	0.0003	0.0010	700	245.4
8 Days	0.6	9.2	3.8	0.0005	0.0002	0.0007	500	183.9
9 Days	0.2	11.9	5.8	0.0003	0.0001	0.0004	300	107.3
10 Days	1.6	8.7	3.0	0.0014	0.0005	0.0019	1,400	481.6
Marijuana
6 Days	0.3	11.0	5.1	0.0003	0.0002	0.0005	400	122.0
7 Days	0.2	18.7	9.1	0.0003	0.0002	0.0005	400	126.4
8 Days	0.2	11.4	9.2	0.0002	0.0001	0.0003	200	780.4
9 Days	0.1	15.7	10.8	0.0001	0.0001	0.0002	100	46.9
10 Days	0.4	12.9	7.5	0.0005	0.0003	0.0008	600	211.2

DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013).

¹ Percentage of respondents who reported that frequency of use (e.g., 1.2 percent of respondents used alcohol on 6 days in the past year).

² Percentage of respondents who used at that frequency who endorsed one SUD criterion. Excludes the DSM-IV (fourth edition; American Psychiatric Association, 1994) legal criterion but does not include craving because NSDUH does not yet collect craving data.

³ Percentage of respondents who used at that frequency who endorsed two or more SUD criteria. Excludes the DSM-IV legal criterion but does not include craving because NSDUH does not yet collect craving.

⁴ Potential sample misclassification based on endorsing a single criterion (i.e., the percentage who used at that level multiplied by the percentage of those respondents who endorsed one criterion). These individuals might be misclassified if the threshold for receiving the module were raised, because they may potentially have two criteria if craving were added.

⁵ The sample misclassification if the threshold for receiving the module were raised because, despite not having craving in the NSDUH module, respondents are already meeting SUD criteria (two or more symptoms). Calculated by multiplying the percentage who used at that level by the percentage who endorsed two or more criteria.

⁶ Total possible misclassification is the total amount of possible misclassification if everyone who endorsed only one criterion would also have endorsed craving (calculated by adding the possible misclassification with the sample misclassification).

⁷ Total possible sample misclassification is the number of NSDUH respondents who might be misclassified at that level of use.

⁸ Population misclassification N represents the estimate of the number of possible misclassifications in population estimates annually.

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2004-2014.

Table 3.2 Summary if Effect of Module Entry Rule is More than 5 Days of Use for Alcohol and Marijuana (DSM-IV)

Row	Description	Source	Alcohol	Marijuana
1	Estimated number (1000s) of population aged 12 or older	Combined 2004-2014 NSDUHs, Annual Average	252,500	252,500
2	Estimated number (1000s) of those aged 12 or older with any past year use	Combined 2004-2014 NSDUHs, Annual Average	167,200	28,600
3	Estimated number (1000s) of those aged 12 or older using 1-5 days in past year	Combined 2004-2014 NSDUHs, Annual Average	25,700	7,300
4	Estimated prevalence of past year disorder among and estimated number (1000s) of those aged 12 or older using 1-5 days in the past year	Best guess based on NESARC estimates for 1-6 day users and projections from NSDUH 6-10 day users, applied to 1-5 day users	2.5% (644)	5% (365)
5	Estimated number (1000s) of those aged 12 or older with disorder among > 5 day users	Combined 2004-2014 NSDUHs, Annual Average	18,000	4,200
6	Estimated number (1000s) of those aged 12 or older with disorder among all past year users	Sum of n in rows 4 and 5	18,700	4,600
9	Estimated relative bias in numerator estimate from entry rule set at > 5 days rather than 1 or more days	Computed from rows 5 and 6	-3.5%	-8.0%
10	Current prevalence of past year disorder among population aged 12 or older	Combined 2004-2014 NSDUHs, Annual Average	7.2%	1.7%
11	Current prevalence of past year disorder among all past year users of that substance aged 12 or older	Combined 2004-2014 NSDUHs, Annual Average	10.8%	14.8%
12	Estimated prevalence of past year disorder among those aged 12 or older, if entry criteria included those who used 1-5 times	Row 6/Row 1	7.4%	1.8%
13	Estimated prevalence of past year disorder among past year users aged 12 or older, if entry criteria included those who used 1-5 times	Row 6/Row 2	11.9%	16.0%
16	Added interview time per interview to overall sample if module entry rule was changed to any past year use	RTI memo on DSM-5 module entry	13 seconds	5 seconds

DSM-IV = Diagnostic and Statistical Manual of Mental Disorders, fourth edition (American Psychiatric Association, 1994); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013).

3.3.2 Use but Not Misuse of Prescription Drugs

A topic not discussed by expert panelists but explored in a short literature review was whether SUDs should be assessed among those who used but did not misuse prescription medications. Currently, NSDUH only collects data on prescription sedative, stimulant, tranquilizer, and pain reliever use disorder for those who reported misuse of the drug (i.e., used a prescription medication in ways not prescribed by a doctor), excluding those who used but did not misuse these drugs. However, under DSM-5, people can meet criteria for an SUD even if they have a legitimate prescription for a drug (stimulants, sedatives/tranquilizers, and/or pain relievers) and are not misusing the substance. The criteria for SUD under these circumstances are the same, except that tolerance and withdrawal symptoms do not apply to those using as medically appropriate. Given this change between DSM-IV and DSM-5, the value of routing NSDUH respondents who used but did not misuse prescription drugs to the appropriate SUD modules was considered. This routing would increase the length of the NSDUH interview, particularly for older adults who take more of these prescription medications. Despite concern over increasing NSDUH length and administration time, collecting SUD data among those who use but do not misuse would provide a more complete picture of the number of SUD symptoms in the household population. Moreover, it could identify whether individuals are experiencing impairing SUD symptoms even when using the drug as prescribed. In line with DSM-5, the routing to the SUD modules will be changed to include those who use but do not misuse prescription sedatives/tranquilizers, stimulants, and opioids. The memo discussing these issues and the current status of the literature is provided in Appendix B.

3.3.3 Sedatives and Tranquilizers

Currently, NSDUH provides separate assessments for sedative and tranquilizer use disorder. Neither DSM-IV nor DSM-5 distinguish between sedatives and tranquilizers. Under DSM-IV and DSM-5, sedatives and tranquilizers form a class of drugs due to similarities in their pharmacological properties, tolerance liabilities, and withdrawal characteristics. Treating sedatives and tranquilizers as a single class of drugs could shorten the NSDUH administration time by assessing only one rather than two separate use disorders and also by measuring use only once for those sedatives/tranquilizers that currently appear in both the sedative and the tranquilizer lists. Combination would further align NSDUH measures with DSM-5 use disorder criteria.

3.4 SUD Criteria

3.4.1 Existing Items

In this section, the criteria based on DSM-5 are discussed in the order assessed by the 2017 NSDUH. For each individual criterion, background on the DSM-5 definition and NSDUH operationalization are provided, followed by a summary of comments, concerns, and recommendations from the expert panel review; an overview of findings from prior validation studies; an examination of other surveys' assessments of the specific criterion; and a description of existing or required cognitive interviewing and validation.

Ten of the 11 DSM-5 criteria for SUD are currently assessed in NSDUH: time spent; larger amounts used; tolerance; persistent desire, unsuccessful control; withdrawal criterion A (except for marijuana); continued use despite health problems; giving up activities; role failure; hazardous use; and use despite social problems. Table 3.3 shows the current NSDUH assessment ordering and corresponding DSM-5 criteria.

Table 3.3 Crosswalk of NSDUH, DSM-IV-TR, and DSM-5 Substance Use Disorder Criteria

2017 NSDUH Assessment	DSM-IV-TR	DSM-5	Description of Criterion
1. Time Spent	Dependence 5	Criterion A3	Spending a lot of time getting, using, or recovering from use of the substance
2. Larger Amounts Used	Dependence 3	Criterion A1	Taking the substance in larger amounts or for longer than you are meant to
3. Tolerance	Dependence 1A, 1B	Criterion A10	Needing more of the substance to get the effect you want OR not getting effect you want using the same amount of the substance
4. Persistent Desire, Unsuccessful Control	Dependence 4	Criterion A2	Wanting to cut down or stop using the substance but not managing to
5. Withdrawal Criterion A (NSDUH did not assess Withdrawal Criterion B)	Dependence 2A, 2B	Criterion A11a and A11b	Development of withdrawal symptoms after stopping or reducing substance use (DSM-5 A11a), which can be relieved by taking more of the substance (DSM-5 A11b)
6. Continued Use Despite Health Problems	Dependence 7	Criterion A9	Continuing to use, even when you know you have a physical or psychological problem that could have been caused or made worse by the substance
7. Giving Up Activities	Dependence 6	Criterion A7	Giving up or reducing important social, occupational, or recreational activities because of substance use
8. Role Failure	Abuse 1	Criterion A5	Not managing to do what you should at work, home, or school because of substance use
9. Hazardous Use	Abuse 2	Criterion A8	Using substances again and again, even when it puts you in danger
10. Use Despite Social Problems	Abuse 4	Criterion A6	Continuing to use, even when it causes or exacerbates problems in relationships

DSM-IV-TR = Diagnostic and Statistical Manual of Mental Disorders, fourth edition, text revision (American Psychiatric Association, 2000); DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013).

NOTE: New items developed for NSDUH will be evaluated in subsequent sections, including items measuring the newly added DSM-5 criterion for cravings (criterion 4) and for marijuana withdrawal. Later sections also propose cognitive testing for items intending to measure DSM-5 withdrawal criterion B (use of a similar substance).

1. Time Spent

The first criterion assessed by NSDUH is time spent, which corresponds with DSM-IV substance dependence criterion 5 and DSM-5 SUD criterion A3: "A great deal of time is spent in activities necessary to obtain the substance (such as visiting multiple doctors or driving long distances), use the substance (e.g., chain-smoking), or recover from its effects."

NSDUH operationalized this item by asking the following pair of questions for each substance assessed:⁰

• DRALC01: "During the past 12 months, was there a month or more when you spent a lot of your time getting or drinking alcohol?" [Y/N]

• DRALC02 [IF DRALC01 = N, DK/REF]: "During the past 12 months, was there a month or more when you spent a lot of time getting over the effects of the alcohol you drank?" [Y/N]

a. Expert Panel Review

The 2017 expert panel review identified three main areas of focus in evaluating the current NSDUH question for this criterion: time frame wording ("a month or more," "a lot of time"), adolescent versus adult considerations, and substance-specific differences.

Time frame wording ("a month or more"): The expert panelists noted concerns about the question phrase "a month or more" because DSM-5 does not specify a concrete time frame. Moreover, it is difficult for people to keep track of the time they spent obtaining, using, and getting over a substance. It was noted that questions assessing this criterion should clearly indicate (1) spending "a lot of time" getting or using a substance and (2) spending a lot of time does not need to occur across all of the past 12 months. Other questionnaires and surveys have used the phrase "when you were drinking/using the substance the most" or "when you had the most symptoms" to query the period when respondents were most symptomatic (e.g., the Composite International Diagnostic Interview; Kessler & Üstün, 2004).

Time frame wording ("a lot of time"): The expert panel also noted that the meaning of the phrase "a lot of time" is very subjective and that different respondents may judge its meaning differently. Similarly, the 2015 written expert feedback suggested that an example or definition be provided for "a lot of your time," similar to examples included in the DSM-IV criterion description (e.g., driving long distances). The 2017 expert panel noted that the heart of the criterion is saliency of the behavior (i.e., a general feeling of spending "a lot of time"—not spending a specific period of time). The panel advised trying to find a way to bring the sense of saliency into the NSDUH measures rather that asking about "a month or more" specifically. In addition, the "month or more" quantification is based on alcohol research specifically. It is difficult to operationalize the quantification equally across all substances because there is no benchmark for the other assessed substances of abuse.

Adolescent versus adult considerations: The panel noted that children would not easily understand the wording of the current NSDUH question. In the National Institute of Mental Health Diagnostic Interview Schedule for Children (DISC), the following question for this criterion is asked: "In the last year, were there many days where you felt sick or hungover after drinking?" The panel advised that, generally, if the questions are designed for adolescents to understand, they will work for adults as well (at least when assessing SUDs). The applicability of the criterion on time spent obtaining the substance might differ for adults and adolescents. For example, adults typically do not need to spend a lot of time getting alcohol, but adolescents who cannot legally buy alcohol might need to spend a lot of time getting alcohol before using it.

Substance-specific differences: The time spent criterion may also have different levels of salience for different drugs, including legal versus illegal drugs, prescription drugs, and variations among the biological effects of different drugs (e.g., how long the effects last).

b. Prior Validation Study

A prior validation study (Jordan, Karg, Batts, Epstein, & Wiesen, 2008) examining the concordance between results from the current NSDUH and results from a clinical interview for alcohol, marijuana, and cocaine use disorders found that this criterion had moderate inter-rater reliability with kappas⁰ ranging from 0.45 (standard error [SE] = 0.05) for alcohol to 0.55 (SE = 0.00) for cocaine (Landis & Koch, 1977). Across all substances, the ratio of false positives to false negatives was 167 to 24, suggesting that respondents are systematically more likely to mistakenly endorse this criterion rather than mistakenly deny it.⁰ Overall, discrepancies appear to be due to the clinical interviewers' ability to ask follow-up probes to determine whether symptoms met the severity necessary for a clinical diagnosis.

c. Other Surveys' Assessment of Criterion A3

Table 3.4 shows how two other surveys have worded items assessing this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.4 Assessment by Other Surveys of DSM-5 SUD Criterion A3: Time Spent

Other Survey	Question Wording	Answer Choices	Comments
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You spent a great deal of time drinking or recovering from drinking?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never	The NMHS CIDI is still under development and uses a checklist symptom set for the SUD assessment. The following stem question repeats for all items in the checklist: "In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems?" Fully structured ACASI.

Table 3.4 Assessment by Other Surveys of DSM-5 SUD Criterion A3: Time Spent (continued)

Other Survey	Question Wording	Answer Choices	Comments
AUDADIS-5	a. In your entire life, did you EVER... b. Did this happen in the last 12 months? Have a period when you spent a lot of time drinking? Have a period when you spent a lot of time being sick or getting over the bad aftereffects of drinking?	[Y/N] [Y/N]	AUDADIS-5 refers to the Wave 3 AUDADIS-5, which assesses for lifetime, followed by past year. Fully structured ACASI.
NCS-A CIDI	Did you ever have times of several days or more when you spent so much time drinking or getting over the effects of alcohol that you had little time for anything else?

ACASI = audio computer-assisted self-interviewing; AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Overall, the revised questions have been simplified from the original NSDUH items (Table 3.5), but there were minimal conceptual changes made to this item. Several possible revisions were created based on feedback from the expert reviewers; however, it was determined that in addressing the feedback, additional complexity and concerns were introduced into the revised items. Therefore, the original question content was retained with simplifications to address complexity, and the question was tested through cognitive interviewing. The revised versions performed well in English (and in Spanish during Round 2 interviews). The only changes made between cognitive interviewing rounds was to bold the qualifier "great deal of your time" to draw respondents' attention to that component of the criterion.

Table 3.5 Cognitive Interviewing Question Versions Assessing Criterion A3 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC01: "During the past 12 months, was there a month or more when you spent a lot of your time getting or drinking alcohol?" [Y/N]	DRALC01 During the past 12 months, did you spend a great deal of your time drinking alcohol, feeling its effects, or getting over the effects of drinking? [Y/N]	DRALC01 During the past 12 months, did you spend a great deal of your time [bolded] drinking alcohol, feeling its effects, or getting over the effects of drinking? [Y/N]	DPALFEEL During the past 12 months, did you spend a great deal of your time drinking alcohol, feeling its effects, or getting over the effects of drinking?

Table 3.5 Cognitive Interviewing Question Versions Assessing Criterion A3 for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC02 [IF DRALC01 = N, DK/REF]: "During the past 12 months, was there a month or more when you spent a lot of time getting over the effects of the alcohol you drank?" [Y/N]	DRALC02 [IF DRALC01=2 OR DK/REF] During the past 12 months, did you spend a great deal of your time getting or trying to get alcohol? [Y/N]	DRALC02 [IF DRALC01=2 OR DK/REF] During the past 12 months, did you spend a great deal of your time [bolded] getting or trying to get alcohol? [Y/N]	DPALGET [IF DPALFEEL=2 OR DK/REF] During the past 12 months, did you spend a great deal of your time getting or trying to get alcohol?

2. Larger Amounts

The second criterion assessed by NSDUH is larger amounts, which corresponds with DSM-IV dependence criterion A3 and DSM-5 SUD criterion A1: "Substance is taken in larger amount or for longer period than intended." The current NSDUH operationalizes this item by asking the following questions for each substance assessed:

• DRALC04: During the past 12 months, did you try to set limits on how often or how much alcohol you would drink? [Y/N]

• DRALC05 [IF DRALC04 = Y]: Were you able to keep to the limits you set, or did you often drink more than you intended to?

1. Usually kept to the limits set

2. Often drank more than intended

DK/REF

a. Expert Panel Review

Expert panelists and prior expert feedback identified four main areas of focus in evaluating the current original NSDUH question for this criterion: double-barreled questioning within item DRALC05, assessment of respondent limit setting, adolescent versus adult considerations, and the impact of the criterion on meeting SUD criteria.

Double-barreled questioning within item DRALC05: Prior written feedback in 2015 from expert reviewers noted that DRALC05 may be "impossible" for some respondents to answer because of its double-barreled nature: It asks about "keeping the limits you set" and "often drinking more than intended" in the same question. A respondent could both "usually keep to the limits set" and often drink more than intended. For example, a person might occasionally try to set limits and keep to them but drink more than intended on the many occasions where no limits were set. Furthermore, the phrasing lacks consistency with DSM-5 criteria that specify "often taken in larger amounts or over a longer period than was intended." The expert panel recommended having the following single combined question for DRMJ04 and DRMJ05: "During the past 12 months, did you often use marijuana or hashish in larger amounts or over a much longer period of time than you planned?"

Assessment of respondent limit setting: Panelists generally liked the approach of assessing whether the respondent had set a substance use limit and was able to keep to the limit when using. This assessment of limit setting is intended to operationalize the "intent" component of the DSM-5 criterion and help differentiate criterion A1 (larger amounts than intended) from criterion A10 on tolerance (needing to use more to get the same effect). However, additional written expert review raised concerns about including an assessment of respondent limit setting, stating that the additional step of attempting to set a limit was not necessarily required clinically. Rather, the following rewording of DRALC04 was suggested, which excludes the limit setting piece: "During the past 12 months, have there been instances when you drank more than you intended and/or drank for longer periods? DRALC05 (if DRALC04=Y) "Were you able to stop drinking, once you detected increased/longer period of use, or not?"

Adolescent versus adult considerations: The expert panel noted that setting limits on use, particularly for alcohol, might be harder to assess among adolescents because they might not set limits. For example, young adult social drinkers often expect to use a certain amount and end up using more.

Impact of the criterion on meeting SUD criteria: Because SUD diagnoses in DSM-5 require only two criteria, the threshold for this criterion is important. The expert panel noted that this criterion is challenging to measure because it blends use and abuse by implying a quantity of use as part of the symptom (more than intended for someone who drinks only occasionally may be different than for someone who drinks often). The construction of the question should keep in mind that the criterion is supposed to represent a pattern of behavior. The panel noted that care should be taken in writing this question because the trajectory from use to abuse is on a continuum. However, a threshold must be chosen with respect to "larger amounts" than intended. That is, consider what constitutes a "larger amount" and whether that threshold is consistent across all levels of use. There is not strong consensus on where the threshold should be set because the item usually is not examined on its own. Typically, the "larger amount" construct is examined in the context of co-occurrence with other problematic behaviors or consequences.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to moderate inter-rater reliability with kappas ranging from 0.21 (SE = 0.07) for marijuana to 0.58 (SE = 0.09) for cocaine. Across all substances, the ratio of false positives to false negatives was 42 to 128, suggesting that respondents were much more likely to mistakenly not endorse this criterion rather than mistakenly endorse it.

c. Other Studies' Assessment of Criterion A1

Table 3.6 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.6 Assessment by Other Surveys of DSM-5 SUD Criterion A1: Larger Amounts

Other Survey	Question Wording	Answer Choices	Comments
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You either drank more or spent more time drinking than you intended when you started?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Have a period when you ended up drinking more than you meant to? Have a period when you kept on drinking for longer than you had intended to?	[Y/N] [Y/N]
NCS-A CIDI	Did you have times when you started drinking even though you promised yourself you wouldn't, or when you drank a lot more than you planned to? [If NO to previous question:] Were there ever times when you drank more often or for more days in a row than you planned to? [If NO to previous question:] Did you have times when you started drinking and became drunk when you didn't want to?	[Y/N]	The second and third questions are asked only if the respondent answers NO to the immediately preceding question.

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Based on experts' concerns that people may not set limits and on the high false negative rates documented in prior studies, the NSDUH items measuring criterion A1 were drastically changed. The question revision reduced the two questions to one question and revised the wording to match the DSM-5 language (Table 3.7). Findings of cognitive interviewing found that the initial question revision worked well in both English and Spanish; therefore, no additional changes were made.

Table 3.7 Cognitive Interviewing Question Versions Assessing Criterion A1 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC04: During the past 12 months, did you try to set limits on how often or how much alcohol you would drink? [Y/N]	DRALC03 During the past 12 months, were there many times when you ended up drinking alcohol in larger amounts or for a longer time than you meant to?	DRALC03 During the past 12 months, were there many times when you ended up drinking alcohol in larger amounts or for a longer time than you meant to?	DPALLRGR During the past 12 months, were there many times when you ended up drinking alcohol in larger amounts or for a longer time than you meant to?

Table 3.7 Cognitive Interviewing Question Versions Assessing Criterion A1 for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC05 [IF DRALC04 = Y]: Were you able to keep to the limits you set, or did you often drink more than you intended to? [Usually kept to the limits set] [Often drank more than intended]	Follow-up item unnecessary	Follow-up item unnecessary	Follow-up item unnecessary

3. Tolerance

The third criterion assessed by NSDUH is tolerance, which corresponds with DSM-IV dependence criteria 1A and 1B and DSM-5 SUD criterion A10. DSM-5 criterion A10 defines tolerance as "a need for markedly increased amounts of the substance to achieve intoxication or the desired effect, OR markedly diminished effect with continued use of the same amount of the substance." As of 2017, NSDUH operationalized this item by asking the following questions for each substance assessed:

• DRALC06: During the past 12 months, did you need to drink more alcohol than you used to in order to get the effect you wanted? [Y/N]

• DRALC07 [IF DRALCO6 = N OR DK/REF]: During the past 12 months, did you notice that drinking the same amount of alcohol had less effect on you than it used to? [Y/N]

a. Expert Panel Review

The 2017 expert panel review identified two specific wording concerns in the current NSDUH question for this criterion: absence of language capturing "markedly" increased amounts or "markedly" diminished effect and use of the word "effect."

Specific wording concerns: The expert panelists noted that NSDUH items DRALC06 and DRALC07 contain wording that is inconsistent with the DSM criterion, which specifies a "markedly" increased amount and a "markedly" diminished effect, respectively. The panelists suggested that the phrases "much more" and "much less" be added to the question to capture the "markedly" component of the DSM criterion. Even if the addition of "much" does not affect the prevalence estimates, it would contribute to face validity against the DSM criterion.

The panelists also expressed concern that some of the questions contained clinical language that may be difficult for a layperson to fully understand. They were concerned that the term "effect" may not be equally applicable to all substances and suggested replacing the word "effect" in DRALC06 with the word "feeling." There was consensus that this would work better. The panel noted that the construct of "amount" is also variable across substances (e.g., number of hits vs. number of drinks). However, if prevalence of the symptom is compared across substances, then it is important to minimize substance-specific variation.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to moderate inter-rater reliability with kappas ranging from 0.35 (SE = 0.06) for alcohol to 0.45 (SE = 0.09) for cocaine. Across all substances, the ratio of false positives to false negatives was 142 to 64, indicating that respondents were much more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Studies' Assessment of Criterion A10

Table 3.8 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording.

Table 3.8 Assessment by Other Surveys of DSM-5 SUD Criterion A10: Tolerance

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You developed alcohol tolerance, that is, either the same amount no longer had the same effect or you needed to drink a lot more to get the same effect?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. Did this happen in the last 12 months? Find that your usual number of drinks had much less effect on you than it once did? Find that you had to drink much more than you once did to get the effect you wanted? Increase your drinking because the amount you used to drink didn't give you the same effect anymore?	[Y/N] [Y/N] [Y/N]
NCS-A CIDI	Did you ever need to drink more than you used to, to get buzzed or drunk, or did you ever find that you could no longer get buzzed or drunk on the amount you used to drink?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Changes to the assessment of tolerance in NSDUH focused mainly on simplifying question wording, emphasizing a marked increase, and using the word "feeling" rather than "effect" (Table 3.9). During Round 1, many participants thought that the increase in use had to have initiated in the past year. Per DSM-5, tolerance is based on changes in physiological responses to a drug starting from when a person begins to use regularly. This was incorporated in the second round of cognitive interviewing. However, participants did not consider their substance use to be regular. Therefore, a balance was sought by using the Round 1 language but removing the reference period so that it was clear that the change in tolerance did not have to initiate in the past 12 months.

Table 3.9 Cognitive Interviewing Question Versions Assessing Criterion A10 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC06: During the past 12 months, did you need to drink more alcohol than you used to in order to get the effect you wanted? [Y/N]	DRALC06 During the past 12 months, did you need to drink a lot more alcohol than you used to in order to get the feeling you wanted? [Y/N]	DRALC06 During the past 12 months, did you need to drink a lot more alcohol than you used to in order to get the feeling you wanted than you did when you first started drinking regularly? [Y/N/I never drank alcohol regularly]	DPALMORE Do you need to drink a lot more alcohol than you used to in order to get the feeling you want wanted? than you did when you first started drinking regularly? [Y/N/I never drank alcohol regularly]?
DRALC07 [IF DRALCO6 = N OR DK/REF]: During the past 12 months, did you notice that drinking the same amount of alcohol had less effect on you than it used to? [Y/N]	DRALC07 [IF DRALCO6 = 2 OR DK/REF] During the past 12 months, did you notice that drinking the same amount of alcohol have much less effect on you than it used to? [Y/N]	DRALC07 [IF DRALCO6 = 2 OR DK/REF] During the past 12 months, did drinking the same amount of alcohol have much less effect on you than it used to when you first started drinking regularly? [Y/N/I never drank alcohol regularly]	DPALLESS [IF DPALMORE = 2 OR DK/REF] During the past 12 months, did drinking Does drinking the same amount of alcohol have much less effect on you than when you first started drinking regularly it used to?

4. Persistent Desire, Unsuccessful Control

The fourth criterion assessed by NSDUH is persistent desire to control substance use and/or unsuccessful control. The fourth criterion corresponds to DSM-IV dependence criterion A4 and DSM-5 SUD criterion A2. DSM-5 criterion A2 states that "there is a persistent desire or unsuccessful efforts to cut down or control substance use." Currently, NSDUH operationalizes this item by asking the following questions:

• DRALC08: During the past 12 months, did you want to or try to cut down or stop drinking alcohol? [Y/N]

• DRALC09 [IF DRALC08 = Y]: During the past 12 months, were you able to cut down or stop drinking alcohol every time you wanted to or tried to? [Y/N]

a. Expert Review

In-person and written expert reviews identified four areas of focus for revising the current NSDUH questions assessing this criterion: inconsistency with the DSM criteria "persistent" attempts, specific wording concerns, adolescent versus adult considerations, and question restructuring.

Inconsistency with the DSM criterion "persistent" attempts: The written feedback provided by external experts noted that NSDUH's question DRALC08 is not consistent with the DSM criteria's specification of "persistent" attempts to cut down or stop use. Although NSDUH DRALC08 asks about wanting or trying to cut down or stop use of the substance, it does not capture whether the desire or efforts were persistent or recurring. In addition, the expert review panel noted that NSDUH item DRALC09 does not assess the "unsuccessful efforts" part of the DSM criteria. Although a "YES" response would rule out the unsuccessful efforts portion of the criterion, a "NO" response fails to capture whether multiple efforts were unsuccessful because NSDUH item DRALC09 asks only whether the respondent was able "every time" to cut down or stop substance use. In other words, respondents could answer "no" to item DRAL09 when they were unsuccessful only on rare occasions.

Specific wording concerns: There was significant objection to the threshold of being able to cut down "every time," as currently imposed by the DRAL09 item. The DSM does not support this very high bar. The item needs to address the repeatable nature of failure. The expert panel suggested rewording the NSDUH items to read: "During the past 12 months, did you more than once want to or try to cut down or stop using marijuana or hashish but found you couldn't?" One suggestion was to replace "every time" in DRALC09 with "when" or to modify the question to use "over a period of time" to address this concern.

Adolescent versus adult considerations: There was general agreement among experts that the language used in the current DRALC09 question is too complicated for youths.

Question restructuring: The experts suggested restructuring the question by asking "Did you ever cut down?" If the respondent answers "NO," then ask "Did you want to?" If the respondent then answers "YES," ask "Were you successful?" Some experts acknowledged that these questions do not assess whether this was a repeated behavior and suggested changing "ever" in the first question to "more than once." "Persistent" is usually only applied to desire, whereas unsuccessful control usually applies to more than one effort.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to moderate inter-rater reliability with kappas ranging from 0.28 (SE = 0.08) for marijuana to 0.42 (SE = 0.06) for alcohol. Across all substances, the ratio of false positives to false negatives was 58 to 93, indicating that respondents were somewhat more likely to mistakenly not endorse this criterion rather than mistakenly endorse it.

c. Other Studies' Assessment of Criterion A2

Table 3.10 shows how other studies have worded their assessments of this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.10 Assessment by Other Surveys of DSM-5 SUD Criterion A2: Persistent Desire, Unsuccessful Control

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You tried to cut down or control your drinking but were unable to do so?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. Did this happen in the last 12 months? More than once WANT to stop or cut down on your drinking? More than once TRY to stop or cut down on your drinking but found you couldn't do it?	[Y/N] [Y/N]
NCS-A CIDI	Were there times when you tried to stop or cut down on your drinking and found that you were not able to do so?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Revisions to the items assessing criterion A2 focused on incorporating persistent desire to stop or cut down using, independent of attempts to do so (Table 3.11). Additionally, revisions attempted to incorporate the concept of repeated failure to cut down or stop using. During cognitive interviewing, respondents found the concept of wanting and trying to cut down difficult to distinguish when asked in a single item. Therefore, the concepts were asked about in two separate items in the final version of the questions, and the language was simplified.

Table 3.11 Cognitive Interviewing Question Versions Assessing Criterion A2 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC08: During the past 12 months, did you want to or try to cut down or stop drinking alcohol? [Y/N]	DRALC08 During the past 12 months, did you often want to cut down or stop drinking alcohol? [Y/N]	[Reordered to be last] DRALC10 [IF DRALC08=2 OR DK/REF] During Even though you did not try to cut down or stop drinking alcohol in the past 12 months, did you often wish that you could want to cut down or stop drinking alcohol? [Y/N]	DPALWISH [IF DPALSTOP=2 OR DK/REF] Even though you did not try to cut down or stop drinking alcohol In the past 12 months, did you often wish that you could cut down or stop drinking alcohol?
	DRALC09 During the past 12 months, did you try to cut down or stop drinking alcohol? [Y/N]	[Reordered to be first] DRALC08 During the past 12 months, did you try to cut down or stop drinking alcohol? [Y/N]	DPALSTOP During the past 12 months, did you try to cut down or try to stop drinking alcohol?

Table 3.11 Cognitive Interviewing Question Versions Assessing Criterion A2 for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC09 [IF DRALC08 = Y]: During the past 12 months, were you able to cut down or stop drinking alcohol every time you wanted to or tried to? [Y/N]	DRALC10a [IF DRALC09 = 1] In the past 12 months, were you able to cut down or stop drinking alcohol every time you tried? [Y/N]	DRALC9a [IF DRALC08 = 1] In the past 12 months, were you able to cut down or stop drinking alcohol every time you tried? [Y/N]	-DRALC9a [IF DRALC08 = 1] In the past 12 months, were you able to cut down or stop drinking alcohol every time you tried? [Y/N]
	DRALC10b [IF DRALC10a = 2 OR DK/REF] Was there more than one time in the past 12 months when you tried but were unable to cut down or stop drinking alcohol? [Y/N]	[Reordered to be in the middle] DRALC9b [IF DRALC9a = 2 OR DK/REF] Was there more than one time in the past 12 months when you tried but were unable to cut down or stop drinking alcohol? [Y/N]	DPALCANT [IF DPALSTOP=1] Some people who drink alcohol try to cut down or stop but find they can't. Was there more than one time in the past 12 months when you tried but were unable to cut down or stop drinking alcohol?

5. Withdrawal Criterion A

The fifth criterion assessed by NSDUH is withdrawal criterion A, which corresponds with DSM-IV dependence criteria 2A and 2B and DSM-5 SUD criterion A11a. Withdrawal criterion A is defined as the characteristic withdrawal syndrome for the substance: (A) after cessation of use (DSM sometimes specifies heavy and prolonged use, which varies by substance), and (B) the number of symptoms (varies by substance) that develop within a certain period of time (varies by substance) after cessation of or reduction in use.

Currently, NSDUH operationalizes this item by asking the following series of questions for each substance assessed, with slight variations by substance. Examples of the withdrawal criterion A operationalizations for NSDUH alcohol and NSDUH cocaine follow:

Withdrawal Criterion A: NSDUH Alcohol

• DRALC10 [IF DRALC08 = N OR DK/REF OR DR ALC09 = N OR DK/REF]:⁰ During the past 12 months, did you cut down or stop drinking at least one time? [Y/N]

• DRALC11 [IF DRALC09 = Y OR DRALC10 = Y]:⁰ Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms after you cut back or stopped drinking alcohol? [Y/N]

• Sweating or feeling that your heart was beating fast

• Having your hands tremble

• Having trouble sleeping

• Vomiting or feeling nauseous

• Seeing, hearing, or feeling things that weren't really there

• Feeling like you couldn't sit still

• Feeling anxious

• Having seizures or fits

• DRALC12 [IF DRALC11 = Y]: Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms at the same time that lasted for longer than a day after you cut back or stopped drinking alcohol? [SAME SYMPTOM LIST AS DRALC11]

Withdrawal Criterion A: NSDUH Cocaine

• DRCC10 [IF DRCC8 = N OR DK/REF OR DRCC9 = N OR DK/REF]:⁰ During the past 12 months, did you cut down or stop using [COKEFILL] at least one time?

• DRCC10a [IF DRCC09 = Y OR DRCC10 = Y]:⁰ During the past 12 months, have you felt kind of blue or down when you cut down or stopped using [COKEFILL]?

• DRCC11 [IF DRCC10a = Y]: Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms after you cut back or stopped using [COKEFILL]?

• Feeling tired or exhausted

• Having bad dreams

• Having trouble sleeping or sleeping more than you normally do

• Feeling hungry more often

• Feeling either very slowed down or like you couldn't sit still

• DRCC12 [IF DRCC11 = Y]: Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms at the same time that lasted for longer than a day after you cut back or stopped using [COKEFILL]? [SAME SYMPTON LIST AS DRCC11]

a. Expert Panel Review

Withdrawal assessment was specifically reviewed by both the external experts via written reviews in 2015 and by the in-person panelists in 2017. In-person and written expert feedback identified several areas of focus for revising the current NSDUH questions assessing this criterion: item wording for "cessation of use," other wording concerns, adolescent versus adult consideration, and question restructuring.

Cessation of use: Written feedback from the 2015 external reviewers noted that item DRALC10 is not needed because parts of the question can be incorporated into the subsequent two questions that assess withdrawal. For example, a response of "I did not cut back or stop" could be added to item DRALC11, and respondents endorsing this response could skip out of the remaining withdrawal items. However, this strategy is generally recognized as bad survey practice (Lessler & Forsyth, 1996; Forsyth, Lessler, & Hubbard, 1992; Olsen, Smyth, & Ganshert, 2019). Therefore, the survey methodologists involved in the SUD redesign had concerns about this suggestion. Both sets of expert reviewers also noted that a quit attempt is not needed for withdrawal. To meet withdrawal criteria, there only needs to be a period of no use when the effects from prior use had worn off or were stating to wear off. If respondents are only reporting times that their cutting down or quitting was purposeful, it is problematic. The DISC uses "when you hadn't drunk/used for a while."

The 2015 review provided several suggestions on how to eliminate references to deliberate attempts to cut back or stop:

• Suggestion 1: Item wording could be revised to add the phrase "because you wanted to or because it wasn't available" when referencing cutting down or stopping use.

• Suggestion 2: "During the past 12 months, whether you wanted to or not, did you cut back or stop drinking alcohol at least one time?" Internal experts noted that this version was more consistent with the pharmacological concept of withdrawal because it captured times that the person could not use (e.g., when they were sleeping or at work).

• Suggestion 3: "During the past 12 months, for any reason, did you cut back or stop using marijuana or hashish at least one time?"

Specific wording concerns: The 2015 expert reviewers noted that item DRACL12 (two or more symptoms at the same time that lasted for longer than a day after you cut back or stopped drinking alcohol) is not representative of the DSM-5 criteria, because symptoms do not necessarily occur at the same time but rather develop in stages.

The 2017 expert panelists and 2015 expert reviewers generally thought that wording of the questions was complicated and that some of the time period references could be removed or simplified. There also was concern about use of the word "symptoms," although prior cognitive testing found that respondents generally understood and were not bothered by the word "symptoms."

Assessing multiple symptoms: There was unanimous feedback from the in-person expert panelist and written expert reviews that respondents should not have to count symptoms, times, or other entities to answer a question. In a strong design, symptoms would be assessed one at a time, but this could cause underreporting or a patterned response (where a respondent registers the same response over and over without reading the question). There was discussion of randomizing the individual withdrawal symptoms to reduce underreporting, but the panel eventually suggested that the symptoms be listed in order from least to most severe.

Time frame of symptom occurrence: Experts expressed concern that there was no requirement regarding the time frame in which withdrawal symptoms occur. For opiates, symptoms will occur very rapidly, but for marijuana, symptoms could take a week to appear. Pharmacological study findings might lend information about these time periods in cases where the DSM does not. This might help decrease the likelihood that respondents report symptoms occurring long after they stopped using and that might not be linked to withdrawal specifically.

Hangover versus withdrawal symptoms: There was concern that hangover was being reported as withdrawal. In-person expert panelists suggested adding a note to the question: "We do not mean symptoms of a hangover."

Differences in International Classification of Diseases (ICD) system symptoms: In-person expert panelists noted that ICD-10 withdrawal symptoms differ from those included in DSM. Because ICD is typically not used in the United States, it is advisable to include only the symptoms listed in DSM.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to moderate inter-rater reliability with Cohen's kappas ranging from 0.35 (SE = 0.10) for cocaine to 0.49 (SE = 0.09) for alcohol. Across all substances, the ratio of false positives to false negatives was 48 to 45, indicating that respondents were equally likely to mistakenly endorse this criterion or mistakenly deny it.

c. Other Studies' Assessment of Criterion A11a

Table 3.12 shows how several other studies have worded their assessments of this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording.

Table 3.12 Assessment by Other Surveys of DSM-5 SUD Criterion A11a: Withdrawal Criterion A11a

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You experienced withdrawal symptoms like fatigue, headaches, diarrhea, the shakes, or emotional problems when you tried to cut down your drinking?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never

Table 3.12 Assessment by Other Surveys of DSM-5 SUD Criterion A11: Withdrawal Criterion A11 (continued)

Other Survey	Question Wording	Answer Choices
AUDADIS-5	The next few questions are about the bad aftereffects of drinking that people may have when the effects of alcohol are wearing off. This includes the morning after drinking or in the first few days after stopping or cutting down. a. Did you EVER... b. Did this happen in the last 12 months? Have trouble falling asleep or staying asleep (when the effects of alcohol were wearing off)? Find yourself shaking or your hands trembling? Feel anxious or nervous? Feel sick to your stomach or vomit (when the effects of alcohol were wearing off)? Feel more restless than is usual for you? Find yourself sweating or your heart beating fast? See, feel, or hear things that weren't really there (when the effects of alcohol were wearing off? Have fits or seizures? You just mentioned that you had SOME bad aftereffects when stopping or cutting down on drinking in the last 12 months. Did at least 2 of these experiences happen around the same time DURING the last 12 months?	[Y/N] [Y/N] [Y/N] [Y/N] [Y/N] [Y/N] [Y/N] [Y/N] [Y/N]
NCS-A CIDI	People who all of a sudden cut down or stop drinking may not feel well. These feelings are much stronger and can last longer than the usual hangover. Did you ever get tired or have headaches, diarrhea, the shakes, or emotional problems when you stopped, cut down, or went without drinking?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

NOTE: NSDUH does not assess withdrawal criterion B.

d. 2015-2016 Cognitive Interviewing and Validation

Cognitive interviewing feedback on the items assessing withdrawal criterion A11a indicated that participants were not necessarily considering all times that they cut back or stopped use of the substance. For example, participants reported thinking only about intentionally stopping and not just periods of time when they might have gone without the substance for other reasons. One consistent finding was that participants did not think to include time in rehabilitation as times they cut down or stopped use. In addition, participants thought DRALC10 (and similar items for other substances), asking about cutting down or stopping use during the past 12 months, was asking about successfully cutting down or stopping use. Other cognitive interviewing findings were that some participants did not use the substance often enough to have cut down; that is, cutting down implied a regular level of use. Finally, cognitive interviewing found that respondents were not consistently attributing the queried withdrawal symptoms to having cut back or stopped use of the substance. This appeared to be particularly problematic for the alcohol withdrawal assessment. For example, after positive endorsement of withdrawal, a respondent described a hangover rather than withdrawal.

e. 2018-2019 Cognitive Testing

Table 3.13 shows the changes in the items assessing criterion A11a from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. The primary changes were defining withdrawal symptoms (and for alcohol, differentiating withdrawal from a hangover), eliminating the requirement that respondents have tried to cut down or stop using in the past year, and adapting the question to use a yes/no grid format. Cognitive testing found that the initial revision of the item performed well in English and Spanish and was not further revised.

Table 3.13 Cognitive Interviewing Question Versions Assessing Criterion A11a for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC10 [IF DRALC08 = N OR DK/REF OR DR ALC09 = N OR DK/REF].1 During the past 12 months, did you cut down or stop drinking at least one time? [Y/N] DRALC11 [IF DRALC09 = Y OR DRALC10 = Y] Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms after you cut back or stopped drinking alcohol? [Y/N] Sweating or feeling that your heart was beating fast Having your hands tremble Having trouble sleeping Vomiting or feeling nauseous Seeing, hearing, or feeling things that weren't really there Feeling like you couldn't sit still Feeling anxious Having seizures or fits	[Formatting changed to one item with symptom grid] DRALC22 People may experience withdrawal symptoms when they drink less or stop drinking alcohol. Withdrawal symptoms are stronger and last longer than a hangover. During the past 12 months, did you cut down or stop drinking at least one time have the following withdrawal symptoms after you drank less or stopped drinking alcohol for a while?	DRALC22 People may experience withdrawal symptoms when they drink less or stop drinking alcohol. Withdrawal symptoms are stronger and last longer than a hangover. During the past 12 months, did you have the following withdrawal symptoms after you drank less or stopped drinking alcohol for a while?	DRALC22 People may experience withdrawal symptoms when they drink less or stop drinking alcohol. Withdrawal symptoms are stronger and last longer than a hangover. During the past 12 months, did you have the following withdrawal symptoms after you drank less or stopped drinking alcohol for a while?
	DRALC22_1 Sweating or feeling that your heart was beating fast	DRALC22_1 Sweating or feeling that your heart was beating fast	DRALC22_1 Sweating or feeling that your heart was beating fast
	DRALC22_2 Having your hands tremble	DRALC22_2 Having your hands tremble	DRALC22_2 Having your hands tremble
	DRALC22_3 Having trouble sleeping	DRALC22_3 Having trouble sleeping	DRALC22_3 Having trouble sleeping
	DRALC22_4 Vomiting or having an upset stomach	DRALC22_4 Vomiting or having an upset stomach	DRALC22_4 Vomiting or having an upset stomach
	DRALC22_5 Seeing, hearing, or feeling things that weren't really there	DRALC22_5 Seeing, hearing, or feeling things that weren't really there	DRALC22_5 Seeing, hearing, or feeling things that weren't really there
	DRALC22_6 Feeling like you couldn't sit still	DRALC22_6 Feeling like you couldn't sit still	DRALC22_6 Feeling like you couldn't sit still

Table 3.13 Cognitive Interviewing Question Versions Assessing Criterion A11a for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC12 [IF DRALC11 = Y]: Please look at the symptoms listed below. During the past 12 months, did you have 2 or more of these symptoms at the same time that lasted for longer than a day after you cut back or stopped drinking alcohol? [SAME SX LIST AS DRALC11]	DRALC22_7 Feeling anxious	DRALC22_7 Feeling anxious	DRALC22_7 Feeling anxious
	DRALC22_8 Having seizures or fits	DRALC22_8 Having seizures or fits	DRALC22_8 Having seizures or fits

¹DRALC10 is asked only if the respondent reported not having wanted or tried to cut down or reported having been unable to cut down every time he or she tried, or if he or she answered "don't know" or refused these questions.

²DRALC11 is asked only if the respondent reported that he or she was able to cut down every time he or she tried or that he or she cut down or stopped using at least one time.

6. Continued Use Despite Health Problems

The sixth criterion assessed by NSDUH is continued use despite health problems, which corresponds with DSM-IV dependence criterion A7 and DSM-5 SUD criterion A9. The DSM-5 definition for this criterion states that "substance use is continued despite knowledge of having a persistent physical or psychological problem that is likely to have been caused or exacerbated by the substance (e.g., current cocaine use despite recognition of cocaine-induced depression or continued drinking despite recognition that an ulcer was made worse by alcohol consumption)."

NSDUH operationalizes this item by asking the following series of questions for each substance assessed:

• DRALC13: During the past 12 months, did you have any problems with your emotions, nerves, or mental health that were probably caused or made worse by drinking alcohol? [Y/N]

• DRALC14 [IF DRALC13 = Y]: Did you continue to drink alcohol even though you thought drinking was causing you to have problems with your emotions, nerves, or mental health? [Y/N]

• DRALC15 [IF DRALC13 = N OR DK/REF OR DRALC14 = N OR DK/REF]: During the past 12 months, did you have any physical health problems that were probably caused or made worse by drinking alcohol? [Y/N]

• DRALC16 [IF DRALC15 = Y]: Did you continue to drink alcohol even though you thought drinking was causing you to have physical problems? [Y/N]

a. Expert Panel Review

The 2017 in-person expert panel review identified three areas of focus for revising the current NSDUH question assessing criterion A9 on continued use despite health problems: question complexity, knowledge that use caused health problems, and inclusion of substance-specific examples.

Complexity: The panelists noted that these questions were overly complex as written.

Knowledge that use caused health problems: The panelists suggested that the key to this item is that people continue to use even though they know their substance use is causing or has caused a health problem. Therefore, the word "probably" should be removed. Although problems may be caused by a person's substance use and they continue using, experts stated that if the respondent does not know that the health problems were caused or made worse by his or her substance use, it does not "count" as meeting this criterion.

Inclusion of substance-specific examples: Multiple panelists said substance-specific examples might be useful for this question (e.g., paranoia for cocaine/stimulants); however, other panelists were not convinced (e.g., the Alcohol Use Disorder and Associated Disabilities Interview Schedule [AUDADIS] uses the same item across all substances). Several panelists noted that very few people would consider alcohol-induced blackouts to be an example of "emotions, nerves, or mental health." However, blackouts after alcohol use are an important indicator of this criterion. Not all panelists thought that specifically mentioning blackouts was a good idea unless it was asked separately for alcohol, because most substances do not induce blackouts and this could confuse respondents.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to substantial inter-rater reliability with kappas ranging from 0.35 (SE = 0.08) for marijuana to 0.59 (SE = 0.08) for cocaine. Across all substances, the ratio of false positives to false negatives was 90 to 61, indicating that respondents were slightly more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Surveys' Assessment of Criterion A9

Table 3.14 shows how other studies have worded their assessments of this criterion. Features of the items used by other nationally representative studies that aligned with the expert panelists' feedback were considered in revised item wording. It is important to note, however, that not all survey items aligned well with all expert feedback. For example, the National Mental Health Survey, Composite International Diagnostic Interview (NMHS CIDI) and the National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview (NCS-A CIDI) combine physical and emotional problems in a single item.

Table 3.14 Assessment by Other Surveys of DSM-5 SUD Criterion A9: Continued Use Despite Health Problems

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You continued to drink even when it caused physical or emotional health problems?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Continue to drink even though you knew it was making you feel depressed, uninterested in things, or suspicious or distrustful of other people? Continue to drink even though you knew it was causing you a health problem or making the health problem worse? Continue to drink even though you had experienced a prior blackout, that is, awakened the next day not being able to remember some of the things you did while drinking or after drinking?	[Y/N] [Y/N] [Y/N]
NCS-A CIDI	Did you ever continue to drink even though you knew you had a serious physical or emotional problem that might have been caused by or made worse by drinking?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Changes to the items assessing criterion A9 included removing the word "nerves," reducing unnecessary words, adding a question about experiencing blackouts for alcohol, and comparing two wording options for measuring recurrent and chronic type health conditions. Additional revisions included presenting the physical health problem questions first (starting in the second round of cognitive interviewing) and testing whether the word "probably" was needed by leaving it in the mental health problem questions and removing it in the physical health version of the question. Table 3.15 shows the changes in the items assessing criterion A9 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. English cognitive interviewing results indicated that most of the revised phrasing worked well, including the removal of the term "probably," but that respondents needed a definition of blackouts to distinguish a blackout from losing consciousness or simple forgetfulness. In different variations, cognitive interviewing tested whether respondents interpreted "long-lasting" and "repeated" health problems similarly or whether both phrases were necessary. Results indicated that these two phrases represented different constructs to respondents and both should be included. Spanish-language cognitive testing also found that adult Spanish speakers did not understand the word "emotions" as used in the phrase "problems with emotions or mental health." After consultation with the Spanish-language SUD expert, it was determined that for the Spanish-language version only, the word "emotions" was dropped so that the items ask about "problems with mental health" only. Although this may lead to a slight challenge of English to Spanish comparability, it was determined that the effect of the inequality on the small number of NSDUH Spanish-language respondents was less concerning than the effect of changing wording that was functioning well for most respondents. Moreover, if the difference in wording makes symptom assessment more accurate within each population, then comparability may be improved.

Table 3.15 Cognitive Interviewing Question Versions Assessing Criterion A9 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC13: During the past 12 months, did you have any problems with your emotions, nerves, or mental health that were probably caused or made worse by drinking alcohol? [Y/N]	DRALC13 During the past 12 months, did you have any long-lasting [IF VERSION=2: or repeated] problems with emotions nerves or mental health that were probably caused or made worse by drinking alcohol? [Y/N]	Note question ordering was changed DRALC11 During the past 12 months, did you have any long-lasting [IF VERSION=2: or repeated] physical health problems that were caused or made worse by drinking alcohol? [Y/N]	Note question ordering was changed DPALPHYS During the past 12 months, did you have any long-lasting or repeated physical health problems that were caused or made worse by drinking alcohol?
DRALC14 [IF DRALC13 = Y]: Did you continue to drink alcohol even though you thought drinking was causing you to have problems with your emotions, nerves, or mental health? [Y/N]	DRALC14 [IF DRALC13 = 1]: Did you continue to drink alcohol even though you thought drinking it was causing these problems with your emotions nerves or mental health or making them worse? [Y/N]	DRALC12 [IF DRALC11 = 1]: Did you continue to drink alcohol even though it was causing these long-lasting or repeated physical health problems or making them worse? [Y/N]	DPALPCNT [IF DPALPHYS = 1]: Did you continue to drink alcohol even though it was causing these long-lasting or repeated physical health problems or making them your physical health problems worse?
DRALC15 [IF DRALC13 = N OR DK/REF OR DRALC14 = N OR DK/REF]: During the past 12 months, did you have any physical health problems that were probably caused or made worse by drinking alcohol? [Y/N]	DRALC11 During the past 12 months, did you have any long-lasting [IF VERSION=2: or repeated] physical health problems that were probably caused or made worse by drinking alcohol? [Y/N]	DRALC13 [IF DRALC11 = 2 OR DK/REF OR DRALC12 = 2 OR DK/REF] During the past 12 months, did you have any long-lasting [IF VERSION=2: or repeated] problems with emotions or mental health that were caused or made worse by drinking alcohol? [Y/N]	DPALMNTL [IF DPALPHYS = 2 OR DK/REF OR DPALPCNT = 2 OR DK/REF] During the past 12 months, did you have any long-lasting or repeated problems with emotions or mental health that were caused or made worse by drinking alcohol?
DRALC16 [IF DRALC15 = Y]: Did you continue to drink alcohol even though you thought drinking was causing you to have physical problems? [Y/N]	DRALC12 [IF DRALC11 = 1]: Did you continue to drink alcohol even though you thought drinking it was causing you to have these physical health problems or making them worse? [Y/N]	DRALC14 [IF DRALC13 = 1]: Did you continue to drink alcohol even though it was causing these long-lasting or repeated problems with your emotions or mental health or making them worse? [Y/N]	DPALMCNT [IF DPALMNTL = 1]: Did you continue to drink alcohol even though it was causing these long-lasting or repeated problems with your emotions or mental health or making them your emotions or mental health worse?

Table 3.15 Cognitive Interviewing Question Versions Assessing Criterion A9 for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
	[new item Alcohol only]. DRALC15 During the past 12 months, did you have blackouts, that is, woke the next day not being able to remember some of the things that happened while drinking or after drinking alcohol? [Y/N]	DRALC15 A blackout is lack of memory. That is, you were awake, but you have no recall of the things you did or that were done to you. During the past 12 months, did you have blackouts, that is, woke the next day not being able to remember some of the things that happened while drinking or after drinking alcohol? [Y/N]	DPALBLCK [IF (DPALPHYS = 2 OR DK/REF OR DPALPCNT = 2 OR DK/REF) AND (DPALMENT=2 OR DK/REF OR DPALMCNT = 2 OR DK/REF)] A blackout is lack of memory. That is, you were awake, but you have no recall of the things you did or that were done to you. During the past 12 months, did you repeatedly have blackouts that is, woke the next day not being able to remember some of the things that happened while drinking or after drinking alcohol?
	[new item] Alcohol only. DRALC16 [IF DRALC15 = 1]: Did you continue to drink alcohol even though drinking gave you blackouts? [Y/N]	DRALC16 [IF DRALC15 = 1]: Did you continue to drink alcohol even though drinking gave you blackouts? [Y/N]	DPALBCNT [IF DPALBLCK = 1] Did you continue to drink alcohol even though drinking gave you repeated blackouts?

7. Giving Up Activities

The seventh criterion assessed by NSDUH is giving up activities, which corresponds with DSM-IV dependence criterion A6 and DSM-5 SUD criterion A7. The DSM-5 definition for criterion A7 states that "important social, occupational, or recreational activities are given up or reduced because of substance use." NSDUH operationalizes this item by asking the following question for each substance assessed:

• DRALC17: This question is about important activities such as working, going to school, taking care of children, doing fun things such as hobbies and sports, and spending time with friends and family. During the past 12 months, did drinking alcohol cause you to give up or spend less time doing these types of important activities? [Y/N]

a. Expert Panel Review

The expert reviewers identified three areas of focus for revising the current NSDUH question assessing criterion A7 on giving up activities: specific wording concerns, categorizing types of activities, and adolescent versus adult considerations.

Specific wording concerns: The panel members unanimously agreed that the current item in NSDUH used to assess criterion A7 was too long and contained too much information. They suggested the need for multiple items to assess this criterion. Because meeting this criterion requires giving up activities in only one area of life (e.g., work, leisure), respondents who endorse the first one assessed could skip out of the assessments of the other items. Doing this would reduce interview time and fatigue, but the trade-off would be that the data could be less comprehensive because it would not assess all areas of life activities.

Categorizing types of activities: Some experts recommended splitting the assessment of this criterion into two sets of questions: one for work and the other for leisure activities. Others suggested breaking the criterion into separate work, social, and recreation activity assessments with bulleted examples for each.

Adolescent versus adult considerations: The panel members also noted that some wording suggestions proposed in the meeting would be too complex for young adolescents. They suggested replacing the word "associating" with "doing things with" or "getting together with." Another panel participant noted that some adolescents may not consider school to be "important," so they might not endorse this item. The criterion does not require the respondent to acknowledge the importance of these different domains of activities but instead requires that a clinician would consider them important. Therefore, including the word "important" may be unadvisable.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had moderate to substantial inter-rater reliability with kappas ranging from 0.41 (SE = 0.06) for alcohol to 0.61 (SE = 0.09) for cocaine. Across all substances, the ratio of false positives to false negatives was 113 to 27, indicating that respondents were much more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Surveys' Assessment of Criterion A7

Table 3.16 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies aligned in some ways with the expert panelists' feedback. The questions are worded more simply, and AUDADIS-5 breaks up the types of activities into two groups.

Table 3.16 Assessment by Other Surveys of DSM-5 SUD Criterion A7: Giving Up Activities

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You greatly reduced important activities with family, friends, or at work because of your drinking?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never

Table 3.16 Assessment by Other Surveys of DSM-5 SUD Criterion A7: Giving Up Activities (continued)

Other Survey	Question Wording	Answer Choices
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Give up or cut down on activities that were important to you in order to drink—like work, school, or associating with friends or relatives? Give up or cut down on activities that you were interested in or that gave you pleasure in order to drink?	[Y/N] [Y/N]
NCS-A CIDI	Did you ever have a time lasting a month or longer when you gave up or cut down on important activities because of your drinking—like sports, work, or seeing friends and family?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Table 3.17 shows the changes in the items assessing criterion A7 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. Revisions included listing the activities in bulleted form to improve readability, simplifying the question wording, and formatting and differentiating the list of activities. Cognitive interviewing found the English and Spanish revision largely worked as intended. The activities were reordered so that most frequent activity (spending time with friends and family) was listed first to ensure that all respondents saw it.

Table 3.17 Cognitive Interviewing Question Versions Assessing Criterion A7 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC17: This question is about important activities such as working, going to school, taking care of children, doing fun things such as hobbies and sports, and spending time with friends and family. During the past 12 months, did drinking alcohol cause you to give up or spend less time doing these types of important activities? [Y/N]	DRALC17 This question is about important activities such as: Attending special events at work or school Participating in hobbies and sports Attending religious services and events Spending time with friends and family During the past 12 months, did drinking alcohol cause you give up or spend a lot less time doing any of these types of important activities because of your alcohol use? [Y/N]	DRALC17 This question is about important activities such as: -Spending time with friends and family [reordered to come at beginning of list] Attending special events at work or school Participating in hobbies and sports Attending religious services and events During the past 12 months, did you give up or spend a lot less time doing any of these types of important activities because of your alcohol use? [Y/N]	DPALACTV This question is about important activities such as: Spending time with friends and family Attending special events at work or school Participating in hobbies and sports Attending religious services and events During the past 12 months, did you give up or spend a lot less time doing any of these types of important activities because of your alcohol use?

8. Role Failure

The eighth criterion assessed by NSDUH is role failure, which corresponds with DSM-IV abuse criterion A1 and DSM-5 SUD criterion A5. The DSM-5 criterion A5 definition for role failure is "recurrent substance use resulting in a failure to fulfill major role obligations at work, school, or home." NSDUH operationalizes this item by asking the following question for each substance assessed:

• DRALC18: Sometimes people who drink alcohol have serious problems at home, work, or school—such as:

• neglecting their children

• missing work or school

• doing a poor job at work or school

• losing a job or dropping out of school

• During the past 12 months, did drinking alcohol cause you to have serious problems like this either at home, work, or school?

a. Expert Panel Review

Expert panelists identified three areas of focus for revising the current NSDUH question assessing Criterion A5 on role failure: importance of impairment, differentiating between role failure during versus after substance use, and adolescent versus adult considerations.

Degree of impairment: Experts noted that endorsing role failure reflects a fair amount of impairment and severity of associated problems. For example, missing work or school from time to time is not considered to be a failure to fulfill a major role obligation. Meeting this criterion would instead require missing a lot of school or work (e.g., missing so often that a respondent might get suspended, drop out, or be disciplined or fired). It should be a recurrent pattern and needs to be differentiated from the prior item about giving up activities.

Inclusivity of role failure: DSM criteria require measuring role failure during the entire process of substance use (obtaining, using, feeling the effects of use, the period after the effects of use wear off), not just role failure during substance use itself. Therefore, any revisions should be evaluated to ensure that it was not explicitly or implicitly suggested that role failure counted only during use.

Adolescent versus adult considerations: Examples of role failure may differ between adolescents and adults, but because individuals' situations differ, it is challenging to know in advance which items will apply to an individual respondent (e.g., some adolescents do have responsibilities taking care of home or family). Some expert panels noted that the AUDADIS items do a good job assessing this criterion, but there is some concern about using the AUDADIS items with adolescents. This is because the instrument has been developed for adults, and there were some concerns about how the items would apply to adolescents (e.g., responsibilities for taking care of home and family).

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair to moderate inter-rater reliability with kappas ranging from 0.33 (SE = 0.09) for cocaine to 0.46 (SE = 0.07) for marijuana. Across all substances, the ratio of false positives to false negatives was 106 to 46, indicating that respondents were much more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Surveys' Assessment of Criterion A5

Table 3.18 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies that aligned with the expert panelists' feedback were considered in revised item wording. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.18 Assessment by Other Surveys of DSM-5 SUD Criterion A5: Role Failure

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? Your drinking or being hung over interfered with your responsibilities at school, home, or work?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Have a period when your drinking or being sick from drinking often interfered with taking care of your home or family? Have job or school troubles because of your drinking or being sick from drinking—like missing too much work, not doing your work, being demoted or losing a job, or being suspended, expelled or dropping out of school? Continue to drink even though it was causing you problems at school or at work?	[Y/N] [Y/N] [Y/N]
NCS-A CIDI	The next questions are about problems you may have had because of drinking. First, was there ever a time in your life when your drinking or being hung over often caused problems at school or work or at home?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Table 3.19 shows the changes in the items assessing criterion A5 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. Revisions included simplifying the question wording, reordering the list of serious problems, expanding the description of each problem listed, and changing the wording of "drinking alcohol" to "alcohol use" to avoid implying the question was only about times when the respondent was physically drinking. Based on Round 1 findings, an additional serious problem—not being able to get or keep a job—was added.

Table 3.19 Cognitive Interviewing Question Versions Assessing Criterion A5 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC18: Sometimes people who drink alcohol have serious problems at home, work, or school—such as: -neglecting their children -missing work or school -doing a poor job at work or school -losing a job or dropping out of school During the past 12 months, did drinking alcohol cause you to have serious problems like this either at home, work, or school?	DRALC18 Sometimes people who drink alcohol have serious problems at work, school, or home—such as: missing a lot of work or school getting demoted, having your hours cut, or losing a job getting suspended, expelled, or dropping out of school Failing to take care of family During the past 12 months, did drinking alcohol cause you have serious problems like this either at work, school, or home because of your alcohol use? [Y/N] Reordered work, school, home	DRALC18 Sometimes people who drink alcohol have serious problems at work, school, or home—such as: missing a lot of work or school getting demoted, having your hours cut, or losing a job not being able to get a job or keep a job [added] getting suspended, expelled, or dropping out of school failing to take care of family During the past 12 months, did you have serious problems like this either at work, school, or home because of your alcohol use? [Y/N]	DPALSERI Sometimes people who drink alcohol have serious problems at work, school, or home—such as: missing a lot of work or school getting demoted, having your hours cut, or losing a job not being able to get a job or keep a job getting suspended, expelled, or dropping out of school failing to take care of family During the past 12 months, did you have any serious problems like these either at work, school, or home because of your alcohol use?

9. Hazardous Use

The ninth criterion assessed by NSDUH is hazardous use, which corresponds with DSM-IV abuse criterion A2 and DSM-5 SUD criterion A8. The DSM-5 criterion A8 definition for hazardous use is "recurrent substance use in situations in which it is physically hazardous." NSDUH operationalizes this item by asking the following question:

• DRALC19: During the past 12 months, did you regularly drink alcohol and then do something where being drunk might have put you in physical danger? [Y/N]

a. Expert Panel Review

Expert panelists identified two areas of focus for revising the current original NSDUH question assessing Criterion A8 on hazardous use: revisions of language around being "drunk" and other effects and use of the word "regular" in assessing recurrence.

Specific wording concerns: The expert reviewers noted that (for alcohol) the NSDUH question refers to being "drunk" but that being "drunk" is not actually part of the criterion. Furthermore, some respondents may say they were not drunk even if they were. For marijuana, some respondents believe that they are better drivers when high on marijuana than when sober. The experts suggested rewording the NSDUH item to include "while feeling the effects of ___." Reviewers also noted that the criterion requires recurrent use but that NSDUH used the word "regular," which could have had different meanings to different respondents. To assess recurrent use, some experts recommended adding "more than one time" to the question, but this was not universal because there was concern over whether recurrent use was equivalent to two or more times.

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had fair inter-rater reliability with kappas ranging from 0.25 (SE = 0.05) for alcohol to 0.35 (SE = 0.09) for cocaine. Across all substances, the ratio of false positives to false negatives was 144 to 39, indicating that respondents were much more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Surveys' Assessment of Criterion A8

Table 3.20 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.20 Assessment by Other Surveys of DSM-5 SUD Criterion A8: Hazardous Use

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You were under the influence in hazardous situations, like when driving or operating a machine?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? More than once drive a car or other vehicle WHILE you were drinking? Drive a car, motorcycle, truck, boat or other vehicle and have an accident WHILE you were under the influence of alcohol? More than once drive a car, motorcycle, truck, boat, or other vehicle AFTER having too much to drink? Get into situations while drinking or after drinking that increased your chances of getting hurt—like swimming, using machinery, or walking in a dangerous area or around heavy traffic?	[Y/N] [Y/N] [Y/N] [Y/N]

Table 3.20 Assessment by Other Surveys of DSM-5 SUD Criterion A8: Hazardous Use (continued)

Other Survey	Question Wording	Answer Choices
NCS-A CIDI	Were there times in your life when you were often buzzed or drunk in situations where you could get hurt; for example, when riding a bicycle, driving, playing sports, operating a machine, or anything else?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Table 3.21 shows the changes in the items assessing criterion A8 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. Initial changes included revising the question to focus on "risky situations" (specifically situations where the chance of danger was increased) to better align with the DSM-5 criteria and to include situations when respondents were at increased risk of having dangerous things done to them (e.g., being sexually assaulted). In addition, "regularly" was revised to "repeatedly" to better align with DSM-5's use of the term "recurrent." Per reviewer feedback, the term "drunk" was removed from the question, but the phrase "feeling the effects of" was not included in the question because of concerns that it made the question wording too long and complex. The initial revision performed well in English and Spanish and was not changed after cognitive interviewing.

Table 3.21 Cognitive Interviewing Question Versions Assessing Criterion A8 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC19: During the past 12 months, did you regularly drink alcohol and then do something where being drunk might have put you in physical danger? [Y/N]	DRALC21 During the past 12 months, did you regularly repeatedly get into situations where drinking alcohol increased your chances of getting physically hurt? [Y/N]	DRALC21 During the past 12 months, did you repeatedly get into situations where drinking alcohol increased your chances of getting physically hurt? [Y/N]	DPALHURT During the past 12 months, did you repeatedly get into situations where drinking alcohol increased your chances of getting physically hurt?

10. Use Despite Social Problems

The 10th criterion assessed by NSDUH is use despite social problems, which corresponds with DSM-IV abuse criterion 4 and DSM-5 SUD criterion A6. The DSM-5 criterion A6 definition for use despite social problems is "continued substance use despite having persistent or recurrent social or interpersonal problems caused by or exacerbated by the effects of the substance." NSDUH operationalizes this item by asking the following questions for each substance assessed:

• DRALC21: During the past 12 months, did you have any problems with family or friends that were probably caused by your drinking? [Y/N]

• DRALC22 [IF DRALC21 = Y]: Did you continue to drink alcohol even though you thought your drinking caused problems with family or friends? [Y/N]

a. Expert Panel Review

Expert panelist review of the original NSDUH items on use despite social problems identified specific wording concerns around the assessment of "any use" versus the effects of use, assessment of recurrence, and assessment of the "exacerbated" construct.

Specific wording concerns: The expert reviewers noted that the current NSDUH question only assesses the "use" but not the "effect" (i.e., social problems) component of this criterion. Specifically, they noted that it is not so much the arguments over use that are the focus of this criterion, but rather the focus is arguments incited over the person's behavior while under the effect of the substance. This is particularly a concern for adolescents whose use of any of these substances even once is likely to cause problems with family if it becomes known. Similar to the criterion focused on continued use despite health problems, this criterion focuses on continuation despite knowledge of the social problems caused by use. The current NSDUH item contained the word "probably," which the experts recommended should be replaced with "you thought" to capture use despite knowledge. Some panelists thought that adding "you thought" was redundant because the respondent would only be reporting on things "they thought" were a result of substance use. In addition, panel members had concern with the potential for misreporting to occur when the respondent affirms social problems that occurred as the result of any use rather than the effects of use. For example, if a respondent's wife does not approve of any use and the respondent uses at a low level, social problems may occur between the husband and wife. The panel members suggested making it clear that the problems occurred as the result of drinking by adding "caused by the effects of your drinking" to the question. However, internal methodologists expressed concern that this phrasing might cause more confusion by increasing the complexity of an already complex question. They felt that cognitive interviewing might determine whether this was more than a theoretical concern. In addition, some panel members suggested simplifying some of the language by using the word "repeated" to assess the recurrent nature of the problems and using "increased" or "caused or made worse" to assess the construct of "exacerbated."

b. Prior Validation Study

The prior validation study examining concordance between results from the current NSDUH and results from a clinical interview found that this criterion had moderate inter-rater reliability with kappas ranging from 0.46 (SE = 0.06) for alcohol and marijuana to 0.49 (SE = 0.09) for cocaine. Across all substances, the ratio of false positives to false negatives was 90 to 45, indicating that respondents were somewhat more likely to mistakenly endorse this criterion rather than mistakenly deny it.

c. Other Surveys' Assessment of Criterion A6

Table 3.22 shows how other studies have worded their assessment of this criterion. Features of the items used by other nationally representative studies were considered in revised item wording. However, the distinction between any use and the effects of use is not represented in any items. Cognitive interviewing and validation will be needed to inform final wording choice.

Table 3.22 Assessment by Other Surveys of DSM-5 SUD Criterion A6: Use Despite Social Problems

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You continued drinking even when it caused problems with your family, friends, neighbors, or co-workers?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Have arguments or problems with your spouse or partner or family or friends because of your drinking? Continue to drink even though it was causing you trouble with your family or friends? Get into physical fights while drinking or right after drinking?	[Y/N] [Y/N] [Y/N]
NCS-A CIDI	a. Was there ever a time in your life when your drinking caused arguments or other serious or repeated problems with your family, friends, teachers, neighbors, or co-workers? b. Did you continue to drink even though it caused problems with these people?	[Y/N] [Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

The goals of item revisions for criterion A6 were to simplify the language, incorporate the persistent or recurrent quality of the behavior (the word "often" was thought to best capture these two constructs), revise "problems" to "arguments or other problems," and clarify the link between the behavior of drinking and the effect on social problems (i.e., removing the word "probably"). Table 3.23 shows the changes in the items assessing criterion A6 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. The initial revisions functioned well in English and Spanish cognitive interviewing and were not further changed.

Table 3.23 Cognitive Interviewing Question Versions Assessing Criterion A6 for NSDUH

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC21: During the past 12 months, did you have any problems with family or friends that were probably caused by your drinking? [Y/N]	DRALC19 During the past 12 months, did you often have any arguments or other problems with family or friends that were probably caused or made worse by your drinking alcohol use? [Y/N]	DRALC19 During the past 12 months, did you often have arguments or other problems with family or friends that were caused or made worse by your alcohol use? [Y/N]	DPALARGU During the past 12 months, did you often have arguments or other problems with family or friends that were caused or made worse by your alcohol use?

Table 3.23 Cognitive Interviewing Question Versions Assessing Criterion A6 for NSDUH (continued)

Original NSDUH Item	First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC22 [IF DRALC21 = Y]: Did you continue to drink alcohol even though you thought your drinking caused problems with family or friends? [Y/N]	DRALC20 [IF DRALC19 = 1]: Did you continue to drink alcohol even though you thought your drinking it often caused arguments or problems with family or friends? [Y/N]	DRALC20 [IF DRALC19 = 1]: Did you continue to drink alcohol even though it often caused arguments or problems with family or friends? [Y/N]	DPALACNT [IF DPALARGU = 1]: Did you continue to drink alcohol even though it often caused arguments or problems with family or friends?

3.4.2 New Items

New items evaluated for NSDUH included an assessment of the newly added DSM-5 criterion for "craving" (all substances), the newly added DSM-5 criterion for marijuana withdrawal, and cognitive testing to capture DSM withdrawal criterion 11B.

1. Craving (DSM-5 SUD Criterion A4)

The first criterion to be added for assessment in NSDUH was craving, which corresponds with the newly added DSM-5 SUD criterion A4. This criterion was assessed across all substances. The DSM-5 definition for criterion A4 is "craving, or a strong desire or urge to use the substance." The DSM-5 describes craving in several ways. In the criterion tables, it is described as a "strong desire or urge"; in the Introduction text (American Psychiatric Association, 2013, p. 483), it is described as an "intense desire or urge"; and in the Introduction text (American Psychiatric Association, 2013, p. 483), it is also described as having "had such strong urges they could not think of anything else."

There were two main concerns related to the operationalization of this criterion for NSDUH. First, there was no clear consensus as to the "strength" of desire or urge needed to consider a respondent as having a craving. Second, there was concern about the criterion's impact on the overall SUD diagnosis, given the requirement of only two criteria for a diagnosis. The concern was that too low a threshold for the new craving criterion could result in an overestimation of the individual criterion and an overestimation of the larger SUD diagnosis.

Written input on question wording was obtained from external experts and used in the development of draft items that were subsequently tested in cognitive interviewing. In addition to recognizing the challenges of operationalizing the criterion, the reviewers noted that assessing craving should not be predicated on a period of nonuse, because that is not part of the DSM criteria. They also noted that craving is often assessed using a two-item structure, which was incorporated into the items that were tested in cognitive interviewing.

a. 2015 Cognitive Interviewing Results

The draft craving items underwent three rounds of cognitive interviewing. The first two rounds of cognitive interviewing tested the following proposed wording for assessing DSM-5 cravings:

• DRALC23a: During the past 12 months, was there ever a time when you wanted to drink alcohol so much that you couldn't think of anything else? [Y/N]

• DRALC23b [If DRALC23a = N, DK/REF]: During the past 12 months, was there ever a time when you had a strong desire or urge to drink alcohol? [Y/N]

Rounds 1 and 2. Findings from the first two rounds of cognitive interviewing suggested that participants understood the first of the two questions assessing craving as intended. However, in the second round, it was noted that the second in the pair of craving items had a high risk of false positives. Interpreting these results is complicated by the vague and inconsistent description of craving in DSM-5. False positives are of great concern because of the impact they might have on the broader diagnostic results. Because respondents needed to endorse only two items to meet diagnostic criteria for an SUD, it is was critical to prevent over-endorsement of this criterion.

Participants differed in their views of the intensity of question 23b. When participants answered "YES," they tended to indicate that it was because they "really wanted to do it," "looked forward to it," or "really enjoyed it." In addition, participants were asked about the phrase "very strong desire or urge" to drink alcohol. Regardless of how they answered question 23b, about half of the participants indicated that adding the word "very" would not change their answer.

Cognitive Interviewing Round 3. The third round of cognitive interviewing used a revised version of question 23b that omitted the word "desire" to test a simpler version, as well as to determine whether including the word "desire" altered how respondents interpreted the severity of what was being asked:

• DR(DRUG)23a: During the past 12 months, was there ever a time when you wanted to drink alcohol so much that you couldn't think of anything else? [Y/N]

• DR(DRUG)23b [If DRALC23a = N, DK/REF]: During the past 12 months, was there ever a time when you had a strong urge to use [DRUG]? [Y/N]

Findings from the third round of cognitive interviewing suggested that the revised language of "strong urge" rather than "strong desire or urge" improved the reporting on question 23b. When participants explained why they answered "YES," it appeared to fit the definition of craving. When participants answered "NO," their responses seemed consistent with having a desire but not a strong desire or strong urge. The phrase "strong urge" seemed clearer to respondents than "strong desire or urge."

b. Expert Panel Review

In-person expert panel reviewers identified three areas of focus for revising the proposed new NSDUH question assessing this criterion: specific wording concerns, the need to include triggers of craving, and adolescent versus adult considerations.

Specific wording concerns: The experts suggested rewording to include "were there times" (plural) to reduce false positives. There was consensus about this suggested rewording.

Need to include triggers of craving: Some experts noted that the item does not include mention of craving triggers (e.g., around places or people associated with use). Prior 2015 cognitive interviewing indicated that people were considering these situations when answering this question, particularly because craving can occur without use. Although DSM-5 describes how craving can be influenced by cues and triggers, it is not a formal part of the diagnostic criterion; therefore, it was not added as part of the questions.

Adolescent versus adult considerations: Panel members recommended keeping "strong" before the word "urge," particularly when assessing this criterion among adolescents because they tend to experience a lot of urges and desires.

c. Other Surveys' Assessment of Craving

Table 3.24 shows how other studies have worded their assessment of craving. Features of the items used by other nationally representative studies that closely align with the expert panelists' feedback were considered in revised item wording.

Table 3.24 Assessment by Other Surveys of DSM-5 SUD Criterion A4: Craving

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You had such a strong desire or craving to drink that you couldn't think of anything else?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. In your entire life, did you EVER... b. [If YES] Did this happen in the last 12 months? Feel a very strong urge or desire to drink? Want a drink so badly that you couldn't think of anything else?	[Y/N] [Y/N]
NCS-A CIDI	(The next questions are about some other problems you may have had because of drinking.) Was there ever a time in your life when you often felt like drinking so badly that you couldn't stop yourself from drinking or couldn't stop thinking about drinking?	[Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Craving items had already been developed and tested prior to the 2018-2019 cognitive testing rounds. Prior testing in 2015 had developed items that functioned well in Spanish and English and were therefore included unchanged in the 2018-2019 testing. They continued to perform well and remained only slightly changed (the word "time" was changed to "times" per expert feedback).

Table 3.25 shows the changes in the items assessing criterion A4 from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study.

Table 3.25 Cognitive Interviewing Question Versions Assessing Criterion A4 for NSDUH

First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC04 During the past 12 months, were there times when you wanted to drink alcohol so badly that you couldn't think of anything else? [Y/N]	DRALC04 During the past 12 months, were there times when you wanted to drink alcohol so badly that you couldn't think of anything else? [Y/N]	DPALBDLY During the past 12 months, were there times when you wanted to drink alcohol so badly that you couldn't think of anything else?
DRALC05 [IF DRALC04 = 2 OR DK/REF] During the past 12 months, were there times when you had a strong urge to drink alcohol?	DRALC05 [IF DRALC04 = 2 OR DK/REF] During the past 12 months, were there times when you had a strong urge to drink alcohol?	DPALURGE [IF DPALBDLY = 2 OR DK/REF] During the past 12 months, were there times when you had a strong urge to drink alcohol?

2. Marijuana Withdrawal (Criterion A11a)

The second newly added criterion for assessment in NSDUH is marijuana withdrawal, which corresponds with DSM-5 criterion A11a. Although withdrawal was assessed across other substances in DSM-IV and in NSDUH, the marijuana withdrawal criterion is new to DSM-5 and was not previously assessed in NSDUH. The DSM-5 definition for marijuana withdrawal is slightly different from the withdrawal definition for most other substances, which have separate lists of physical and mental symptoms. The DSM-5 criterion for marijuana withdrawal requires endorsement of three or more of the following symptoms occurring within approximately 1 week of cessation of heavy and prolonged marijuana use: irritability/anger/aggression; nervousness/anxiety; sleep difficulty (insomnia or disturbing dreams); decreased appetite or weight loss; restlessness; depressed mood; or a physical symptom that causes significant discomfort: abdominal pain, shakiness/tremors, sweating, fever, chills, or headache.

a. 2015 Cognitive Interviewing Results

The new NSDUH item for marijuana withdrawal was developed based on the format of the withdrawal items for other substances of abuse. Before convening the 2017 in-person expert panel, draft marijuana withdrawal items underwent three rounds of cognitive interviewing.

Round 1. The following questions were used for testing in the first round of cognitive interviewing:

• DRMJ11a. Please look at the symptoms listed below. During the past 12 months, did you have any of these symptoms after you cut down or stopped using marijuana or hashish? [Pain in the stomach area; Shaking or tremors; Sweating; Fever; Chills; Headache]

• 1DRMJ11b. During the past 12 months, did you have [IF DRMJ11a = Yes (i.e., endorsed experiencing 1 or more of the listed symptoms after cutting down or stopping use of marijuana or hashish in the past 12 months), then fill 2; IF DRMJ11a = No, DK/REF (i.e., did not endorse that they experienced any of the listed symptoms after cutting down or stopping use of marijuana or hashish in the past 12 months), then fill 3] or more of these symptoms after you cut down or stopped using marijuana or hashish? [Feeling irritable or angry; Feeling anxious; Having trouble sleeping; Losing your appetite or losing weight without trying to; Feeling like you couldn't sit still; Feeling depressed]

Findings from the first round of cognitive interviewing suggested that question DRMJ11a appeared to work well. For question DRMJ11b, however, it was unclear whether symptoms for some participants were a result of withdrawal because they indicated that they had some of these symptoms already and that using marijuana/hashish helped alleviate those symptoms.

Rounds 2 and 3. The following revised questions were used for testing in the second and third rounds of cognitive interviewing:

• DRMJ11a. Please look at the symptoms listed below. During the past 12 months, did you have any of these symptoms after you cut down or stopped using marijuana or hashish? [Stomach ache, Shaking or tremors, Sweating, Fever, Chills, Headache]

• DRMJ11b. During the past 12 months, did you have [IF DRMJ11a = Yes then fill 2; IF DRMJ11a = No, DK/REF, then fill 3] or more of these symptoms after you cut down or stopped using marijuana or hashish? [Feeling irritable or angry, Feeling anxious or nervous, Having trouble sleeping, Losing your appetite or losing weight without trying to, Feeling like you couldn't sit still, Feeling depressed]

In Rounds 2 and 3, participants were specifically probed on whether they experienced these symptoms as a result of cutting down or stopping marijuana use. All participants agreed symptoms were from cutting back or stopping use. Participants were also asked whether these symptoms were "withdrawal" symptoms or something else. All participants agreed that they were withdrawal symptoms and that "withdrawal" was the correct term for these feelings or experiences. The participants had no difficulty with these items. It may be that respondents had some of these symptoms prior to using marijuana but that participants considered these symptoms withdrawal when they appeared after quitting or stopping. Finally, the use of boldface type fixed the problem of participants properly counting symptoms.

b. Expert Panel Review

Expert panelists identified four areas of focus for developing and revising the new NSDUH questions assessing this criterion: recommendation against having respondents count symptoms, assessment of symptoms occurring after heavy use, wording around withdrawal, and emerging marijuana delivery systems.

Recommendation against having respondents count symptoms: The panelists repeated their earlier recommendation to avoid requiring respondents to count their symptoms in order to respond to the question.

Assessment of symptoms occurring after heavy use: The draft NSDUH question does not establish that symptoms needed to occur after a heavy period of use as the criterion requires. The panelists generally preferred using a modification of the AUDADIS question for this part of the criterion by adding "after a lot of use" and "in the first few days" to the question. However, the timing for withdrawal symptoms can vary by half-life of the substance (e.g., marijuana's half-life is approximately 3 to 7 days), so modifications for marijuana items were needed.

Wording around withdrawal: Panelists suggested that the wording of the questions needed to be such that the respondent would not need to understand the symptoms of withdrawal. Thus, easy-to-understand words should be used to describe the symptoms of withdrawal instead of asking respondents about "withdrawal" specifically.

Emerging marijuana delivery systems: The experts discussed whether the instrument would be able to assess marijuana use disorders associated with emerging forms of marijuana such as edibles, vaping, and synthetics. They acknowledged that it can be difficult to develop questions that will keep up with and assess these new forms of use without requiring frequent modification, which may introduce measurement bias into the assessment of trends over time.

c. Other Surveys' Assessment of Marijuana Withdrawal

Table 3.26 shows how other studies have worded their assessment of marijuana withdrawal. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording.

Table 3.26 Assessment by Other Surveys of DSM-5 Marijuana Withdrawal

Other Survey	Question Wording	Answer Choices
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You experienced withdrawal symptoms like trouble sleeping, emotional problems, restlessness, sweating or nausea when you tried to cut down your use?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	The next few questions are about the bad aftereffects that people may have when the effects of a medicine or drug are wearing off. This includes the morning after using it or in the first few days after stopping. a. Did you EVER… b. [If YES] Did this happen in the last 12 months? Sleep more than usual (when the effects of a medicine or drug were wearing off)? Feel weak or tired? You just mentioned that you had SOME bad aftereffects when stopping or cutting down on your use of medicines or drugs in the last 12 months. Did at least 2 of these experiences happen around the same time DURING the last 12 months?	[Y/N] [Y/N] [Y/N]

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Questions for marijuana withdrawal were modeled on the first round cognitive interviewing version of the withdrawal question used for other substances but modified for marijuana-specific symptoms. This included not avoiding the term "withdrawal," because the 2015 cognitive interviewing had already shown that the word "withdrawal" did not introduce concerns over stigma or comprehension. Table 3.27 shows that no further changes were made to the items assessing criterion A11a for marijuana use disorder throughout cognitive testing and from the original NSDUH version through the final version for implementation in the 2020 Clinical Validation Study. Cognitive testing found no concerns specific to marijuana withdrawal that were not also observed for the withdrawal items for other substances (see Section 3.4.1.5). Therefore, the final version is similar to other substances in formatting and language (with slight modification for the marijuana-specific withdrawal symptoms).

Table 3.27 Cognitive Interviewing Question Versions Assessing Criterion A11a for NSDUH

First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRMJ22a People may experience withdrawal symptoms when they use less or stop using marijuana or hashish. During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?	DRMJ22a People may experience withdrawal symptoms when they use less or stop using marijuana or hashish. During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?	DRMJ22a People may experience withdrawal symptoms when they use less or stop using marijuana or hashish. During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?
Feeling irritable or angry [Y/N] Feeling anxious or nervous [Y/N] Having trouble sleeping [Y/N] Losing your appetite or losing weight without trying to [Y/N] Feeling like you couldn't sit still [Y/N] Feeling depressed [Y/N]	Feeling irritable or angry [Y/N] Feeling anxious or nervous [Y/N] Having trouble sleeping [Y/N] Losing your appetite or losing weight without trying to [Y/N] Feeling like you couldn't sit still [Y/N] Feeling depressed [Y/N]	Feeling irritable or angry [Y/N] Feeling anxious or nervous [Y/N] Having trouble sleeping [Y/N] Losing your appetite or losing weight without trying to [Y/N] Feeling like you couldn't sit still [Y/N] Feeling depressed [Y/N]
DRMJ22b [IF DRMJ22a=1] During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?	DRMJ22b [IF DRMJ22a=1] During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?	DRMJ22b [IF DRMJ22a=1] During the past 12 months, did you have the following withdrawal symptoms after you used less or stopped using marijuana or hashish for a while?
Stomach ache [Y/N] Shaking or tremors [Y/N] Sweating [Y/N] Fever [Y/N] Chills [Y/N] Headache [Y/N]	Stomach ache [Y/N] Shaking or tremors [Y/N] Sweating [Y/N] Fever [Y/N] Chills [Y/N] Headache [Y/N]	Stomach ache [Y/N] Shaking or tremors [Y/N] Sweating [Y/N] Fever [Y/N] Chills [Y/N] Headache [Y/N]

3. Withdrawal Criterion B (DSM-5 Criterion A11b)

Although withdrawal criterion B was listed as a DSM-IV SUD criterion, NSDUH has not previously included items to assess this criterion. Withdrawal criterion B corresponds with DSM-5 criterion A11b, which states that "The same (or closely related) substance is taken to relieve or avoid withdrawal symptoms." As part of the formative research before convening the in-person expert panel in 2017, draft items to assess withdrawal criterion B were developed and underwent three rounds of cognitive interviewing.

a. 2015 Cognitive Interviewing Results

Round 1. Two items underwent cognitive testing in round one:

• DRMJX1 [IF DRMJ11a=1 OR DRMJ11b=1]: You just mentioned that you experienced symptoms after you cut down or stopped using marijuana or hashish. During the past 12 months, did you use marijuana or hashish, or any illegal substance to avoid or get over these symptoms?

• DRMJX2 [IF (DRMJ11a=2 OR DK/REF) AND (DRMJ11b=2 OR DK/REF)]: During the past 12 months, did you use marijuana or hashish, or any illegal substance to avoid these symptoms?

Findings from the first round of cognitive interviewing suggested that there was a high number of false positives for item DR(DRUG)X1. Reasons for false positives included the following: The respondent used a substance that was not pharmacologically similar, the respondent used another substance to continue to get high (as opposed to avoiding or getting over withdrawal symptoms), or the respondent was thinking of symptoms that were not a result of withdrawal.

Most respondents correctly answered question DR(DRUG)X2. However, most indicated that they found the question confusing. For alcohol, some participants seemed to be thinking about hangover symptoms instead of withdrawal symptoms.

Round 2. The second round of cognitive interviewing tested the following revised versions of the two questions. In addition, participant instructions were added to press F2 to see the list of symptoms.

• DR(DRUG)X1: You just mentioned that you had symptoms after you cut down or stopped [using DRUG]. During the past 12 months, did you use [DRUG] again, methamphetamine, prescription stimulants, or any illegal substance to avoid or get over these symptoms?

• DR(DRUG)X2: After you cut down or stopped [using DRUG] during the past 12 months, did you use [DRUG] again or any illegal substance to avoid these symptoms?

Findings from the second round of cognitive interviewing suggested that most respondents appeared to answer question DR(DRUG)X1 correctly. It was sometimes hard for participants to determine whether they had used another substance to get over withdrawal symptoms or simply to get high again. When asked what symptoms they were thinking of, most said they were thinking of "general withdrawal symptoms," not the specific symptoms listed for each substance. Some respondents who used multiple substances seemed to think of all the withdrawal symptoms they might have experienced, not just those associated with the specific substance being reported. Some respondents received the question in error because they indicated that they had had alcohol withdrawal symptoms, when the symptoms were really a side effect of drinking (e.g., hangover).

Most participants answered "NO" to the tested revised version of question DR(DRUG)X2. Some respondents appeared to misunderstand the question. For example, they based their answers on whether they used the substances mentioned and not on the use of these substances to avoid symptoms.

Round 3. The third round of cognitive interviewing tested the following re-revised versions of the two question items:

• DR(DRUG)X1: You just mentioned that you had symptoms after you cut down or stopped [using DRUG]. Did you use [DRUG] again, [drug list], or any illegal substance to avoid or get over these symptoms?

• DR(DRUG)X2: After you cut down or stopped [using DRUG], did you use [DRUG] again or any illegal substance to prevent these symptoms?

Findings from the third round of cognitive interviewing suggested that question DR(DRUG)X1 appeared to be confusing to respondents. To some, it was not clear if they used a substance to "get over or avoid" withdrawal symptoms. Participants did not consciously decide to use again to get over or avoid a symptom, but this may have contributed to their wanting to get high or use again.

Like the second round of cognitive interviewing, most participants answered "NO" to question DR(DRUG)X2, and most respondents found these items confusing. A small number of participants answered "YES" incorrectly for two main reasons: They thought it was asking if they had experienced any of the symptoms listed, or they answered "YES" because they had used the substance again.

b. Expert Panel Review

In 2017, expert panelists were asked to review the draft items used in the final round of the 2015 cognitive interviewing. Panelists identified three areas of focus for revising the proposed new NSDUH questions assessing the withdrawal criterion B: specific wording concerns, the importance of assessing pharmacologically similar substances, and adolescent versus adult considerations.

Specific wording concerns: The experts came to a consensus about suggested rewording to include "were there times" (plural) to reduce false positives. Several panelists noted that the questions were worded in a confusing manner. Some disliked the use of the word "symptoms" and suggested replacing it with "aftereffects."

Importance of assessing pharmacologically similar substances: Consistent with feedback from written reviews provided by external experts in 2015, the in-person experts pointed out that the substance used to ward off withdrawal symptoms must be pharmacologically similar to the substance being assessed. The item includes "or any illegal substance," which does not require that a similar substance be used. Thus, "or any illegal substance" was removed.

Adolescent versus adult considerations: Experts discussed whether respondents might get confused over the difference between using to avoid withdrawal and using to get high again and whether this would pertain mostly to adolescents or perhaps some adults as well.

c. Other Surveys' Assessments

Table 3.28 shows how other studies have worded their assessment of marijuana withdrawal. Features of the items used by other nationally representative studies that closely aligned with the expert panelists' feedback were considered in revised item wording.

Table 3.28 Assessment by Other Surveys of DSM-5 Withdrawal Criterion B

Other Survey	Question Wording	Answer Choices	Comments
NMHS CIDI	In answering the next questions, think of the one year in your life when your use of alcohol interfered most with your life. During that year, how often did you have each of the following problems? You continued to use in order to avoid having withdrawal symptoms?	4-7 days a week, 2-3 days a week, 2-4 days a month, monthly or less, never
AUDADIS-5	a. Did you EVER… b. [If YES] Did this happen in the last 12 months? Take a drink or use any drug or medicine, other than aspirin, Advil or Tylenol, to GET OVER any of the bad aftereffects of drinking? Take a drink or use any drug or medicine, other than aspirin, Advil or Tylenol, to KEEP FROM having any of these bad aftereffects of drinking?	[Y/N] [Y/N]
NCS-A CIDI	Did you ever drink to keep from having problems like these? NOTE: This is a follow-up to withdrawal criterion A question: People who all of a sudden cut down or stop drinking may not feel well. These feelings are much stronger and can last longer than the usual hangover. Did you ever get tired or have headaches, diarrhea, the shakes, or emotional problems when you stopped, cut down, or went without drinking?		This question is only asked if respondent answered NO to the previous question that assesses withdrawal criterion A.

AUDADIS-5 = Alcohol Use Disorder and Associated Disabilities Interview Schedule 5; NCS-A CIDI = National Comorbidity Survey-Adolescents, Composite International Diagnostic Interview; NMHS CIDI = National Mental Health Survey, Composite International Diagnostic Interview.

d. 2018-2019 Cognitive Testing

Although prior versions for withdrawal criterion A11b had been developed in the last round of revisions that were part of item development activities conducted in 2015, they did not perform well during cognitive testing. Therefore, new versions were developed for deployment in the 2018-2019 cognitive testing. The new version combined feedback from the prior 2015 cognitive interviewing and the 2018 expert panel. The result was the development of an item that performed well in the first round of English cognitive interviewing and in Round 2 Spanish. As a result, the final version included only a small grammatical change (Table 3.29).

Table 3.29 Cognitive Interviewing Question Versions Assessing Criterion A10b for NSDUH

First Round Cognitive Interviewing	Second Round Cognitive Interviewing	Final Version
DRALC23 During the past 12 months, did you use alcohol or another drug to get over or avoid having alcohol withdrawal symptoms? [Y/N]	DRALC23 During the past 12 months, did you use alcohol or another drug to get over or avoid having alcohol withdrawal symptoms? [Y/N]	DPALOVER During the past 12 months, did you use alcohol or another drug to get over or avoid having alcohol withdrawal symptoms?
DRALC24 [IF DRALC23=1] Did you use any of the following to get over or avoid having alcohol withdrawal symptoms during the past 12 months? DRALC24_1 Alcohol [Y/N] DRALC24_2 Prescription sedatives, tranquilizers, sleeping pills, or downers DRALC24_3 Something else [Y/N]	DRALC24 [IF DRALC23=1] Did you use any of the following to get over or avoid having alcohol withdrawal symptoms during the past 12 months? DRALC24_1 Alcohol [Y/N] DRALC24_2 Prescription sedatives, tranquilizers, sleeping pills, or downers DRALC24_3 Something else [Y/N]	DPALUSE [IF DPALOVER=1] Which Did you use any of the following did you use to get over or avoid having alcohol withdrawal symptoms during the past 12 months? DPALUSE _1 Alcohol DPALUSE _2 Prescription sedatives, tranquilizers, sleeping pills, or downers DPALUSE _3 Something else
DRALC25 [IF DRALC24_3=1] You indicated that you took something else to get over or avoid having alcohol withdrawal symptoms during the past 12 months. What did you take?	DRALC25 [IF DRALC24_3=1] You indicated that you took something else to get over or avoid having alcohol withdrawal symptoms during the past 12 months. What did you take?	DPALOTH [IF DPALUSE_3=1] You indicated that you took something else to get over or avoid having alcohol withdrawal symptoms during the past 12 months. What did you take?

3.5 Validation

3.5.1 Importance

During the discussion of each of the criterion items, panelists emphasized the importance of gathering data to assess the validity of the SUD questions at the criterion level and at the disorder level. The panelists agreed that the overall construct of SUD might be valid even when individual SUD criteria do not have perfect agreement. Conversely, they noted that although the overall goal may not be to validate the individual diagnostic criteria, it may be difficult to get a good measure of SUD if there is not good criterion agreement, because only two criteria are needed to meet criteria for an SUD diagnosis. Moreover, identifying problems with the SUD construct might require examining agreement at the criterion level.

3.5.2 Mode

The interview mode used for the NSDUH SUD questions should be kept as similar as possible to the mode used in the full administration of NSDUH. It was also suggested that the order of the two assessments (NSDUH and the clinical interview) be randomly assigned to reduce the attenuation bias (i.e., when people recognize that "YES" responses will incur more follow-up and therefore deny symptoms in follow-ups) typically associated with the assessment being administered second. In NESARC, which included about 700 adult respondents in a study to validate the SUD modules, the AUDADIS questionnaire was administered first, then clinicians used the Psychiatric Research Interview for Substance and Mental Disorders (PRISM) to validate the NESARC items. Attenuation bias (when a respondent provides a response designed to shorten the survey rather than answer the question) makes validation harder because it is unclear which assessment represents the "true" state of the respondent.

3.5.3 Gold Standard

In validation studies, the gold standard is defined as a benchmark representing the best diagnostic tool or procedure available under reasonable conditions. In mental health, the gold standard is considered to be a semi-structured clinical interview administered by a well-trained clinician. With SUD, however, errors may exist in the gold standard because of the use of subjective judgement. This issue has led to the criticism that some "validation" work may not be well-founded. Some experts thus favor using the term "clinical reappraisal" instead of "validation." The experts noted that a lot of people in the field talk about inter-rater reliability instead of validity because there exists no true gold standard. That is, if two psychologists assess SUD in the same respondent, they might agree only a percentage of the time. Panelists noted that a semi-structured clinician-administered tool may be a good way to validate a construct but that it has its own shortcomings. Thus, the panelists agreed on the importance of gathering information using multiple, often correlated, measures that look for convergence. Regardless of which clinical interview is used to make determinations about true/false positives and true/false negatives, it is essential that the clinical interviewers understand the diagnostic criteria well. Therefore, high-quality and essential training is critical. That is, although experts indicated that there were errors in gold standards for measuring mental disorders, they also indicated that a clinical validation/reappraisal study should be done and should use a semi-structured clinician-administered tool and well-trained clinicians.

3.5.4 Age

In NSDUH, all respondents aged 12 or older were asked the same questions to assess SUDs. Experts were asked about the suitability and availability of clinical interviews that assess SUDs that would be appropriate for administration to adolescent respondents, as well as clinical interviews that would be appropriate for all respondents aged 12 or older. The experts discussed using the same clinical interview with adults and adolescents and noted that, in general, the decision depends on whether the instrument being used has been previously validated in adult and adolescent samples. The panelists noted that SCID-IV was administered to adolescents as young as age 12, but that the Structured Clinical Interview for DSM-5 (SCID-5) has not been validated among adolescents. Like SCID-5, the PRISM's assessment of SUD criteria in DSM-5 has not been used on samples of adolescents as young as age 12. It was noted that parental reporting is usually considered an important addition for clinical validation. However, in the context of substance use, there may be little gain in SUD assessment from including parental reporting because parents are typically bad reporters of their child's substance use (Fisher et al., 2006). Thus, there are limited options currently available for using a single clinical validation measure in adolescents and adults.

3.5.5 Language

Follow-up with the Spanish-language SUD expert indicated that it is always good to validate Spanish-language instruments, especially if the questions will be administered to a large sample of Spanish-speaking adult and youth respondents. However, if resources are limited, then it is particularly important to validate among Spanish-speaking adults. The expert noted that most Latino youths will have enough command of the English language within a year of moving to the United States and (based on prior experience) will choose to use the English-language version of the survey. Spanish-speaking adults, however, have a more variable command of the English language, with about 40 percent not possessing sufficient English skill to use the English-language version of the survey.⁰ If resources are not available for any Spanish validation, prior experience suggests that it would still be better to include an unvalidated instrument rather than not assess the Spanish-speaking population at all. This is because when the instrument performs well in English and a good translation is provided, the Spanish version of the instrument generally performs almost as well as the English source version. The Spanish-language SUD expert noted a willingness to review any translations to assist in this process. There is a Spanish version of the SCID-IV, but it was initially unclear if a Spanish version of the SCID-5 exists. The lead developer of the SCID-5, later confirmed that a Spanish version of the SCID-5 is in development.

After the conversation with the Spanish-language SUD expert, the numbers and percentages of total respondents, adult respondents, and youth respondents in the 2014 NSDUH who completed the Spanish-language version of the questionnaire and had past AUD, drug use disorder (DUD), and SUD (Table 3.30) were compiled. Among total respondents, 4.6 percent completed the Spanish-language version of the questionnaire, which included 4.8 percent of all adult respondents and 2.4 percent of all youth respondents. About 0.2 percent of NSDUH respondents took the survey in Spanish and had AUD, and less than 0.1 percent took the survey in Spanish and had DUD. This suggests that Spanish-language validation would affect a small number of, primarily adult, NSDUH participants annually. Moreover, the small number of Spanish-speaking respondents who have SUDs would make sample sizes for validation challenging to obtain.

Table 3.30 Spanish-Language Respondents in the 2014 NSDUH, by Age and AUD, DUD, and SUD (Drug or Alcohol Use Disorder) Status: Unweighted n, Weighted N (in Thousands), Percentages, and Standard Errors

Instrument/ Module	Total (n=67,901)				Adults 18 or Older (n=50,894)				Youths 12 to 17 (n=17,007)
	Unweighted n	Weighted N (Thousands)	%	SE	Unweighted n	Weighted N (Thousands)	%	SE	Unweighted n	Weighted N (Thousands)	%	SE
Spanish NSDUH	2,500	12,066	4.6	0.17	2,100	11,479	4.8	0.19	400	587	2.4	0.20
Spanish NSDUH, AUD	100	521	0.2	0.03	100	494	0.2	0.03	<100	27	0.1	0.04
Spanish NSDUH, DUD	<100	124	<0.1	0.01	<100	94	<0.1	0.01	<100	30	0.1	0.04
Spanish NSDUH, SUD	200	579	0.2	0.03	100	536	0.2	0.03	<100	43	0.2	0.05

AUD = alcohol use disorder; DUD = drug use disorder; SE = standard error; SUD = substance use disorder.

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Survey on Drug Use and Health, 2014.

3.6 Other Topics

3.6.1 Emerging Drugs and Modes of Use

As part of the 2017 expert panel meeting, panelists were encouraged to discuss any additions or deletions of NSDUH items that might be appropriate for consideration, even beyond SUD assessment. Several panel members advocated for adding questions to NSDUH on emerging modes of use or abuse (e.g., injection, edibles, vaping) and emerging drugs (e.g., synthetic cannabinoids). They noted that NSDUH lags behind other drug monitoring data sources, such as the National Forensic Laboratory Information System, when emerging drugs are not included in the survey. It was also discussed that NSDUH may not be the most appropriate forum for collecting data on emerging drugs and modes of use. Adding questions or items to the survey takes time. Emerging drugs like synthetic cannabinoids present several measurement challenges: Respondents may not be able to identify what drug they took or may not be aware of what they took, and synthetics are changing so rapidly in response to changing regulations that questions about specific drugs will be outdated quickly. Adding questions alters the context in which other questions are asked and may affect responses to those questions, thereby disrupting measures of trends across time. One panelist recommended that data be collected on the specific drugs in a given drug class (e.g., hallucinogens) that the respondent has used in the past year, which would allow data users to be able to determine which specific drugs within the class were involved in SUD for that class. For example, researchers would be able to determine (1) among people with a past year hallucinogen use disorder, what percentage had been LSD users in the past year, and (2) among past year LSD users, what percentage also had a past year hallucinogen use disorder.

3.6.2 Prescription Drugs

The 2017 panel members noted that it would be helpful to understand what proportion of the population is getting prescriptions from multiple doctors.

3.6.3 Medical Marijuana

The 2017 experts discussed whether the distinction between medical and nonmedical marijuana use still works in the age of marijuana legalization. Some of the panelists noted that health care providers might recommend marijuana use without actually prescribing it. This could explain why a fair number of NSDUH respondents report the use of medical marijuana even though they live in states without legal access. One panelist expressed interest in gathering data to determine the proportion of medical marijuana users who used marijuana nonmedically before using it medically.

3.6.4 Tobacco

The 2017 experts believed that applying the use disorder criteria to tobacco is very different from applying the criteria to other substances. For example, if you smoke regularly, you probably meet tobacco use disorder criteria by virtue of meeting the craving criterion and by meeting the criterion for spending a lot of time obtaining or using the substance (i.e., nicotine's effect does not last long, so many end up chain-smoking). The panelists noted that it would be interesting to understand how the estimates of tobacco/nicotine use disorder have changed over time because smoking policies have made use more restrictive. Panelists were not certain if evaluating tobacco use disorder using the DSM criteria would "catch on" because the field is "wedded" to using the Fagerström test (Heatherton, Kozlowski, Frecker, & Fagerström, 1991) to assess tobacco-related problems (i.e., symptoms of dependence, specifically).

3.6.5 Inhalant Use Disorder

In 2017, several experts noted the low prevalence of inhalant use as well as the low prevalence of inhalant use disorder among past year users. This led to a question about the value of assessing inhalant use disorder considering the potential respondent burden associated with these questions.

3.6.6 Remission

The topic of remission came up during the 2018 discussion of possible explanations for cases where past year substance use frequency was low but criteria were met for SUD. People who are in remission or recovery from SUD may still be dealing with some symptoms related to a pattern of substance use that occurred more than 12 months ago. Experts felt that it would be helpful to estimate the proportion of individuals in remission from SUD among those receiving treatment and those not receiving treatment. Remission from SUD is characterized by the absence of symptoms (except craving, which may persist) for at least 3 months. In order to assess remission from SUD, it would be necessary to assess lifetime SUDs and the recency of individual symptoms in the past year.

3.6.7 Behavioral Addictions

The 2018 expert panelists noted that under DSM-5, more attention is given to behavioral addictions. For example, gambling disorder was added to DSM-5, and several items are noted for future research in DSM-5 (e.g., gaming addiction, compulsive sexual behavior). One expert noted that questions about gambling were included in NESARC-II. In this setting, many questions were required to assess the gambling disorder criteria. Furthermore, gambling disorder criteria were rarely endorsed. Thus, gambling disorder was not included in the NESARC-III. Panelists concluded that each survey will have to weigh the trade-off between gathering more information and the additional resources needed to do so.

4. Summary

As a result of the release of the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013), the Substance Abuse and Mental Health Services Administration (SAMHSA) investigated updating the substance use disorder (SUD) modules in the National Survey on Drug Use and Health (NSDUH), beginning with a technical review of the diagnostic criteria, followed by review by external experts, item development and/or revision, and then cognitive testing. In reviewing the DSM-5 criteria, several changes in the diagnostic criteria were identified that would need to be addressed, including the addition of craving and marijuana withdrawal. Expert reviewers noted that the current NSDUH SUD assessments had many strengths and commended SAMHSA on its rigor and procedures for developing NSDUH items. At the same time, the experts recognized and identified some opportunities for improvement.

After extensive review of existing data, expert feedback, examination of other survey item wording, and findings from cognitive interviewing, the redesigned SUD modules appeared to be understood by potential NSDUH respondents and ready for validation in the 2020 NSDUH Clinical Validation Study. This was the final test for how well the redesigned SUD modules performed. The 2020 Clinical Validation Study consisted of a clinical reappraisal where NSDUH SUD self-reports were compared with diagnoses obtained by clinically trained interviewers. Concordance was used to assess the validity of the redesigned SUD measures.

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.; DSM-IV). Washington, DC: Author.

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.; text rev.; DSM-IV-TR). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.; DSM-5). Washington, DC: Author.

Center for Behavioral Health Statistics and Quality. (2016a). DSM-5 final report (unpublished internal documentation). Rockville, MD: Substance Abuse and Mental Health Services Administration.

Center for Behavioral Health Statistics and Quality. (2016b). Impact of the DSM-IV to DSM-5 changes on the National Survey on Drug Use and Health. Retrieved from https://www.samhsa.gov/data/

Center for Behavioral Health Statistics and Quality. (2016c). Key substance use and mental health indicators in the United States: Results from the 2015 National Survey on Drug Use and Health (HHS Publication No. SMA 16-4984, NSDUH Series H-51). Retrieved from https://www.samhsa.gov/data/

Chávez, L. M., & Canino, G. (2005). Toolkit on translating and adapting instruments. Retrieved from https://www.hsri.org/files/uploads/publications/PN54_Translating_and_Adapting.pdf 

Fisher, S. L., Bucholz, K. K., Reich, W., Fox, L., Kuperman, S., Kramer, J., ... & Bierut, L. J. (2006). Teenagers are right—Parents do not know much: An analysis of adolescent-parent agreement on reports of adolescent substance use, abuse, and dependence. Alcoholism: Clinical and Experimental Research, 30(10), 1699-1710.

Forsyth, B. H., Lessler, J. T., & Hubbard, M. L. (1992). Cognitive evaluation of the questionnaire. In C. F. Turner, J. T. Lessler, & J. C. Gfroerer (Eds.), Survey measurement of drug use: Methodological studies (pp. 13-52). Washington, DC: U.S. Department of Human Services.

Grant, J. E., Potenza, M. N., Weinstein, A., & Gorelick, D. A. (2010). Introduction to behavioral addictions. American Journal of Drug and Alcohol Abuse, 36(5), 233-241.

Heatherton, T. F., Kozlowski, L. T., Frecker, R. C., & Fagerström, K. O. (1991). The Fagerström test for nicotine dependence: A revision of the Fagerström Tolerance Questionnaire. British Journal of Addiction 86(9), 1119-1127.

Jordan, B. K., Karg, R. S., Batts, K. R., Epstein, J. F., & Wiesen, C. (2008). A clinical validation of the National Survey on Drug Use and Health assessment of substance use disorders. Addictive Behaviors, 33(6), 782-798.

Kessler R. C., & Üstün, T. B. (2004). The World Mental Health (WMH) Survey Initiative version of the World Health Organization (WHO) Composite International Diagnostic Interview (CIDI). International Journal of Methods in Psychiatric Research, 13(2), 93-121.

Krogstad, J. M., Stepler, R., & Lopez, M. H. (2015). English proficiency on the rise among Latinos; U.S. born driving language changes. Washington, DC: Pew Research Center.

Lessler, J. T., & Forsyth, B. H. (1996). A coding system for appraising questionnaires. In N. Schwarz & S. Sudman (Eds.), Answering questions: Methodology for determining cognitive and communicative processes in survey research (pp. 259-291). San Francisco, CA: Jossey-Bass.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159-174. https://doi.org/10.2307/2529310 

Olsen, K., Smyth, J. D., & Ganshert, A. (2019). The effects of respondent and question characteristics on respondent answering behaviors in telephone interviews. Journal of Survey Statistics and Methodology, 7(2), 275-308. https://doi.org/10.1093/jssam/smy006 

Robbins, T. W., & Clark, L. (2015). Behavioral addictions. Current Opinion in Neurobiology, 30, 66-72.

List of Contributors

This report was prepared by the Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Behavioral Health Statistics and Quality, and RTI International (a registered trademark and a trade name of Research Triangle Institute). Work by RTI was performed under Contract No. HHSS283201300001C. Peter Tice served as the Government Project Officer, and David Hunter served as the RTI Project Director.

Contributors to this report at SAMHSA include Grace Medley, Sarra Hedden, and Barbara Forsyth.

Contributors to this report at RTI include Cristie Glasheen, Emily Geisen, Douglas Currivan, Valerie Hoffman (formerly with RTI), Kathy Batts, Elizabeth Foley (formerly with RTI), Debbie Bond, and Claudia Clark.

Appendix A: Module Entry Analyses Tables

Table A.1 Potential Misclassification for DSM-5 Cocaine, Heroin, Hallucinogen, and Inhalant Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older)

Substance Use Frequency	Total %	1 Criterion1,2 %	≥ 2 Criteria1,2 %	Potential Sample Misclassification (1 Criterion)	Sample Misclassified (≥ 2 Criteria)	Total Possible Misclassification %	Possible Sample Misclassification n (in Hundreds)	Population Misclassification N (in Thousands)
Cocaine
1 Day	0.4	5.4	3.7	0.0002	0.0002	0.0004	300	98.8
2 Days	0.3	6.8	6.3	0.0002	0.0002	0.0004	300	109.2
3 Days	0.3	9.0	7.0	0.0002	0.0002	0.0004	300	101.0
4 Days	0.2	11.5	10.6	0.0002	0.0002	0.0004	300	89.3
5 Days	0.2	8.9	15.3	0.0001	0.0002	0.0004	300	97.8
6 Days	0.1	15.6	20.4	0.0001	0.0001	0.0003	200	63.6
7 Days	<0.1	12.6	19.6	0.0001	0.0001	0.0001	<100	32.5
8 Days	<0.1	12.3	16.7	0.0001	0.0001	0.0001	<100	29.3
9 Days	<0.1	*	*	*	*	*	*	*
10 Days	0.1	15.1	20.3	0.0001	0.0002	0.0003	200	80.5
Heroin
1 Day	0.1	7.3	16.3	<0.0001	0.0001	0.0001	<100	29.8
2 Days	<0.1	10.9	14.0	<0.0001	<0.0001	<0.0001	<100	12.6
3 Days	<0.1	*	*	*	*	*	*	*
4 Days	*	*	*	*	*	*	*	*
5 Days	*	*	*	*	*	*	*	*
6 Days	*	*	*	*	*	*	*	*
7 Days	*	*	*	*	*	*	*	*
8 Days	*	*	*	*	*	*	*	*
9 Days	*	*	*	*	*	*	*	*
10 Days	*	*	*	*	*	*	*	*
11 Days	*	*	*	*	*	*	*	*
2-5 Days	0.1	12.0	36.8	0.0001	0.0002	0.0002	200	61.6
6-11 Days	<0.1	7.7	33.5	<0.0001	0.0001	0.0001	<100	31.2
Hallucinogens
1 Day	0.9	4.7	2.1	0.0004	0.0002	0.0006	400	149.4
2 Days	0.5	6.2	2.9	0.0003	0.0002	0.0005	400	119.5
3 Days	0.3	8.3	5.2	0.0003	0.0002	0.0004	300	112.5
4 Days	0.2	12.6	6.2	0.0002	0.0001	0.0004	300	90.2
5 Days	0.2	13.4	7.1	0.0002	0.0001	0.0003	300	88.0
6 Days	0.1	13.0	8.0	0.0001	0.0001	0.0002	100	47.7
7 Days	0.1	26.4	9.1	0.0002	0.0001	0.0002	200	53.8
8 Days	<0.1	15.6	11.6	0.0001	<0.0001	0.0001	<100	27.5
9 Days	<0.1	10.9	16.4	<0.0001	<0.0001	0.0001	<100	20.7
10 Days	0.1	14.4	14.7	0.0001	0.0001	0.0002	200	51.4

Table A.1 Potential Misclassification for DSM-5 Cocaine, Heroin, Hallucinogen, and Inhalant Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older) (continued)

Substance Use Frequency	Total %	1 Criterion1,2 %	≥ 2 Criteria1,2 %	Potential Sample Misclassification (1 Criterion)	Sample Misclassified (≥ 2 Criteria)	Total Possible Misclassification %	Possible Sample Misclassification n (in Hundreds)	Population Misclassification N (in Thousands)
Inhalants
1 Day	0.4	5.3	2.1	0.0002	0.0001	0.0003	200	80.4
2 Days	0.2	9.7	2.7	0.0002	0.0001	0.0003	200	68.9
3 Days	0.2	8.8	4.7	0.0001	0.0001	0.0002	200	54.6
4 Days	0.1	10.6	5.2	0.0001	<0.0001	0.0001	100	35.9
5 Days	0.1	11.1	3.8	0.0001	<0.0001	0.0001	100	14.6
6 Days	<0.1	11.1	4.6	<0.0001	<0.0001	0.0001	<100	15.9
7 Days	<0.1	11.9	13.0	<0.0001	<0.0001	0.0001	<100	18.9
8 Days	<0.1	9.1	5.3	<0.0001	<0.0001	<0.0001	<100	0.7
9 Days	*	*	*	*	*	*	*	*
10 Days	0.1	25.2	4.5	0.0001	<0.0001	0.0001	100	37.5

DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013).

*Not presented because of imprecision of the estimate.

¹ Excludes the DSM-IV (fourth edition; American Psychiatric Association, 1994) legal criterion but does not include craving because NSDUH does not yet collect craving data.

² Percentage of respondents who reported within each level of use.

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2004-2014.

Table A.2 Potential Misclassification for DSM-5 Prescription Pain Reliever, Tranquilizer, Stimulant, and Sedative Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older)

Substance Use Frequency	Total %	1 Criterion1,2 %	≥ 2 Criteria1,2 %	Potential Sample Misclassification (1 Criterion)	Sample Misclassified (≥ 2 Criteria)	Total Possible Misclassification %	Possible Sample Misclassification n (in Hundreds)	Population Misclassification N (in Thousands)
Pain Relievers
1 Day	0.8	6.2	4.5	0.0005	0.0003	0.0008	600	208.1
2 Days	0.6	7.1	4.8	0.0004	0.0003	0.0007	600	186.3
3 Days	0.5	9.0	5.3	0.0005	0.0003	0.0007	500	180.6
4 Days	0.3	10.3	6.2	0.0003	0.0002	0.0005	400	129.2
5 Days	0.4	8.9	8.3	0.0004	0.0003	0.0007	500	182.4
6 Days	0.2	9.4	9.1	0.0002	0.0001	0.0003	200	74.8
7 Days	0.1	11.5	10.5	0.0001	0.0001	0.0003	200	66.7
8 Days	0.1	14.9	11.5	0.0001	0.0001	0.0002	200	60.0
9 Days	<0.1	*	11.5	*	<0.0001	*	*	*
10 Days	0.3	14.8	12.0	0.0004	0.0003	0.0007	500	182.7
Tranquilizers
1 Day	0.4	2.9	1.7	0.0001	0.0001	0.0002	100	45.3
2 Days	0.3	5.5	2.0	0.0002	0.0001	0.0002	200	53.0
3 Days	0.2	5.0	1.4	0.0001	<0.0001	0.0001	100	35.6
4 Days	0.1	5.7	3.3	0.0001	<0.0001	0.0001	<100	29.5
5 Days	0.2	5.2	3.6	0.0001	0.0001	0.0002	100	40.0
6 Days	0.1	7.7	7.7	0.0001	0.0001	0.0001	<100	27.2
7 Days	0.1	3.9	8.5	<0.0001	<0.0001	0.0001	<100	15.7
8 Days	<0.1	13.8	12.0	0.0001	<0.0001	0.0001	<100	26.1
9 Days	<0.1	5.6	5.0	<0.0001	<0.0001	<0.0001	<100	5.3
10 Days	0.1	8.2	4.8	0.0001	0.0001	0.0002	100	39.4
Stimulants
1 Day	0.2	6.4	3.2	0.0001	0.0001	0.0002	200	53.3
2 Days	0.2	4.8	4.0	0.0001	0.0001	0.0001	100	37.8
3 Days	0.1	6.2	5.2	0.0001	0.0001	0.0001	100	37.4
4 Days	0.1	11.9	4.5	0.0001	<0.0001	0.0001	100	37.3
5 Days	0.1	9.4	5.7	0.0001	0.0001	0.0002	100	41.9
6 Days	0.1	6.4	8.1	<0.0001	<0.0001	0.0001	<100	18.3
7 Days	<0.1	7.9	15.5	<0.0001	0.0001	0.0001	<100	23.6
8 Days	<0.1	9.3	*	<0.0001	*	*	*	*
9 Days	<0.1	*	2.8	*	<0.0001	*	*	*
10 Days	0.1	8.4	11.5	0.0001	0.0001	0.0001	100	35.2

Table A.2 Potential Misclassification for DSM-5 Prescription Pain Reliever, Tranquilizer, Stimulant, and Sedative Use Disorder in NSDUH, Based on Unweighted Frequency of Use for the 2004-2014 NSDUHs, All Ages (12 or Older) (continued)

Substance Use Frequency	Total %	1 Criterion1,2 %	≥ 2 Criteria1,2 %	Potential Sample Misclassification (1 Criterion)	Sample Misclassified (≥ 2 Criteria)	Total Possible Misclassification %	Possible Sample Misclassification n (in Hundreds)	Population Misclassification N (in Thousands)
Sedatives
1 Day	0.1	5.3	5.7	<0.0001	<0.0001	0.0001	<100	16.7
2 Days	<0.1	9.4	*	<0.0001	*	*	*	*
3 Days	<0.1	6.3	9.6	<0.0001	<0.0001	<0.0001	<100	12.0
4 Days	<0.1	8.6	*	<0.0001	*	*	*	*
5 Days	<0.1	*	*	*	*	*	*	*
6 Days	*	*	*	*	*	*	*	*
7 Days	*	*	*	*	*	*	*	*
8 Days	*	*	*	*	*	*	*	*
9 Days	*	*	*	*	*	*	*	*
10 Days	*	*	*	*	*	*	*	*
11 Days	*	*	*	*	*	*	*	*
6-11 Days	0.1	9.6	11.4	0.0001	0.0001	0.0001	<100	26.5

DSM-5 = Diagnostic and Statistical Manual of Mental Disorders, fifth edition (American Psychiatric Association, 2013).

*Not presented because of imprecision of the estimate.

¹ Excludes the DSM-IV (fourth edition; American Psychiatric Association, 1994) legal criterion but does not include craving because NSDUH does not yet collect craving data.

² Percentage of respondents who reported within each level of use.

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2004-2014.

Appendix B: Prescription Drug Use but Not Misuse Memo

DSM-5 Substance Use Disorder Assessment among Users of Prescription Drugs
Cristie Glasheen, Mark Edlund, Kathy Batts, Michael Pemberton
August 31, 2018

Purpose

The Substance Abuse and Mental Health Services Administration (SAMHSA) is in the process of completing a redesign of the National Survey on Drug Use and Health (NSDUH). This redesign includes updating the existing substance use disorder (SUD) sections from using criteria based on the fourth edition, text revision, of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR; American Psychiatric Association, 2000) to the fifth edition (DSM‑5; American Psychiatric Association, 2013), including determining which NSDUH respondents should be routed into the SUD sections for each substance.

Currently, those who use but do not misuse (e.g., using without a prescription or using in ways not prescribed) prescription pain relievers (i.e., opioids), prescription stimulants, and prescription sedatives/tranquilizers are not routed into the respective SUD sections. However, the DSM-5 does not preclude having opioid use disorder (OUD), stimulant use disorder, or sedative/tranquilizer use disorder even when the drugs are used solely under appropriate medical supervision. This memo provides a short summary of considerations for changing the current NSDUH routing logic to include all users of prescription pain relievers in the OUD sections.

Specific Drugs

Stimulants

To date, there has been little research into the prevalence of SUD among those who use but do not misuse prescription drugs, with more data available for opioids than for stimulants or sedatives/tranquilizers. For prescription stimulants, Biederman et al. found low liability for stimulant use disorder after 10 years of following young adult males with attention-deficit hyperactivity disorder (ADHD) using medically prescribed stimulants (Biederman et al., 2008). Chang et al. made similar findings, with some evidence of protective effects against subsequent addiction among people with ADHD who use prescription stimulants as directed by a doctor (Chang et al., 2014). This suggests that the prevalence of stimulant use disorder among those who use but do not misuse prescription stimulants should be low, although these findings are not always consistent. Weyandt et al. (2014) conducted a review of literature focusing on college students with ADHD and found that those receiving treatment for ADHD were more likely to misuse stimulants than those without ADHD or those not receiving ADHD treatment. This suggests that prescription stimulant use may increase the risk of stimulant use disorder and increase the risk of starting to misuse prescriptions. However, this does not directly answer the question of whether those who use but do not misuse stimulants can develop stimulant use disorder or whether the development of stimulant use disorder only occurs after the person begins to misuse stimulants. Currently, little data exist to assess because most studies (nationally representative and not) have examined stimulant use disorder among those who misused stimulants, excluding those who used as medically instructed.

Sedatives/Tranquilizers

Prescription sedatives/tranquilizer use disorder among those who use but do not misuse sedatives/tranquilizers are also lacking. Sedatives/tranquilizers is a diverse class of drugs, with differing liability for addiction/dependence (Hajak et al., 2003). A group of sedatives/tranquilizers with a higher liability for addiction are benzodiazepines, which are one of the most commonly prescribed medications in the United States—meaning a larger proportion of individuals are exposed to them (Longo & Johnson, 2000). Benzodiazepines are generally considered safe and have low mortality in the cases of overdose but also come with adverse side effects that become more severe with higher doses and longer time usage. Moreover, they are one of the most commonly involved substances in drug-related emergency department visits nationally (Paulozzi et al., 2015). Prescription guidelines recommend short-term use of benzodiazepines only. However, long-term prescriptions for benzodiazepines are not infrequent, despite these guidelines (Sirdifield et al., 2013). Because many surveys only assess sedative/tranquilizer use disorder among those who report misusing sedatives/tranquilizers, it is unclear if those who use but do not misuse (i.e., those who use as medically instructed) can also display the negative behavioral symptoms associated with SUDs.

Opioids

Opioids (referred to in NSDUH as prescription pain relievers) have the largest body of literature about them, partially because of the severity of potential consequences of OUD and partly because of the frequency in which these drugs are prescribed (Paulozzi et al., 2015). In a study of over 560,000 records of patients in a claim database with chronic non-cancer pain, patients prescribed high-dose chronic opioid use had 122.5 times the odds of developing OUD within 18 months of the initial visit for chronic pain compared with those with no opioid use (Edlund et al., 2014). Given the substantial burden that OUD places on patients for possible overdose and death, as well as the system costs of healthcare use, SUD treatment, and loss of work/productivity (Birnbaum et al., 2011), a better understanding of the prevalence of OUD among users as well as misusers of prescription pain relievers may be valuable.

Prescription pain relievers provide needed pain relief for many patients with a variety of medical conditions, but properly prescribed prescription pain reliever use does not preclude the development of OUD among patients (Vowles et al., 2015). Moreover, studies suggest that OUD is not uncommon among adults on long-term opioid therapy. In a 2015 telephone-administered survey of 705 DSM-5-based, lay-administered interviews with patients on long-term opioid therapy, 28.1 percent had mild OUD symptoms (2-3 symptoms), 9.7 percent had moderate (4-5 symptoms), and 3.5 percent had severe OUD (6-9 symptoms) (Boscarino, Hoffman, & Han, 2015). Although the patients were not assessed specifically for misuse of their prescriptions (e.g., taking more than prescribed), a systematic review of studies between 2000 and 2015 about misuse of prescription pain relievers among chronic pain patients found that, on average, between 21 and 29 percent (range = 2.0-56.3 percent) of chronic pain patients had misused opioids (Vowles et al., 2015).

Currently, data on the prevalence of OUD among those who use but do not misuse prescription pain relievers are sparse. Most studies have not differentiated between appropriate users and misusers or have only included misusers. In a study of about 700 pain patients that utilized latent class analysis to categorize pain patient typologies, among the 500 identified "typical" users, 50 percent met the criteria of wanting to stop or cut down, 8 percent reported opioids interfering with role functioning, and 7 percent reported continuing use despite health problems resulting from opioid use (Banta-Green et al., 2009). This study used a probabilistic modeling of a "typical" pain patient typology; therefore, it does not clearly distinguish use versus misuse but is suggestive of an area where more data are needed. Collecting data on OUD among users but not misusers of prescription pain relievers may provide important information about the risk and burden of OUD in the United States. Not only may individuals have an OUD when using prescription pain relievers as prescribed, but those who are using as prescribed and do not yet have an OUD may be at increased risk for OUD development.

Effect on NSDUH

Changes to NSDUH are made periodically to ensure that the survey is collecting the most useful and timely data as possible to provide for the needs of policy makers, public health practitioners, researchers, and the public. However, changes are made with care so as to avoid disrupting trends. If changes are made to the routing logic, context effects may be introduced in the NSDUH items that follow the SUD sections among those who are newly receiving this section. To reduce the impact of context effects on other variables of interest among this subgroup, changing the routing is ideally introduced during a time when trend breaks are already expected, recognizing that this change may disrupt trends in post-SUD measures. If this were incorporated during the 2025 NSDUH redesign, then the effect on trends would not be a concern because other changes necessitate trend disruption already. SUD estimates would not be expected to be disrupted due to the change in routing because estimates could be generated for opioid misusers and users who are not misusers, separately.

Another consideration for changing the routing logic is that NSDUH uses a fully structured interview format, which limits the amount of additional probing of how and why patients use prescription pain relievers. For example, patients with prescriptions for pain relievers that specify use "as needed" may not report misuse of prescription pain relievers even if their motives for use were not strictly to alleviate pain. As a result of these factors, estimates of SUD in NSDUH may be undercounts because respondents with SUD symptoms who did not self-report that they had misused prescription pain relievers are not assessed. Therefore, changing the routing logic could improve the accuracy of estimates of SUD.

An additional consideration is the effect that adding the SUD sections for users but not misusers of prescription pain relievers may have on the timing of the NSDUH interview.

Stimulants

Analyses using existing NSDUH data suggest that if all stimulant users but not misusers (2015-2016 annual average = 3,726 respondents) were to be routed into the stimulant use disorder section, the overall average NSDUH duration per respondent (among all 68,000 respondents) would increase less than 0.1 minutes. Among the 3,726 respondents each year who are users but not misusers, the average duration of NSDUH would increase 0.9 minutes. There is also a concern about the interview length among older respondents because older adults in general take longer to complete NSDUH. Table B.1 demonstrates the effect of including the stimulant use disorder section among users but not misusers of prescription stimulants by age. Among adults aged 65 or older who use but do not misuse prescription stimulants, the average NSDUH length increases by 2.2 minutes if they are routed into the stimulant use disorder section (< 0.1 minutes average among all 65 or older respondents).

Table B.1 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Stimulants If They Were Routed into the Stimulant Disorder Section, by Age Group: NSDUH 2015-2016, Annual Averages

Age Group	Annual Average Sample Size n (Users but Not Misusers)	Mean Minutes without Stimulant Use Disorder Section	Median Minutes without Stimulant Use Disorder Section	Additional Average Minutes per Respondent (Users but Not Misusers)
12 or Older	3,700	58.71	54.33	0.86
12-17	1000	55.57	52.87	0.90
18-25	1,100	51.75	48.73	0.78
26-34	600	54.03	50.82	0.86
35-49	700	59.08	55.30	1.19
50-64	200	68.49	62.97	1.95
65 or Older	100	78.34	72.90	2.18

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2015-2016.

Sedatives/Tranquilizers

Analyses using existing NSDUH data suggest that if all sedative/tranquilizer users but not misusers (2015-2016 annual average = 7,849 respondents) were to be routed into the sedative/tranquilizer use disorder section, the overall average NSDUH duration per respondent (among all 68,000 respondents) would increase 0.1 minutes. Among the 7,849 respondents who are users but not misusers, the average duration of NSDUH would increase 0.9 minutes. There is also a concern about the interview length among older respondents because older adults in general take longer to complete NSDUH. Table B.2 demonstrates the effect of including the sedative/tranquilizer use disorder section among users but not misusers of prescription pain relievers by age. Among adults aged 65 or older who use but do not misuse prescription pain relievers, the average NSDUH length increases by 2.1 minutes if they are routed into the sedative/tranquilizer use disorder section (0.4 minutes average among all 65 or older respondents).

Table B.2 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Sedatives/Tranquilizers If They Were Routed into the Sedative/Tranquilizer Use Disorder Section, by Age Group: NSDUH 2015-2016, Annual Averages

Age Group	Annual Average Sample Size n (Users but Not Misusers)	Mean Minutes without Sedative/Tranquilizer Use Disorder Section	Median Minutes without Sedative/Tranquilizer Use Disorder Section	Additional Average Minutes per Respondent (Users but Not Misusers)
12 or Older	7,800	63.45	58.17	0.90
12-17	700	59.87	57.27	0.83
18-25	1,300	55.81	52.02	0.69
26-34	1,300	56.19	52.48	0.87
35-49	2,300	61.81	57.17	1.24
50-64	1,300	71.70	65.44	1.71
65 or Older	900	80.71	75.18	2.09

NOTE: Number of sedative and tranquilizer users but not misusers combined (if either was used and both were not misused, then the respondent was counted as a user but not a misuser). Timing estimates used the Tranquilizer module because it currently comes before the Sedative module. Because both modules are the same, it was assumed that any timing differences were due to fatigue.

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2015-2016.

Opioids

Analyses using existing NSDUH data suggest that if all opioid users but not misusers (2015-2016 annual average = 18,130 respondents) were to be routed into the OUD section, the overall average NSDUH duration per respondent (among all 68,000 respondents) would increase 0.36 minutes. Among the 18,130 respondents who are users but not misusers, the average duration of NSDUH would increase 1.4 minutes. There is also a concern about the interview length among older respondents because older adults in general take longer to complete NSDUH. Table B.3 demonstrates the effect of including the OUD section among users but not misusers of prescription pain relievers by age. Among adults aged 65 or older who use but do not misuse prescription pain relievers, the average NSDUH length increases by 2.9 minutes if they are routed into the OUD section (1.0 minutes average among all 65 or older respondents).

Table B.3 Effect on NSDUH Average Interview Duration among Users but Not Misusers of Prescription Pain Relievers If They Were Routed into the Opioid Use Disorder (OUD) Section, by Age Group: NSDUH 2015-2016, Annual Averages

Age Group	Annual Average Sample Size n (Users but Not Misusers)	Mean Minutes without OUD Section	Median Minutes without OUD Section	Additional Average Minutes per Respondent (Users but Not Misusers)
12 or Older	18,100	58.71	54.33	1.4
12-17	2,800	55.57	52.87	1.3
18-25	4,200	51.75	48.73	1.0
26-34	3,100	54.03	50.82	1.3
35-49	4,400	59.08	55.30	1.6
50-64	2,200	68.49	62.97	2.2
65 or Older	1,500	78.34	72.90	2.9

Source: SAMHSA, Center for Behavioral Health Statistics and Quality, National Surveys on Drug Use and Health, 2015-2016.

Summary

Currently, estimates of respective SUDs among those who use but do not misuse prescription drugs are not assessed by most studies, and nationally representative studies are unavailable. This is potentially a significant data gap because it leaves unclear the burden of SUD among these populations. Moreover, given the current opioid crisis in the United States, it is critical that OUD estimates from NSDUH are as complete and accurate as possible. Changing the NSDUH routing logic to provide the capability of producing estimates of SUD among those who use but do not misuse prescription drugs could increase the confidence in and usability of the NSDUH data for surveillance in the population. However, this change would result in an increase in survey burden for some NSDUH respondents, particularly among older adults who already have a relatively long mean response time and may experience more respondent fatigue. This could be further explored in the field test by examining whether there is an increase in breakoffs and missing data among older adults.

References

American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed.; text rev.; DSM-IV-TR). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.; DSM-5). Washington, DC: Author.

Banta-Green, C. J., Merrill, J. O., Doyle, S. R., Boudreau, D. M., & Calsyn, D. A. (2009). Opioid use behaviors, mental health and pain—Development of a typology of chronic pain patients. Drug and Alcohol Dependence, 104(1), 34-42.

Biederman, J., Monuteaux, M. C., Spencer, T., Wilens, T. E., MacPherson, H. A., & Faraone, S. V. (2008). Stimulant therapy and risk for subsequent substance use disorders in male adults with ADHD: A naturalistic controlled 10-year follow-up study. American Journal of Psychiatry 165(5), 597-603.

Birnbaum, H. G., White, A. G., Schiller, M., Waldman, T., Cleveland, J. M., & Roland, C. L. (2011). Societal costs of prescription opioid abuse, dependence, and misuse in the United States. Pain Medicine, 12(4), 657-667.

Boscarino, J. A., Hoffman, S. N., & Han, J. J. (2015). Opioid-use disorder among patients on long-term opioid therapy: Impact of final DSM-5 diagnostic criteria on prevalence and correlates. Substance Abuse and Rehabilitation, 6, 83-91.

Chang, Z., Lichtenstein, P., Halldner, L., D'Onofrio, B., Fazel, S., Langstrom, N., & Larsson, H. (2014). Stimulant ADHD medication and risk for substance abuse. Journal of Child Psychology and Psychiatry, 55(8), 878-885.

Edlund, M. J., Martin, B. C., Russo, J. E., Devries, A., Braden, J. B., & Sullivan, M. D. (2014). The role of opioid prescription in incident opioid abuse and dependence among individuals with chronic non-cancer pain: The role of opioid prescription. Clinical Journal of Pain, 30(7), 557.

Hajak, G., Müller, W., Wittchen, H.-U., Pittrow, D., & Kirch, W. (2003). Abuse and dependence potential for the non‐benzodiazepine hypnotics zolpidem and zopiclone: A review of case reports and epidemiological data. Addiction, 98(10), 1371-1378.

Longo, L. P., & Johnson, B. (2000). Addiction: Part I. Benzodiazepines-side effects, abuse risk and alternatives. American Family Physician, 61(7), 2121-2128.

Paulozzi, L. J., Strickler, G. K., Kreiner, P. W., & Koris, C. M. (2015). Controlled substance prescribing patterns—Prescription behavior surveillance system, eight states, 2013. Morbidity and Mortality Weekly Report. Surveillance Summaries, 64(9), 1-14.

Sirdifield, C., Anthierens, S., Creupelandt, H., Chipchase, S. Y., Christiaens, T., & Siriwardena, A. N. (2013). General practitioners' experiences and perceptions of benzodiazepine prescribing: Systematic review and meta-synthesis. BMC Family Practice, 14(1), 191.

Substance Abuse and Mental Health Services Administration. (2010). Reliability of Key Measures in the National Survey on Drug Use and Health. CBHSQ Methodology Report. Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration, Rockville, MD.

Vowles, K. E., McEntee, M. L., Julnes, P. S., Frohe, T., Ney, J. P., & van der Goes, D. N. (2015). Rates of opioid misuse, abuse, and addiction in chronic pain: A systematic review and data synthesis. Pain, 156(4), 569-576.

Weyandt, L. L., Oster, D. R., Marraccini, M. E., Gudmundsdottir, B. G., Munro, B. A., Zavras, B. M., & Kuhar, B. (2014). Pharmacological interventions for adolescents and adults with ADHD: Stimulant and nonstimulant medications and misuse of prescription stimulants. Psychology Research and Behavior Management, 7, 223-249.

Appendix C: 2020 Clinical Validation Study NSDUH Alcohol Use Disorder Module

English

DPALINT [IF AL12MON2 = 1] Think about your use of alcohol during the past 12 months as you answer these next questions.

Press [ENTER] to continue.

(IF AL12MON2 = 2, SKIP TO DPMJINT)

DPALFEEL During the past 12 months, did you spend a great deal of your time drinking alcohol, feeling its effects, or getting over the effects of drinking?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALGET [IF DPALFEEL = 2 OR DK/REF] During the past 12 months, did you spend a great deal of your time getting or trying to get alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALLRGR During the past 12 months, were there many times when you ended up drinking alcohol in larger amounts or for a longer time than you meant to?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALBDLY During the past 12 months, were there times when you wanted to drink alcohol so badly that you couldn't think of anything else?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALURGE [IF DPALBDLY = 2 OR DK/REF] During the past 12 months, were there times when you had a strong urge to drink alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALMORE Do you need to drink a lot more alcohol than you used to in order to get the feeling you want?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALLESS [IF DPALMORE = 2 OR DK/REF] Does drinking the same amount of alcohol have much less effect on you than it used to?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALSTOP During the past 12 months, did you try to cut down or try to stop drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALCANT [IF DPALSTOP = 1] Some people who drink alcohol try to cut down or stop but find they can't. Was there more than one time in the past 12 months when you tried but were unable to cut down or stop drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALWISH [IF DPALSTOP = 2 OR DK/REF] In the past 12 months, did you often wish that you could cut down or stop drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALPHYS During the past 12 months, did you have any long-lasting or repeated physical health problems that were caused or made worse by drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALPCNT [IF DPALPHYS = 1]: Did you continue to drink alcohol even though it was causing long-lasting or repeated physical health problems or making your physical health problems worse?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALMNTL [IF DPALPHYS = 2 OR DK/REF OR DPALPCNT = 2 OR DK/REF] During the past 12 months, did you have any long-lasting or repeated problems with emotions or mental health that were caused or made worse by drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALMCNT [IF DPALMNTL = 1]: Did you continue to drink alcohol even though it was causing long-lasting or repeated problems with your emotions or mental health or making your emotions or mental health worse?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALBLCK [IF (DPALPHYS = 2 OR DK/REF OR DPALPCNT = 2 OR DK/REF) AND (DPALMENT = 2 OR DK/REF OR DPALMCNT = 2 OR DK/REF)] A blackout is lack of memory. That is, you were awake, but you have no recall of the things you did or that were done to you.

During the past 12 months, did you repeatedly have blackouts while drinking or after drinking alcohol?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALBCNT [IF DPALBLCK = 1] Did you continue to drink alcohol even though drinking gave you repeated blackouts?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALACTV This question is about important activities such as:

• Spending time with friends and family

• Attending special events at work or school

• Participating in hobbies and sports

• Attending religious services and events

During the past 12 months, did you give up or spend a lot less time doing any of these types of important activities because of your alcohol use?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALSERI Sometimes people who drink alcohol have serious problems at work, school, or home—such as:

• missing a lot of work or school

• getting demoted, having your hours cut, or losing a job

• not being able to get a job or keep a job

• getting suspended, expelled, or dropping out of school

• failing to take care of family

During the past 12 months, did you have any serious problems like these at work, school, or home because of your alcohol use?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALARGU During the past 12 months, did you often have arguments or other problems with family or friends that were caused or made worse by your alcohol use?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALACNT [IF DPALARGU = 1]: Did you continue to drink alcohol even though it often caused arguments or problems with family or friends?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALHURT During the past 12 months, did you repeatedly get into situations where drinking alcohol increased your chances of getting physically hurt?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALWD People may experience withdrawal symptoms when they drink less or stop drinking alcohol. Withdrawal symptoms are stronger and last longer than a hangover.

During the past 12 months, did you have the following withdrawal symptoms after you drank less or stopped drinking alcohol for a while?

	Yes	No
DPALWD_1 Sweating or feeling that your heart was beating fast	⃝ 1	⃝ 2
DPALWD _2 Having your hands tremble	⃝ 1	⃝ 2
DPALWD _3 Having trouble sleeping	⃝ 1	⃝ 2
DPALWD _4 Vomiting or having an upset stomach	⃝ 1	⃝ 2
DPALWD _5 Seeing, hearing, or feeling things that weren't really there	⃝ 1	⃝ 2
DPALWD _6 Feeling like you couldn't sit still	⃝ 1	⃝ 2
DPALWD _7 Feeling anxious	⃝ 1	⃝ 2
DPALWD _8 Having seizures or fits	⃝ 1	⃝ 2

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALOVER During the past 12 months, did you use alcohol or another drug to get over or avoid having alcohol withdrawal symptoms?

1 Yes

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

PROGRAMMER: DISPLAY IN LOWER RIGHT:

Press [F2] to see these symptoms again.

DPALUSE [IF DPALOVER = 1] Which of the following did you use to get over or avoid having alcohol withdrawal symptoms during the past 12 months?

	Yes	No
DPALUSE _1 Alcohol	⃝ 1	⃝ 2
DPALUSE _2 Prescription sedatives, tranquilizers, sleeping pills, or downers	⃝ 1	⃝ 2
DPALUSE _3 Something else	⃝ 1	⃝ 2

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DPALOTH [IF DPALUSE_3 = 1] You indicated that you took something else to get over or avoid having alcohol withdrawal symptoms during the past 12 months. What did you take?

______________________________________

DK/REF

Spanish

DRALC [IF ALC12MON = 1 - 3] Piense en su consumo de alcohol en los últimos 12 meses al contestar las siguientes preguntas.

Presione [ENTER] para continuar.

(IF ALC12MON = 4, SKIP TO DRMJ)

DRALC01 En los últimos 12 meses, ¿pasó gran parte de su tiempo bebiendo alcohol, sintiendo sus efectos o recuperándose de los efectos de la bebida?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC02 [IF DRALC01=2 OR DK/REF] En los últimos 12 meses, ¿pasó gran parte de su tiempo consiguiendo o tratando de conseguir alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC03 En los últimos 12 meses, ¿hubo muchas veces en que terminó bebiendo alcohol en mayor cantidad o durante un tiempo más prolongado de lo que quería?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC04 En los últimos 12 meses, ¿hubo momentos en los que tenía tantos deseos de beber alcohol que no podía pensar en otra cosa?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC05 [IF DRALC04 = 2 OR DK/REF] En los últimos 12 meses, ¿hubo momentos en los que tuvo un fuerte impulso de beber alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC06 En los últimos 12 meses, ¿necesitó beber mucho más alcohol para obtener la sensación que deseaba que cuando comenzó a beber regularmente?

1 Sí

2 No

3 Nunca bebí alcohol regularmente

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC07 [IF DRALCO6 = 2 OR DK/REF] En los últimos 12 meses, ¿el beber la misma cantidad de alcohol tuvo mucho menos efecto en usted que cuando empezó a beber regularmente?

1 Sí

2 No

3 Nunca bebí alcohol regularmente

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC08 En los últimos 12 meses, ¿trató de reducir la cantidad o dejar de beber alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC9a [IF DRALC08 = 1] En los últimos 12 meses, ¿fue capaz de reducir la cantidad o dejar de beber alcohol cada vez que lo intentó?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC9b [IF DRALC9a = 2 OR DK/REF] ¿Hubo más de una vez en los últimos 12 meses cuando lo intentó, pero no pudo reducir la cantidad ni dejar de beber alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC10 [IF DRALC08=2 OR DK/REF] Aunque no intentó reducir la cantidad ni dejar de beber alcohol en los últimos 12 meses, ¿deseaba con frecuencia que pudiera reducir la cantidad o dejar de beber?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH

DRALC11 En los últimos 12 meses, ¿tuvo problemas de salud física que le duraron mucho o repetidos que fueron causados o empeoraron por el consumo de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH

DRALC12 [IF DRALC11 = 1]: ¿Continuó bebiendo alcohol a pesar de que le estaba causando problemas de salud física que le duraron mucho o repetidos, o empeorándolos?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC13 En los últimos 12 meses, ¿tuvo problemas que le duraron mucho o repetidos con la salud mental que fueron causados o empeoraron por el consumo de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC14 [IF DRALC13 = 1]: ¿Continuó bebiendo alcohol a pesar de que le estaba causando problemas que le duraron mucho o repetidos con su salud mental, o empeorándolos?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC15 La expresión "blackout" en inglés significa pérdida de memoria. Es decir, usted estaba despierto, pero no recuerda las cosas que hizo o que le hicieron.

En los últimos 12 meses, ¿tuvo pérdidas de memoria, es decir, se despertó al día siguiente sin poder recordar algunas de las cosas que sucedieron mientras estaba bebiendo o después de beber alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC16 [IF DRALC15 = 1]: ¿Continuó bebiendo alcohol a pesar de que el beber le causaba pérdidas de memoria?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC17 Esta pregunta se trata de actividades importantes como:

• Pasar tiempo con las amistades y la familia

• Asistir a eventos especiales en el trabajo o la escuela

• Participar en pasatiempos y deportes

• Asistir a servicios y eventos religiosos

En los últimos 12 meses, ¿dejó de hacer o pasó mucho menos tiempo haciendo cualquiera de estos tipos de actividades importantes debido a su consumo de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC18 A veces las personas que beben alcohol tienen serios problemas en el trabajo, la escuela o en el hogar—por ejemplo:

• faltan mucho al trabajo o a la escuela

• les bajan de puesto en el trabajo, ya sea cortándole las horas o perdiendo el empleo

• no pueden conseguir un trabajo o mantener un trabajo

• son suspendidas, expulsadas o abandonan la escuela

• dejan de prestar atención a la familia

En los últimos 12 meses, ¿tuvo problemas serios como estos ya sea en el trabajo, la escuela o en el hogar debido a su consumo de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC19 En los últimos 12 meses, ¿tuvo con frecuencia discusiones u otros problemas con su familia o sus amistades que fueron causados o empeoraron por su consumo de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC20 [IF DRALC19 = 1]: ¿Continuó bebiendo alcohol a pesar de que le causaba con frecuencia discusiones o problemas con su familia o sus amistades?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC21 En los últimos 12 meses, ¿se metió repetidamente en situaciones en las que el consumo de alcohol aumentaba sus posibilidades de lastimarse físicamente?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC22 Las personas que han consumido alcohol de manera excesiva durante mucho tiempo pueden experimentar síntomas de abstinencia cuando beben menos o dejan de beber. Los síntomas de abstinencia son más fuertes y duran más que una resaca o cruda.

En los últimos 12 meses, ¿tuvo los siguientes síntomas de abstinencia después que bebió menos o dejó de beber alcohol por un tiempo?

	Sí	No
DRALC22_1 Sudaba o sentía que su corazón latía rápido	⃝ 1	⃝ 2
DRALC22_2 Le temblaban sus manos	⃝ 1	⃝ 2
DRALC22_3 Tenía problemas para dormir	⃝ 1	⃝ 2
DRALC22_4 Vomitaba o tenía malestar estomacal	⃝ 1	⃝ 2
DRALC22_5 Veía, escuchaba o sentía cosas que no estaban realmente ahí	⃝ 1	⃝ 2
DRALC22_6 Se sentía inquieto	⃝ 1	⃝ 2
DRALC22_7 Se sentía ansioso	⃝ 1	⃝ 2
DRALC22_8 Tenía convulsiones o ataques	⃝ 1	⃝ 2

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC23 En los últimos 12 meses, ¿consumió alcohol o usó otra sustancia para recuperarse o prevenir el tener síntomas de abstinencia de alcohol?

1 Sí

2 No

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

PROGRAMMER: DISPLAY IN LOWER RIGHT:

Presione [F2] para ver estos síntomas otra vez

DRALC24 [IF DRALC23=1] ¿Consumió o usó alguna de las siguientes sustancias para recuperarse o prevenir el tener síntomas de abstinencia de alcohol durante los últimos 12 meses?

	Sí	No
DRALC24_1 Alcohol	⃝ 1	⃝ 2
DRALC24_2 Sedantes, tranquilizantes, pastillas para dormir o calmantes que normalmente se venden con una receta médica	⃝ 1	⃝ 2
DRALC24_3 Alguna otra cosa	⃝ 1	⃝ 2

DK/REF

PROGRAMMER: SHOW 12 MONTH CALENDAR

DRALC25 [IF DRALC24_3=1] Usted indicó que tomó algo diferente para recuperarse o prevenir el tener síntomas de abstinencia de alcohol durante los últimos 12 meses. ¿Qué tomó?

______________________________________

DK/REF

0 In this section, example item wording is framed for alcohol. Wording is similar for other substances assessed.

0 Kappa is a statistic that demonstrates the agreement (i.e., concordance) between two measures of the same construct. Kappa can range from -1 (perfect inverse association) to 1 (perfect direct agreement). The further from 0 kappa is, the stronger the association is. The qualitative thresholds for describing the association (poor, moderate, etc.) are based on Landis & Koch, 1977).

0 Examining the ratio of false positives to false negatives can demonstrate whether an estimate is biased in one direction or the other. More false positives will result in estimates that are too high, whereas more false negatives will result in estimates that are too low.

0 DRALC10 is asked only if the respondent reported not having wanted or tried to cut down, or reported having been unable to cut down every time he or she tried, or if he or she answered "don't know" or refused these questions.

0 DRALC11 is asked only if the respondent reported that he or she was able to cut down every time he or she tried or that he or she cut down or stopped using at least one time.

0 DRCC10 is asked only if the respondent reported not having wanted or tried to cut down or reported not having been able to cut down every time they tried, or if they answered "don't know" or refused these questions.

0 DRCC10a is asked only if the respondent reported that they were able to cut down every time they tried or that they cut down or stopped using at least one time.

0 These percentages were provided by the Spanish-language SUD expert. Other research supports this statement, with variable estimates of proficiency (Krogstad, Stepler, & Lopez, 2015).