2019 OMB addendum CVS memo

National Survey on Drug Use and Health (NSDUH) 2020 Clinical Validation Study (CVS) Details

Overview

The Clinical Validation Study (CVS) is designed to assess NSDUH’s revised substance use disorder (SUD) module, developed by CBHSQ for inclusion in the NSDUH main study questionnaire, consistent with the Diagnostic and Statistical Manual of Mental Disorders (DSM), fifth edition (DSM-5) criteria, published by the American Psychiatric Association (APA). Since 2000, NSDUH has included an assessment of SUD based on the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) as part of its data collection. The DSM is the handbook used by mental health professions and is the authority on diagnosis of mental disorders including SUD. The SUD section in the DSM has been revised in the 5^th edition. To continue producing current, high-quality data, CBHSQ has updated the NSDUH SUD module to reflect the revised criteria in the DSM-5. The CVS will examine the validity of this revised SUD module. Specifically, the estimates computed from the self-administered NSDUH will be compared to a gold standard for SUD diagnoses (a clinically administered interview)Analysis of the CVS data will focus on establishing the validity of the NSDUH DSM-5 measures of alcohol use disorder (AUD), cannabis use disorder, drug use disorder (DUD), and substance use disorder (SUD). Substance use disorder is defined as having an alcohol or drug use disorder. Results from these CVS follow-up clinical interviews will provide critical information about the validity of the data collected using the revised NSDUH SUD module based on DSM-5. That is, the overarching goal of the CVS is to ensure NSDUH’s SUD estimates obtained used self-administered questions have acceptable correspondence with estimates developed using a gold standard for SUD diagnoses, which uses a structural clinical interview.

During NSDUH data collection from January through June 2020, approximately 1,500 NSDUH main study interview respondents will be selected for the CVS. During their NSDUH main study interview, these 1,500 respondents will be routed to the new DSM-5 SUD module instead of the DSM-IV SUD module that will be administered to other main study respondents. The 1,500 respondents who are routed to the DSM-5 SUD module will be recruited at the end of the main study interview to participate in a follow-up clinical interview. The clinical interviews will be conducted via telephone by trained clinicians within two to four weeks of completion of the NSDUH main study interview. The expected number of completed clinical interviews is approximately 826 interviews.

Respondents who agree to participate in the follow-up clinical interview will receive an additional $30 in cash at the end of the NSDUH main study interview (i.e., $30 for the NSDUH interview and $30 for the follow-up clinical interview). The burden for the follow-up clinical interview is expected to be an average of 50 minutes per respondent, in addition to the 60 minutes (on average) per respondent already incurred during the NSDUH main study interview.

Many of the procedures and protocols planned for inclusion in this CVS are based upon those previously employed as part of the 2018 National Mental Health Study (NMHS) and the 2008-2012 NSDUH Mental Health Surveillance Study (MHSS). The MHSS was approved as an add-on to NSDUH under OMB No. 0930-0110 and the NMHS was approved under OMB No. 0930-0380.

Achieving a successful result from the CVS will require gathering NSDUH main study and follow-up clinical interview data from a sufficient number and variety of respondents with regard to age, gender, and past year substance use/substance use disorder status to perform the necessary validity analyses. The number and variety of those respondents will be monitored throughout CVS data collection and if rates of respondent participation, substance use, and/or substance use disorder are lower than expected, selection algorithms will be adjusted in order to reach sample size targets.

From those sample targets, the CVS will determine whether or not estimates of SUD, AUD, DUD, and cannabis use disorder developed using the NSDUH revised DSM-5 SUD module have acceptable correspondence with comparable estimates developed using a semi-structured clinical interview. The commonly-used metric of acceptable agreement put forth by Fleiss, Levin, & Palik (1981) will be applied for that assessment. That metric designates kappa values less than 0.40 reflect poor agreement, those from 0.40 to 0.75 reflect fair to good agreement, and kappa values greater than 0.75 reflect excellent agreement.

Findings from previous clinical validation studies of similar instruments have shown good to fair correspondence between estimates of SUDs and those derived using clinical interview measures. For example, the NSDUH DSM-IV SUD module had kappa values of 0.68, 0.62, 0.74, and 0.59 for SUD, AUD, DUD, and marijuana use disorder, respectively, among adults when compared with ratings made using a clinical interview (Jordan et al., 2008). Another survey measure of SUDs, the Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5) had kappa values of 0.58 and 0.62 for AUD and DUD, respectively, among adults when compared with clinical interview ratings (Goldstein et al., 2015). Because the NSDUH revised DSM-5 SUD draws heavily form both of these instruments, it is expected that kappa values from the CVS will be in a similar range.  

If correspondence between NSDUH and clinical estimates of any of these disorders from the CVS fall in the poor agreement range, additional revisions to the relevant questions and subsequent cognitive testing will be considered to determine whether those revisions will further improve the performance of the questions.

Background

Since 2000, NSDUH questions about SUD have been based on the DSM-IV criteria for substance dependence and abuse. The most recent DSM-5 has further modified the diagnostic criteria associated with SUD. As part of an ongoing process to evaluate and improve survey questions, a revised set of SUD questions was developed for NSDUH by SAMHSA based on the DSM-5 criteria. The development included soliciting expert feedback and conducting cognitive testing of these items. The CVS will examine the validity of this revised NSDUH assessment of SUD by administering the SUD questions to adults and youth who will then be interviewed by clinical interviewers (CIs) and classified as having or not having substance use disorders based on past year DSM-5 disorders, as assessed by the Structured Clinical Interview for DSM-5 (SCID-5) (Follow-up Clinical Interview Questionnaire, Attachment CVS-1). CIs administering the SCID-5 are blind to the individual NSDUH main study responses.

Clinical validation, reliability and concordance studies have been conducted on several instruments used in assessing the prevalence of psychiatric disorders, including SUD (Haro et al., 2006; Hasin et al., 2015; Jordan, Karg, Batts, Epstein, & Wiesen, 2008; Kessler et al., 2006). The findings from this CVS will provide critical information about the validity of data collected using the revised NSDUH SUD module based on DSM-5.

In 2014, CBHSQ began by revising the existing NSDUH SUD module to better align with the updated SUD criteria in DSM-5, including the addition of questions on marijuana withdrawal symptoms, and questions measuring the new symptom of substance craving. That revised module was reviewed by SUD experts and survey methodologists to ensure the questions addressed the DSM-5 criteria and could be easily and accurately answered by respondents. After expert review, the revised SUD items were cognitively tested in 2015 (under NSDUH’s generic clearance, OMB Control No. 0930-0290). As a result of the cognitive interview findings, CBHSQ determined that additional research and testing were needed before updating the current NSDUH questions to reflect DSM-5 criteria.

In 2017, the NSDUH SUD module—the existing items and items cognitively tested in 2015—was reevaluated and updated to improve validity and ensure the questions capture DSM-5 criteria. External expert reviews of the revisions to the NSDUH SUD module questions were conducted in two phases. The first phase involved conducting an in-person expert panel meeting on May 8 and 9, 2017. Experts were chosen to provide expertise in survey methodology, youth and adult SUD assessment, as well as SUD among Spanish-speaking populations. In preparation for the in-person expert panel meeting, experts were provided with an overview of NSDUH, copies of the DSM-5 criteria for SUD, and the existing NSDUH SUD module questions for alcohol use disorder and cocaine use disorder. Experts were also provided an informal report that examined diagnostic criteria changes from the DSM-IV to DSM-5 and evaluated their impact on NSDUH.

The expert panel included the following persons:

• Paul C. Beatty, Ph.D., Chief, Center for Survey Measurement, U.S. Census Bureau, (301) 763-5001;

• Raul Caetano, M.D., Ph.D., Senior Research Scientist, Pacific Institute for Research and Evaluation, (510) 883-5728;

• Michael First, M.D., Professor, Columbia University, (646) 774-7935;

• Prudence Fisher, Ph.D., Associate Professor, Columbia University, (212) 543-5357;

• Deborah Hasin, Ph.D., Professor, Columbia University, (212) 543-5035;

• Aaron Hogue, Ph.D., Director of Adolescent and Family Research, The National Center on Addiction and Substance Abuse, (212) 841-5200;

• Brent Moore, Ph.D., Research Scientist, Yale School of Medicine, (203) 932-5711; and

• Glorisa Canino, Ph.D. (via telephone), Professor, University of Puerto Rico, (787) 758-2525.

Additional written feedback was sought from an expert at NIDA, who was unable to attend the expert panel:

• Wilson M. Compton, M.D., Deputy Director, NIDA, (301) 443-6480.

Once feedback was received from the expert reviewers, the revised NSDUH SUD module was drafted. Then, a second written review of the revised questionnaire module was requested from the experts. They were asked to review the questions for validity concerns, as well as potential sources of participant confusion or language concerns. Based on the feedback provided by the experts, the items were further revised for cognitive testing before their inclusion in the CVS.

In 2018 and 2019, three rounds of cognitive interviews with 44 respondents were conducted to assess how well the revised SUD questions perform (also under NSDUH’s generic clearance, OMB Control No. 0930-0290). These cognitive interviews were conducted with youth (aged 12 to 17) and adult (aged 18 or older) past year users of substances such as alcohol, marijuana, cocaine, and/or heroin. These revised SUD questions were cognitively tested in English and Spanish as is done in the NSDUH main study.

Findings from these cognitive interviews assisted CBHSQ in its determination of final changes needed to the SUD questions prior to the next step: inclusion of these questions in the CVS in 2020 to assess validity, ahead of their use in field tests in 2020 and 2022, and then their potential inclusion in the NSDUH main study.

Respondent Universe and Sampling Methods

In conjunction with the selection of a probability subsample of NSDUH interview respondents for the CVS, the number of NSDUH main study sampled dwelling units (SDUs) will be increased by about 2,300 total dwelling units across the first two quarters of 2020 (January through June). The expansion of SDUs is expected to produce approximately 1,500 additional NSDUH main study interviews. This supplemental sample will be allocated proportionally to states and age groups.

Individuals eligible for the CVS sample will be those aged 12 or older who chose to answer the NSDUH main study interview questions in English and who do not break off before beginning the SUD module, including those with no past year use of alcohol or any drug.¹ Within the NSDUH interview, eligible respondents will be selected for the CVS based on their responses to questions on past year use of cigarettes, alcohol, and marijuana. Eight strata will be constructed from all combinations of past year use of cigarettes, alcohol, and marijuana. Using the initials of these three variables, the stratification design is referred to as the cigarette, alcohol, and marijuana (CAM) design. These three substances were chosen for use in stratification given their known, strong correlation with the SUD outcomes of interest. A random sample will be selected from each CAM stratum based on predetermined sampling rates. In addition, a state and age group adjustment will increase efficiency by selecting a nationally representative sample of respondents aged 12 or older, essentially removing the NSDUH oversampling of residents of small states and younger age groups.

The CVS sample will be administered the new SUD module that uses criteria in the DSM-5 rather than the DSM-IV. Those selected for the CVS will then be recruited by the field interviewer (FI) at the end of the NSDUH main study interview to participate in a follow‑up clinical interview by telephone. It is expected that approximately 826 of the 1,500 CVS sample respondents will complete the CVS clinical follow‑up.

Similar to the 2008-2012 MHSS, Neyman optimal allocation (Lohr, 1999) was applied to the strata (i.e., stratum selection probabilities are proportional to the standard deviation of the measure in question). Neyman allocation results in increased precision of key outcome variables (e.g., SUD, AUD, etc.) compared with other allocations, assuming a fixed sample size and equal costs across strata. Also, in the event that sample sizes fall short in some strata, Neyman allocation produces robust estimates of stratum sample sizes (i.e., meaning that moderate deviations will result in only a small loss of precision).

The stratified random sample based on the CAM design minimizes the sample size required for the CVS in two ways: (1) by increasing the yield of DSM-5 SUD cases within the CVS sample (i.e., the closer the yield is to 50 percent, the smaller the required sample size will be for Kappa, given all else is equal), and (2) by reducing the design effect within the CVS sample due to the Neyman allocation.

Because the representation of AUD is large in the SUD outcome variable, optimization of the stratified design based on SUD favors AUD over most other disorders associated with illicit drug use. Therefore, the DUD² outcome variable was used in the Neyman allocation to increase the number of respondents with disorders associated with illicit drug use in the CVS sample. Table 1 displays CVS sample sizes and sampling rates by stratum with Neyman allocation applied to strata based on DUD.

Table 1. Expected Sampling Rates for the CVS Based on CAM Design with DUD as the Outcome Variable

Stratum	Past Year Cigarette Use	Past Year Alcohol Use	Past Year Marijuana Use	CVS Stratum Sample Size	Unadjusted Sampling Rate
1	0	0	0	223	0.0182
2	0	0	1	25	0.0634
3	0	1	0	337	0.0286
4	0	1	1	206	0.0701
5	1	0	0	90	0.0839
6	1	0	1	36	0.1249
7	1	1	0	229	0.0672
8	1	1	1	355	0.1154
Total	N/A	N/A	N/A	1,500	N/A

Table 2 shows the expected sample sizes and outcome rates for each stage of the 2020 NSDUH main study and CVS data collection. (Outcome rates include eligibility rates, selection rates, agreement rates, and response rates.) Assumed outcome rates for the clinical interviews are based on consideration of both the 2008-2012 MHSS outcomes rates for clinical interviews and the recent NMHS Clinical Reappraisal Study (CRS) (Center for Behavioral Health Statistics and Quality, 2014a, 2014b, 2019b), in which respondents were recruited to participate in a follow-up clinical interview within two weeks of completion of the main study interview but not more than four weeks. The average 2008-2012 MHSS combined agreement and completion rate was 66 percent, and the combined agreement and completion rate for the NMHS CRS was 60 percent. The projected combined agreement rate and completion rate in Table 2 is 55 percent. To ensure the CVS sample will yield the required number of clinical interviews, this assumption also reflects recent trends in declining response rates. The CVS eligibility rate reflects the expected proportion of interviews that will be completed in English and was computed using 2016 NSDUH data.

Assuming an 85.5 percent clinical follow-up agreement rate and a 64.4 percent clinical follow-up completion rate, a CVS sample of size 1,500 will yield approximately 826 completed clinical follow-up interviews. Of those 826 CVS interviews, 743 interviews are expected to be completed with adults aged 18+ and 83 interviews with adolescents aged 12-17. (This total does not include the estimated 40 to 70 respondents needed for CI certification interviews, discussed in detail below in Section 22.)

From this sample design, the CVS will determine whether or not estimates of SUD, AUD, DUD, and cannabis use disorder developed using the NSDUH revised DSM-5 SUD module have acceptable correspondence with comparable estimates developed using a semi-structured clinical interview. The commonly-used metric of acceptable agreement put forth by Fleiss, Levin, & Palik (1981) will be applied for that assessment. That metric designates kappa values less than 0.40 reflect poor agreement, those from 0.40 to 0.75 reflect fair to good agreement, and kappa values greater than 0.75 reflect excellent agreement. If correspondence between NSDUH and clinical estimates of any of these disorders from the CVS fall in the poor agreement range, additional revisions to the relevant questions and subsequent cognitive testing will be considered to determine whether those revisions will further improve the performance of the questions.

Table 2. Expected Sample Sizes and Outcome Rates for 2020 CVS

Item	Quarter 1 and 2 Expanded Sample	Eligible for CVS	Selected for CVS	Agree to Follow-Up	Complete Follow-Up Interview
Sample Size	35,254	33,876	1,500	1,283	826
Outcome Rate	N/A	0.9609	0.0443	0.8550	0.6440

Information Collection Procedures

NSDUH Main Study

Both main study interview respondents selected for the CVS and those not selected for the CVS will complete the 2020 NSDUH main study interview. The only difference between the interviews for these two sets of respondents is that those selected for the CVS will receive the new DSM-5-based SUD module during the main study interview instead of the current DSM-IV-based SUD module. The only exception is the legal criterion question that is part of the DSM-IV criteria but not included in the DSM-5 criteria that will be administered to CVS sample respondents. Including the legal criterion question for all interview respondents will facilitate coding/imputing DSM-IV responses from DSM-5 responses for the CVS subsample.

CVS Recruitment

Respondents aged 12 and older who complete the main study interview in person with a NSDUH FI and are selected for the CVS will be recruited by that FI for a follow-up clinical interview to be conducted by a CI via telephone. FIs will be blind to the respondent selection criteria for the CVS and will not know a respondent has been selected for the CVS until the follow-up clinical interview recruitment scripts (Attachment CVS-2) appear at the end of the main study interview. As part of this recruitment, FIs will obtain agreement to participate in the follow-up clinical interview from adult respondents (18 and older) and parents of youth respondents (aged 12 to 17).

Adult Respondents (aged 18 or older):

When the follow-up clinical interview recruitment scripts appear at the end of the main study interview, the FI will read the scripts from the laptop verbatim to the selected respondent and provide them with a hard copy follow-up clinical interview study description (Attachment CVS-3). The follow-up clinical interview study description includes important details about the follow-up clinical interview and a clear description of the follow-up clinical interview procedures, as well as information about protections afforded by CIPSEA (identical to those discussed for the main study).

If the respondent agrees, the FI will ask for the respondent’s first name, request a primary and secondary telephone number for the CI to call, and the best days of the week and times of the day for a CI to contact them to complete the follow-up clinical interview. FIs will enter that information into the laptop for secure transmission back to the FIPS-Moderate network.

Youth Respondents (aged 12-17):

For youth respondents selected for the CVS, first, parental agreement for the youth to participate will be obtained by the FI as part of the recruitment script before talking with the youth about the follow-up clinical interview. The FI will read the script from the laptop and provide the parent (or legal guardian) with a hard copy youth follow-up clinical interview study description (Attachment CVS-3).

If the parent agrees, the FI will then speak with the youth, reading the script from the laptop and providing each respondent with a hard copy youth follow-up clinical interview study description. If the youth agrees to the follow-up clinical interview, the FI will request from the parent the first names of both the parent and the youth, a primary and secondary telephone number for the CI to call, and best days of the week and times of the day for a CI to contact both the parent and the youth by telephone to complete the follow-up clinical interview. FIs will enter that information into the laptop for secure transmission back to the FIPS-Moderate network.

All Respondents

After agreeing to participate in the follow-up clinical interview, FIs will give a $30 cash incentive and a Follow-up Clinical Interview Incentive Receipt (Attachment CVS-4) to all respondents (adults and youths). This is in addition to the $30 and receipt already given for completing the NSDUH main study interview. Respondents will also be given a follow-up clinical interview study reminder card (Attachment CVS-14), filled out by the FI to note the best days and times information provided by the respondent for the CI to call for the follow-up clinical interview, with the understanding that CIs may call on other days or times outside of those on the card.

If potential CVS adult or youth respondents (or the parents/legal guardians of youth respondents) refuse to participate in the follow-up clinical interview, FIs will not attempt to convert that refusal. Instead, FIs will be prompted by the laptop to ask the reason for the refusal and enter the response into the computer. Also, if a parent (or legal guardian) is not available at the time of recruitment (only another adult family member is present in the home), the FI will not proceed with recruiting that youth respondent for the follow-up clinical interview. The case will be closed and the FI will enter notes with details.

Follow-up Clinical Interview

Adult Respondents (aged 18 or older):

Before starting the follow-up clinical interview on the telephone, CIs will read scripts to obtain informed consent from adult CVS respondents (Follow-up Clinical Interview Introduction and Informed Consent Scripts, Attachment CVS-15). The scripts will remind respondents that information gathered during the interview will be kept completely confidential. To further safeguard the privacy of CVS follow-up clinical interviews, an additional script will be read after the consent process to confirm respondents are in a safe and private location before beginning the interview.

CIs will also read scripted text to the respondent asking for permission to audio record the interview for quality control purposes. The CI will explain to the respondent that they know only the respondent’s first name and telephone number and this identifying information will not be recorded with the respondent’s answers or on any audio files created. The CI will repeat the confidentiality assurances and ask for permission to audio record the interview for quality control purposes. If the respondent refuses to allow the interview to be recorded, the CI will proceed with the interview without recording. All recorded clinical interviews will be stored only within the FIPS-Moderate secured network and will never be stored locally on CI computers. All audio files will be discarded no later than 24 months after the end of the CVS data collection period.

Youth Respondents (aged 12-17):

Before speaking with a CVS youth respondent to complete the follow-up clinical interview via telephone, the CI will first ask to speak with the parent who provided agreement for the youth to participate at the end of the main study interview. If that parent is not available at the time of the call, the CI will ask to speak with another parent or legal guardian. If no parent or legal guardian is available, the CI will ask for a good time to call again and then end the call.

When a parent or legal guardian is available, CIs will read a script to gain parental consent to speak to the youth and to administer the interview, if the youth subsequently agrees. After gaining parental consent, the CI will read scripts to the youth respondent to obtain informed assent and confirm they are in a safe and private location at home to complete the interview (Follow-up Clinical Interview Introduction and Informed Consent Scripts, Attachment CVS-15). CIs will also read scripted text to the parent and then the youth asking for permission to audio record the interview. If the parent or youth respondent refuses to allow the interview to be recorded, the CI will proceed with the interview without recording.

If the youth is not available during the telephone call with the parent, the CI will ask for a good time to call back to speak with the youth. The CI will also ask if the parent wants to be present in the home when the youth ultimately completes the telephone interview and records the parent’s response. If the parent agrees that the CI can interview the youth on a subsequent call when a parent is not present, the CI can call the youth at another time and proceed with informed assent and then the interview, if the youth agrees. If the parent wants to be present when the youth completes the interview, on the subsequent call with the youth the CI must confirm the parent is present before reading the assent script and conducting the interview.

Additional CVS Procedures

For planning purposes with all respondents, the average combined interview time for CVS respondents is expected to be 110 minutes. This includes approximately 60 minutes for the NSDUH main study interview and approximately 50 minutes for the follow-up clinical interview. The estimated length of the clinical follow-up interviews of 50 minutes is based primarily on the NMHS CRS. The abbreviated SCID-5 used for the NMHS CRS, which included the overview and up to three diagnostic modules, averaged about 54 minutes. The abbreviated SCID-5 being used for the CVS includes only the overview and one diagnostic module, so it the total administration time is expected to be shorter than the NMHS CRS.

The goal will be to complete follow-up clinical interviews with CVS participants within two weeks of completion of the NSDUH main study interview but not more than four weeks. The follow-up clinical interviews will be conducted by CIs over the telephone using the SCID-5, in English only. This follow-up clinical interview will include three main sections: the overview, the SUD module, and end of interview procedures. The SUD module contains question on substance use disorder for each class of substances. For example, should a respondent use both alcohol and cannabis in the past year, they will be asked the questions on substance use disorder separately for both alcohol and cannabis. The total number of questions asked of each respondents will depend on the number of substances a respondent has used in the past year. At the end of the follow-up clinical interview, CIs will read a script to the respondent ending the call and then, after the call has ended, answer debriefing questions about the interview quality and confidentiality on their own.

CIs will record respondent’s answers on a paper SCID instrument and will keep the questionnaire secure until shipping for overnight delivery to a designated individual at the Contractor’s office in North Carolina. No PII will be recorded on the SCID instrument. Each clinical interview case will only be identified by a randomly-generated seven-digit ID number, unique to only that case.

If the respondent (or the parent of a youth respondent) refuses to participate or schedule a follow-up clinical interview at recruitment, or refuses to participate at the time of the follow-up clinical interview, the FI or CI will not recontact the household. CI recontact attempts will only be made for respondents who are difficult to contact due to their busy schedules or personal circumstances. As necessary, adult respondents or the parents of youth respondents will be mailed a Follow-up Clinical Interview Unable to Contact letter (Attachment CVS-5) for those cases where CIs experience difficulty getting respondents who agreed to participate in the follow-up clinical interview to answer the phone. No attempts will be made to convert individuals who elect not to participate when contacted to complete the telephone follow-up clinical interview.

In addition, if a CI becomes concerned a respondent is not cognitively capable of answering clinical interview questions, the CI will administer the Short Blessed scale to determine how well the respondent is functioning cognitively. The CI will typically utilize the Short Blessed scale (Attachment CVS-16) in situations where a respondent sounds intoxicated, is rambling, or is giving answers that do not tell a coherent story. The Short Blessed scale consists of six questions that test the respondent’s orientation to time, ability to perform simple tasks (e.g., count backwards) and short-term memory. If the respondent scores below the scale’s set cut-off point, the CI discontinues the interview. If the impairment appears temporary (i.e., respondent is intoxicated), the CI will try to schedule another time to complete the interview. If the impairment is not temporary, the CI will tell the respondent they have finished the interview, read the end of interview script, and end the call.

CVS Certification Respondents (aged 12 or older):

As part of their training prior to beginning CVS data collection, all CI candidates must successfully complete a certification interview over the telephone with a volunteer respondent, following all protocols and procedures planned for the CVS follow-up clinical interview.

Certification respondents will be adult and youth volunteers recruited from outpatient treatment centers, support groups and through online ads (e.g. craigslist). Based on the anticipated number of CIs to be hired, a maximum of 70 total respondents will be required for these certification interviews. Respondents will be screened for eligibility over the telephone by Contractor staff or through a web screener (for adults only). The respondent eligibility criteria for these certification interviews are those who report alcohol, marijuana and/or illegal drug use in the past 12 months and who have also received some form of treatment during that time. Respondents who are on active military duty, are current employees of the Contractor or have a family member who is a current Contractor employee are not eligible to participate in certification interviews.

Contractor staff will obtain agreement to participate from these certification respondents. Volunteer respondents will be provided a copy of the Follow-up Clinical Interview Certification Study Description (Attachment CVS-8) prior to their interview, sent to an email address collected from the respondent during screening. When CIs contact the respondents for the certification interview, they will complete the same informed consent process as the actual follow-up clinical interviews: they will obtain informed consent/assent, parental consent for youth respondents, permission to record the interview, and ensure the privacy of the interview (Attachment CVS-9). Following completion of the interview, certification respondents will be mailed $40 as a token of appreciation for their time and thank you letter (Attachment CVS-10).

Certification instruments will be shipped and stored at the Contractor’s office in North Carolina, following the same procedures described for the follow-up clinical interviews. Audio recordings and questionnaire booklets from each certification interview will be evaluated by a clinical supervisor to determine whether the candidate properly administered the instrument. Each CI must successfully pass the certification interview before being formally hired. CIs will be given three opportunities to successfully pass the certification interview. All certification instruments will be destroyed on the same schedule as all other follow-up clinical interview instruments.

For those CIs who successfully pass certification, once data collection begins, the first two CVS interviews completed by each CI, plus an additional 10% random sample of each CI’s completed interviews thereafter, will be reviewed in detail along with the corresponding audio file. After each review by a clinical supervisor, CIs will receive performance feedback via teleconference and coaching as necessary to ensure all CIs are administering the interview appropriately and consistently.

Information Technology Use

The selection of NSDUH main study interview respondents for the CVS follow-up clinical interview will be pre-programmed into the NSDUH main study questionnaire. Individuals eligible for the CVS will be those aged 12 or older who chose to answer the NSDUH main study interview questions in English and who do not break off before beginning the SUD module, including those with no past year use of alcohol or any drug. Within the NSDUH main study interview, eligible respondents will be selected for the CVS based on their responses to questions on past year use of cigarettes, alcohol, and marijuana. Eight strata will be constructed from all combinations of past year use of cigarettes, alcohol, and marijuana.

For those selected for the follow-up clinical interview, follow-up clinical interview recruitment scripts (Attachment CVS-2) are programmed within the 2020 NSDUH main study questionnaire and will be administered at the end of the main study interview using Computer Assisted Personal Interviewing (CAPI). The FI will not know if the respondent is selected for the follow-up clinical interview until the recruitment scripts appear on the laptop screen. Contact information for those who agree to participate in the follow-up clinical interview will be entered into the laptop. This information will only be available on a secure website for access by the CI assigned to contact the respondent for the follow-up clinical interview. The follow-up clinical interview will be administered via telephone on a paper and pencil (PAPI) SCID-5 instrument.

When ready to administer a follow-up clinical interview, CIs will dial into a telephone number to connect to an Interactive Voice Response (IVR) system to make contact with respondents for the follow-up clinical interview. The IVR resides within the Contractor’s FIPS-Moderate network and will also initiate audio recordings of the telephone interviews, subject to respondent (and parent) permission, and will store each recording on a secure NSDUH file share. The IVR system will also create metadata (i.e., time, duration etc.) about the call on a SQL Server database that can be made available for reporting and/or case management purposes.

Payment to Respondents

The follow-up clinical interview conducted via telephone for the CVS will constitute an additional burden on respondents and may make it more difficult to obtain respondent participation. To maintain adequate response rates, SAMHSA believes it is necessary to offer CVS respondents an additional $30 incentive for completing the follow-up clinical interview. The clinical interview will take about the same amount of time as the initial interview, so an equitable incentive is necessary. Research studies have shown that providing incentives before the interview increases the likelihood that participants will complete the interview (Groves & Couper, 1998).

Therefore, SAMHSA believes it is necessary to provide the additional $30 incentive for the follow-up clinical interview at the end of the NSDUH main study interview, once the participant agrees to the follow-up interview. Respondents who agree to complete the follow-up interview will receive a total of $60 at the end of the initial interview – consistent with incentive amounts provided to respondents as part of the 2008-2012 NSDUH MHSS. The additional incentive for the follow-up interview is mentioned in the following respondent materials: Follow-up Clinical Interview Recruitment Scripts (Attachment CVS-2), Follow-up Clinical Interview Study Descriptions (Attachment CVS-3), Follow-up Clinical Interview Incentive Receipt (Attachment CVS-4) and Follow-Up Clinical Interview Unable to Contact Letters (Attachment CVS-5).

In addition, as part of training prior to CVS data collection, each CI candidate must successfully complete a certification interview with a volunteer respondent. These certification interviews will be administered in the same manner as CVS follow-up clinical interviews, except that following completion of the certification interview, those respondents will be mailed $40 as a token of appreciation for their time. (The $40 incentive for certification respondents is consistent with the amount given to respondents during the 2008-2012 NSDUH MHSS.) The incentive for the certification interview is mentioned in the following materials: Follow-up Clinical Certification Recruitment Flyer (Attachment CVS-6), Follow-up Clinical Certification Recruitment Scripts (Attachment CVS-7), Follow-up Clinical Certification Study Descriptions (Attachment CVS-8), Follow-up Clinical Certification Introduction and Informed Consent (Attachment CVS-9) and Follow-up Clinical Certification Thank You Letter (Attachment CVS-10).

Assurance of Confidentiality

The CVS will incorporate several procedures to ensure respondents’ rights will be protected, including procedures developed for the NSDUH main study. The FI will introduce the follow-up clinical interview with recruitment scripts (Attachment CVS-2). These scripts will appear on the computer screen at the end of the main study interview and will be read out loud by the FI to each interview respondent selected for the CVS. As part of the process for obtaining informed consent for the follow-up clinical interview, respondents will be given a follow-up clinical interview study description (Attachment CVS-3) by the FI, which includes information on CIPSEA and the protection it affords. Specifically, the Follow-up Study Description states that respondents’ answers will only be used by authorized personnel for statistical purposes and cannot be used for any other purpose. The dwelling unit address information, which is maintained in a separate file for Contractor use in sampling, fielding, and weighting cases, as well as the respondent’s first name and phone number, will be destroyed when all final data files are delivered to SAMHSA.

Although the respondent’s first name and phone number will be collected within the main study interview, it will be used only for recontact purposes. Once the data are transmitted and arrive in the Contractor’s FIPS-Moderate network, the respondent’s name, phone number, and information regarding the best time to call will be split off into a separate database with only a random, seven-digit ID number for linkage. The rest of the data will be converted into a SAS data file format and merged onto the master data file.

CIs will be assigned cases via the NSDUH Case Management System (CMS) based on availability and the respondent’s preferred time for completing the follow-up clinical interview. The CMS will reside within the Contractor’s FIPS-Moderate network where access requires two-factor authentication. Respondent contact information will be encrypted in the CMS database. Cases will be accessed by CIs via the CMS on a project-issued laptop computer. CIs will be required to confirm that project-issued computers will not contain any electronic notes from interviews and will not have other programs or websites running in the background while they are accessing the CMS.

The follow-up clinical interview will be conducted over the telephone by clinicians trained in the administration of the SCID using the paper and pencil SCID instrument. Each clinical interview case will be identified by a randomly-generated, seven-digit ID number, unique to that case. No personally identifiable information (PII) including respondent name, telephone number, or contact information will be recorded on the SCID instrument.

To initiate interviews, CIs will dial into the IVR system residing in the Contractor’s FIPS-Moderate network, that will prompt the CI for the seven-digit case ID unique to each case and a respondent phone number, then initiate an outbound call to the respondent. The IVR system will also initiate audio recordings of the telephone interviews, subject to respondent (and parent) permission, and will store each recording on a secure NSDUH file share in the Contractor’s FIPS-Moderate network.

CIs will keep the completed SCID secure until shipping for overnight delivery to a designated individual in the data receipt department at the Contractor’s office in North Carolina, where the instruments will be logged as received and stored in a locked location. The CIs will update the case status information on the CMS to notify Contractor staff that a CVS interview has been completed and shipped, generating a notification email that will be distributed to the Contractor’s project manager. Completed CVS SCID instruments will be shipped by the CI with signature required for receipt at the Contractor’s office in North Carolina. Each paper SCID instrument will be sealed in a manila envelope within the shipping package. The cover sheet for the SCID (Follow-up Clinical Interview Cover Sheet, Attachment CVS-11) will not contain any of the respondent’s identifying information but will be used to track the SCID. Additionally, a notice stating the contents in the manila envelope are confidential with an instruction to contact the Contractor’s Project Director if found will be printed directly on the outside of the manila envelope (Confidentiality Notice, Attachment CVS-12).

All paper CVS instruments will be secured in a locked cabinet by Contractor staff when not being keyed. When removed from locked file cabinets for keying, the CVS instruments will remain in the possession of trained Contractor staff who have completed mandatory CIPSEA training and have signed confidentiality pledges. Paper CVS instruments (and associated audio files) will be destroyed within 24 months of the end of CVS data collection.

Once the SCIDs are keyed, those resulting data are protected by the same corporate security controls used to protect NSDUH main study interview data. Access to the Contractor’s FIPS-Moderate network is restricted by the use of specific user names and passwords, in addition to 2-factor authentication tokens. Access permissions to files and data sets on the network are carefully controlled using a combination of domain usernames/passwords and other forms of authentication, such as SQL Server logins. User names and passwords on the Contractor’s networks are subject to enforced security checks, password length requirements, and corporate policies requiring adherence to security procedures. As a project operating policy, all sensitive and PII NSDUH data are stored only on file servers running and implementing full Microsoft or UNIX security, within the Contractor’s FIPS-Moderate network.

Following the conclusion of the CVS data collection, analyses of data will be conducted. The Restricted Use Data File (RUF) is stored securely on file storage appliances within the Contractor’s FIPS-Moderate network and can be accessed by only a small group of authorized NSDUH staff. Any use of RUF files is contained entirely within the FIPS-Moderate network.

The permanent sampling records will contain no record of which addresses were selected for the CVS.

Questions of a Sensitive Nature

There is the possibility the questions contained in the CVS follow-up clinical interview may cause some respondents emotional distress. All CIs will be trained mental health professionals who can recognize signs of cognitive impairment or significant distress in respondents. A Distressed Respondent Protocol (Attachment CVS-13) will be used to help respondents who disclose intent to harm themselves or others. CIs will remain alert for and utilize their professional mental health training to identify when a respondent exhibits active psychotic or suicidal symptoms or seems traumatized during a follow-up clinical interview. In these circumstances, the CI will stop the interview and follow the appropriate guidelines in the Distressed Respondent Protocol. The protocol will also include provisions for respondents who may not be in imminent danger of harm, but who become distressed or agitated during the follow-up clinical interview. CIs will not report any information about the respondent to anyone except in accordance with this protocol. The Distressed Respondent Protocol also includes procedures to be followed if a respondent reveals information that leads the CI to believe that a child is being abused or neglected. The Distressed Respondent Protocol for the CVS was developed based on similar versions used in the NSDUH MHSS from 2008-2012 and the NMHS in 2018.

The CVS interview is designed to reduce the risk associated with revealing potentially-sensitive information. First, the CI will administer the follow-up clinical interview over the telephone from a private location in his/her home or office. During the informed consent/assent process, the CI also ensures the respondent is in a private location (and that any youth respondents are also at home). Secondly, similar to the main study FIs, all CVS CIs complete mandatory CIPSEA training and sign a Data Collection Agreement, which states they agree to treat as confidential all information secured during interviews or obtained in any project-related way.

In the CVS, there are questions related to alcohol and illicit drug use, including questions related to illegal activities, such as driving while impaired. However, all CVS questions about alcohol and drug use query behaviors have already been asked of respondents during the preceding main study NSDUH interview.

Estimates of Annualized Hour Burden

For the CVS, the sample has been designed to yield approximately 826 completed clinical interviews. It will be necessary to supplement the NSDUH sample with approximately 1,500 additional interviews to obtain the requisite number of CVS clinical interviews. Based on previous experience with the NMHS and MHSS, administration of the follow-up clinical interview questions is expected to take an average of 50 minutes per respondent.

The data collection field period for the 2020 CVS is six months, spanning the period from January through June 2020. The annualized estimated respondent burden for the CVS is shown in Table 3. The hourly wage of $18.48 was calculated based on weighted data from the 2017 NSDUH and respondents’ reported personal annual income.

For the CVS, approximately 23 CIs will be hired to conduct the follow-up clinical interviews. Minimum CI credentials require either enrollment in a third year doctoral-level psychology program or a doctoral degree in clinical or counseling psychology, a medical degree with a specialty in psychiatry, or an advanced degree in a related field such as clinical social work.

In addition to attending a training session prior to CVS data collection, all CIs will be required to complete and pass a certification interview before being assigned cases for the study. The certification interview will involve conducting the CVS interview over the telephone with a volunteer respondent. This process will require CIs to properly administer the instrument and follow study protocols and is also expected to take an average of 50 minutes per respondent.

Each CI will get up to three opportunities to successfully pass the certification interview process. That will require a maximum of 70 respondents to volunteer for and complete a certification interview. Certification respondents will be adult and youth volunteers recruited from outpatient treatment centers and support groups who report alcohol, marijuana and/or illegal drug use in the past 12 months who have also received some form of treatment during that time.

Table 3. Annualized Estimated Respondent Burden for 2020 CVS

Instrument	No. of respondents	Responses per respondent	Total number of responses	Hours per response	Total burden hours	Hourly wage rate	Total hour cost
Follow-up Clinical Certification	70	1	70	0.83	58	$18.48	$1,072
Follow-up Clinical Interview	826	1	826	0.83	686	$18.48	$12,677
Total	896		896		744		$13,749

Estimates of Annualized Cost to the Government

Total costs associated with the 2020 NSDUH are estimated to be $69,436,795 over a 48-month contract performance period, including the CVS. Of those total costs, $2,661,392 are contract costs for the CVS.

Changes in Burden

SAMHSA is requesting 722 burden hours for the CVS (of the 82,604 hours total for 2020).

List of Attachments

Attachment CVS-1. Follow-up Clinical Interview Questionnaire

Attachment CVS-2. Follow-up Clinical Interview Recruitment Scripts

Attachment CVS-3. Follow-up Clinical Interview Study Descriptions

Attachment CVS-4. Follow-up Clinical Interview Incentive Receipt

Attachment CVS-5. Follow-up Clinical Interview Unable to Contact Letters

Attachment CVS-6. Follow-up Clinical Certification Recruitment Flyer

Attachment CVS-7. Follow-up Clinical Certification Recruitment Scripts

Attachment CVS-8. Follow-up Clinical Certification Study Descriptions

Attachment CVS-9. Follow-up Clinical Certification Introduction and Informed Consent

Attachment CVS-10. Follow-up Clinical Certification Thank You Letter

Attachment CVS-11. Follow-up Clinical Interview Cover Sheet

Attachment CVS-12. Confidentiality Notice

Attachment CVS-13. Distressed Respondent Protocol

Attachment CVS-14. Follow-up Clinical Interview Study Reminder Card

Attachment CVS-15. Follow-up Clinical Interview Introduction and Informed Consent

Attachment CVS-16. Short Blessed Scale

References

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.) (DSM-IV). Washington, DC: Author.

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA: American Psychiatric Publishing.

Center for Behavioral Health Statistics and Quality. (2014a). 2012 National Survey on Drug Use and Health: Methodological resource book (Mental Health Surveillance Study operations report, 2008-2012). Retrieved from https://www.samhsa.gov/data/report/2012-nsduh-mental-health-surveillance-study-data-collection

Center for Behavioral Health Statistics and Quality. (2014b). 2012 National Survey on Drug Use and Health: Methodological resource book (Section 16a, 2012 Mental Health Surveillance Study: Design and estimation report). Retrieved from https://www.samhsa.gov/data/report/2012-nsduh-mental-health-surveillance-study-design-and-estimation-report

Center for Behavioral Health Statistics and Quality. (2019b). National Mental Health Study-Field Test (NMHS-FT): Final report. Rockville, MD: Substance Abuse and Mental Health Services Administration.

First, M. B., Williams, J. B. W., Karg, R. S., & Spitzer, R. L. (2015). Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). Arlington, VA: American Psychiatric Association.

Fleiss, J. L., Levin, B., & Paik, M. C. (1981). The measurement of interrater agreement. Statistical methods for rates and proportions, 2(212-236), 22-23.

Goldstein, R. B., Chou, S. P., Smith, S. M., Jung, J., Zhang, H., Saha, T. D., ... & Grant, B. F. (2015). Nosologic comparisons of DSM-IV and DSM-5 alcohol and drug use disorders: Results from the National Epidemiologic Survey on Alcohol and Related Conditions–III. Journal of studies on alcohol and drugs, 76(3), 378-388.

Groves, R. M., and M.P. Couper. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.

Haro, J. M., Arbabzadeh‐Bouchez, S., Brugha, T. S., De Girolamo, G., Guyer, M. E., Jin, R., ... & Sampson, N. A. (2006). Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys. International journal of methods in psychiatric research, 15(4), 167-180.

Hasin, D. S., Greenstein, E., Aivadyan, C., Stohl, M., Aharonovich, E., Saha, T., ... & Grant, B. F. (2015). The Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5): procedural validity of substance use disorders modules through clinical re-appraisal in a general population sample. Drug and alcohol dependence, 148, 40-46.

Jordan, B. K., Karg, R. S., Batts, K. R., Epstein, J. F., & Wiesen, C. (2008). A clinical validation of the National Survey on Drug Use and Health assessment of substance use disorders. Addictive Behaviors, 33(6), 782-798.

Kessler, R. C., Adler, L., Barkley, R., Biederman, J., Conners, C. K., Demler, O., ... & Spencer, T. (2006). The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. American Journal of psychiatry, 163(4), 716-723.

Lohr, S. L. (1999). Sampling: Design and analysis. Belmont, CA: Duxbury Press.

1 The new SUD module and the clinical follow-up interview will be conducted only in English. Therefore, respondents who chose to answer the NSDUH main study questions in Spanish will be excluded from the CVS.

2 DUD refers to any disorder associated with illicit drug use; that is, it refers to any SUD except AUD.

13