NSDUH-SSB _Revised_9-29-21

2022 National Survey on Drug Use and Health

SUPPORTING STATEMENT

B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS

1. Respondent Universe and Sampling Methods

The respondent universe for the 2022 NSDUH is the civilian, noninstitutionalized population aged 12 or older within the 50 states and the District of Columbia. The NSDUH universe includes residents of noninstitutional group quarters (e.g., shelters, rooming houses, dormitories), and civilians residing on military bases. Persons excluded from the universe include those with no fixed household address (e.g., homeless transients not in shelters, and residents of institutional group quarters such as jails and hospitals).

Given the public health emergency related to COVID-19 and considering the safety of field staff and the public, NSDUH in-person data collection was suspended on March 16, 2020. For the remainder of 2020, the COVID-19 pandemic severely restricted the areas where data could be collected in person. National in-person data collection did not resume until the fourth calendar quarter of the 2020 NSDUH. Because of the limited number of counties in which data could be collected in person in the fourth quarter, the respondent sample size was at risk of being reduced to an unacceptable level. To minimize the impact of COVID-19 on the 2020 NSDUH respondent sample size, web-based data collection was added on October 30, 2020 and the fourth quarter of 2020 became a period of multimode (web and in-person) data collection. The combination of in-person and web-based data collection modes continued through 2021.

For 2022, NSDUH data will continue to be collected from respondents using a combination of in-person and web-based data collection modes. The web-based data collection will be self-administered by eligible respondents in all states, while the in-person data collection effort, administered by FIs, will occur in those states and/or counties only as permitted by current COVID-19 metrics.

If the COVID-19 pandemic subsides to such levels to allow in-person data collection to resume nationwide, SAMHSA may reassess that multimode data collection model as part of the 2022 NSDUH.

However, similar to previous NSDUHs, the overall sample design will continue to consist of a stratified, multi-stage area probability design (See Attachment A for a detailed presentation of the Sample Design). The 2022 Sample Design Plan will undergo a peer review to help inform the design of future NSDUHs. In particular, the review will help ensure the ability of the NSDUH design to support comparisons of population subgroups defined by the intersection of race and gender, or other demographic variables that may be associated with disparities in survey outcomes. In the meantime, design options will be investigated for these important demographic subpopulations. As with most area household surveys, the NSDUH design continues to offer the advantage of minimizing interviewing costs by clustering the sample. This design also maximizes coverage of the respondent universe since an adequate dwelling unit and/or person-level sample frame is not available. Although the main concern of area surveys is the potential variance-increasing effects due to clustering and unequal weighting, these potential problems are directly addressed for NSDUH by selecting a relatively large sample of clusters at the early stages of selection and by selecting these clusters with probability proportionate to a composite size measure. This selection maximizes precision by allowing one to achieve an approximately self-weighting sample within strata at the latter stages of selection. Furthermore, it is appealing because the design of the composite size measure makes the FI workload roughly equal among clusters within strata.

A coordinated design was developed for the 2014-2017 NSDUHs and has been extended to the 2018-2022 NSDUHs. To accurately measure drug use and related mental health measures among the aging drug use population, this design allocates the sample to the 12 to 17, 18 to 25, and 26 or older age groups in proportions of 25 percent, 25 percent, and 50 percent, respectively.

While age group is currently the only stratification variable at the person level, stratification by other demographic subgroups has been considered in the past. Prior to the 2014 sample redesign, estimates by four-level race and ethnicity (Hispanic, NonHispanic White, NonHispanic Black, and NonHispanic Other) were deemed to have sufficient precision based on a sample of 67,500 completed interviews without needing to stratify or oversample these subgroups. Because NSDUH continues to target 67,500 interviews each year, estimates by four-level race and ethnicity continue to have sufficient precision without stratification or oversampling. Further, four of the 25 “key” measures for which precision is monitored each year are among the American Indian and Alaska Native (AIAN) and Asian or Pacific Islander (API) populations. Finally, four of the annual detailed tables report estimates by four-level race and ethnicity (Hispanic, NonHispanic White, NonHispanic Black, and NonHispanic Other) by gender using data from two years combined (e.g. 2019 and 2020): numbers (in thousands) of people aged 12 or older by age group and demographic characteristics; numbers (in thousands) of people aged 12 or older by underage and legal drinking age groups and demographic characteristics; alcohol use in lifetime, past year, and past month and binge alcohol and heavy alcohol use in past month among people aged 12 to 20 by demographic characteristics; and alcohol use in lifetime, past year, and past month and binge alcohol and heavy alcohol use in past month among people aged 21 or older by demographic characteristics. Estimates in these tables typically have sufficient sample and precision to be reported. Thus, there is no need to stratify and oversample by four-level race/ethnicity and gender based on current reporting needs. However, options for stratifying the sample to facilitate finer race/ethnicity categories (e.g., subgroups under “Asian”) and gender or oversampling areas with higher concentrations of the demographic subgroups of interest will be researched for future consideration.

In anticipation of the next decennial census data not being available at the time the 2018-2022 sample would be selected, a large reserve sample was selected when the 2014–2017 NSDUH sample was selected. Thus, the 2018-2022 sample has been selected down to the area segment level. The sample selection procedures begin by geographically partitioning each state into roughly equal size state sampling regions (SSRs). Regions are formed so that each area within a state yields, in expectation, roughly the same number of interviews during each data collection period. As shown in Table 1 of Attachment A, this partition divides the U.S. into 750 SSRs.

Within each of the 750 SSRs formed for the 2014-2022 NSDUHs, a sample of Census tracts is selected. Then, within sampled Census tracts, Census block groups are selected. This stage of selection facilitates transitioning to an address-based sampling (ABS) design, which is planned for the 2022 NSDUH. Finally, within Census block groups, smaller geographic areas, or segments, are selected. A total of 48 segments per SSR are selected: 20 to field the 2014-2018 surveys and 28 “reserve” segments which are available to field the 2018-2022 NSDUHs. In general, segments consist of adjacent Census blocks and are equivalent to area segments selected at the second stage of selection in the 2005-2013 NSDUHs.

In summary, the first-stage stratification for the 2014-2022 NSDUHs is states and SSRs within states, the first-stage sampling units are Census tracts, the second-stage sampling units are Census block groups, and the third-stage sampling units are small area segments. This design for the 2014-2022 NSDUHs at the first stages of selection is desirable because of (a) the large person-level sample required at the latter stages of selection in the design and (b) continued interest among NSDUH data users and policymakers in state and other local-level statistics.

The coordinated design facilitates 50 percent overlap in third-stage units (area segments) between each two successive years from 2014 through 2022. The primary benefit of the sample overlap is the cost savings achieved from being able to reuse the list frames for half of the area segments in the 2015 through 2022 surveys. In addition, the expected precision of difference estimates generated from consecutive years (e.g., the year-to-year difference in past month marijuana use among 12- to 17-year-old respondents) is slightly improved because of the expected positive correlation resulting from the overlapping sample.

Within area segments, NSDUH has traditionally used counting and listing (also known as field enumeration) to construct dwelling unit frames for the fourth stage of sample selection. For the 2022 NSDUH, a combination of field enumeration and ABS will be used. ABS refers to the sampling of residential addresses from a list based on the U.S. Postal Service's Computerized Delivery Sequence file. Relative to field enumeration, ABS has the potential to greatly reduce costs, improve timeliness, and improve frame accuracy in areas with controlled access. However, ABS has some limitations. To maximize coverage and minimize costs, many studies use a hybrid ABS design that uses both ABS and field enumeration, depending on the expected coverage of the ABS frame for particular areas.¹

For the 2022 NSDUH hybrid ABS design, ABS coverage estimates will first be computed for all selected second-stage sampling units (SSUs; one or more census block groups).² In addition to having high expected ABS coverage (95 percent or more), SSUs will be required to meet separate criteria for group quarters and drop points.³ SSUs that meet the ABS coverage criteria will then serve as sample segments and use the ABS frame. SSUs that do not meet the ABS coverage criteria will require field enumeration. For cost efficiency, the smaller geographic area that was previously selected at the third stage of selection will be field enumerated and considered the sample segment. The planned approach allows SAMHSA to deploy a hybrid ABS design on a limited basis, without significant risk, while realizing some of the cost and timeliness benefits. Based on findings from a comprehensive ABS research report⁴ and preliminary results from an ABS field test, and because of the limited number of areas that will use the ABS frame, the planned approach is expected to have minimal impact on 2022 NSDUH survey estimates.

Similar to previous NSDUHs, at the latter stages of selection, five age group strata are sampled at different rates. These five strata are defined by the following age group classifications: 12 to 17, 18 to 25, 26 to 34, 35 to 49, and 50 or older. Adequate precision for race/ethnicity estimates at the national level is achieved with the larger sample size and the allocation to the age group strata. Consequently, race/ethnicity groups are not over-sampled. However, consistent with previous NSDUHs, the 2022 NSDUH is designed to over-sample the younger age groups.

Table 1 in Attachment A shows the projected number of person respondents by state and age group. Table 2 (Attachment A) shows sample sizes and the projected number of completed interviews by sample design stage. Table 3 (Attachment A) shows the expected precision for key measures by demographic domain.

In addition, all estimates ultimately released as NSDUH data must meet published precision levels. Through a thorough review of all data, any direct estimates from NSDUH designated as unreliable are not shown in reports or tables and are noted by an asterisk (*). The criteria used to define unreliability of direct estimates from NSDUH is based on the prevalence (for proportion estimates), relative standard error (RSE) (defined as the ratio of the standard error (SE) over the estimate), nominal (actual) sample size, and effective sample size for each estimate. These suppression criteria for various NSDUH estimates are summarized in the table below.

Summary of NSDUH Suppression Rules

Estimate	Suppress if:
Prevalence Estimate, , with Nominal Sample Size, n, and Design Effect, deff	(1) The estimated prevalence estimate, , is < 0.00005 or > 0.99995,1 or (2) when , or when , or (3) Effective n < 68, where  or (4) n < 100. Note: The rounding portion of this suppression rule for prevalence estimates will produce some estimates that round at one decimal place to 0.0 or 100.0 percent but are not suppressed from the tables.2
Estimated Number (Numerator of )	The estimated prevalence estimate, , is suppressed. Note: In some instances when is not suppressed, the estimated number may appear as a 0 in the tables. This means that the estimate is greater than 0 but less than 500 (estimated numbers are shown in thousands). Note: In some instances when totals corresponding to several different means that are displayed in the same table and some, but not all, of those means are suppressed, the totals will not be suppressed. When all means are suppressed, the totals will also be suppressed.
Means not bounded between 0 and 1 (i.e., Mean Age at First Use, Mean Number of Drinks), , with Nominal Sample Size, n	(1) , or (2) n < 10.

deff = design effect; RSE = relative standard error; SE = standard error.

NOTE: The suppression rules included in this table are used for detecting unreliable estimates and are sufficient for confidentiality purposes in the context of the first findings reports and detailed tables.

¹   Starting with the 2015 NSDUH, the close to 100 percent portion of the rule was changed to  > 0.99995 instead of the old rule, which was greater than or equal to 0.99995. This was done so the close to 0 and close to 100 rules were both strict inequalities.

2. Information Collection Procedures

Prior to the FI’s arrival at the SDU, a Lead Letter (see Attachment C) will be mailed to the resident(s) briefly explaining the survey and requesting their cooperation. In this letter, adult residents of each SDU will be presented with the option to participate in NSDUH via the web-based instruments (screening and interview) or to contact the Contractor directly about participating in-person with screenings and interviews administered by an FI – but only in those areas where FIs are approved to work based on current COVID-19 metrics.

Respondents may select to participate via web or in-person at their choosing, regardless of current eligibility status for in-person data collection determined by the COVID-19 metrics in the state/county where the SDU is located (meaning those respondents in areas approved for in-person data collection retain the option to participate via web for the entire data collection period).

Those SDUs in areas where FIs are approved to conduct in-person data collection will receive a visit from an FI to attempt the screening and/or any yielded interviews if a resident/selected respondent of an SDU does not respond to the initial lead letter. As part of in-person data collection in 2022, FIs will continue to wear PPE as directed while working, monitor their health before starting the workday, and follow sanitation procedures based on current best practices for COVID-19 prevention (such as use of hand sanitizer, safe storage of materials, cleaning of surfaces, etc.)

Also, throughout the data collection period, respondents residing in areas approved for in-person data collection will retain the option to participate via web at their discretion, completing screenings and/or interviews online instead of doing so in-person with an FI or vice versa, as well as transitioning between modes when available.

In-Person Data Collection

Upon arrival at the SDU, the FI will refer the resident to this letter and answer any questions. If the resident has no knowledge of the Lead Letter, the FI will provide another copy, explain that one was previously sent, and then answer any questions. If no one is home during the initial visit to the SDU, the FI may leave a Sorry I Missed You Card (Attachment F) informing the resident(s) that the FI plans to make another callback at a later date/time. Callbacks will be made as soon as feasible following the initial visit. FIs will attempt to make at least four callbacks (in addition to the initial call) to each SDU in order to complete the screening process and complete an interview, if yielded.

If the FI is unable to contact anyone at the SDU after repeated attempts, that FI’s Field Supervisor (FS) may send one of the Unable-to-Contact (UTC) letters (Attachment J). These UTC letters reiterate information contained in the Lead Letter and present a plea for the resident to participate in the study.

When in‑person contact is made with an adult member of the SDU and introductory procedures are completed, the FI will present a Study Description (Attachment G) and answer any questions that person might have concerning the study. Once the FIs have informed the screening respondent about NSDUH, each FI will then provide the respondent with a printed copy of the COVID-19 risk information form to keep – included as Attachment Q. This document provides details specific to COVID-19, the risks associated with COVID-19 (including populations most at risk from the disease – older adults, those with certain pre-existing medical conditions, etc.) as well as steps NSDUH is taking to mitigate those risks. If the COVID-19 pandemic subsides to sufficient levels, SAMHSA may reassess the need to provide this risk form to respondents as part of in-person data collection.

FIs may also give respondents a Question & Answer Brochure (Attachment D) that provides answers to commonly-asked questions may also be given. In addition, FIs are supplied with copies of the NSDUH Highlights & Newspaper Articles (Attachment R) for use in eliciting participation, which can be left with the respondent.

Also, the FI may utilize the multimedia capability of the touch-screen tablet to display one of two short videos (approximately 60 seconds total run time per video) for members of the SDU to view, which provides a brief explanation of the study and why participation is important. The scripts for these videos are included as Attachment E.

If a potential respondent refuses to be screened, the FI has been trained to accept the refusal in a positive manner, thereby minimizing the possibility of creating an adversarial relationship that might preclude future opportunities for contact. The FS may then request that one of several Refusal Letters (Attachment K) be sent to the residence. The letter sent is tailored to the specific concerns expressed by the potential respondent and asks him or her to reconsider participation. Refusal letters are customized and also include the FS’s phone number in case the potential respondent has questions or would like to set up an appointment with the FI. Unless the respondent calls the FS, CBHSQ or the Contractor’s office to refuse participation, an in-person conversion is then attempted by specially-selected FIs with successful conversion experience.

With respondent cooperation, the FI will begin screening the SDU by asking either the Housing Unit Screening questions, or the Group Quarters Unit Screening questions, as appropriate. The screening questions are administered using a 7-inch touch screen Android tablet computer. A paper representation of the housing unit and group quarters unit screening process is shown in Attachment I.

Once all household members aged 12 or older have been rostered, the hand-held computer performs the within-dwelling-unit sampling process, selecting zero, one, or two members to complete the interview. For cases with no one selected, the FI asks for a name and phone number for use in verifying the quality of the FI’s work, thanks the respondent, and concludes the household contact.

For each person selected to complete the interview, the FI follows these steps:

• If the selected individual is aged 18 or older and is currently available, the FI immediately seeks to obtain informed consent. Once consent is obtained, the FI begins to administer the questionnaire in a private setting within the dwelling unit. As necessary and appropriate, the FI may make use of the Appointment Card (in Attachment F) for scheduled return visits with the respondent.

• If the selected individual is 12 to 17 years of age, the FI reads the parental introductory script (Attachment O) to the parent or guardian before speaking with the youth about the interview. Subsequently, parental consent is sought from the selected individual’s parent or legal guardian using the parent section of the youth version of the Introduction and Informed Consent Scripts (Attachment H). Once parental consent is granted, the minor is then asked to participate using the youth section of the same document. If assent is received, the FI begins to administer the questionnaire in a private setting within the dwelling unit with at least one parent, guardian or another adult remaining present in the home throughout the interview.

Prior to beginning the interview, the FI will provide the COVID-19 risk information form to interview respondents and/or parents of youth respondents regarding the risks of participation and allow each to make their own decision about participating.

As mentioned in section A.3, the FI administers the interview in a prescribed and uniform manner with sensitive portions of the interview completed via ACASI. The changes to the 2022 NSDUH questionnaire are summarized in section A.1.

Race/ethnicity questions are FI-administered and meet all of the guidelines for the OMB minimum categories. The addition of the finer delineation of Guamanian or Chamorro and Samoan, which collapse into the OMB standard Native Hawaiian/Other Pacific Islander category, were a requirement of the new HHS Data Collection Standards and will continue to be included in the 2022 questionnaire.⁵ 

In order to facilitate the respondent’s recollection of prescription-type drugs and their proper names, pill images will appear on the laptop screen during the ACASI portions of interviews as appropriate in 2022. Also, respondents will use an electronic reference date calendar, which displays automatically on the computer screens when needed throughout the ACASI parts of the interview. Finally, in the FI-administered portion of the questionnaire, showcards are included in the Showcard Booklet (Attachment N) that allow the respondent to refer to information necessary for accurate responses.

After the interview is completed, each respondent is given a $30 cash incentive and an Interview Incentive Receipt (Attachment L) signed by the FI.

For verification purposes, interview respondents are asked to complete an electronic Quality Control Form that requests his/her telephone number so Contractor staff can potentially call to ensure the FI did his or her job appropriately. Respondents are informed that completing the Quality Control Form is voluntary. If he or she agrees, the FI completes this form using the handheld tablet computer and those data are securely delivered directly to the Contractor’s office as part of that FI’s next successful transmission.

FIs may give a Certificate of Participation (Attachment S) to interested respondents after the interview is completed. Respondents may attempt to use these certificates to earn school or community service credit hours. As stated on the certificate, no guarantee of credit is made by SAMHSA or the Contractor. The respondent’s name is not written on the certificate. The FI signs his or her name and dates the certificate, but for confidentiality reasons the section for recording the respondent’s name is left blank. The respondent can fill in his/her name at a later time so the FI will not be made aware of the respondent’s identity. It is the respondent’s choice whether he or she would like to be identified as a NSDUH respondent by using the certificate in an attempt to obtain school or community service credit.

A random sample of those who complete Quality Control Forms are contacted via telephone to answer a few questions verifying that the interview took place, that proper procedures were followed, and that the amount of time required to administer the interview was within expected parameters. The CATI Verification Scripts (Attachment T) contain the scripts for these interview verification contacts via telephone, as well as the scripts used when verifying a percentage of certain completed screening cases in which no one was selected for an interview or the SDU was otherwise ineligible (vacant, not primary residence, not a dwelling unit, dwelling unit contains only military personnel, respondents living at the sampled residence for less than half of the quarter).

As noted above, all interview data are transmitted on a regular basis via secure encrypted data transmission to the Contractor’s offices in a FIPS-Moderate environment, where the data are subsequently processed and prepared for reporting and data file delivery.

In-Person Questionnaire

As explained in section A.3, the version of the NSDUH questionnaire to be fielded in 2022 is a computerized (CAPI/ACASI) instrument based on the 2021 questionnaire.

As in past years, two versions of the instrument will be prepared: an English version and a Spanish translation. Both versions will have the same essential content.

The proposed CAI Questionnaire is shown in Attachment B. While the actual administration will be electronic, the document shown is a paper representation of the content that is to be programmed. The interview process is designed to retain respondent interest, ensure confidentiality, and maximize the validity of responses. The questionnaire is administered in such a way that FIs do not know respondents’ answers to sensitive questions, including those on illicit drug use and mental health. These questions are self-administered using ACASI. The respondent listens to the questions privately through headphones so even those who have difficulty seeing or reading are able to complete the self-administered portion. Topics administered by the FI (i.e., the CAPI section) are limited to Demographics, Health Insurance, and Income. Respondents are given the option of designating an adult proxy who is at home to provide answers to questions in the Health Insurance and Income sections.

The interview consists of a combination of FI-administered and self-administered questions. FI-administered questions at the beginning of the interview consist of initial demographic items. The first set of self-administered questions pertain to the use of tobacco, alcohol, marijuana, cocaine, crack cocaine, heroin, hallucinogens, inhalants, methamphetamine, and prescription psychotherapeutic drugs (i.e., pain relievers, tranquilizers, stimulants, and sedatives). Similar questions are asked for each substance or substance class, ascertaining the respondent’s history in terms of age of first use, most recent use, number of times used in lifetime, and frequency of use in past 30 days and past 12 months. These substance use histories allow estimation of the incidence, prevalence, and patterns of use for licit and illicit substances.

Additional self-administered questions or sections follow the substance use questions and ask about a variety of sensitive topics related to substance use and mental health issues. These topics include (but are not limited to) injection drug use, perceived risks of substance use, substance use disorders, arrests, treatment for substance use problems, pregnancy, mental illness, the utilization of mental health services, disability, and employment and workplace issues. Additional FI-administered CAPI sections at the end of the interview ask about the household composition, the respondent’s health insurance coverage, and the respondent’s personal and family income.

The detailed specifications for the proposed in-person questionnaire for 2022 are provided in Attachment I.

Web-based Data Collection

The lead letter received at all SDUs will contain the mailing address and 10-digit alphanumeric participant code unique to each dwelling unit for use by those selecting to participate via web. The letter also contains the address to the NSDUH respondent web site, which provides general information about the study for all respondents, and the toll-free number for the NSDUH respondent call line. The letter also emphasizes that a resident must be 18 or older to participate as a screening respondent and that parental permission is required for interviews with youth respondents. Using the website address provided in the lead letter, an adult resident of the SDU (aged 18 or older) will access the NSDUH web screening from any device with internet access (computer, tablet, phone, etc.). The website’s HTTPS encryption will provide sufficient security for all information entered from any device via any internet connection (public Wi-Fi, cellular, at-home Wi-Fi, etc.).

The letter also includes a QR code that respondents can scan to access the web screening site without having to type in the web address – though respondents will need to enter their unique participant code to begin the screening.

The web-based screening (and any subsequent interviews) will be available in English and Spanish. Those living in SDUs who are not able to read English and Spanish well enough to complete the screening (and interview) without assistance will not be able to participate as they will not be able to read and understand the lead letter they receive.

If the resident chooses to complete the online screening, included as WEB-4, all household members aged 12 and older will be rostered, which involves the screening respondent providing basic demographic information for each, and the selection algorithm will then determine if any eligible respondents are selected for the interview. At the end of the web-based screening, just like the in-person screening, 0, 1, or 2 residents aged 12 or older may be selected to complete the NSDUH interview.

The interview respondents will be identified on the interview selections screen according to their age and relationship to the SR (i.e., 14-year-old son, 46-year-old wife, etc.) and clearly labeled as Interview A and/or Interview B.

If one or both of the adult interview respondents are not the screening respondent, the screening respondent will be asked to inform those adult interview respondent(s) verbally of the interview selection. (Also, as described below, each adult interview respondent will receive a follow-up letter about completing the interview.)

If one or both of the interview respondents are youth respondents (aged 12-17), the screening respondent will be asked to verbally inform the parent(s) of the youth (if the screening respondent is not also the parent) that: 1) the youth was selected for an interview; 2) verbal parental permission is required via telephone before the youth can participate in the interview; and 3) the toll-free phone number to call to conduct the parental permission and youth assent process for the interview.

In addition to the lead letter, SDUs will receive a series of follow-up letters reminding them of their selection for NSDUH. A few days after receiving the lead letter, all SDUs will receive a follow-up letter (Attachment WEB-5) as a reminder to complete screening. This letter will contain the website address to the screening website (including a QR code for quick access) and the SDU’s participant code.

Then once each week for the next three weeks following the mailing of the first lead letter, residents at SDUs who have not yet completed the web screening will receive additional follow up correspondence (included in Attachment WEB-5). Once an adult resident of an SDU completes the web screening, the screening follow-up letters will no longer be mailed to that SDU.

If the screening respondent is one of the adult interview respondents, he/she has the option to immediately begin the web interview once selected. Interview respondents will be able to access their interview at a later time of their choosing (until the end of the quarterly data collection period) by using the website address provided on the lead letter – the same address for the screening – and inserting the participant code unique to that SDU.

Once the interview respondent accesses the interview, each interview respondent will first have to review the informed consent documentation for the web-based interview. This text provides many of the same elements as the informed consent text read to in-person interview respondents by field interviewers but has been updated to incorporate text specific to the web mode administration.

This Intro to CAI text – included in the Web Interview Specs, Attachment WEB-1 – asks each adult interview respondent to confirm they are 18 years old or older and that they are at home in a private location where no one else can see their answers. Interview respondents are also encouraged to conduct the interview in one sitting, rather than conducting a break-off.

Also, the screen with this Intro to CAI text will contain a link to the NSDUH Study Description; when clicked, a separate window will display the Study Description for an interview respondent to review, download as a PDF and/or print at their discretion.

At the end of the Intro to CAI, each respondent will be asked to confirm he/she is 18 years old or older, is a current resident of the SDU, is at home in a private location and has read and understands the information provided in the Intro to CAI about participating in the web interview. After clicking an acceptance box, the IR can then advance to the next screen. Those who do not wish to proceed are then given the option to exit the interview at that time.

As an additional layer of security, after advancing past the Intro to CAI text, each adult respondent will be required to set a unique 4-digit PIN code of their own choosing before they begin self-administering interview questions. This will prevent anyone else within the dwelling unit from accessing the interview and seeing answers to questions.

Before a web-based interview with a youth respondent can begin, a parent/guardian of that youth respondent (and the youth selected as the respondent) must speak with a telephone interviewer at the Contractor’s call center. Parents will be given a telephone number displayed at the end of the web screening (and in follow-up letters mailed after the screening) to place a call to speak with the telephone interviewer.

Once the parent and telephone interviewer are on the phone, the telephone interviewer will ask the parent to confirm the SDU address and the unique Participant Code printed on the lead letter (and any follow-up letters) mailed to that SDU to verify the telephone interviewer is speaking with residents of the dwelling unit. All required elements of this call, interactions with parents and youth respondents, will be scripted for each telephone interviewer to read and are included as Attachment WEB-3.

The telephone interviewer will read an informed consent script to the parent over the telephone. The script read by the telephone interviewer first to the parent includes the expectation the parent and youth be at home, youth must be in a private location, parents cannot view answers, nature of the questions, etc. After speaking with the parent, the telephone interviewer will then read an assent script to the youth about the content of the interview.

A text version of this youth assent script will also be displayed again within the online interview for the youth to read and acknowledge before they begin the interview (described in detail below).

Once parental permission and youth assent have been given verbally, at that point the youth interview can be accessed from the NSDUH website at any time. Until that permission is logged, the youth interview cannot be accessed.

Similar to the adult interviews, as an additional layer of security, after advancing past the Intro to CAI text, each youth respondent will be required to set a unique 4-digit PIN code of their own choosing before they begin self-administering interview questions. This will prevent anyone else within the dwelling unit from accessing the interview and seeing answers to questions.

Every respondent who completes the web-based interview, either an adult or youth respondent – will be eligible for the $30 incentive and will select a preferred method for receiving that incentive – either an electronic Visa or MasterCard gift card or a physical Visa or MasterCard gift card delivered to the SDU. Each respondent will also receive a thank you message for their participation (Attachment WEB-6).

Each IR will also be offered a link to the web version of the NSDUH Q&A brochure – similar the brochure given by FIs during in-person data collection but with a format optimized for web without mention of an in-person interview (included as Attachment WEB-7).

Web-based Questionnaire

Those respondents who choose to take part in the web-based interview will be asked to complete the interview while at home, in a private location within the home and to allow sufficient time to complete the interview in one sitting. Respondents can access the interview from any device with internet access (smartphone, tablet, computer, etc.), but it will be recommended that they self-administer the interview from a laptop or desktop computer in their home to allow for the optimal viewing experience with the interview program via Blaise 5. As mentioned in Section A.3, the 2022 web-based instrument will continue to include user testing to ensure functionality across multiple platforms and devices (e.g., tablets and smartphones). The entire questionnaire for the web interview is included as Attachment WEB-1.

Also, those respondents who have difficulty accessing the web interview, need other assistance with the interview or have any questions about their participation can call the toll-free telephone number provided on the lead letter and receive technical support and answers to their questions about participation in NSDUH.

For the transition from in-person to web-based interviewing, various design changes have been implemented to optimize the experience for each respondent, in particular those with lower literacy levels who may have benefited from the ACASI text-to-speech element of the in-person interview not available in the web interview. These changes include the following:

• use of bold and blue-colored text to draw attention to keywords;

• transition of several screens to a more user-friendly grid format to limit the repetitiveness of some questions; and

• spreading text-heavy items, such as the Intro to CAI, across multiple screens to avoid excessive scrolling, etc.

The decision to not include ACASI as part these web-based interviews was made following consultation with subject matter experts which concluded that offering audio would result in a larger risk of breeching confidentiality (allowing others to potentially hear items positively endorsed previously by a respondent and “filled” in a new item query) than not having audio at all.

Prior to particularly sensitive modules in the “ACASI section” of the web interview prompts are available to remind respondents to be sure they are in a private location. Also, similar to the in-person interview, respondents will have the option to enter “Don’t Know” or “Refused” to any question, with those options shown on screen with the other answer choices.

For those respondents who answer “Yes” to any of the questions about past year suicidal ideation or attempts, that respondent will receive a follow-up screen at the end of the series of questions with a reminder of the contact information for the helpline if that respondent would like to speak with a mental health professional.

The detailed specifications for the proposed web-based questionnaire for 2022 are provided in Attachment WEB-1.

3. Methods to Maximize Response Rates

In 2019, the weighted response rates were 71 percent for screening and 65 percent for interviews, with an overall response rate (screening * interview) of 46 percent. Although the $30.00 incentive will be continued for the 2022 survey year, SAMHSA expects the weighted response rates for 2022 to be slightly lower than the 2019 rates because of the potential for the continued impact of COVID-19.

SAMHSA and Contractor staff work continuously with FIs to ensure all necessary steps are taken so that the best response rates possible are achieved. Despite those efforts, as shown in Table 2 (“Screening, Interview, and Overall NSDUH Weighted Response Rates, by Year” in Section A.9), NSDUH response rates continue to decline. This decline of the screening, interview and overall response rates over the past 10 years can be attributed to three main factors:

• Increase in respondent refusals;

• Decrease in refusal conversion; and

• Increase in controlled access barriers (i.e., gated communities, secured buildings, etc.).

Refusals at the screening and interviewing level have historically been a problem for NSDUH due to: 1) respondents feeling they are too busy to participate; 2) respondents feeling inundated with market research and other survey requests; 3) growing concerns about providing personal information due to raised awareness of identity theft and leaks of government and corporate data; and 4) concerns about privacy and increased anti-government sentiment.

The $30.00 cash incentive for interview completion was implemented beginning with the 2002 NSDUH (Wright et al., 2005). The decision to offer an incentive was based largely on an experiment conducted in 2001, which showed that providing incentives of this amount appeared to increase response rates. Wright and his coauthors explored the effect that the incentive had on nonresponse bias. The sample data were weighted by likelihood of response between the incentive and non-incentive cases. Next, a logistic regression model was fit using substance use variables and controlling for other demographic variables associated with either response propensity or drug use. The results indicate that for past year marijuana use, the incentive either encourages users to respond who otherwise would not respond or encourages respondents who would have participated without the incentive to report more honestly about drug use. Therefore, it is difficult to determine whether the incentive money is reducing nonresponse bias, response bias, or both. However, reports of past year and lifetime cocaine did not increase in the incentive category, and report of past month use of cocaine actually was lower in the incentive group than in the control group.

In addition to the $30.00 cash incentive and contact materials, to achieve the expected response rates, the 2022 NSDUH will continue utilizing study procedures designed to maximize respondent participation. This begins with assignment of the cases prior to the start of field data collection, accompanied by weekly response rate goals that are conveyed to the FIs by the FS. When making assignments, FSs take into account which FIs are in closest proximity to the work, FI skill sets, and basic information about the segment (demographics, size, etc.). FSs assign cases to the FIs in order to ensure maximum production levels at the start of the data collection period. To successfully complete work in remote segments or where no local FI is available, a traveling FI (i.e., a veteran NSDUH FI with demonstrated performance and commitment to the study) or a “borrowed” FI from another FS region can be utilized to prevent delays in data collection.

Once FIs transmit their work, data are processed and summarized in daily reports posted to a web-based case management system (CMS) accessed by FSs, which requires two-factor authentication for log-in as part of a FIPS-Moderate environment. On a daily basis, FSs use reports on the CMS to review response rates, production levels, and record of call information to determine an FI’s progress toward weekly goals, to determine when FIs should attempt contact with a case, and to develop plans to handle challenging cases such as refusal cases and cases where an FI is unable to access the dwelling unit. FSs discuss this information with FIs on a weekly basis. Whenever possible, cases are transferred to available FIs with different skill sets to assist with refusal conversion attempts or to improve production in areas where the original FI has fallen behind weekly response rate goals.

Additionally, FSs hold group calls with FI teams to continuously improve FIs’ refusal conversion skills. FIs report the types of refusals they are currently experiencing, and the FS leads the group discussion on how to best respond to the objections and successfully address these challenges. FSs regularly, on group and individual FI calls, work to increase the FI’s ability to articulate pertinent study information in response to common respondent questions.

Periodically throughout the year, response rate patterns are analyzed by state. States with significant changes are closely scrutinized to uncover possible reasons for the changes. Action plans are put in place for states with significant declines. Response and nonresponse patterns are also tracked by various demographics on an annual basis in the NSDUH Data Collection Final Report. The report provides detailed information about noncontacts versus refusals, including reasons for refusals. This information is reviewed annually for changes in trends.

As noted in section B.2 above, FIs may use a Sorry I Missed You Card (Attachment F), NSDUH Highlights and Newspaper Articles (Attachment R), and a Certificate of Participation (Attachment S) to help make respondent contact and encourage participation. To aid in refusal conversion efforts, Refusal Letters (Attachment K) tailored to specific refusal reasons can be sent to any case that has refused. Similarly, an Unable-to-Contact Letter (Attachment J) may be sent to a selected household if the FI has been unable to contact a resident after multiple attempts. For cases where FIs have been unable to gain access to a group of SDUs due to some type of access barrier, such as a locked gate or doorperson, Controlled Access Letters (Attachment J) can be sent to the gatekeeper to obtain his or her assistance in gaining access to the units. In situations where a doorperson is restricting access, the FI can provide that doorperson a card to read to selected residents over the phone or intercom (Attachment U) seeking permission to allow the FI into the building.

Aside from refusals and controlled access barriers, other field challenges attributing to a decline in response rates include:

• An increase of respondents who are home but simply do not answer the door. In these instances, FIs try various conversion strategies (i.e., sending unable to contact/conversion letters) to address potential concerns that a respondent may have, while also respecting their right to refuse participation.

• An increase in homes with video doorbells (RING, NEST, etc.), where a respondent can view someone at their door via a Smartphone. In these situations, FIs introduce themselves, show their ID badge and note that they are following up on the lead letter the respondent should have received in the mail.

• An increase in respondents broadcasting messages across neighborhood social media platforms, such as Nextdoor.com, Facebook, etc. about FIs working in the area and making false claims that the study is a scam and warning residents not to open their door. In these situations, FIs inform the local police department they are working in the general area to address any concerns raised by the residents.

• An increase in the number of respondents calling into the project’s toll-free number to refuse participation before ever being visited in person by an FI. In these instances, both the FS and FI are notified and the case is coded as a final refusal and no subsequent visits are made to that address. These respondents will also be reminded of their option to participate via web.

• An increase in hostile and adamant refusals. These cases are coded out as a final refusal and no subsequent visits are made to that address.

In addition to procedures outlined in this section, NSDUH currently employs the following procedures to avert refusal situations in an effort to increase response rates:

• All aspects of NSDUH are designed to project professionalism and thus enhance the legitimacy of the project, including the NSDUH respondent website and materials. FIs are instructed to always behave professionally and courteously.

• The NSDUH Field Interviewer Manual includes specific instructions to FIs for introducing both themselves and the study. Additionally, an entire chapter is devoted to “Obtaining Participation” and lists the tools available to field staff along with tips for answering questions and overcoming objections.

• During new-to-project FI training, trainers cover details for contacting dwelling units and how to deal with reluctant respondents and difficult situations. During exercises and mock interviews, FIs practice answering questions and using letters and handouts to obtain cooperation.

• During veteran FI training, time is spent reviewing various techniques for overcoming refusals. The exercises and ideas presented help FIs improve their skills and thus increase their confidence and ability to handle the many situations encountered in the field.

Data have also shown that experienced FIs are more successful in converting refusals. Beginning in 2018, NSDUH implemented a yearly FI Tenure-Incentive Plan to reward FIs monetarily for their dedication to the project and ultimately to reduce FI attrition and ensure more experienced FIs are retained.

Also, for 2022, NSDUH data will continue to be collected from respondents using a combination of in-person and web-based data collection modes. Like respondent incentives and FI training and procedures, the web option has the potential to address nonresponse factors observed in recent NSDUH data. First, a second mode for completing the survey offers an alternative to respondents who would prefer not to complete the survey with an FI. A significant portion of respondents might simply have a strong preference for participating via web mode vs. with an FI-administered mode (Olson, Smyth & Wood, 2012). Others might be uncomfortable allowing an FI in their home and be more comfortable completing the survey via web mode. The COVID-19 pandemic could, at least temporarily, continue to increase respondent concerns about allowing FIs into their homes. Web mode provides an option to respondents who would otherwise not participate due to their preferences or concerns regarding FI administration.

Second, adding web mode has the potential to address historical response rate challenges affecting NSDUH data collection. As noted in section B.3, controlled access barriers such as gated communities and secured buildings have increased significantly in the past 10 years. Despite the controlled access procedures and materials field developed and refined for overcoming barriers, FIs are frequently unable to access these properties. Web mode provides an alternative means of response for members of sample dwelling units in controlled access settings to which FIs cannot gain access. Other increasingly common barriers include people not answering door to unexpected visitors and community forums that broadcasting false information about NSDUH of the FI assigned to their area. Web mode provides members of SDUs another option to determine the legitimacy and purpose of NSDUH and how to participate in the study without interacting in person with an FI.

Nonresponse Bias Studies

In addition to the investigations noted above, several studies have been conducted over the years to assess nonresponse bias in NSDUH. For example, the 1990 NSDUH⁶ was one of six large federal or federally-sponsored surveys used in the compilation of a dataset that then was matched to the 1990 decennial census for analyzing the correlates of nonresponse (Groves and Couper, 1998). In addition, data from surveys of NSDUH FIs were combined with those from these other surveys to examine the effects of FI characteristics on nonresponse. One of the main findings was that those households with lower socioeconomic status were no less likely to cooperate than those with higher socioeconomic status; there was instead a tendency for those in high-cost housing to refuse survey requests, which was partially accounted for by residence in high-density urban areas. There was also some evidence that FIs with higher levels of confidence in their ability to gain participation achieved higher cooperation rates.

In follow-up to this research, a special study was undertaken on a subset of nonrespondents to the 1990 NSDUH to assess the impact of the nonresponse (Caspar, 1992). The aim was to understand the reasons people chose not to participate, or were otherwise missed in the survey, and to use this information in assessing the extent of the bias, if any, that nonresponse introduced into the 1990 NSDUH estimates. The study was conducted in the Washington, DC, area, a region with a traditionally high nonresponse rate. The follow-up survey design included a $10 incentive and a shortened version of the instrument. The response rate for the follow-up survey was 38 percent. Follow-up respondents appeared to have similar demographic characteristics to the original NSDUH respondents. Estimates of drug use for follow-up respondents showed patterns that were similar to the regular NSDUH respondents. Another finding was that among those who participated in the follow-up survey, one-third were judged by FIs to have participated in the follow-up because they were unavailable for the main survey request. Finally, 27 percent were judged to have been swayed by the incentive, and another 13 percent were judged to have participated in the follow-up due to the shorter instrument. Overall, the results did not demonstrate definitively either the presence or absence of a serious nonresponse bias in the 1990 NSDUH. Based on these findings, no changes were made to NSDUH procedures.

CBHSQ produced a report to address the nonresponse patterns obtained in the 1999 NSDUH (Eyerman et al., 2002). In 1999, the NSDUH changed from PAPI to CAI instruments. The report was motivated by the relatively low response rates in the 1999 NSDUH. The analyses presented in that report were produced to help provide an explanation for the rates in the 1999 NSDUH and guidance for the management of future projects. The report describes NSDUH data collection patterns from 1994 through 1998. It also describes the data collection process in 1999 with a detailed discussion of design changes, summary figures and statistics, and a series of logistic regressions comparing 1998 with 1999 nonresponse patterns. The results of this study were consistent with conventional wisdom within the professional survey research field and general findings in survey research literature: the nonresponse can be attributed to a set of FI influences, respondent influences, design features, and environmental characteristics. The nonresponse followed the demographic patterns observed in other studies, with urban and high crime areas having the worst rates. Finally, efforts taken in 1999 to improve the response rates were effective. Unfortunately, the tight labor market combined with the large increase in sample size caused these efforts to lag behind the data collection calendar. The authors used the results to generate several suggestions for the management of future projects. No major changes were made to NSDUH as a result of this research, although it—along with other general survey research findings—has led to minor tweaks to respondent cooperation approaches.

In 2004, focus groups were conducted with NSDUH FIs on the topic of nonresponse among the 50 or older age group to gather information on the root causes for differential response by age. The study examined the components of nonresponse (refusals, noncontacts, and other incompletes) among the 50 or older age group. It also examined respondent, environmental, and FI characteristics in order to identify the correlates of nonresponse among the 50 or older group, including relationships that are unique to this group. Finally, they considered the root causes for differential nonresponse by age, drawing from focus group sessions with NSDUH FIs on the topic of nonresponse among the 50 or older group. The results indicated that the high rate of nonresponse among the 50 or older age group was primarily due to a high rate of refusals, especially among sample members aged 50 to 69, and a high rate of physical and mental incapability among those 70 or older. It appeared that the higher rate of refusals among the 50 or older age group may, in part, have been due to fears and misperceptions about the survey and FIs’ intentions. It was suggested that increased public awareness about the study may allay these fears (Murphy et al., 2004).

In 2005, Murphy et al. sought a better understanding of nonresponse among the population 50 or older in order to tailor methods to improve response rates and reduce the threat of nonresponse error (Attachment V, Nonresponse among Sample Members Aged 50 and Older Report). Nonresponse to the NSDUH is historically higher among the 50 or older age group than lower age groups. Focus groups were again conducted, this time with potential NSDUH respondents to examine the issue of nonresponse among persons 50 or older. Participants in these groups recommended that the NSDUH contact materials focus more on establishing the legitimacy of the sponsoring and research organizations, clearly conveying the survey objectives, describing the selection process, and emphasizing the importance of the selected individual’s participation.

Another examination of nonresponse was done in 2005. The primary goal was to develop a methodology to reduce item nonresponse to critical items in the ACASI portion of the NSDUH questionnaire (Caspar et al., 2005). Respondents providing “Don’t know” or “Refused” responses to items designated as essential to the study's objectives received tailored follow-up questions designed to simulate FI probes. Logistic regression was used to determine what respondent characteristics tended to be associated with triggering follow-up questions. The analyses showed that item nonresponse to the critical items is quite low, so the authors caution the reader to interpret the data with care. However, the findings suggest the follow-up methodology is a useful strategy for reducing item nonresponse, particularly when the nonresponse is due to “Don’t know” responses. In response, follow-up questions were added to the survey and asked when respondents indicated they did not know the answer to a question or refused to answer a question. These follow-up items encouraged respondents to provide their best guess or presented an assurance of data confidentiality in order to encourage response.

Biemer and Link (2007) conducted additional nonresponse research to provide a general method for nonresponse adjustment that relaxed the ignorable nonresponse assumption. Their method, which extended the ideas of Drew and Fuller (1980), used level-of-effort (LOE) indicators based on call attempts to model the response propensity. In most surveys, call history data are available for all sample members, including nonrespondents. Because the LOE required to interview a sample member is likely to be highly correlated with response propensity, this method is ideally suited for modeling the nonignorable nonresponse. The approach was first studied in a telephone survey setting and then applied to data from the 2006 NSDUH, where LOE was measured by contact attempts (or callbacks) made by FIs.

The callback modeling approach investigation confirmed what was known from other studies on nonresponse adjustment approaches (i.e., there is no uniformly best approach for reducing the effects of nonresponse on survey estimates). All models under consideration were the best in eliminating nonresponse bias in different situations using various measures. Furthermore, possible errors in the callback data reported by FIs, such as underreporting of callback attempts, raise concerns about the accuracy of the bias estimates. Unfortunately, it is very difficult to apply uniform callback reporting procedures amongst the large NSDUH interviewing staff, which is spread across the country. An updated study of the callback modeling approach was completed in 2013 and similar conclusions were reached on errors in the callback data reported by FIs (Biemer, Chen, and Wang, 2013). For these reasons, the callback modeling approach was not implemented in the NSDUH nonresponse weighting adjustment process (Biemer and Link, 2007; Biemer, Chen, and Wang, 2013).

Methods to assess nonresponse bias vary and each has its limitations (Groves, 2006). Some methods include follow-up studies, comparisons to other surveys, comparing alternative post-survey adjustments to examine imbalances in the data, and analysis of trend data that may suggest a change in the characteristics of respondents. When comparing to other surveys such as Monitoring the Future (MTF), comparable trends were found even though the estimates themselves differ in magnitude (mostly due to differences in survey designs). For example, trends in NSDUH and MTF cigarette use between 2002 and 2012 show a consistent pattern:

https://www.samhsa.gov/data/sites/default/files/NSDUHresults2012/NSDUHresults2012.htm#fig8.2

Using a different approach, information in a NSDUH report suggests that alternative weighting methods based on variables correlated with nonresponse were not better or only slightly better than the current weighting procedure:

https://www.samhsa.gov/data/sites/default/files/NSDUHCallbackModelReport2013/NSDUHCallbackModelReport2013.pdf

In April 2015, a study was conducted on the potential for declines in the screening response rate to introduce nonresponse bias for trend estimates. The primary purpose of these analyses was to examine any potential impact of steep declines in the screening response rate observed in 2013 and the first three quarters of 2014 on nonresponse bias. The common approach in these analyses was to examine changes over time from 2007 through 2014, rather than estimating nonresponse bias only at one point in time.

The first analysis evaluated whether there were any apparent changes in the reliance on variables in the screening nonresponse weighting adjustment models over time.

The second analysis used household level demographic estimates from the Current Population Survey (CPS) as a “gold standard” basis of comparison for similar estimates constructed from the NSDUH screening. CPS quarterly estimates related to household size, age, gender, race, Hispanic origin and active military duty status were compared with NSDUH selection weighted estimates (i.e. NSDUH base weights that do not adjust for nonresponse and are not adjusted to population control totals).

The third analysis used data from the National Health Interview Survey (NHIS) to compare estimates of cigarette smoking and alcohol consumption over time with estimates from NSDUH. Overall, the results of these analyses provided little evidence of any systematic change in trend estimates during a period of relatively steeper declines in screening response rates. In January 2018, NSDUH benchmark comparisons with CPS and NHIS were reproduced through the 2016 survey year which confirmed earlier findings.

SAMHSA plans to conduct a new nonresponse bias study in the spring of 2022, using 2021 NSDUH data. As part of the study, data collected in-person from web nonrespondents will be used to evaluate potential nonresponse bias in estimates during periods with severely restricted or suspended in-person data collection. Demographic characteristics of web and in-person respondents have already been compared using 2020 data, and weighting methodology was adjusted to compensate for the observed differences between the two modes in levels of educational attainment. Prior to the pandemic, a nonresponse bias analysis was begun in which small area estimates of both outcomes and response propensities were tested for correlations, which would indicate nonresponse bias. These and other approaches may be used to evaluate nonresponse bias.

The report from the nonresponse bias investigation conducted in 2022 will be peer-reviewed. Two experts in the area of survey nonresponse, steeped in both survey theory and practical operations, will be recruited to review the report, much like a journal article is peer-reviewed. The feedback will help ensure that NSDUH’s sample design meets the evolving needs of its stakeholders.

Multimode Response Rate Improvement Ideas for Future NSDUHs

The use of multi-mode data collection, with the mix of in-person and web-based options, was initially implemented in October 2020 (and carried into 2021) in response to the COVID-19 pandemic. This multi-mode data collection allowed for successful collection of NSDUH data to provide nationally representative estimates on the effects of COVID-19 on substance use and mental health in the United States. NSDUH continues to explore ideas for maximizing respondent participation across modes. In addition to the methods already discussed in this section, SAMHSA is considering the following ideas for improving NSDUH multimode response rates in the future:

• Conducting a screening and/or interview incentive experiment (See Incentive Experiment Addendum);

• Conducting nonresponse bias analyses;

• Conducting expert reviews and focus groups on multimode materials for potential improvements;

• Conducting experiments involving methodology for parental permission for web-based interviews;

• Implementing phone and/or text interview and parental consent reminders; and

• Implementing an online scheduler for in-person interviews.

4. Tests of Procedures

There are no plans for additional field testing prior to the start of 2022 data collection. The use of multi-mode data collection, with the mix of in-person and web-based options, was initially implemented in October 2020 (and carried into 2021) in response to the COVID-19 pandemic. This multi-mode data collection allowed for successful collection of NSDUH data to provide nationally representative estimates on the effects of COVID-19 on substance use and mental health in the United States. In addition, many of the recent modifications to the questionnaire have already been tested under NSDUH Methodological Field Tests generic OMB clearance (OMB No. 0930-0290).

5. Statistical Consultants

The basic NSDUH design was reviewed by statistical experts, both within and outside SAMHSA. Statistical experts reviewing portions of prior NSDUHs designs include William Kalsbeek, PhD, University of North Carolina; Robert Groves, PhD, Georgetown University; and Michael Hidiroglou, PhD, Statistics Canada. Monroe Sirken, PhD, National Center for Health Statistics (NCHS) (deceased); Arthur Hughes, CBHSQ, SAMHSA (retired); James Massey, PhD, (deceased) also of NCHS; Douglas Wright, CBHSQ, SAMHSA (retired); and Joseph Gfroerer, CBHSQ, SAMHSA (retired) were consulted on the 1992 and subsequent survey designs. Rong Cai, CBHSQ, SAMHSA is the Government Project Officer responsible for overall project management, (240) 276-1288. Dr. Ahmed Khago, Chief, Population Surveys Branch, Division of Surveillance and Data Collection, CBHSQ, SAMHSA is the primary mathematical statistician, (240) 276-1257. RTI senior statisticians contributing to the design are Ralph Folsom, PhD (retired), Rachel Harter, PhD, and Akhil Vaish, PhD.

The 2018–2022 National Survey on Drug Use and Health contract was awarded to RTI International (RTI) on December 15, 2016, with only the Base Award (2018 NSDUH) exercised initially. The 2022 NSDUH was exercised by SAMHSA for RTI on October 1, 2020. Contractor personnel will implement the sample design; recruit FSs and FIs; train FIs; conduct data collection; conduct data receipt, editing, coding, and keying; conduct data analysis; and develop and deliver to CBHSQ statistical reports and data files. CBHSQ will provide direction and review functions to the Contractor. Data collection will be conducted throughout the 2022 calendar year.

Appendix A

Potential NSDUH Consultants

Michael Arthur, PhD, Research Associate Professor (206) 685-3858

Social Development Research Group, School of Social Work

University of Washington

Raul Caetano, M.D., PhD, Dean (214) 648-1080

School of Health Professions

University of Texas Southwestern

John Carnevale, PhD, President and CEO (301) 977-3600

Carnevale Associates, LLC

William D. Kalsbeek, PhD, Professor/Founder (919) 962-3249

Carolina Survey Research Laboratory, Biostatistics

University of North Carolina at Chapel Hill

Graham Kalton, PhD, (301) 251-8253

Chairman of the Board

Westat

Philip Leaf, PhD, Professor (410) 955‑3962

Department of Mental Health Policy and Management

School of Public Health

Johns Hopkins University

Patrick O’Malley, PhD, Senior Research Scientist (734) 763-5043

Population Studies Center, The Institute for Social Research

University of Michigan

Peter Reuter, PhD, Professor (301) 405-6367

School of Public Policy and Department of Criminology

University of Maryland

References

Biemer, P., & Link, M. (2007). Evaluating and modeling early cooperator bias in RDD surveys. In Lepkowski, J. et al. (Eds.), Advances in telephone survey methodology. Hoboken, NJ: John Wiley & Sons.

Biemer, P., Chen, P., and Wang, K. (2013). Incorporating level of effort paradata in the NSDUH nonresponse adjustment process. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Behavioral, Health Statistics and Quality.

Caspar, R.A. (1992). A follow-up study of nonrespondents to the 1990 National Household Survey on Drug Abuse. In Proceedings of the1992 American Statistical Association, Survey Research Section, Boston, MA (pp. 476-481). Alexandria, VA: American Statistical Association.

Caspar, R. A., Penne, M. A., & Dean, E. (2005). Evaluation of follow-up probes to reduce item nonresponse in NSDUH. In J. Kennet & J. Gfroerer (Eds.), Evaluating and improving methods used in the National Survey on Drug Use and Health (HHS Publication No. SMA 05-4044, Methodology Series M-5, pp. 121-148). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

Drew, J.H. and Fuller, W.A. (1980). Modeling Nonresponse in Surveys with Callbacks. Proceedings of the American Statistical Association, Survey Research Methods Section, 639–642.

Eyerman, J., Odom, D., Butler, D., Wu, S., and Caspar, R. (2002). Nonresponse in the 1999 NHSDA. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.

Gfroerer, J., Wright, D., & Kopstein, A. (1997). Prevalence of youth substance use: The impact of methodological differences between two national surveys. Drug and Alcohol Dependence, 47, 19–30.

Groves, R. M., and M.P. Couper. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.

Groves, R. M. (2006). Nonresponse Rates and Nonresponse Bias in Household Surveys, Public Opinion Quarterly, 70(5), pp. 646-675.

Groves, R. (1989). Survey Errors and Survey Costs. New York: Wiley.

Grucza, R. A., Abbacchi, A. M., Przybeck, T. R., & Gfroerer, J. C. (2007). Discrepancies in estimates of prevalence and correlates of substance use and disorders between two national surveys. Addiction, 102, 623-629.

Haro, J. M., Arbabzadeh‐Bouchez, S., Brugha, T. S., De Girolamo, G., Guyer, M. E., Jin, R., ... & Sampson, N. A. (2006). Concordance of the Composite International Diagnostic Interview Version 3.0 (CIDI 3.0) with standardized clinical assessments in the WHO World Mental Health surveys. International journal of methods in psychiatric research, 15(4), 167-180.

Harrison, L. D. (2001). Understanding the differences in youth drug prevalence rates produced by the MTF, NHSDA, and YRBS studies. Journal of Drug Issues, 31, 665-694.

Hasin, D. S., Greenstein, E., Aivadyan, C., Stohl, M., Aharonovich, E., Saha, T., ... & Grant, B. F. (2015). The Alcohol Use Disorder and Associated Disabilities Interview Schedule-5 (AUDADIS-5): procedural validity of substance use disorders modules through clinical re-appraisal in a general population sample. Drug and alcohol dependence, 148, 40-46.

Hennessy, K., & Ginsberg, C. (Eds.). (2001). Substance use survey data collection methodologies [Special issue]. Journal of Drug Issues, 31(3), 595–727.

Jordan, B. K., Karg, R. S., Batts, K. R., Epstein, J. F., & Wiesen, C. (2008). A clinical validation of the National Survey on Drug Use and Health assessment of substance use disorders. Addictive Behaviors, 33(6), 782-798.

Kessler, R. C., Adler, L., Barkley, R., Biederman, J., Conners, C. K., Demler, O., ... & Spencer, T. (2006). The prevalence and correlates of adult ADHD in the United States: results from the National Comorbidity Survey Replication. American Journal of psychiatry, 163(4), 716-723.

Miller, J. W., Gfroerer, J. C., Brewer, R. D., Naimi, T. S., Mokdad, A., & Giles, W. H. (2004). Prevalence of adult binge drinking: A comparison of two national surveys. American Journal of Preventive Medicine, 27, 197-204.

Murphy, J., Eyerman, J., & Kennet, J. (2004). Nonresponse among persons aged 50 or older in the National Survey on Drug Use and Health. In S. B. Cohen & J. M. Lepkowski (Eds.), Eighth Conference on Health Survey Research Methods (HHS Publication No. PHS 04-1013, pp. 73-78). Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.

National Center for Health Statistics. (2018, June). 2017 National Health Interview Survey public use data release: Survey description. Retrieved from ftp://ftp.cdc.gov/pub/Health_Statistics/NCHS/Dataset_Documentation/NHIS/2017/srvydesc.pdf

Olson, K., Smyth, J., and Wood, H. (2012). Does giving people their preferred survey mode actually increase survey participation rates? An experimental examination. Public Opinion Quarterly 76: 611-635

Wright, D., Bowman, K., Butler, D., & Eyerman, J. (2005). Non-response bias from the National Household Survey on Drug Abuse incentive experiment. Journal for Social and Economic Measurement, 30(2-3), 219-231.

Attachments

Attachment A. Sample Design

Attachment B. In-Person CAI Questionnaire

Attachment C. Lead Letter

Attachment D. Question & Answer Brochure

Attachment E. Tablet Screening Video Scripts

Attachment F. Contact Cards – Sorry I Missed You Card and Appointment Cards

Attachment G. Study Description

Attachment H. Introduction and Informed Consent Scripts

Attachment I. In-Person Screening Questions

Attachment J. Unable-to-Contact and Controlled Access Letters

Attachment K. Refusal Letters

Attachment L. Interview Incentive Receipt

Attachment M. Federalwide Assurance

Attachment N. Showcard Booklet

Attachment O. Parental Introductory Script

Attachment P. Confidentiality Agreement and Data Collection Agreement

Attachment Q. COVID-19 Risk Information Form

Attachment R. NSDUH Highlights and Newspaper Articles

Attachment S. Certificate of Participation

Attachment T. CATI Verification Scripts

Attachment U. Doorperson Card

Attachment V. Nonresponse among Sample Members Aged 50 and Older Report

Attachment WEB-1. Web-based CAI Questionnaire

Attachment WEB-2. Web Study Description

Attachment WEB-3. Parental Permission Script

Attachment WEB-4. Web-based Screening Questions

Attachment WEB-5. Follow-up Letters

Attachment WEB-6. Incentive Thank You Letters

Attachment WEB-7. Web Question & Answer Brochure

1 Other studies that have employed a hybrid ABS design include the National Health Interview Survey, the Residential Energy Consumption Survey, and the U.S. Food & Drug Administration Tobacco Consumer Studies Panel.

2 Compared with geocoding at the census block level, geocoding accuracy improves significantly at the census block group level in both rural and urban areas. Thus, in the planned hybrid ABS design, census block groups will serve as geographic clusters in areas with sufficient mailing address coverage.

3 A drop point is a mail receptacle that is shared by multiple housing units (drop units). The ABS frame indicates the number of units at a drop point but does not include unit identifiers (e.g., apartment numbers). Drop points present additional challenges for sample implementation because of their one-to-many relationship to dwelling units.

4 Substance Abuse and Mental Health Services Administration. (2019). Address-based sampling research report. Rockville, MD: Center for Behavioral Health Statistics and Quality, Substance Abuse and Mental Health Services Administration. Retrieved from https://www.samhsa.gov/data/.

5 http://aspe.hhs.gov/datacncl/standards/ACA/4302/index.shtml

6 Prior to 2002, the NSDUH was referred to as the National Household Survey on Drug Abuse (NHSDA). In this document the term NSDUH is used for all survey years.

14