2015 National Survey on Drug Use and Health
SUPPORTING STATEMENT
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
The respondent universe for the 2015 NSDUH is the civilian, noninstitutionalized population aged 12 or older within the 50 States and the District of Columbia. The NSDUH universe includes residents of noninstitutional group quarters (e.g., shelters, rooming houses, dormitories), residents of Alaska and Hawaii, and civilians residing on military bases. Persons excluded from the universe include those with no fixed household address (e.g., homeless transients not in shelters, and residents of institutional group quarters such as jails and hospitals).
Similar to previous NSDUHs, the sample design consists of a stratified, multi-stage area probability design (see Attachment A for a detailed presentation of the Sample Design). As with most area household surveys, the NSDUH design continues to offer the advantage of minimizing interviewing costs by clustering the sample. This type of design also maximizes coverage of the respondent universe since an adequate dwelling unit and/or person-level sample frame is not available. Although the main concern of area surveys is the potential variance-increasing effects due to clustering and unequal weighting, these potential problems are directly addressed in the NSDUH by selecting a relatively large sample of clusters at the early stages of selection and by selecting these clusters with probability proportionate to a composite size measure. This type of selection maximizes precision by allowing one to achieve an approximately self-weighting sample within strata at the latter stages of selection. Furthermore, it is appealing because the design of the composite size measure makes the FI workload roughly equal among clusters within strata.
A coordinated design has been developed for the 2014-2017 NSDUHs. To accurately measure drug use and related mental health measures among the aging drug use population, this design allocates the sample to the 12 to 17, 18 to 25, and 26 or older age groups in proportions of 25 percent, 25 percent, and 50 percent, respectively.
The 2014-2017 sample has been selected down to the area segment level. The sample selection procedures begin by geographically partitioning each State into roughly equal size State sampling regions (SSRs). Regions are formed so that each area within a State yields, in expectation, roughly the same number of interviews during each data collection period. As shown in Table 1 of Attachment A, this partition divides the U.S. into 750 SSRs.
Within each of the 750 SSRs formed for the 2014-2017 NSDUHs, a sample of Census tracts is selected. Then, within sampled Census tracts, Census block groups are selected. This stage of selection facilitates possible transitioning to an address-based sampling (ABS) design in the future. Finally, within Census block groups, smaller geographic areas, or segments, are selected. A total of 48 segments per SSR are selected: 20 to field the 2014-2017 surveys and 28 to support any supplemental studies embedded within NSDUH or to extend the sample beyond 2017, if desired. In general, segments consist of adjacent Census blocks and are equivalent to area segments selected at the second stage of selection in the 2005-2013 NSDUHs.
In summary, the first-stage stratification for the 2014-2017 Studies is States and SSRs within States, the first-stage sampling units are Census tracts, the second-stage sampling units are Census block groups, and the third-stage sampling units are small area segments. This design for the 2014-2017 NSDUHs at the first stages of selection is desirable because of (a) the large person-level sample required at the latter stages of selection in the design and (b) continued interest among NSDUH data users and policymakers in State and other local-level statistics.
The coordinated design facilitates 50 percent overlap in third-stage units (area segments) between each two successive years from 2014 through 2017. The primary benefit of the sample overlap is the cost savings achieved from being able to reuse the list frames for half of the area segments in the 2015 through 2017 surveys. In addition, the expected precision of difference estimates generated from consecutive years (e.g., the year-to-year difference in past month marijuana use among 12- to 17-year-old respondents) is improved because of the expected positive correlation resulting from the overlapping sample.
Similar to previous NSDUHs, at the latter stages of selection, five age group strata are sampled at different rates. These five strata are defined by the following age group classifications: 12 to 17, 18 to 25, 26 to 34, 35 to 49, and 50 or older. Adequate precision for race/ethnicity estimates at the national level is achieved with the larger sample size and the allocation to the age group strata. Consequently, race/ethnicity groups are not over-sampled. However, consistent with previous NSDUHs, the 2015 NSDUH is designed to over-sample the younger age groups.
Table 1 in Attachment A shows the projected number of person respondents by State and age group. Table 2 (Attachment A) shows main study sample sizes and the projected number of completed interviews by sample design stage. Table 3 (Attachment A) shows the expected precision for key measures by demographic domain.
Unless otherwise specified, the 2015 NSDUH procedures described in this section follow the same processes as used on the 2014 NSDUH.
Prior to the FI’s arrival at the SDU, a Lead Letter (see Attachment E) will be mailed to the resident(s) briefly explaining the survey and requesting their cooperation. As described in section A.1, this letter has been updated for 2015 to reflect changes made for and tested as part of the QFT and DR. It will be printed on HHS letterhead with the signature of the HHS National Study Director and the Contractor’s National Field Director.
Upon arrival at the SDU, the FI will refer the resident to this letter and answer any questions. If the resident has no knowledge of the Lead Letter, the FI will provide another copy, explain that one was previously sent, and then answer any questions. If no one is home during the initial visit to the SDU, the FI may leave a Sorry I Missed You Card (Attachment J) informing the resident(s) that the FI plans to make another callback at a later date/time. Callbacks will be made as soon as feasible following the initial visit. FIs will attempt to make at least four callbacks (in addition to the initial call) to each SDU in order to complete the screening process and complete an interview, if yielded.
If the FI is unable to contact anyone at the SDU after repeated attempts, the FS may send one of the Unable-to-Contact (UTC) letters (Attachment N). These UTC letters reiterate information contained in the Lead Letter and present a plea for the resident to participate in the study. If after sending the UTC letter, an FI is still unable to contact anyone at an SDU, a Call-Me letter (Attachment N) may be sent to the SDU requesting that the resident(s) call the FS as soon as possible to set up an appointment for the FI to visit the resident(s).
When in‑person contact is made with an adult member of the SDU and introductory procedures are completed, the FI will present a Study Description (Attachment K) and answer any questions that person might have concerning the study. A Question & Answer Brochure (Attachment F) that provides answers to commonly asked questions may also be given. As described in section A.1, this brochure is new for 2015 but was successfully used in the QFT and DR. In addition, FIs are supplied with copies of the NSDUH Highlights & Newspaper Articles (Attachment V) for use in eliciting participation, which can be left with the respondent.
Also, new for 2015, the FI may utilize the multimedia capability of the touch screen tablet to display a short video (approximately 50 seconds total run time) for members of the SDU to view, which provides a brief explanation of the study and why participation is important. The script for that video is included as Attachment G.
If a potential respondent refuses to be screened, the FI has been trained to accept the refusal in a positive manner, thereby minimizing the possibility of creating an adversarial relationship that might preclude future opportunities for contact. The FS may then request that one of several Refusal Letters (Attachment O) be sent to the residence. The letter sent is tailored to the specific concerns expressed by the potential respondent and asks him or her to reconsider participation. Refusal letters are customized and also include the FS’s phone number in case the potential respondent has questions or would like to set up an appointment with the FI. Unless the respondent calls the FS or the Contractor’s office to refuse participation, an in-person conversion is then attempted by specially-selected FIs with successful conversion experience.
With respondent cooperation, the FI will begin screening the SDU by asking either the Housing Unit Screening questions, or the Group Quarters Unit Screening questions, as appropriate. The screening questions are administered using a 7-inch touch screen Android tablet computer. A paper representation of the housing unit and group quarters unit screening process is shown in Attachment M.
Once all household members aged 12 or older have been rostered, the hand-held computer performs the within-dwelling-unit sampling process, selecting zero, one, or two members to complete the interview. For cases with no one selected, the FI asks for a name and phone number for use in verifying the quality of the FI’s work, thanks the respondent, and concludes the household contact.
For each person selected to complete the full interview, the FI follows these steps:
If the selected individual is aged 18 or older, or aged 17 and living independently from his or her parent or guardian, and is currently available, the FI immediately seeks to obtain informed consent. Once consent is obtained, the FI begins to administer the questionnaire in a private setting within the dwelling unit. As necessary and appropriate, the FI may make use of the Appointment Card (in Attachment J) for scheduled return visits with the respondent.
If the selected individual is 12 to 17 years of age, except in rare instances where a 17-year-old is living independently from his or her parent or guardian, in which case the 17-year-old provides his or her own consent, the FI will read the parental introductory script (Attachment H) to the parent or guardian before speaking with the youth about NSDUH. Subsequently, parental consent is sought from the selected individual’s parent or legal guardian using the Parent section of the youth version of the Introduction and Informed Consent Scripts (Attachment L). Once parental consent is granted, the minor is then asked to participate using the Youth section of the same document. If assent is received, the FI begins to administer the questionnaire in a private setting within the dwelling unit with at least one parent, guardian or another adult remaining present in the home throughout the interview.
As mentioned in section A.3, the FI administers the interview in a prescribed and uniform manner with sensitive portions of the interview completed via ACASI. The changes to the 2015 NSDUH questionnaire are described in section A.1.
Race/ethnicity questions are FI-administered and meet all of the guidelines for the OMB minimum categories. The addition of the finer delineation of Guamanian or Chamorro and Samoan, which collapse into the OMB standard Native Hawaiian/Other Pacific Islander category, were a requirement of the new HHS Data Collection Standards and were added to the 2013 and 2014 NSDUH interviews. They will continue to be included in the 2015 questionnaire.1
In order to facilitate the respondent’s recollection of prescription-type drugs and their proper names, as tested during the QFT and DR, pill images will now appear on the laptop screen during the ACASI portions of interviews as appropriate in 2015. Also, respondents will now use an electronic reference date calendar, which displays automatically on the computer screens when needed throughout the ACASI parts of the interview. Finally, in the FI-administered portion of the questionnaire, showcards are included in the Showcard Booklet (Attachment R) that allow the respondent to refer to information necessary for accurate responses. Due to changes throughout the questionnaire, there are subsequently fewer showcards required for 2015.
After the interview is completed and before the verification procedures begin, each respondent is given a $30.00 cash incentive and an Interview Incentive Receipt (P) signed by the FI.
For verification purposes, interview respondents are asked to complete a Quality Control Form (Attachment T) that requests his/her current address and phone number for possible follow‑up to ensure that the FI did his or her job appropriately. Respondents are informed that completing the Quality Control Form is voluntary. If he or she agrees, the respondent completes this form, places it in an envelope and seals it. The form is then mailed to the Contractor’s office for processing. In previous NSDUHs, less than one percent of the verification sample refused to fill out Quality Control Forms.
FIs may give a Certificate of Participation (Attachment W) to interested respondents after the interview is completed. Respondents may attempt to use these certificates to earn school or community service credit hours. As stated on the certificate, no guarantee of credit is made by SAMHSA or the Contractor. The respondent’s name is not written on the certificate. The FI signs his or her name and dates the certificate, but for confidentiality reasons the section for recording the respondent’s name is left blank. The respondent can fill in his/her name at a later time so the FI will not be made aware of the respondent’s identity. It is the respondent’s choice whether he or she would like to be identified as a NSDUH respondent by using the certificate in an attempt to obtain school or community service credit.
A random sample of those who complete Quality Control Forms are contacted via telephone to answer a few questions verifying that the interview took place, that proper procedures were followed, and that the amount of time required to administer the interview was within expected parameters. The CATI Verification Scripts (Attachment X) contain the scripts for these interview verification contacts via telephone, as well as the scripts used when verifying a percentage of certain completed screening cases in which no one was selected for an interview or the SDU was otherwise ineligible (vacant, not primary residence, not a dwelling unit, dwelling unit contains only military personnel, respondents living at the sampled residence for less than half of the quarter). For verification purposes, a Quality Control letter (Attachment Y) is mailed to a respondent’s address when a phone number is not available.
As noted above, all interview data are transmitted on a regular basis via secure encrypted data transmission to the Contractor’s offices, where the data are subsequently processed and prepared for reporting and data file delivery.
Questionnaire
As explained in section A.3, the version of the questionnaire to be fielded in 2015 is a computerized (CAPI/ACASI) instrument based on the QFT and DR questionnaires, unless otherwise noted.
As in past years, two versions of the instrument will be prepared: an English version and a Spanish translation. Both versions will have the same essential content.
The proposed CAI Questionnaire is shown in Attachment U. While the actual administration will be electronic, the document shown is a paper representation of the content that is to be programmed. The interview process is designed to retain respondent interest, ensure confidentiality, and maximize the validity of response. The questionnaire is administered in such a way that FIs do not know respondents’ answers to sensitive questions, including those on illicit drug use and mental health. These questions are self-administered using ACASI. The respondent listens to the questions privately through headphones so even those who have difficulty seeing or reading are able to complete the self-administered portion. Topics that are administered by the FI (i.e., the CAPI section) are limited to Demographics, Health Insurance, and Income. Respondents are given the option of designating an adult proxy who is at home to provide answers to questions in the Health Insurance and Income sections.
The ACASI portion of the questionnaire is divided into sections based on specific substances or other main topics. The same questions are asked for each substance or substance class, ascertaining the respondent’s history in terms of age of first use, most recent use, number of times used in lifetime, and frequency of use in past 30 days and past 12 months. These substance use histories allow estimation of the incidence, prevalence, and patterns of use for licit and illicit substances.
A key feature of the questionnaire is a core-supplement structure. Core questions that are critical for basic trend measurement of substance use incidence and prevalence rates remain in the survey every year and comprise the main part of the questionnaire. The core is comprised of the initial demographic questions and the Tobacco through Sedatives modules. Supplemental questions, or modules, which can be revised, dropped, or added from year to year comprise the remainder of the questionnaire. Supplemental items include the remaining modules, such as the education and employment sections. Some of the supplemental portion of the questionnaire remains in the survey, essentially unchanged, every year (e.g., Consumption of Alcohol).
As explained in section A.1, the 2015 questionnaire has few changes from the previously-tested QFT and DR questionnaires discussed in Attachments B and C (QFT and DR Final Reports). The term “Molly” has been added to questions about Ecstasy in the Hallucinogens module. In addition, questions about health insurance and income that were tested in an ACASI module in the QFT and DR were changed back to being administered by FIs due to increased item nonresponse. Also, several routing updates within the questionnaire will improve the flow of the instrument.
The detailed specifications for the proposed CAI Questionnaire for 2015 are provided in Attachment U. A summary of all specific NSDUH questionnaire and materials changes for 2015, as compared with the 2014 NSDUH, is included in Attachment I (2015 NSDUH Questionnaire and Materials Changes).
In 2013, the weighted response rates were 84 percent for screening and 72 percent for interviews, with an overall response rate (screening * interview) of 60 percent. With the continuation of the $30.00 cash incentive for the 2015 survey year, the Contractor expects the weighted response rates for 2014 to be about the same as the 2013 rates.
The $30.00 cash incentive for interview completion was implemented beginning with the 2002 NSDUH (Wright et al., 2005). The decision to offer an incentive was based largely on an experiment conducted in 2001, which showed that providing incentives appeared to increase response rates. Wright and his coauthors explored the effect that the incentive had on nonresponse bias. The sample data were weighted by likelihood of response between the incentive and nonincentive cases. Next, a logistic regression model was fit using substance use variables and controlling for other demographic variables associated with either response propensity or drug use. The results indicate that for past year marijuana use, the incentive either encourages users to respond who otherwise would not respond, or encourages respondents who would have participated without the incentive to report more honestly about drug use. Therefore, it is difficult to determine whether the incentive money is reducing nonresponse bias, response bias, or both. However, reports of past year and lifetime cocaine did not increase in the incentive category, and report of past month use of cocaine actually was lower in the incentive group than in the control group.
As a way to maximize response rates, the 2015 NSDUH will use the revised Lead Letter (Attachment E) and Question & Answer Brochure (Attachment F) that were used in the DR. Prior to the QFT and DR, CBHSQ revised these two items through a review of contact materials used on other government-sponsored surveys, expert review, and feedback from 17 focus groups conducted in both English and Spanish in five metropolitan areas (OMB No. 0930-0290). The primary focus of these revisions was to improve the 2015 NSDUH materials in ways likely to generate positive reactions from members of sampled households and, therefore, maximize participation.
As shown in table 4.1 of Attachment C, DR Final Report, response rates in the DR were slightly lower than in the QFT and main study in 2012 but comparable to Quarters 3 and 4 main study in 2013. We believe this is due to: (a) an increase in the proportion of sample respondents 26 or older than in the main study; this group historically has lower response rates than individuals under 26; (b) a decrease in the number of DR FIs available to work the widely dispersed DR sample compared to the main study; this restricted field managers’ flexibility to transfer refusal cases to FIs more successful at converting refusals than the typical FI and enabled fewer callback attempts to remote work areas; and (c) the oversampling of areas with historically-high concentrations of Spanish-language interviews to allow for testing of the Spanish-language questionnaire. Despite these findings, we have no reason to believe the revised contact materials negatively impacted participation, and in fact as noted above we believe they are improved over the main study versions; therefore, we will use the revised contact materials for the 2015 NSDUH.
Additional information on revisions to these materials, including details from the expert review and focus groups is provided in the Contact Materials Study Report (Attachment Z), as well as information in the QFT and DR Final Reports (Attachments B and C).
In addition to the $30.00 cash incentive and contact materials, to achieve the expected response rates, the 2015 NSDUH will continue utilizing study procedures designed to maximize respondent participation. This begins with assignment of the cases prior to the start of data collection, accompanied by weekly response rate goals that are conveyed to the FIs by the FS. When making assignments, FSs take into account which FIs are in closest proximity to the work, FI skill sets, and basic information (demographics, size, etc.) about the segment. FSs assign cases to the FIs in order to ensure maximum production levels at the start of the data collection period. To successfully complete work in remote segments or where no local FI is available, a traveling FI (i.e., a veteran NSDUH FI with demonstrated performance and commitment to the study) or a “borrowed” FI from another FS region can be utilized to prevent delays in data collection.
Once FIs transmit their work, data are processed and summarized in daily reports posted to a web-based case management system (CMS) accessed by FSs. On a daily basis, FSs use reports on the CMS to review response rates, production levels, and record of call information to determine an FI’s progress toward weekly goals, to determine when FIs should attempt contact with a case, and to develop plans to handle challenging cases such as refusal cases and cases where an FI is unable to access the dwelling unit. FSs discuss this information with FIs on a weekly basis. Whenever possible, cases are transferred to available FIs with different skill sets to assist with refusal conversion attempts or to improve production in areas where the original FI has fallen behind weekly response rate goals.
Additionally, FSs hold group calls with FI teams to continuously improve staff’s refusal conversion skills. FIs report the types of refusals they are currently experiencing and the FS leads the group discussion on how to best respond to the objections and successfully address these challenges. FSs regularly, on group and individual FI calls, work to increase the FI’s ability to articulate pertinent study information in response to common respondent questions.
Periodically throughout the year, response rate patterns are analyzed by State. States with significant changes are closely scrutinized to uncover possible reasons for the changes. Action plans are put in place for States with significant declines. Response and nonresponse patterns are also tracked by various demographics on an annual basis in the NSDUH Data Collection Final Report. The report provides detailed information about noncontacts versus refusals, including reasons for refusals. This information is reviewed annually for changes in trends.
As noted in section B.2 above, FIs may use a Sorry I Missed You Card (in Attachment J), NSDUH Highlights and Newspaper Articles (Attachment V), and a Certificate of Participation (Attachment W) to help make respondent contact and encourage participation. To aid in refusal conversion efforts, Refusal Letters (Attachment O) tailored to specific refusal reasons can be sent to any case that has refused. Similarly, an Unable-to-Contact Letter (in Attachment N) may be sent to a selected household if the FI has been unable to contact a resident after multiple attempts. For cases where FIs have been unable to gain access to a group of SDUs due to some type of access barrier, such as a locked gate or doorperson, Controlled Access Letters (in Attachment N) can be sent to the gatekeeper to obtain his or her assistance in gaining access to the units. If those attempts fail, a Call-Me Letter (in Attachment N) may be sent directly to a selected household. These letters inform the residents that an FI has been trying to contact them and asks that they contact the FS by phone. If the resident calls the FS, the FS attempts to get the resident to agree to an appointment so the FI can return to that address and screen the household in person.
Nonresponse Bias Studies
In addition to the investigations noted above, several studies have been conducted over the years to assess nonresponse bias in NSDUH. For example, the 1990 NSDUH2 was one of six large Federal or Federally sponsored surveys used in the compilation of a dataset that then was matched to the 1990 decennial census for analyzing the correlates of nonresponse (Groves and Couper, 1998). In addition, data from surveys of NSDUH FIs were combined with those from these other surveys to examine the effects of FI characteristics on nonresponse. One of the main findings was that those households with lower socioeconomic status were no less likely to cooperate than those with higher socioeconomic status; there was instead a tendency for those in high-cost housing to refuse survey requests, which was partially accounted for by residence in high-density urban areas. There was also some evidence that FIs with higher levels of confidence in their ability to gain participation achieved higher cooperation rates.
In follow-up to this research, a special study was undertaken on a subset of nonrespondents to the 1990 NSDUH to assess the impact of the nonresponse (Caspar, 1992). The aim was to understand the reasons people chose not to participate, or were otherwise missed in the survey, and to use this information in assessing the extent of the bias, if any, that nonresponse introduced into the 1990 NSDUH estimates. The study was conducted in the Washington, DC, area, a region with a traditionally high nonresponse rate. The follow-up survey design included a $10 incentive and a shortened version of the instrument. The response rate for the follow-up survey was 38 percent. Follow-up respondents appeared to have similar demographic characteristics to the original NSDUH respondents. Estimates of drug use for follow-up respondents showed patterns that were similar to the regular NSDUH respondents. Another finding was that among those who participated in the follow-up survey, one-third were judged by FIs to have participated in the follow-up because they were unavailable for the main survey request. Finally, 27 percent were judged to have been swayed by the incentive, and another 13 percent were judged to have participated in the follow-up due to the shorter instrument. Overall, the results did not demonstrate definitively either the presence or absence of a serious nonresponse bias in the 1990 NSDUH. Based on these findings, no changes were made to NSDUH procedures.
CBHSQ produced a report to address the nonresponse patterns obtained in the 1999 NSDUH (Eyerman et al., 2002). In 1999, the NSDUH changed from PAPI to CAI instruments. The report was motivated by the relatively low response rates in the 1999 NSDUH. The analyses presented in this report were produced to help provide an explanation for the rates in the 1999 NSDUH and guidance for the management of future projects. The report describes NSDUH data collection patterns from 1994 through 1998. It also describes the data collection process in 1999 with a detailed discussion of design changes, summary figures and statistics, and a series of logistic regressions comparing 1998 with 1999 nonresponse patterns. The results of this study are consistent with conventional wisdom within the professional survey research field and general findings in survey research literature: the nonresponse can be attributed to a set of FI influences, respondent influences, design features, and environmental characteristics. The nonresponse followed the demographic patterns observed in other studies, with urban and high crime areas having the worst rates. Finally, efforts taken in 1999 to improve the response rates were effective. Unfortunately, the tight labor market combined with the large increase in sample size caused these efforts to lag behind the data collection calendar. The authors used the results to generate several suggestions for the management of future projects. No major changes were made to NSDUH as a result of this research, although it—along with other general survey research findings—has led to minor tweaks to respondent cooperation approaches.
In 2004, focus groups were conducted with NSDUH FIs on the topic of nonresponse among the 50 or older age group to gather information on the root causes for differential response by age. The study examined the components of nonresponse (refusals, noncontacts, and other incompletes) among the 50 or older age group. It also examined respondent, environmental, and FI characteristics in order to identify the correlates of nonresponse among the 50 or older group, including relationships that are unique to this group. Finally, they considered the root causes for differential nonresponse by age, drawing from focus group sessions with NSDUH FIs on the topic of nonresponse among the 50 or older group. The results indicated that the high rate of nonresponse among the 50 or older age group was primarily due to a high rate of refusals, especially among sample members aged 50 to 69, and a high rate of physical and mental incapability among those 70 or older. It appeared that the higher rate of refusals among the 50 or older age group may, in part, have been due to fears and misperceptions about the survey and FIs' intentions. It was suggested that increased public awareness about the study may allay these fears (Murphy et al., 2004).
In 2005, Murphy et al. sought a better understanding of nonresponse among the population 50 or older in order to tailor methods to improve response rates and reduce the threat of nonresponse error (Attachment AA, Nonresponse among Sample Members Aged 50 and Older Report). Nonresponse to the NSDUH is historically higher among the 50 or older age group than lower age groups. Focus groups were again conducted, this time with potential NSDUH respondents to examine the issue of nonresponse among persons 50 or older. Participants in these groups recommended that the NSDUH contact materials focus more on establishing the legitimacy of the sponsoring and research organizations, clearly conveying the survey objectives, describing the selection process, and emphasizing the importance of the selected individual’s participation, which helped inform contact materials changes tested in the QFT and DR prior to implementation for the 2015 NSDUH.
Another examination of nonresponse was done in 2005. The primary goal was to develop a methodology to reduce item nonresponse to critical items in the ACASI portion of the NSDUH questionnaire (Caspar et al., 2005). Respondents providing "Don't know" or "Refused" responses to items designated as essential to the study's objectives received tailored follow-up questions designed to simulate FI probes. Logistic regression was used to determine what respondent characteristics tended to be associated with triggering follow-up questions. The analyses showed that item nonresponse to the critical items is quite low, so the authors caution the reader to interpret the data with care. However, the findings suggest the follow-up methodology is a useful strategy for reducing item nonresponse, particularly when the nonresponse is due to "Don't know" responses. In response, follow up questions were added to the survey and asked when respondents indicated that they did not know the answer to a question or refused to answer a question. These follow-up items encouraged respondents to provide their best guess, or presented an assurance of data confidentiality in order to encourage response.
Biemer and Link (2007) conducted additional nonresponse research to provide a general method for nonresponse adjustment that relaxed the ignorable nonresponse assumption. Their method, which extended the ideas of Drew and Fuller (1980), used level-of-effort (LOE) indicators based on call attempts to model the response propensity. In most surveys, call history data are available for all sample members, including nonrespondents. Because the LOE required to interview a sample member is likely to be highly correlated with response propensity, this method is ideally suited for modeling the nonignorable nonresponse. The approach was first studied in a telephone survey setting and then applied to data from the 2006 NSDUH, where LOE was measured by contact attempts (or callbacks) made by FIs.
The callback modeling approach investigation confirmed what was known from other studies on nonresponse adjustment approaches (i.e., there is no uniformly best approach for reducing the effects of nonresponse on survey estimates). All models under consideration were the best in eliminating nonresponse bias in different situations using various measures. Furthermore, possible errors in the callback data reported by FIs, such as underreporting of callback attempts, raise concerns about the accuracy of the bias estimates. Unfortunately, it is very difficult to apply uniform callback reporting procedures amongst the large NSDUH interviewing staff, which is spread across the country. An updated study of the callback modeling approach was completed in 2013 and similar conclusions were reached on errors in the callback data reported by FIs (Biemer, Chen, and Wang, 2013).For these reasons, the callback modeling approach was not implemented in the NSDUH nonresponse weighting adjustment process (Biemer and Link, 2007; Biemer, Chen, and Wang, 2013).
ONDCP has repeatedly emphasized that trends be maintained for the first three modules in the survey: Alcohol, Tobacco, and Marijuana. Therefore, all survey protocols administered before these modules needed to remain relatively unchanged. SAMHSA shares OMB’s concerns about decreasing response rates. Initial plans were assessed to adjust the incentive to $40 for the 2015 redesign. However, given the experience with the unexpected break in trends when the original incentive was introduced in 2002 (despite a field test indicating continuity), and ONDCP’s request to maintain trends on those three modules, SAMHSA recommends retaining the $30 incentive. SAMHSA will soon be providing a briefing for ONDCP on the recent declines in NSDUH response rates.
Methods to assess nonresponse bias vary and each has its limitations (Groves, 2006). Some methods include follow-up studies, comparisons to other surveys, examining alternative post-survey adjustments to examine imbalances in the data, and examination of trend data that may suggest a change in the characteristics of respondents. When comparing to other surveys such as the Monitoring the Future (MTF), we have found comparable trends even though the estimates themselves differ in magnitude (mostly due to differences in survey designs). For example, trends in NSDUH and MTF cigarette use between 2002 and 2012 show a consistent pattern http://samhsa.gov/data/NSDUH/2012SummNatFindDetTables/NationalFindings/NSDUHresults2012.htm#fig8.1. Using a different approach, information in a NSDUH report suggests that alternative weighting methods based on variables correlated with nonresponse were not better or only slightly better than the current weighting procedure (http://samhsa.gov/data/NSDUH/NSDUHCallbackModelReport2013.pdf ). Other approaches to indirectly assess nonresponse bias will be assessed in study described below.
SAMHSA is in the early stages of a Nonresponse Trends Analysis Study to take a closer look at how the recent drop in response rates may be affecting the data. The study will be conducted in phases with the results of each phase informing the investigation for the next phase. OMB has been provided a schedule of the proposed study. A final report should be available in Winter 2015.
The 2015 questionnaire and data collection protocols were field tested in 2012 and 2013 as part of the QFT and DR – OMB No. 0930-0334. Information about the rationale for questions, materials and equipment can be found in section A.1. Additional details on the results of the QFT and DR can be found in the final reports for each, included as Attachments B and C.
The basic NSDUH design was reviewed by statistical experts, both within and outside SAMHSA. Statistical experts reviewing portions of prior NSDUHs designs include William Kalsbeek, PhD, University of North Carolina; Robert Groves, PhD, Georgetown University; and Michael Hidiroglou, PhD, Statistics Canada. Monroe Sirken, PhD, National Center for Health Statistics (NCHS); Arthur Hughes, CBHSQ, SAMHSA; James Massey, PhD, (deceased) also of NCHS; Douglas Wright, CBHSQ, SAMHSA (retired); and Joseph Gfroerer, CBHSQ, SAMHSA (retired) were consulted on the 1992 and subsequent survey designs. Peter Tice, CBHSQ, SAMHSA is the Government Project Officer, (240) 276-1254. Arthur Hughes, CBHSQ, SAMHSA is the primary mathematical statistician responsible for overall project management, (240) 276-1262. RTI senior statisticians contributing to the design are James Chromy, PhD, and Ralph Folsom, PhD.
The 2014–2017 National Survey on Drug Use and Health contract was awarded to Research Triangle Institute (RTI) on January 14, 2013, with only the Base Award (2014 NSDUH) exercised initially. The additional Option Year Award for the 2015 NSDUH was exercised, with a start date of October 1, 2013. Contractor personnel will implement the sample design; recruit FSs and FIs; train FIs; conduct data collection; conduct data receipt, editing, coding, and keying; conduct data analysis; and develop and deliver to CBHSQ statistical reports and data files. CBHSQ will provide direction and review functions to the Contractor. Data collection will be conducted throughout the 2015 calendar year.
Potential NSDUH Consultants
a. Consultants on NSDUH Design
Michael Arthur, PhD, Research Associate Professor (206) 685-3858
Social Development Research Group, School of Social Work
University of Washington
Raul Caetano, M.D., PhD, Dean (214) 648-1080
School of Health Professions
University of Texas Southwestern
John Carnevale, PhD, President and CEO (301) 977-3600
Carnevale Associates, LLC
William D. Kalsbeek, PhD, Professor/Founder (919) 962-3249
Carolina Survey Research Laboratory, Biostatistics
University of North Carolina at Chapel Hill
Graham Kalton, PhD, (301) 251-8253
Chairman of the Board
Westat
Philip Leaf, PhD, Professor (410) 955‑3962
Department of Mental Health Policy and Management
School of Public Health
Johns Hopkins University
Patrick O’Malley, PhD, Senior Research Scientist (734) 763-5043
Population Studies Center, The Institute for Social Research
University of Michigan
Peter Reuter, PhD, Professor (301) 405-6367
School of Public Policy and Department of Criminology
University of Maryland
b. NSDUH Consultant for the Tobacco Module
Gary A. Giovino, PhD, Professor and Chair (716) 845-8444
Department of Health Behavior
University at Buffalo - SUNY
c. NSDUH Consultants for Mental Health Modules
Jeffrey Buck, PhD (301) 443-0588
Director of Office of Managed Care
Center for Mental Health Services
Marilyn Henderson (retired) (301) 443-2293
Center for Mental Health Services
Kimberly Hoagwood, PhD, Vice Chair for Research (410) 573-0228
NYU Child Study Center, Department of Child and Adolescent Psychiatry
NYU
Ronald C. Kessler, PhD, Professor (617) 423-3587
Department of Health Care Policy
Harvard Medical School
Christopher P. Lucas, MD, Clinical Associate Professor (212) 263-2499
NYU Study Center, Department of Child and Adolescent Psychiatry
NYU
Michael Schoenbaum, PhD (301) 435-8760
Senior Advisor for Mental Health Services,
Epidemiology and Economics
National Institute of Mental Health
Philip Wang, MD, PhD, Deputy Director (301) 443-6233
National Institute of Mental Health
Attachment A. Sample Design
Attachment B. Questionnaire Field Test (QFT) Final Report
Attachment C. Dress Rehearsal (DR) Final Report
Attachment D. Redesign Pretesting Summary Report
Attachment E. Lead Letter
Attachment F. Question & Answer Brochure
Attachment G. Tablet Screening Video Script
Attachment H. Parental Introductory Script
Attachment I. 2015 NSDUH Questionnaire and Materials Changes
Attachment J. Contact Cards – Sorry I Missed You Card and Appointment Cards
Attachment K. Study Description
Attachment L. Introduction and Informed Consent Scripts
Attachment M. Screening Questions
Attachment N. Unable-to-Contact, Controlled Access, and Call-Me Letters
Attachment O. Refusal Letters
Attachment P. Interview Incentive Receipt
Attachment Q. Federalwide Assurance
Attachment R. Showcard Booklet
Attachment S. Confidentiality Agreement and Data Collection Agreement
Attachment T. Quality Control Form
Attachment U. CAI Questionnaire
Attachment V. NSDUH Highlights and Newspaper Articles
Attachment W. Certificate of Participation
Attachment X. CATI Verification Scripts
Attachment Y. Quality Control Letter
Attachment Z. Contact Materials Study Report
Attachment AA. Nonresponse among Sample Members Aged 50 and Older Report
Attachment BB. Comment from the Campaign for Tobacco-Free Kids
References
Biemer, P., & Link, M. (2007). Evaluating and modeling early cooperator bias in RDD surveys. In Lepkowski, J. et al. (Eds.), Advances in telephone survey methodology. Hoboken, NJ: John Wiley & Sons.
Biemer, P., Chen, P., and Wang, K. (2013). Incorporating level of effort paradata in the NSDUH nonresponse adjustment process. Rockville, MD: U.S. Department of Health and Human Services, Substance Abuse and Mental Health Services Administration, Center for Behavioral, Health Statistics and Quality.
Caspar, R.A. (1992). A follow-up study of nonrespondents to the 1990 National Household Survey on Drug Abuse. In Proceedings of the1992 American Statistical Association, Survey Research Section, Boston, MA (pp. 476-481). Alexandria, VA: American Statistical Association.
Caspar, R. A., Penne, M. A., & Dean, E. (2005). Evaluation of follow-up probes to reduce item nonresponse in NSDUH. In J. Kennet & J. Gfroerer (Eds.), Evaluating and improving methods used in the National Survey on Drug Use and Health (HHS Publication No. SMA 05-4044, Methodology Series M-5, pp. 121-148). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
Center for Behavioral Health Statistics and Quality. (2013). Results from the 2012 National Survey on Drug Use and Health: Summary of national findings (HHS Publication No. SMA 13-4795, NSDUH Series H-46). Rockville, MD: Substance Abuse and Mental Health Services Administration.
Colliver, J.D., Kroutil, L.A., Dai, L., & Gfroerer, J.C. (2006). Misuse of prescription drugs: Data from the 2002, 2003, and 2004 National Surveys on Drug use and Health (HHS Publication No. SMA 06-4192, Analytic Series A-28.) Rockville, MD; Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
Dahlhamer, J., Cain, V., Cynamon, M., Galinsky, A., Joestl, S., Mandans, J., Miller, K. (2013). Asking about Sexual Identity in the National Health Interview Survey: An Experimental Mode Comparison. National Center for Health Statistics.
de Leeuw, E., Hox, J., & Kef, S. (2003). Computer-assisted self-interviewing tailored for special populations and topics. Field Methods, 15(3), 223-251. doi: 10.1177/1525822X03254714
Drew, J.H. and Fuller, W.A. (1980). Modeling Nonresponse in Surveys with Callbacks. Proceedings of the American Statistical Association, Survey Research Methods Section, 639–642.
Eyerman, J., Odom, D., Butler, D., Wu, S., and Caspar, R. (2002). Nonresponse in the 1999 NHSDA. In J. Gfroerer, J. Eyerman, & J. Chromy (Eds.), Redesigning an ongoing national household survey: Methodological issues (HHS Publication No. SMA 03-3768). Rockville, MD: Substance Abuse and Mental Health Services Administration, Office of Applied Studies.
Gfroerer, J., Wright, D., & Kopstein, A. (1997). Prevalence of youth substance use: The impact of methodological differences between two national surveys. Drug and Alcohol Dependence, 47, 19–30.
Groves, R. M., and M.P. Couper. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.
Groves, R. M. (2006). Nonresponse Rates and Nonresponse Bias in Household Surveys, Public Opinion Quarterly, 70(5), pp. 646-675.
Groves, R. (1989). Survey Errors and Survey Costs. New York: Wiley.
Grucza, R. A., Abbacchi, A. M., Przybeck, T. R., & Gfroerer, J. C. (2007). Discrepancies in estimates of prevalence and correlates of substance use and disorders between two national surveys. Addiction, 102, 623-629.
Hennessy, K., & Ginsberg, C. (Eds.). (2001). Substance use survey data collection methodologies [Special issue]. Journal of Drug Issues, 31(3), 595–727.
Huang, B., Dawson, D. A., Stinson, F. S., Hasin, D. S., Ruan, W. J., Saha, T. D., Smith, S. M., Goldstein, R. B., & Grant, B. F. (2006). Prevalence, correlates and comorbidity of nonmedical prescription drug use and drug use disorders in the United States: Results of the National Epidemiological Survey on Alcohol and Related Conditions. Journal of Clinical Psychiatry, 67, 1062-1073.
Miller, J. W., Gfroerer, J. C., Brewer, R. D., Naimi, T. S., Mokdad, A., & Giles, W. H. (2004). Prevalence of adult binge drinking: A comparison of two national surveys. American Journal of Preventive Medicine, 27, 197-204.
Murphy, J., Eyerman, J., & Kennet, J. (2004). Nonresponse among persons aged 50 or older in the National Survey on Drug Use and Health. In S. B. Cohen & J. M. Lepkowski (Eds.), Eighth Conference on Health Survey Research Methods (HHS Publication No. PHS 04-1013, pp. 73-78). Hyattsville, MD: U.S. Department of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics.
National Research Council. (2013). Nonresponse in Social Science Surveys: A Research Agenda. Roger Tourangeau and Thomas J. Plewes, Editors. Panel on a Research Agenda for the Future of Social Science Data Collection, Committee on National Statistics. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60(2), 275‑304.
Tourangeau, R., & Yan, T. (2007). Sensitive questions in surveys. Psychological Bulletin, 133(5), 859-883.
Wright, D., Bowman, K., Butler, D., & Eyerman, J. (2005). Non-response bias from the National Household Survey on Drug Abuse incentive experiment. Journal for Social and Economic Measurement, 30(2-3), 219-231.
1 http://aspe.hhs.gov/datacncl/standards/ACA/4302/index.shtml
2 Prior to 2002, the NSDUH was referred to as the National Household Survey on Drug Abuse (NHSDA). In this document the term NSDUH is used for all survey years.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 2003 National Survey on Drug Use and Health |
| Author | lky |
| File Modified | 0000-00-00 |
| File Created | 2021-01-27 |