The respondent universe for the DR is the civilian, noninstitutionalized population aged 12 years old and older within the 48 U.S. contiguous states and the District of Columbia. This universe includes residents of noninstitutional group quarters (e.g., shelters, rooming houses, dormitories) and civilians residing on military bases. Persons excluded from the universe include those with no fixed household address (e.g., homeless transients not in shelters, and residents of institutional group quarters such as jails and hospitals). To control costs, persons residing in Alaska and Hawaii will be excluded from the DR sample. We excluded these two states due to travel and costs associated with interviewing in these states. They compromise only 2.7 percent of the annual NSDUH respondent sample and therefore, should not have a large impact.
The interview will only be conducted in English and Spanish. The sample design accounts for this DR target population and there are no other exclusions.
Dwelling units not selected for the 2011, 2012 and 2013 NSDUH main study will be eligible for selection in the DR. After accounting for eligibility, nonresponse, and the person-level sample selection procedures, it is estimated that 5,146 dwelling units will need to be selected in order to complete 3,673 screenings and 2,000 completed interviews for the DR. The expected number of selected persons per completed screening and eligibility and screening response rates were estimated using data from the QFT. The QFT used the same age group distribution and a data collection period similar to that planned for the DR. However, the expected interview response rate was adjusted for the oversampling of high Hispanic interview areas using response rates from the 2011 NSDUH.
Attachment N provides the Sample Design while Table 3 below shows a summary of the sampling plans for the DR.
Table 3. Summary of the Dress Rehearsal Sample Design
Statistic |
Total |
Expected Rate |
State Sampling Regions (SSRs) |
200 |
n/a |
Segments |
200 |
n/a |
Dwelling Units Selected |
5,146 |
n/a |
Expected Eligible Dwelling Units |
4,426 |
0.86 |
Expected Completed Screening Interviews |
3,673 |
0.83 |
Expected Selected Persons |
2,703 |
0.74 |
Expected Completed Interviews |
2,000 |
0.74 |
Based on the annual NSDUH sample yielding 67,500 interviews from 7,200 segments (Morton, Martin, Shook-Sa, Chromy, and Hirsch, 2012)1 an estimated 200 segments will be needed to yield 2,000 completed interviews for the DR.
To ensure that a sufficient number of DR interviews are completed in Spanish, a special certainty stratum consisting of state sampling regions (SSRs) with a historically high proportion of Spanish interviews will be created. The high Spanish interview stratum is defined as having 10 percent or more of interviews from the 2011 NSDUH conducted in Spanish. One hundred and one of the SSRs fall into this stratum and will be selected with certainty. The other 99 SSRs will be selected with probability proportional to size (PPS) from the remaining SSRs. This design has the benefit of placing the sample in heavily populated areas where a sufficient mix of FIs with various experience levels will be available to meet the DR staffing needs. In addition, the majority of the Spanish interviews are expected to be completed in states where bilingual NSDUH FIs are currently employed.
Within each selected SSR, a sample of dwelling units will be drawn from the segment to be retired from use in Quarter 1 of the 2013 NSDUH. If an insufficient number of eligible dwelling units remains in any segment, this segment will be replaced with the Quarter 4 2012 retired segment in the same SSR. Similarly, if a significant amount of controlled access problems were encountered in any Quarter 1 2013 retired segment, this segment will be replaced with the Quarter 4 2012 retired segment in the same SSR. The retired segments to be used for the DR were formed and selected at the same time as the main study NSDUH segments, but randomly assigned to a different survey year and quarter.
Table 4 shows the expected number of SSRs and interviews by state. Two hundred and seven of the 2,000 DR interviews are expected to be completed in Spanish.
Table 4. Expected Number of Dress Rehearsal State Sampling Regions and Interviews by State
State |
Hispanic Interview SSRs |
Total SSRs |
Expected Number of DR SSRs |
Expected Interviews |
||||
Low |
High |
Low Hispanic |
High Hispanic |
Total |
Total |
Spanish |
||
AL |
12 |
0 |
12 |
2 |
0 |
2 |
18 |
0 |
AR |
12 |
0 |
12 |
1 |
0 |
1 |
11 |
0 |
AZ |
7 |
5 |
12 |
1 |
5 |
6 |
65 |
8 |
CA |
16 |
32 |
48 |
5 |
32 |
37 |
369 |
60 |
CO |
11 |
1 |
12 |
2 |
1 |
3 |
27 |
2 |
CT |
11 |
1 |
12 |
1 |
1 |
2 |
23 |
2 |
DC |
11 |
1 |
12 |
0 |
1 |
1 |
12 |
2 |
DE |
11 |
1 |
12 |
0 |
1 |
1 |
13 |
2 |
FL |
37 |
11 |
48 |
6 |
11 |
17 |
168 |
33 |
GA |
12 |
0 |
12 |
4 |
0 |
4 |
36 |
1 |
IA |
12 |
0 |
12 |
1 |
0 |
1 |
12 |
0 |
ID |
11 |
1 |
12 |
1 |
1 |
2 |
15 |
2 |
IL |
38 |
10 |
48 |
4 |
10 |
14 |
139 |
15 |
IN |
12 |
0 |
12 |
2 |
0 |
2 |
25 |
0 |
KS |
12 |
0 |
12 |
1 |
0 |
1 |
11 |
0 |
KY |
12 |
0 |
12 |
2 |
0 |
2 |
16 |
0 |
LA |
12 |
0 |
12 |
2 |
0 |
2 |
17 |
0 |
MA |
11 |
1 |
12 |
2 |
1 |
3 |
34 |
2 |
MD |
12 |
0 |
12 |
2 |
0 |
2 |
22 |
0 |
ME |
12 |
0 |
12 |
1 |
0 |
1 |
5 |
0 |
MI |
48 |
0 |
48 |
4 |
0 |
4 |
38 |
0 |
MN |
12 |
0 |
12 |
2 |
0 |
2 |
20 |
0 |
MO |
12 |
0 |
12 |
2 |
0 |
2 |
23 |
0 |
MS |
12 |
0 |
12 |
1 |
0 |
1 |
11 |
0 |
MT |
12 |
0 |
12 |
0 |
0 |
0 |
4 |
0 |
NC |
12 |
0 |
12 |
4 |
0 |
4 |
36 |
0 |
ND |
12 |
0 |
12 |
0 |
0 |
0 |
3 |
0 |
NE |
11 |
1 |
12 |
1 |
1 |
2 |
16 |
1 |
NH |
12 |
0 |
12 |
1 |
0 |
1 |
5 |
0 |
NJ |
12 |
0 |
12 |
3 |
0 |
3 |
34 |
1 |
NM |
9 |
3 |
12 |
1 |
3 |
4 |
36 |
5 |
(continued)
Table 4. Expected Number of Dress Rehearsal State Sampling Regions and Interviews by State
State |
Hispanic Interview SSRs |
Total SSRs |
Expected Number of DR SSRs |
Expected Interviews |
||||
Low |
High |
Low Hispanic |
High Hispanic |
Total |
Total |
Spanish |
||
NV |
9 |
3 |
12 |
1 |
3 |
4 |
38 |
6 |
NY |
39 |
9 |
48 |
6 |
9 |
15 |
152 |
22 |
OH |
48 |
0 |
48 |
4 |
0 |
4 |
44 |
0 |
OK |
11 |
1 |
12 |
1 |
1 |
2 |
23 |
2 |
OR |
12 |
0 |
12 |
2 |
0 |
2 |
15 |
0 |
PA |
47 |
1 |
48 |
5 |
1 |
6 |
58 |
1 |
RI |
9 |
3 |
12 |
0 |
3 |
3 |
33 |
6 |
SC |
12 |
0 |
12 |
2 |
0 |
2 |
18 |
0 |
SD |
12 |
0 |
12 |
0 |
0 |
0 |
3 |
0 |
TN |
12 |
0 |
12 |
2 |
0 |
2 |
24 |
0 |
TX |
34 |
14 |
48 |
7 |
14 |
21 |
210 |
29 |
UT |
10 |
2 |
12 |
1 |
2 |
3 |
29 |
3 |
VA |
12 |
0 |
12 |
3 |
0 |
3 |
30 |
1 |
VT |
12 |
0 |
12 |
0 |
0 |
0 |
2 |
0 |
WA |
12 |
0 |
12 |
3 |
0 |
3 |
26 |
1 |
WI |
12 |
0 |
12 |
2 |
0 |
2 |
22 |
0 |
WV |
12 |
0 |
12 |
1 |
0 |
1 |
7 |
0 |
WY |
12 |
0 |
12 |
0 |
0 |
0 |
2 |
0 |
Total |
775 |
101 |
876 |
99 |
101 |
200 |
2000 |
207 |
SSR = State Sampling Region; DR=Dress Rehearsal.
Similar to the main study, a small reserve of SDUs from each segment will be selected and the total sample will be partitioned into equal probability subsamples. While the majority of the sample will be released at the beginning of the DR data collection period, having the additional sample and sample partitions will allow for greater flexibility in controlling the sample size. This procedure will ensure data collection goals are attained within the field period. Sample partitions may be released in the event of lower than expected response rates.
The DR respondent sample will be allocated to three major age groups in the following proportions: 25 percent aged 12 to 17, 25 percent aged 18 to 25, and 50 percent aged 26 or older. This sample allocation matches the planned allocation for the 2015 NSDUH redesign. Details of this allocation can be found in Attachment N (Sample Design).
Unless otherwise specified, the DR procedures described within this section follow the processes used on the main study NSDUH and the QFT; however the materials referenced in the attachments are specific for the DR.
Prior to the FI’s arrival at the SDU, a Lead Letter (Attachment B) is mailed to the resident(s) briefly explaining the DR and requesting their cooperation. This letter is printed on HHS letterhead with the signature of the CBHSQ National Study Director and the Contractor’s National Field Director.
Upon arrival at the SDU, the FI refers the respondent to this letter and answers any questions. If the respondent has no knowledge of the letter, the FI provides another copy, explains that one was previously sent, and then answers any questions. If no one is at home during the initial visit to the SDU, the FI may leave a Sorry I Missed You Card (in Attachment C) informing the resident(s) that the FI plans to make another callback at a later date/time. Callbacks are made as soon as possible. FIs attempt to make at least four callbacks (in addition to the initial call) to each SDU in order to complete the screening process and complete an interview, if yielded.
If the FI is unable to contact anyone at the SDU after repeated attempts, the field supervisor (FS) may send an Unable to Contact Letter (in Attachment H). These letters reiterate information contained in the lead letter while encouraging the respondent to participate in the study. If after sending that letter an FI is still unable to contact anyone at an SDU, another letter, a Call Me Letter (in Attachment H) may be sent to the SDU requesting that the resident(s) call the FS as soon as possible to set up an appointment for the FI to visit the resident(s).
When in‑person contact is made with an adult resident of the SDU and introductory procedures are completed, the FI presents a Study Description (Attachment D) and answers any questions the person may have concerning the study. A Question & Answer Brochure (Attachment G) that provides answers to commonly asked questions may also be given. In addition, FIs are supplied with copies of the NSDUH Highlights & Newspaper Articles (Attachment O), which can be left with the respondent.
If a resident refuses to be screened, FIs have been trained to accept the refusal in a positive manner, thereby avoiding the possibility of creating an adversarial relationship and precluding future opportunities for conversion. One of several Refusal Letters (Attachment I) may then be requested by the FS. The letter sent is tailored to the specific concerns expressed by the resident and asks him/her to reconsider participation. An in-person conversion is then attempted by a specially-selected FI with successful conversion experience.
With respondent cooperation, the FI begins screening the SDU by asking either the Housing Unit Screening questions or the Group Quarters Unit Screening questions (Attachment F). The screening questions are administered using a hand-held computer, which for the DR will be an Android tablet.
Once all household members 12 or older have been rostered, the tablet performs the within dwelling unit sampling process, selecting zero, one, or two members to participate in the study by completing the interview. For cases with no one selected, the FI asks for a name and phone number for use in verifying the quality of the FI’s work, and then thanks the respondent and concludes the household contact.
For each person selected to complete the full survey, the FI follows these steps:
If the selected individual is 18 years of age or older, or 17 years of age and living independently from his/her parent or guardian, and is currently available, the FI immediately seeks to obtain informed consent and begins to administer the questionnaire in a private setting within the dwelling unit. As necessary and appropriate, the FI may make use of the Appointment Card (in Attachment C) for scheduled return visits with the respondent.
If the selected individual is 12 to 17 years of age, except in rare instances where a 17 year old does not live with a parent or guardian in which case the 17 year old provides his/her own consent, parental consent is obtained from the selected individual’s parent or legal guardian using the Parent section of the youth version of the Introduction and Informed Consent Scripts (Attachment E). The minor is then asked to participate using the Youth section of the same document.
As discussed in section A.3, for all selected respondents, the FI administers the interview in a prescribed and uniform manner with sensitive portions of the interview completed via ACASI. For the DR, CBHSQ is currently evaluating two different models of light ultrabook-style laptop computers, one of which will be selected as the platform for the DR interview.
In order to facilitate the respondent’s recollection of prescription-type drugs and their proper names, electronic pill images for these questions appear on the screen for DR interviews. As in the main study, showcards are included in the Showcard Booklet (Attachment P) and allow the respondent to refer to information necessary for accurate responses, such as education, in the FI-administered portion of the questionnaire.
After the interview is completed and before the verification procedures begin, each respondent is given a $30.00 cash incentive and an Interview Incentive Receipt (Attachment J) signed by the FI.
For verification purposes, interview respondents are asked to complete a Quality Control Form (Attachment M) that requests his/her current address and phone number for possible follow‑up to ensure that the FI did his/her job appropriately. Respondents are informed that completing the Quality Control Form is voluntary. If the respondent agrees, this form is completed, placed in an envelope and sealed by the respondent, and then mailed to the Contractor’s offices for processing. In previous NSDUH surveys, less than one percent of the verification sample refused to complete Quality Control Forms.
FIs may give a Certificate of Participation (Attachment Q) to interested respondents, primarily adolescents, after the interview is completed. Respondents may attempt to use these certificates to earn school or community service credit hours. As stated on the certificate, no guarantee of credit is made by CBHSQ or the Contractor. FIs sign and date the certificate, but for confidentiality reasons, the section for recording the respondent’s name is left blank. The respondent can fill in his/her name at a later time so the FI will not be made aware of the respondent’s identity. It the respondent’s choice if they would like to identify themself as a NSDUH respondent by using the certificate in an attempt to obtain community service credit.
A random sample of those who complete Quality Control Forms are contacted via telephone to answer a few questions verifying that the interview took place, that proper procedures were followed, and that the amount of time required to administer the interview was within expected parameters. Attachment R contains the Telephone Verification Scripts for these interview verification contacts, as well as the scripts used when verifying a percentage of certain completed screening cases in which no one was selected for an interview or the SDU was otherwise ineligible (vacant, not primary residence, not a dwelling unit, dwelling unit contains ONLY military personnel, respondents living at the sampled residence for less than half of the quarter).
All screening and interview data are transmitted securely to the Contractor’s servers on a daily basis.
Questionnaire
As explained in more detail in section A.1, the questionnaire for the DR is based on the main study and QFT questionnaires. Unless otherwise specified, both the QFT and DR follow the main study NSDUH questionnaire and interview methods.
The interview process is designed to retain respondent interest, ensure confidentiality, and maximize the validity of response. The questionnaire is administered in such a way that FIs do not know respondents’ answers to sensitive questions, including those on illicit drug use and mental health. These questions are self-administered using ACASI. The respondent listens privately through headphones, so even those who have difficulty seeing or reading are able to complete the self-administered portion. For the QFT and DR, the only topics that are administered by the FI (i.e., the CAPI section) are an initial section on Demographics and a listing of the residents in the home.
The questionnaire is divided into sections based on specific substances or other main topics. The same questions are asked for each substance or substance class, ascertaining the respondent’s history in terms of age of first use, most recent use, and frequency of use in the past 30 days and past 12 months. These substance use histories allow estimation of the incidence, prevalence, and patterns of use for licit and illicit substances.
A key feature of the questionnaire is a core-supplement structure. Core questions that are critical for basic trend measurement of substance use incidence and prevalence rates remain in the survey every year and comprise the main part of the questionnaire. The core is comprised of the initial demographic questions and the Tobacco through Sedatives modules.
As explained in section A.1, the DR questionnaire has very few changes from the QFT questionnaire. Two new questions asking about sexual identity and attraction have been added to the ACASI section for adults 18 and older. One QFT question about number of employees in the workplace has been removed. Several routing updates within the questionnaire will improve the flow.
The detailed specifications for the proposed CAI Questionnaire Content for the DR are provided in Attachment A.
Procedures to maximize response rates for the DR will be based on past and current experience with NSDUH’s main study and QFT data collection. In this section, any differences from the main study and QFT are indicated.
Given a $30.00 cash incentive offered for the interview, experience on the main study and QFT (described in B.1), and the various processes in place to encourage participation and noted below, the Contractor expects to obtain an 83% unweighted screening response rate and a 74% unweighted interview response rate (IRR) among selected respondents. Combined, this represents an overall response rate (ORR) of approximately 61.4%.
As a way to maximize response rates, the DR will continue to use the redesigned NSDUH Lead Letter and Question & Answer Brochure initially tested in the QFT. Prior to the QFT, CBHSQ revised these two contact materials through review of contact materials used for other government-sponsored surveys, expert review, and feedback from 17 focus groups conducted in both English and Spanish in five metropolitan areas (OMB No. 0930-0290). Focus group participants were tasked with reviewing three different lead letters and providing feedback about content, layout and features of the letter. Participants assessed how the letters were addressed, the level of details about the survey topic, graphics in the letter, and the signature of the letter. Attachment S contains the final Contact Materials Report providing details about this effort.
Feedback was also provided about the Question & Answer Brochure. Focus Group participants reviewed two mock ups of the brochure. They were asked for opinions on graphics, color schemes, content, and how well the brochures addressed potential concerns about the NSDUH survey. Results from these focus groups informed the development of the Question & Answer Brochure and Lead Letter that were initially tested in the QFT.
A graphic in the Question &Answer Brochure has been replaced for the DR. This graphic pictured an FI presenting a paper reference date calendar to a respondent. As the paper calendar has been converted to an electronic calendar in the QFT and DR, this image was replaced with another image. The Lead Letter used in the DR will be the same as the letter used in the QFT.
The primary focus of redesigning the contact materials was to improve the materials in ways likely to generate positive reactions from members of sampled households and, therefore, maximize participation. It is difficult to determine whether the changes in the contact materials had an impact on QFT response rates. Response rates in the QFT were slightly lower than in the main study in 2011 and Quarters 3 and 4 2012. We believe this is due to: (a) an increase in the proportion of sample respondents 26 or older than in the main study; this group historically has lower response rates than individuals under 26; (b) a decrease in the number of QFT FIs being available to work the widely dispersed QFT sample than in the main study; this restricted field managers’ flexibility to transfer refusal cases to FIs more successful at converting refusals than the typical FI and enabled fewer callback attempts to remote work areas; and (c) additional QFT sample was released approximately halfway through the data collection period to ensure the target of 2,000 completed interviews was met; this did not allow for as much time to attempt contact and convert refusals as in the main study or with other QFT cases. Despite these findings, we have no reason to believe the revised contact materials negatively impacted participation, and in fact as noted above we believe they are improved over the main study versions; therefore, we will use the revised contact materials for the DR.
Study procedures to maximize response rates begin with assignment of the cases prior to the start of data collection, accompanied by weekly response rate goals that are conveyed to the FIs by the FS. When making assignments, FSs take into account which FIs are in closest proximity to the work, FI skill sets, and basic information (demographics, size, etc) about the segment. FSs assign cases to the FIs in order to ensure maximum production levels at the start of the data collection period. To successfully complete work in remote segments or where no local FI is available, a traveling FI (i.e., a veteran NSDUH FI with demonstrated performance and commitment to the study) or a “borrowed” FI from another FS region can be utilized to work in locations needed to prevent delays in data collection.
Once FIs transmit their work, data are processed and summarized in daily reports posted to a web-based case management system (CMS) accessed by project management. Managers review production levels and record of call information in order to determine an FI’s progress toward weekly goals, when FIs should attempt contact with a case, and to develop plans to handle challenging cases such as refusal cases and cases where an FI is unable to access the dwelling unit. FSs discuss this information with FIs on a weekly basis. Whenever possible, cases are transferred to available FIs with different skill sets to assist with refusal conversion attempts or to improve production in areas where the original FI has fallen behind weekly response rate goals. A smaller number of FIs will be trained to work on the DR, so options for transferring cases may be limited compared to the main study.
As noted in section B.2 above, FIs may use a Sorry I Missed You Card (in Attachment C), NSDUH Highlights & Newspaper Articles (Attachment O), and a Certificate of Participation (Attachment Q) to help make respondent contact and encourage participation. To aid in refusal conversion efforts, Refusal Letters (Attachment I) tailored to refusal reason can be sent to any case that has refused. Similarly, a Call Me Letter (in Attachment H) may be sent to a selected household if the FI has been unable to contact a resident after multiple attempts. For cases where FIs have been unable to gain access to SDUs due to some type of access barrier, such as a locked gate or doorperson, Controlled Access Letters (in Attachment H) can be sent to the gate keeper to obtain his/her assistance in gaining access to the units. If those attempts fail, a Call Me Letter (in Attachment H) may be sent directly to a selected household. These letters inform the residents that an FI has been trying to contact them and asks that they contact the FS by telephone.
The DR activities to be conducted under this approval are in themselves tests of main study and DR NSDUH procedures. Information about the rationale for new and revised questions can be found in section A.1.
Dicy Painter (240) 276-1264 Statistician Population Surveys Branch CBHSQ, SAMHSA |
Peter Tice (240) 276-1254 Project Officer Population Surveys Branch CBHSQ, SAMHSA |
Joseph Gfroerer (240) 276-1262 Division Director/Supervisory Mathematical Statistician Population Surveys Branch CBHSQ, SAMHSA |
Joel Kennet (240) 276-1265 NSDUH National Study Director Population Surveys Branch CBHSQ, SAMHSA |
Grace O’Neill (240) 276-0513 Statistician Population Surveys Branch CBHSQ, SAMHSA |
(240) 276-0522 Statistician Population Surveys Branch CBHSQ, SAMHSA |
Contractor Project Management:
Tom Virag (941) 776-0797 Project Director, NSDUH RTI |
|
David Hunter (919) 541-1261 Associate Director, NSDUH RTI |
|
Rebecca Granger (919) 541-6000 Associate Director, NSDUH RTI |
|
A CAI Questionnaire Content
B Lead Letter
C Contact Cards – Sorry I Missed You & Appointment Cards
D Study Description
E Introduction and Informed Consent Scripts
F Housing Unit and Group Quarters Unit Screening Questions
G Question & Answer Brochure
H Unable to Contact, Call Me, & Controlled Access Letters
I Refusal Letters
J Interview Incentive Receipt
K Federalwide Assurance
L Confidentiality Agreement & Data Collection Agreement
M Quality Control Form
N Sample Design
O NSDUH Highlights & Newspaper Articles
P Showcard Booklet
Q Certificate of Participation
R Telephone Verification Scripts
S Contact Materials Report
T Comments on 60-Day Federal Register Notice
1 Morton, K. B., Martin, P. C., Shook-Sa, B. E., Chromy, J. R., & Hirsch, E. L. (2012). 2011 National Survey on Drug Use and Health: Sample design report (prepared for the Substance Abuse and Mental Health Services Administration, Office of Applied Studies, under Contract No. HHSS283200800004C, Phase II, Deliverable No. 8, RTI/0211838.203). Research Triangle Park, NC: RTI International.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | Global Comments |
| Subject | RTI: |
| Author | rebecca |
| File Modified | 0000-00-00 |
| File Created | 2021-01-29 |