National Survey on Drug Use and Health:
DSM-5 Cognitive Interviews
SUPPORTING STATEMENT
A. JUSTIFICATION
The Substance Abuse and Mental Health Services Administration (SAMHSA) is requesting OMB approval to conduct 51 cognitive interviews to test questions designed to produce prevalence estimates of substance use disorders (SUD) based on the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) for the National Survey on Drug Use and Health (NSDUH). This DSM-5 cognitive interview package is submitted under the NSDUH Methodological Field Tests generic OMB clearance (OMB No. 0930-0290).
This submission is a revision to OMB 0930-0290 Generic IC (National Survey on Drug Use and Health: Methodological Field Tests) that was previously approved on January 2, 2015.
NSDUH is sponsored by SAMHSA’s Center for Behavioral Health Statistics and Quality (CBHSQ) and approved under OMB No. 0930-0110. The data collection is a national survey of the U.S. civilian, non-institutionalized population aged 12 and older. This survey is paramount in meeting a critical objective of SAMHSA’s mission—to maintain current data on the incidence and prevalence of substance use and mental health problems in the United States. NSDUH has been conducted on a periodic basis from 1971 to 1988, and annually since 1990.
NSDUH is authorized by Section 505 of the Public Health Service Act (42 USC 290aa4 – Data Collection). Section 505 specifically authorizes annual data collection for monitoring the incidence and prevalence of illicit substance use and mental health problems, as well as the abuse of licit substances in the U.S. population.
The DSM is the manual used by clinicians and researchers to diagnose and classify mental disorders, including SUD. The American Psychiatric Association (APA) published the DSM-5 in 2013, culminating in a 14-year revision process.
Since 2000, NSDUH has included an assessment of DSM-IV-based SUD as part of its data collection. In order to continue producing current mental health data, CBHSQ plans to examine the potential impact of the newly-released DSM-5 (APA, 2013) on the NSDUH assessments of SUD and to identify changes to the NSDUH questionnaire that would be needed to produce SUD estimates that are aligned with the DSM-5 definitions.
In an earlier phase of this project, CBHSQ drafted an updated SUD module that assesses SUD aligned to the new DSM-5 definitions and it was reviewed by three substantive experts in mental health as well as two survey methodologists. The goal of the expert review was to ensure the survey questions addressed the DSM-5 criteria, and that they would be easily and accurately answered by respondents. After the expert review, the proposed revised and new items were updated to incorporate the reviewers’ feedback.
The next phase of this project consists of conducting cognitive interviews to further assess the revised SUD questions. The cognitive interviews will be conducted with individuals who have used substances such as alcohol, marijuana, cocaine, and/or heroin in the past 12 months. Since the NSDUH is designed to produce SUD estimates for the civilian population aged 12 and older, cognitive interviews will be conducted with both adolescents (aged 12 to 17) and adults (aged 18 and older). In addition, the revised and possibly new survey questions will be assessed in both English and Spanish.
The findings from the 51 DSM-5 cognitive interviews will be delivered by January 8, 2016 to assist CBHSQ in a determination of specific changes for the 2017 NSDUH questionnaire.
The purpose of the DSM-5 cognitive interviews is to collect and analyze qualitative data to evaluate how well potential NSDUH respondents understand the concepts and language of the new and revised SUD questions, whether they find them difficult to answer, and whether they are able to provide accurate reports of their substance use behaviors.
A total of 51 participants (12 English-speaking adolescents, 27 English-speaking adults, and 12 Spanish-speaking adults) will be interviewed in this study. It is expected that approximately 150 screenings with potential participants will be completed to obtain the 51 study participants.
The new and revised NSDUH survey questions for use in these cognitive interviews focus on dependence and withdrawal from the following substances: alcohol, marijuana, cocaine, heroin, methamphetamine, and prescription drugs. As a result, all cognitive interview participants will be past 12 month users of these substances. A majority of the participants will be past year marijuana users in order to adequately assess the marijuana withdrawal questions that are entirely new to NSDUH. The remaining participants will be past year users of alcohol, cocaine, heroin, methamphetamine, prescription drugs, or a combination of those substances. The questions about withdrawal from these substances are not new to the NSDUH questionnaire, but are being revised. New questions on craving will be added for all substances.
As explained in Section A.1, the findings from the cognitive interviews will be used to revise the survey questions that will be included in the 2017 NSDUH to produce estimates of SUD based on the DSM-5.
The revisions to procedures for these cognitive interviews made since the January 2, 2015 approved version included for this submission are as follows:
In the Recruitment Flyers (Attachment A), for the adolescent ad, specified that parental permission is needed and changed the location of the interviews from “our offices” to the treatment facility.
In the Web Recruitment Screener (Attachment B), added a question how respondents heard about this study.
In the Telephone Recruitment Screener (Attachment C), under AVAIL, changed “Are you and your child interested in participating?” to “Is it ok if I ask your child some questions to see if he/she is eligible for the interview?” and added information to cover situations where the child and parent are in different locations and/or have different phones.
In the Parental Permission Form (Attachment D), added text about risks and benefits.
In the Participant Informed Consent/Assent (Attachment E), under Confidentiality/Your Rights, changed “No one else…” to “Only people working on or with the study…”
Developed a separate document for use by interviewers with procedures for distressed respondents, included as Attachment J.
Information technology will be used for screening potential adult participants via the web. Potential adolescent participants will be screened over the telephone (once parental consent has been received). Information technology will also be used for both adolescents and adults in conducting the cognitive interviews via computer-assisted personal interviewing (CAPI) and audio computer-assisted self-interviewing (ACASI). Details on the use of information technology are described below.
Adult participants who are recruited using online advertisements, such as those placed on www.craigslist.com, will complete a web screener that will be used to determine if they are eligible to participate in the study. The web screener will be programmed using Survey Gizmo or a similar program. Survey Gizmo allows surveys to be accessed by users via secure (https) share links, which keeps responses secure. It also has a Project Data Encryption feature that allows projects to encrypt all survey data that are received, so those data cannot be accessed without a password key.
During the screening, information will be collected on the age, race/ethnicity, education, phone number, city of residence, any significant physical limitations that would preclude participation, and past substance use. The telephone numbers collected will be used to remind the participants about their upcoming appointments and to recruit additional participants placed on a “will call” list in the event that any of the originally-recruited participants are no longer available or do not show for their appointment.
Other electronic files containing screening information will be password protected, with the password set to expire within four weeks after the final memo with the cognitive interview findings is completed and approved by CBHSQ. Each individual screened will be assigned a unique case number so that a participant’s name will not be stored with their responses to the cognitive interview.
In all rounds of the DSM-5 cognitive interviews, light-weight, ultra-book laptops, identical to those which will be used for the 2015 NSDUH will be used for the cognitive interviews. The interview will use a combination of CAPI and ACASI. The interview will focus on questions in the revised SUD module.
The interview will commence after the participant has given his or her consent. The interview will begin with the interviewer asking the participant for demographic information such as age (to determine how the interview program should route the participant through questions in the interview). After the CAPI portion of the interview, the interviewer will show each respondent how to navigate through the interview program. Participants will then complete a tutorial that teaches participants how to complete the ACASI portion of the survey.
Following the tutorial, participants will complete an abridged version of the 2015 NSDUH core drug screening modules for alcohol, marijuana, cocaine, heroin, and methamphetamines, as well as medical and nonmedical use of prescription drugs. These modules will include the same questions that are in the 2015 NSDUH, except some questions not necessary for cognitive testing (age at first use) have been removed. No new questions were added to these modules for cognitive testing. Answers that participants give to questions about use of substances will determine whether they are asked more detailed questions about substance dependence and withdrawal. Some of the SUD questions have been reworded and are being presented in a new format for these interviews.
For all rounds, participants will complete the core drug screening modules via ACASI. For the SUD module, participants will be given the option of having the questions played over the computer’s speakers or turning off the sound and reading the questions aloud. These procedures are required so that the cognitive interviewers can stop the participant after certain questions to ask cognitive interview probes. Providing the participants the option of hearing the questions or reading them allows the cognitive interview process to more closely mimic an actual interview where participants can turn down the volume and read the questions if desired.
With participants’ permission, the sessions will be audio recorded so the interviewer can reference the recordings when refining his/her notes. The audio of the entire cognitive interview sessions will be recorded, including the interview questions and participants’ responses. The audio will be recorded directly onto interviewer laptops using digital recording software (such as Audacity). The digital files of the recordings will be labeled with the respondent case number on laptops secured with Checkpoint Endpoint disk encryption software. The links between the numeric file names and respondent identities will be kept separately from the audio recordings at all times. The audio files will not be transcribed, and they will not be transferred to any removable media, such as a CD.
Cognitive interviewers may take notes electronically on their laptops, which are also secured with Checkpoint Endpoint disk encryption software. Thus, the data on the laptops will be encrypted. Any hardcopy notes that are then transcribed electronically will be stored only on interviewers’ laptops. Interviewers will use a unique case number rather than the participant’s name on all interview notes. Other electronic files containing personal information such as telephone numbers will be password protected, with the password set to expire within four weeks after the final memo with the cognitive interview findings is completed and approved by CBHSQ. Both the electronic files and hardcopies will be destroyed at that time. The audio recordings will also be destroyed within four weeks of the final memo.
CBHSQ is in contact with all major Federal health survey managers and is aware of no other efforts to assess how potential respondents may react to changes made to the NSDUH questionnaire based on DSM-5 revisions. To date, no duplication of effort has been identified.
This survey does not involve small businesses or other such entities.
The changes to the NSDUH survey based on the findings of these cognitive interviews will be implemented in January 2017 when data collection begins. In order to meet this deadline, collection and reporting of the DSM-5 interview results must conclude by December 2015 so that the new questions can be included in the 2017 NSDUH OMB package. This project is a one-time collection and will not be repeated.
This information collection fully complies with 5 CFR 1320.5(d)(2).
External expert reviews were conducted in two phases. The first phase involved a substantive review by substance abuse experts at the National Institute of Drug Abuse and Dartmouth University. As part of that review, experts were provided with an overview of the NSDUH, copies of the DSM-5 criteria for cannabis use disorder, and additional text on substance use disorder measurement in general. Experts were also provided a report that examined diagnostic criteria changes from the DSM-IV to DSM-5 and evaluated their impact on the NSDUH, as well as a draft of the revised cannabis use disorder module with the draft items and specific questions regarding how well the substantive experts felt the questions aligned with the DSM-5 criteria. The substantive reviewers included:
Maureen Boyle, Ph.D., Branch Chief
Science
Policy Branch, National Institute on Drug Abuse
(301) 443-6071
Alan J. Budney, Ph.D., Professor
Geisel
School of Medicine, Dartmouth University
(603)
653-1821Wilson M. Compton, M.D., Deputy Director
National
Institute on Drug Abuse
(301) 443-6480
Once feedback was received from the substantive reviewers, the draft questionnaire was revised prior to the second phase of external review, which focused on methodological concerns. For this second review, the revised questionnaire and other materials were sent to external methodologists for their review and feedback. They were asked to address possible contextual issues, concerns over word choices and respondent comprehension, and skip pattern and question structure. The methodological reviewers included:
Paul C. Beatty, Ph.D., Chief
Center for
Survey Measurement, U.S. Census Bureau
(301)
763-5001
Gordon Willis, Ph.D., Cognitive
Psychologist
Office of the Associate Director of the Applied
Research Program, National
Cancer Institute
(240) 276-6788
Based on the feedback provided by the methodologists, the items were further revised into their current form.
Both adult (aged 18 and older) and youth (aged 12 to 17) cognitive interview participants will be given $40 cash for completion of the interview. The interviews will last, on average, 60 minutes. This incentive amount will be sufficient to compensate for the participants’ time and any travel expenses incurred (Willis, 2005). This amount is consistent with the amount requested for other studies of this length.
This incentive amount is recommended for adolescents as well as adults based on experience recruiting participants from a specific population – in this case, adolescent drug users. Given the relationship of the schedule for cognitive interviewing to the schedule for fielding the revised NSDUH instrument by January 2017, significant delays in recruiting participants with a lower incentive amount could adversely affect the timely implementation of these future activities.
The incentive for the cognitive interview is mentioned in the following materials: Recruitment Flyers (Attachment A), Web Recruitment Screener (Attachment B), Telephone Recruitment Screener (Attachment C), Parental Permission Form (Attachment D), Participant Informed Consent Forms (Attachment E), Protocol for Cognitive Interviews (Attachment F), and Cognitive Interview Receipt for Participation (Attachment G).
Concern for the confidentiality and protection of respondents’ rights has always played a central part in the implementation of NSDUH and will continue to be given the utmost emphasis.
The Contractor’s Institutional Review Board (IRB) was granted a Federalwide Assurance (Attachment H) by the Office for Human Research Protections (OHRP) and HHS in compliance with the requirements for the protection of human subjects (45 CFR 46). The Contractor’s IRB will approve the protocols and consent forms for these DSM-5 cognitive interviews prior to any respondent contact. The IRB’s primary concern is protecting respondents’ rights, one of which is maintaining the confidentiality of respondent information. By obtaining IRB approval for NSDUH procedures and materials, CBHSQ is assured that respondent confidentiality will be maintained.
The cognitive interviews for the DSM-5 study will incorporate several procedures to ensure that respondents’ rights will be protected. The recruitment flyers (Attachment A) will advertise to the participants that “All responses will be kept confidential under federal law”. Also, the recruitment screeners (Attachments B and C), parental permission form (Attachment D), and the participant informed consent forms (Attachment E) all indicate to the participants that the interview will be conducted in private to ensure the following:
no one else will overhear their answers;
all of their answers will be kept private and confidential;
information given by the participants will not be shared with any persons outside the project staff;
their name will never be connected with the answers they provide;
and that federal law (CIPSEA) requires that their answers be kept confidential and used only for statistical purposes.
In these same study materials, participants are informed that their responses are voluntary and are assured there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular question.
During the recruitment process, information will be collected on the age, race/ethnicity, education, phone number, city of residence, any significant physical limitations that would preclude participation, and past substance use. The telephone numbers collected will be used to remind the participants about their upcoming appointments and to recruit additional participants placed on a “will call” list in the event that any of the originally-recruited participants are no longer available or do not show for their appointment to be interviewed.
All internal communication regarding a participant will include only the first name of the participant and time of interview. An example of internal communication would be when the recruiter notifies the interviewer of a scheduled appointment with a participant.
All recruitment materials connecting the first name of the participant with his/her last name and other personal information will be locked in a cabinet (if in hardcopy form) or password protected (if in electronic form). Both the electronic files and hardcopies containing identifying information will be destroyed within four weeks after CBHSQ approves the final report.
If selected for the cognitive interview, recruiters will schedule a time to conduct one-on-one interview appointments at the contractor’s cognitive laboratory facilities, substance use treatment facility or other private location such as a private room in a public library or community center.
For interviews conducted with adolescents, parents will accompany adolescents to the interview. Upon arrival at the interview, the interviewer will review the consent form and assent form (Attachment D, Parental Permission Form; Attachment E, Participant Informed Consent) with both the adolescent and the guardian and will receive verbal consent.
For interviews conducted with adults, interviewers will review the consent form (Attachment E, Participant Informed Consent Form) with the participant and collect verbal consent. To protect respondent anonymity, the informed consent/assent form will be signed only by the interviewer after receiving verbal consent/assent from the participant. Participants will receive a copy of the consent, assent, and parental consent forms.
Only those respondents who give verbal consent/assent to participate will be interviewed. Participants will also be asked to provide consent to have the interview audio recorded. In the event that observers are present, participants will provide consent for observation to take place. If participants decline to have the interview recorded or observed, the interview will still be conducted without any recording or observations.
During the cognitive interviews, the age and gender will be collected from all participants but only to inform the interview program on the laptop computer which questions to display and to tailor wording. No links to individual participants will be preserved in the cognitive interview report, and personal identifying information will not be included in the data or final memo delivered to CBHSQ.
Many of the questions to be tested concern topics that are likely to be of a sensitive nature, including alcohol use by persons under the age of 21, use of illegal drugs by participants of all ages, and questions about substance use dependence and withdrawal. Consequently, some of these questions could be distressing for some respondents. The questions that will be included in the cognitive interview are provided in Attachment I, Cognitive Interview Study Questions. The cognitive interview protocol is provided in Attachment F.
The cognitive interview format could also increase the risk of distress beyond what would occur in a regular NSDUH survey in part because these participants are known substance users who therefore may have increased prevalence for comorbid mental illness and exposure to trauma in individuals with substance abuse disorders. The risk is also increased because participants are asked to discuss the questions and their thinking about the questions directly by the interviewers and at times in an open-ended fashion.
Measures will be taken to reduce risks to the respondents. For participants interviewed on-site at treatment facilities, arrangements will be made with the programs to have a counselor on call at the facility during the times when interviews will be conducted, in case any treatment clients participating in the study become upset by the interview questions and want to speak with a counselor. For participants being interviewed at the Contractor’s facilities who become upset with the questions and would like to speak with someone, referral options include their health care provider (if applicable) and also the Lifeline Network (1-800-273-TALK [8255]). Participants recruited from outside of substance abuse treatment who request information on substance abuse treatment options will be referred to SAMHSA's 24-hour toll-free Treatment Referral Helpline (1-800-662-HELP). These procedures are outlined in more detail in the Distressed Respondent Protocol in Attachment J.
In addition, all participants regardless of age will be reminded periodically not to report anything that could identify another person, such as referencing individuals who sold them or gave them illegal drugs. At the conclusion of the tutorial prior to the main interview (Attachment F, page 1), the interviewer will give examples of ways that a participant may talk about another person and examples of ways not to talk about someone. As further protection, the interviewer will interrupt participants who appear ready to report identifying information about themselves or someone else in response to a question/probe.
As noted in Section 10, potential participants and the actual participants will be assured at all stages of the recruiting and interviewing process that the information they provide is voluntary and will be handled in a confidential manner. These efforts will be made to help participants feel more comfortable with the interview situation and more at ease with the interviewer.
Raw data from the screening questionnaires, cognitive interviewing protocols, and audio recordings that include sensitive information will be stored in locked cabinets (if in hardcopy form) or password protected (if in electronic form) during the recruiting and interview process. None of this information will be retained once the data have been extracted and aggregated; nor will the information become part of a system of records containing permanent identifiers that can be used for retrieval.
A total of 51 participants (12 English-speaking adolescents, 27 English-speaking adults, and 12 Spanish-speaking adults) will be interviewed in this study. It is expected that approximately 150 screenings with potential participants will be completed to obtain the 51 study participants.
Administration of the screening questionnaire during the recruitment process will take an average of 5 minutes per participant. It is estimated that the average amount of time required to conduct each cognitive interview will be approximately 60 minutes.
The recruitment, cognitive interviewing, and analysis phases for all rounds of interviews for this study will span approximately 10 months, from February 2015 through December 2015.
The respondent burden for this study is shown in Table 1 below. The hourly wage of $14.61 was calculated based on weighted data from the 2012 NSDUH respondents’ personal annual income.
Table 1. Estimated Burden for Cognitive Interviews for Questionnaire Redesign
Activity |
Number of Respondents |
Responses per Respondent |
Average Burden per Response (Hours) |
Total Burden (Hours) |
Hourly Wage Rate |
Total Hour Cost |
Screening |
150 |
1 |
0.083 |
12.45 |
$14.61 |
$181.89 |
Full Cognitive Interviews |
51 |
1 |
1.000 |
51.00 |
$14.61 |
$745.11 |
TOTAL |
150 |
– |
– |
63.45 |
– |
$927.00 |
There are no capital, startup, operational, or maintenance costs to respondents.
Total costs associated with the cognitive interviews are estimated to be $215,886 over a 10-month period. Of the total costs, $195,301 are for contract costs (e.g. recruiting for, conducting, analyzing, and reporting on cognitive interviews), and approximately $20,585 represents CBHSQ costs to manage the task.
Currently there are 2,664.71 total burden hours in the OMB inventory. For the DSM-5 cognitive testing, SAMHSA is requesting 63.45 burden hours.
The DSM-5 cognitive interviews will be used to test the wording changes and question additions that are being considered for the 2017 NSDUH questionnaire. The sample size and design do not allow for statistical inference to be conducted, and therefore, the analyses will be qualitative. Debriefings with the cognitive interviewers will be conducted to learn from their experiences about participants’ reactions and responses to the survey questions and interviewer probes. The results will be summarized in a report and used to make recommendations for questions to be revised and/or included for the 2017 NSDUH.
The schedule for the cognitive interviews for the NSDUH DSM-5 Study is included in Table 2 below.
Table 2. Schedule for Cognitive Interviews for Questionnaire Redesign
Subtask |
Date |
Recruiting for cognitive interviews begins |
4/8/2015 |
Round 1 cognitive interviews begin |
4/8/2015 |
Recruiting for Round 2 cognitive interviews begins |
7/9/2015 |
Round 2 cognitive interviews begin |
7/16/2015 |
Recruiting for Round 3 cognitive interviews begins |
8/31/2015 |
Round 3 cognitive interviews begin |
9/7/2015 |
All cognitive interviews completed |
9/24/2015 |
Final cognitive interview report completed |
1/8/2016 |
The OMB expiration date will be displayed on the Parental Permission Form (Attachment D) and Participant Informed Consent Forms (Attachment E).
The certifications are included in this submission and fully comply with 5 CFR 1320.9.
B. COLLECTIONS OF INFORMATION EMPLOYING STATISTICAL METHODS
The Contractor will recruit a total of 51 participants (39 English-speaking and 12 Spanish-speaking) from a non-probability based sample. The 51 participants will consist of volunteers who are screened and determined to meet recruitment criteria.
English-Speaking Participants
A total of 39 cognitive interviews will be conducted with English-speaking participants in each of the following four geographic areas: Research Triangle Park (RTP), NC; Washington, DC; Chicago, IL, and Portland, OR.
The objectives mentioned previously for the cognitive interviewing require the study to include people who match selected characteristics of the target population the NSDUH is trying to reach. Specifically, all participants must meet the criteria for alcohol use, marijuana use, or at least one other substance in the past 12 months. Eligibility targets for adults will seek a majority of marijuana users (67%), so that participants will be routed to the new marijuana withdrawal items during the interview. Table 3 shows the sample targets for English-speaking participants in each round of cognitive interviewing.
Table 3. Targeted Sample Sizes per English Cognitive Interview Round by Location and Age Group
Group/Recruitment Source |
Cognitive Interview Round |
Total |
||
Round 1 |
Round 2 |
Round 3 |
||
Total |
13 |
13 |
13 |
39 |
Location |
|
|
|
|
Research Triangle Park, NC |
3 |
3 |
3 |
9 |
Washington, DC |
4 |
3 |
4 |
11 |
Chicago, IL |
3 |
3 |
3 |
9 |
Portland, OR |
3 |
4 |
3 |
10 |
Age |
|
|
|
|
12-17 |
4 |
4 |
4 |
12 |
18-49 |
4 |
4 |
4 |
12 |
50+ |
5 |
5 |
5 |
15 |
Past 12 month substance use |
|
|
|
|
Marijuana |
|
|
|
|
12-17 |
3 |
3 |
3 |
9 |
18+ |
7 |
5 |
5 |
17 |
Other substances |
|
|
|
|
12-17 |
1 |
1 |
1 |
3 |
18+ |
2 |
4 |
4 |
10 |
Youth: English-speaking participants aged 12 to 17 years old will be recruited from outpatient drug treatment centers in the relevant geographic areas surrounding Research Triangle Park, NC, Washington, DC, Chicago, IL and Portland, OR. Contractor staff will contact the treatment centers via email, telephone or in person. Staff will explain the purpose of the study and provide any additional details about NSDUH, SAMHSA or the cognitive interviews as requested. Contractor staff will ask treatment center staff to identify adolescents who meet the eligibility criteria (past 12 month users of alcohol, marijuana or other drugs), and provide these individuals with a copy of the advertisement (Attachment A), which references the $40 incentive. Interested adolescents can then call the number provided and complete a brief telephone screener to verify eligibility (Attachment C). Prior to completing the telephone screener with the adolescents, recruiters will obtain consent to complete the screener from a parent or guardian. Because adolescents are required to have parental consent to participate, and must have used substances in the past 12 months, all adolescents will be recruited from drug treatment facilities. This will allow us to access a population where parents are aware of the youth’s illicit substance use.
In addition, all youth interviews will be conducted in a private room at the treatment facility. Adolescents will be required to have a parent or guardian accompany them to the interview.
Adults: English-speaking adult participants will be recruited from a variety of sources including outpatient drug treatment centers in the relevant geographic areas, from advertisements posted in the classified sections of internet sites, and from advertisements playing on the radio (Attachment A). All recruitment advertisements reference the $40 incentive. Participants who are recruited from outpatient substance abuse treatment programs will be given the option to be interviewed on-site at their program, at a nearby Contractor’s office (in Research Triangle Park, NC; Washington, DC; or Chicago, IL), or in a private location such as a private room in a public library or community center.
If these methods fail to produce adequate numbers of adult participants as described in Table 3, the Contractor will place newspaper ads or distribute flyers in other locations (e.g., medical clinics, student unions) after having received appropriate permissions. Participants who are recruited from these sources will be interviewed at the Contractor’s offices or a private location such as a private room in a public library or community center.
Individuals who responded to the recruitment in Round 1 or Round 2 but were placed on a waiting list could still be eligible to be interviewed in a subsequent round. They will be rescreened to verify they are past 12 month substance users.
Spanish-speaking participants
Spanish-speaking participants will be recruited from outpatient treatment centers and Hispanic community center organizations and via word of mouth. If these methods fail to produce adequate numbers as described in Table 4, the Contractor will place newspaper ads or distribute flyers in other locations (e.g., medical clinics, student unions) after having received appropriate permissions. These advertisements will reference the $40 incentive for participation.
Interviews with Spanish-speaking participants will only be recruited for Rounds 2 and 3. The Contractor will conduct Round 2 interviews in the Research Triangle Park, NC area. Round 3 interviews will be conducted in Chicago, IL. If needed, alternate geographic locations such as Miami, FL may be used to recruit additional Spanish-speaking participants. Participants will be interviewed at the Contractor’s offices or a private location such as a private room in a public library or community center.
Spanish-speaking participants must meet the criteria for alcohol use, marijuana use, or at least one other substance in the past 12 months. Eligibility targets will seek approximately 50 percent marijuana users so that participants will be routed to the new marijuana withdrawal items during the interview.
Table 4 shows the sample targets for Spanish-speaking participants in each round of cognitive interviewing.
Table 4. Targeted Sample Sizes per Spanish Cognitive Interview Round by Location and Age Group
Group/Recruitment Source |
Cognitive Interview Round |
Total |
||
Round 1 |
Round 2 |
Round 3 |
||
Total |
0 |
6 |
6 |
12 |
Location |
|
|
|
|
Research Triangle Park, NC |
0 |
6 |
0 |
6 |
Chicago, IL |
|
|
6 |
6 |
Age |
|
|
|
|
18-25 |
0 |
2 |
2 |
4 |
26-49 |
0 |
2 |
2 |
4 |
50+ |
0 |
2 |
2 |
4 |
Past 12 month substance use |
|
|
|
|
Marijuana |
0 |
3 |
3 |
6 |
Other substance |
0 |
3 |
3 |
6 |
Recruitment and Screening: There will be up to 39 adult participants and 12 adolescent participants recruited from advertisements posted in the classified sections of internet sites, flyers, and through local outpatient drug treatment centers, and community organizations.
Potential English-speaking adult participants recruited from online advertisements will complete a web screener (Attachment B), and adults recruited from via other means (outpatient treatment center, flyer) will complete a telephone screener to determine if they are eligible. Potential participants will be screened for demographic information (age, sex, race/ethnicity, and education), the presence of any physical limitations that would preclude their effective participation, and past alcohol, marijuana or illegal drug use in the past 12 months. A recruiter will contact eligible adults to schedule a 60-minute interview at the treatment center (for those recruited there), at one of the Contractor’s private offices or another private setting such as a private room in a public library or community center. Callers will be notified that they will receive $40 for completing the interview.
Spanish-speaking adults may either complete the screening online or over the telephone. A recruiter will contact eligible participants to schedule a 60-minute interview at the Contractor’s private offices, or at another private setting agreed upon by the cognitive interviewer and participant (such as the participant’s home). Callers will be notified that they will receive $40 for completing the interview.
Potential adolescent participants or adults calling for their adolescent child will be directed to call into the study line to verify eligibility rather than complete a web screener. For adolescent callers, the consent of a parent or guardian will be required before the adolescent can be asked any screening questions. Potential participants will be screened for demographic information (age, sex, race/ethnicity, and education), the presence of any physical limitations that would preclude their effective participation, and confirm that they are receiving outpatient treatment for alcohol or drug use (using Attachment C). The recruiter will inform potential participants that a parent or guardian must accompany the adolescent to the interview to sign the consent form in person prior to the start of the interview. Eligible adolescent participants will then be scheduled for a 60-minute interview to be held at the treatment center. Callers will be notified that they will receive $40 for completing the interview.
Potential participants who call the Contractor’s designated telephone number or complete the screener online after the requisite number of participants has been recruited will be placed on a wait list with their permission, but only for the duration of the interviewing phase. During the interviewing phase, individuals on the “wait list may be called to ask if they are available, such as if one or more originally scheduled participants did not arrive for a scheduled interview. Recruitment and contact information will be kept in locked cabinets and via password protected electronic files and not shared except with those who are assigned to complete the interviews.
Interview Process: Cognitive interviews will be conducted by Contractor staff who are survey methodologists trained in conducting cognitive interviews. Prior to the first round, a cognitive interviewer training will be held to discuss the goals of the project and train interviewers on the procedures specific to this study.
When each participant arrives for their interview, he or she will be greeted and asked to listen to instructions and informed consent/assent information from the interviewer, and parental permission when applicable (Attachments D and E). All cognitive interviews will be audio recorded upon consent of each participant and a subset may be observed by a staff member, again upon consent of each participant. Participants will have the right to decline to be audio recorded and/or observed without being excluded from participation.
Before the interview begins, the participant will be read the Participant Informed Consent Form (Attachment E). If the participant appears to be cognitively impaired or under the influence of alcohol, then informed consent cannot be given, and the interview will not be conducted. In this event, the interviewer will follow the procedures described in the Distressed Respondent Protocol (Attachment J).
The interview will commence after the participant has given his or her consent. The interviewer will begin by asking the participant for demographic information (to determine how the interview program should route the participant through questions in the interview) and then will show each respondent how to navigate through the interview program. Participants then will complete a tutorial that teaches participants how to complete the ACASI portion of the survey.
Following the tutorial, participants will complete an abridged version of the 2015 NSDUH core drug screening modules for alcohol, marijuana, cocaine, heroin, and methamphetamines, as well as medical and nonmedical use of prescription drugs. These modules will include the same questions that are in the 2015 NSDUH, except some questions not necessary for cognitive testing (age at first use) have been removed. No new questions were added to these modules for cognitive testing. Answers that participants give to questions about use of substances will determine whether they are asked more detailed questions about substance dependence and withdrawal. Some of the SUD questions have been reworded and are being presented in a new format for these interviews.
Cognitive interviews will be iterative with changes made to the questionnaire based on the findings from the previous round. For all rounds, participants will complete the core drug screening modules via ACASI. For the SUD module, participants will be given the option of having the questions played over the computer’s speakers or turning off the sound and reading the questions aloud. These procedures are required so the cognitive interviewers can stop the participant after certain questions to ask cognitive interview probes. Providing participants the option of hearing the questions or reading them allows the cognitive interview process to more closely mimic an actual interview where participants can turn down the volume and read the questions if desired.
Interviewers will be provided with a series of probes and questions that will further explore the quality of responses and whether these responses are meeting the researchers’ goals. Interviewers are instructed to use the probes as a guideline, but are not required to use all of them, be limited by them, or to read them exactly as written (Beatty 2004). Interviewers may probe based upon the content of the interview and participant responses. Examples of the types of pre-scripted and spontaneous probes that will be used in cognitive interviewing as recommended by Willis (2005) are found in Attachment F (with the probes highlighted to differentiate them from them the rest of the questionnaire).
For Rounds 1 and 2, the cognitive probes will be administered concurrently, that is immediately after the participant answers the survey question. For Round 3, cognitive interview probes will be administered retrospectively after the participant has answered all SUD questions for a particular substance. This approach allows the cognitive interview to more closely mimic that of an actual interview, yet still allow for feedback about how participants understand the questions to be collected.
After the interview, participants will be thanked, given $40 cash as appropriate for their time, and will be given a participation receipt form (Attachment G).
To assure the participation of the recruited cognitive interviewing participants, each selected person will receive a reminder telephone call the day before the interview with directions to the interview location and an opportunity to ask any questions about the purpose or logistics of the study. The $40 incentive will also help ensure participation.
The activities to be conducted under this approval are in themselves tests of procedures.
The basic NSDUH design was reviewed by statistical experts, both within and outside SAMHSA. Statistical experts reviewing portions of prior NSDUH designs include William Kalsbeek, PhD, University of North Carolina; Robert Groves, PhD, Georgetown University; and Michael Hidiroglou, PhD, Statistics Canada. Monroe Sirken, PhD, National Center for Health Statistics (NCHS) (retired); James Massey, PhD, (deceased) also of NCHS; Douglas Wright, CBHSQ, SAMHSA (retired); Joseph Gfroerer, CBHSQ, SAMHSA (retired); and Arthur Hughes, CBHSQ, SAMHSA were consulted on the 1992 and subsequent survey designs. Peter Tice, CBHSQ, SAMHSA is the Government Project Officer, (240) 276-1254. Arthur Hughes, CBHSQ, SAMHSA is the primary mathematical statistician responsible for overall project management, (240) 276-1262. RTI senior statisticians contributing to the design are Paul Biemer, PhD, James Chromy, PhD, Ralph Folsom, PhD, and Rachel Harter, PhD.
Attachment A Recruitment Flyers
Attachment B Web Recruitment Screener
Attachment C Telephone Recruitment Screener
Attachment D Parental Permission Form
Attachment E Participant Informed Consent/Assent Forms
Attachment F Protocol for Cognitive Interviews
Attachment G Cognitive Interview Receipt for Participation
Attachment H Federalwide Assurance
Attachment I Cognitive Interview Survey Questions
Attachment J Distressed Respondent Protocol
References
American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author.
Beatty, P. (2004). The dynamics of cognitive interviewing. In S. Presser, J. Rothgeb, M. Couper, J. Lessler, E. Martin, J. Martin, and E. Singer (Eds.), Methods for Testing and Evaluating Survey Questionnaires (pp. 45-66). New York: John Wiley & Sons.
Forsyth, B. H. & Lessler, J. T. (1991) Cognitive Laboratory Methods: A Taxonomy. In Measurement Error in Surveys. P.P Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowitz, & S. Sudman (eds). New York, Wiley, 393-418.
Willis, G. B. (2005). Cognitive Interviewing: A Tool for Improving Questionnaire Design (p. 144). Sage Publications, Inc.: Thousand Oaks, CA.
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| File Title | 2003 National Survey on Drug Use and Health |
| Author | lky |
| File Modified | 0000-00-00 |
| File Created | 2021-01-25 |