OMB Submission
2008 National Survey on Drug Use and Health
Cognitive Interviews for Marijuana Consumption Questions
2008 National Survey on Drug Use and Health:
Cognitive Interviews for Marijuana Consumption Questions
SUPPORTING STATEMENT
1. Purpose and Use of Information
The National Survey on Drug Use and Health (NSDUH), sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA), is a national survey of the U.S. civilian non-institutionalized population aged 12 and older. The conduct of the NSDUH is paramount in meeting a critical objective of SAMHSA’s mission, i.e., to maintain current data on the prevalence of substance use in the United States.
This package is submitted under the NSDUH Methodological Field Tests generic OMB clearance (OMB No. 0930-0261).
In June of 2007, a meeting of expert consultants was convened by the Office of National Drug Control Policy (ONDCP) and the Substance Abuse and Mental Health Services Administration (SAMHSA) to discuss the potential for collecting items on drug consumption within the NSDUH survey. Recommendations for various methods to collect this information were discussed thoroughly; however, at the time of this meeting, it was not feasible to implement any of them into the 2008 NSDUH. These recommendations will be explored further for the 2009 survey. In the meantime, ONDCP has requested that two questions on marijuana consumption within the past 30 days and within the past 12 months be added to the 2008 survey.
Prior to implementation, qualitative cognitive interview data from 30 participants will be analyzed and reported on to evaluate how well potential NSDUH respondents understand the concept and language of the questions, whether they find them difficult or sensitive to answer, and whether they are able to provide accurate reports of marijuana consumption.
Data will be collected in a face-to-face interview setting. Information will be collected electronically via audio computer-assisted self-interview (ACASI) for the majority of the interview. The sessions will be audiotaped so the interviewer can reference the recordings when refining his/her notes. The sessions will involve showing participants questions that are added to the 2008 NSDUH Instrument as well as other questions within the marijuana modules. Respondents will be asked to answer these questions aloud and discuss them with the interviewer.
The OAS, SAMHSA, is in contact with all major Federal health survey managers and is aware of no other efforts to assess how potential respondents may react to changes made to the NSDUH instrument. To date, no duplication of effort has been identified.
4. Consequences if Information Collected Less Frequently
The timetable for finalizing the study participant materials for the 2008 NSDUH dictates the project be completed by October, 2007. This project will not be repeated.
It is DHHS policy that all national surveys are reviewed by the Office of the Assistant Secretary for Planning and Evaluation (ASPE). The review was coordinated by Dale Hitchcock, Director, Division of Data Policy, Office of Science Policy, ASPE, (202) 690-7100.
There are no unresolved issues resulting from these consultations.
Participants will be paid $40 in the form of a debit card made out to “RTI Respondent” for a forty-five minute session. This amount should be enough to compensate for the respondent’s time and any travel expenses incurred (Willis, 2005).
The payment for the interview is mentioned in the following materials: Recruitment Flyers [Attachment A], Recruitment Scripts [Attachment B], Parental Informed Consent Form [Attachment C], Participant Informed Consent Forms [Attachment D], Protocols for Cognitive Interviews [Attachment E], and Receipt for Participation [Attachment F].
Concern for the confidentiality and protection of respondents’ rights has always played a central part in the implementation of the NSDUH and will continue to be given emphasis.
The Cognitive Interviews for Marijuana Consumption Questions will incorporate several procedures to ensure that respondents’ rights will be protected. The recruitment flyers [Attachment A] will advertise to the participants that “Confidentiality is guaranteed”. Also, the recruitment scripts [Attachment B], parental informed consent form [Attachment C], and the participant informed consent forms [Attachment D] all indicate to the participants that the interview will be conducted in private to ensure nobody else overhears their answers; all of their answers will be kept private and confidential; information given by the participants will not be shared with any persons outside the project staff; their name will never be connected with the answers they provide; and that federal law (CIPSEA) requires that their answers be kept confidential and used only for statistical purposes. In these same study materials, participants are informed that their responses are voluntary and are assured that there will be no penalties if they decide not to respond, either to the information collection as a whole or to any particular question.
During the recruitment process, potential participants are told that we would like to audiotape the interactions between them and the interviewer during the cognitive interview so we can make sure we have all of the information from them on how our new questions work. They are told that the audiotapes will be heard only by members of the research team and that having their interview audiotaped is voluntary so they can decline (see the recruitment scripts, [Attachment B]). Additional information on the audiotapes is provided in the Parental Informed Consent Form [Attachment C] and the Participant Informed Consent Forms [Attachment D]. These documents inform the participants that the audiotape will be transferred to a CD, which will be protected by a password, to protect their privacy. We will store the CDs and tapes in a locked cabinet until the password on the CD expires (within 4 weeks after the last cognitive interview is completed). At that time both the CDs and tapes will be destroyed.
All internal communication regarding a participant will only include the first name of the participant and time of interview. An example of internal communication would be when the recruiter notifies the interviewer of a scheduled appointment with a participant. Any other materials (including the recruitment screeners, “will call” list, and informed consent forms) connecting the first name of the participant with his/her last name, telephone number, etc. will be locked in a cabinet (if in hardcopy form) or password protected (if in electronic form). This password will be set to expire within 4 weeks after the last cognitive interview is completed and both the electronic files and hardcopies will be destroyed at that time.
Although some personal information will be collected during the recruitment process and the cognitive interviews, data will not be retrieved by personal identifiers, and thus the Privacy Act does not apply to these activities. More specifically, during the recruitment process, we will collect the age, phone number, city of residence, and consumption of marijuana in the past 30 days and 12 months of potential participants who are aged 12 to 17. Other potential participants aged 18 or older will also be asked about their marijuana use within the past month and year. The telephone numbers collected will be used to either remind the participants about their upcoming appointments or to recruit additional participants who were placed on a “will call” list in the event that spaces opened up for them to be interviewed. For potential participants aged 12 to 17, the telephone number will also be used to verify with Directory Assistance that their parents are who they say they are. During the cognitive interviews, we will collect the age and gender of all participants only to inform the CAI program which questions to display and to tailor wording. No links to individuals will be preserved in the cognitive interview report.
The questions tested for the 2008 NSDUH deal with consumption of marijuana during the past 30 days and the past 12 months. To thoroughly test these questions, the target population for the cognitive interviews will need to be adults and adolescents who have used marijuana during the past 30 days or in the past 12 months. The initial screening questionnaire used for recruiting [Attachment B] will ask all potential participants, including adolescents and other adults, for their age and whether or not they have used marijuana at any point in the past 30 days, and if not, during the past 12 months.
During the cognitive interview, the participants are instructed on how to employ ‘think-aloud’ techniques when answering the questions that have been added to the instrument. The interviewer will use a standard set of interviewing protocols to best understand how the participants understand each question and how features of the question impact responses.
As noted in Section 7 potential participants and the actual participants will be assured at all stages of the recruiting and interviewing process that the information they provide is voluntary and will be handled in a confidential manner. These efforts will be made to help participants feel more comfortable with the interview situation and more at ease with the interviewer.
Raw data from the screening questionnaires, paper cognitive interviewing protocols, and audio tapes that include sensitive information will be stored in locked cabinets (if in hardcopy form) or password protected (if in electronic form) during the recruiting and interview process. None of this information will be retained once the data has been extracted and aggregated; nor will the information become part of a system of records containing permanent identifiers that can be used for retrieval.
Thirty participants will be interviewed in this study. It is expected that approximately 180 screenings with potential participants will be completed to obtain the 30 study participants.
Administration of the screening questionnaire during the recruitment process will take an average of 5 minutes per participant. The length of each cognitive interview will vary depending on whether the participant is recruited as an adolescent or an adult. Based on this information, it is estimated that the average amount of time required to conduct each cognitive interview will be approximately 45 minutes.
Both the recruitment and cognitive interviewing phases of this study will span for approximately three weeks over the months of August and September. The recruitment phase will begin one week prior to the cognitive interviews.
The respondent burden for this study is shown in the following table. The hourly wage of $14.02 was calculated based on weighted data from the 2006 NSDUH respondents’ personal annual income.
Estimated Burden for 2008 Cognitive Interviews for Marijuana Consumption Questions
Activity |
Number of Respondents |
Responses per Respondent |
Average Burden per Response |
Total Burden (Hrs.) |
Hourly Wage Rate |
Total Hour Cost |
Screening |
180 |
1 |
0.083 hr. |
14.94 |
$14.02 |
$209.46 |
Cognitive Interviews |
30 |
1 |
.75 hr. |
22.5 |
$14.02 |
$315.45 |
TOTAL |
210 |
– |
– |
37.44 |
– |
$524.91 |
Total costs associated with the 2008 Cognitive Interviews for Marijuana Consumption Questions are estimated to be $30,000 over a 12-month performance period. Of the total costs, $26,400 is for study design, preparation of materials for laboratory testing, recruiting, conducting the cognitive interviews, analysis and report/publication writing, and approximately $3,600 represents SAMHSA costs to manage/administrate the survey.
The 2008 Cognitive Interviews for Marijuana Consumption Questions results will be used to gauge the impact of wording changes and question additions required for the 2008 NSDUH survey. The sample size and design do not allow for statistical inference to be conducted, and therefore, the analyses will be observational and anecdotal. Debriefings with the cognitive interviewers will be conducted to learn from their experiences about respondents’ reactions and responses to the additional questions in the 2008 NSDUH instrument.
When the study has been completed, the SAMHSA and RTI liaisons will discuss opportunities for presenting results of the study to the external research community at a professional conference and/or via external publication.
The time schedule for the 2008 Cognitive Interviews for Marijuana Consumption Questions is included below.
2007 COGNITIVE INTERVIEWS FOR MARIJUANA CONSUMPTION QUESTIONS SCHEDULE
Subtask |
Due Date |
RTI sends cognitive interview materials (including recruitment ads and scripts and the interviewing protocol) to SAMHSA for review |
July 30, 2007 |
SAMHSA provides comments to RTI on cognitive interview materials |
July 31, 2007 |
RTI delivers final package (3 sets) for request of OMB blanket clearance to SAMHSA |
August 3, 2007 |
RTI submits IRB materials for approval |
August 3, 2007 |
OMB package sent from DHHS to OMB |
August 9, 2007 |
RTI IRB meeting |
August 14, 2007 |
RTI IRB approval obtained |
August 14, 2007 |
OMB notifies SAMHSA concerning outcome of review |
August 14, 2007 |
RTI begins recruiting for cognitive interviews |
August 23, 2007 |
RTI conducts cognitive interviews |
August 24-September 21, 2007 |
RTI sends final cognitive interview report to SAMHSA |
September 24, 2007 |
The sample of 30 participants to be recruited is non-probability based. It will consist of volunteers who are screened and determined to meet recruitment criteria. Adult volunteers will primarily be recruited from advertisements posted at drug treatment centers in the three interview areas, and in the classified sections of internet sites. If these methods fail to produce adequate numbers, we will place newspaper ads. Adolescent respondents will be recruited from drug treatment centers in the relevant geographic areas.
The above mentioned objectives require the study to include people who match selected characteristics of the target population SAMHSA is trying to reach for the NSDUH. Details regarding the composition of potential participants for the cognitive interviews are as follows:
10 interviews will be conducted in each of the RTI offices in Research Triangle Park (RTP), NC Washington DC and Chicago, IL.
One of the main focuses of the NSDUH is to gather national data on the use of illicit substances. The particular questions being tested measure amounts of marijuana consumption in the past 30 days and 12 months. In order to yield the most insightful and valid feedback about these questions, each respondent, including adolescents, must report having used marijuana in either the past 30 days or 12 months.
The majority of the NSDUH interviews are conducted in English. Thus we will focus our efforts on the English speaking population for all of our cognitive interviews.
Exhibit 1 below details the estimated number of participants to be recruited according to the characteristics detailed above for the cognitive interviews. If after 180 screenings (see Section 9), some of the cells specified in the table are unfilled, potential participants who give permission to be put on a “will call” list (because the requisite number of participants for their criteria had already been recruited) will be re-contacted to fill the vacant interview slots to maximize the attempt to obtain 30 study participants.
Exhibit 1 – Recruiting Matrix
Site |
Characteristics |
Estimated Number of Participants |
RTP, NC |
Aged 12-17, marijuana user in past 30 days |
1 |
RTP, NC |
Aged 12-17, marijuana user in past 12 months |
1 |
RTP, NC |
Aged 18+, marijuana user in past 30 days |
4 |
RTP, NC |
Aged 18+, marijuana user in past 12 months |
4 |
Washington DC |
Aged 12-17, marijuana user in past 30 days |
1 |
Washington DC |
Aged 12-17, marijuana user in past 12 months |
1 |
Washington DC |
Aged 18+, marijuana user in past 30 days |
4 |
Washington DC |
Aged 18+, marijuana user in past 12 months |
4 |
Chicago |
Aged 12-17, marijuana user in past 30 days |
1 |
Chicago |
Aged 12-17, marijuana user in past 12 months |
1 |
Chicago |
Aged 18+, marijuana user in past 30 days |
4 |
Chicago |
Aged 18+, marijuana user in past 12 months |
4 |
Up to 24 adult participants will be recruited from advertisements posted in the classified sections of internet sites, and through local drug treatment centers. All six adolescents will be recruited exclusively through flyers posted at drug treatment centers. [See Attachment A for recruitment flyers]. Potential participants who call into the study line will be screened for eligibility [Attachment B]. Adolescents who call in will be screened for their geographical location and age, whether they have, in the past 30 days or 12 months, used marijuana, and will require permission from their parent or legal guardian before being recruited. Screening questions will ask potential participants aged 18 or older whether they have, in the past 30 days or 12 months, used marijuana. Eligible respondents will be scheduled for a forty five minute cognitive interview.
Potential participants who call the RTI study line after the requisite number of participants has been recruited will be placed on a “will call” list with their permission, but only for the duration of the interviewing phase. During this phase, individuals on the “will call” list may be called to ask if they are available, should one or more originally scheduled participants not arrive for a scheduled interview. As specified in Section 7, recruitment and contact information will be kept in locked cabinets and via password protected electronic files and not shared except with those who are assigned to complete the interviews.
When each participant arrives for their interview, they will be greeted and asked to listen to instructions and information from the interviewer, including the signing of a confidentiality form [Attachment D]. All cognitive interviews will be audiotaped upon consent of each participant and a subset in Washington DC will be observed by SAMHSA staff from a separate room, again upon consent of each participant. Participants will have the right to decline to be audiotaped or observed without being excluded from participation.
Before the interview begins, the participant will be read the Participant Informed Consent. The interviewer will then show each respondent how to navigate through the NSDUH CAI program, and then they will complete a tutorial, along with the core marijuana and marijuana marketing modules from the main NSDUH survey [See Attachment E]. These modules will be completed via ACASI. At the end of the marijuana marketing module, the respondent will be instructed to read the new questions aloud and discuss them with the interviewer. Participants will deliver concurrent verbal reports of their thought process, or will provide thoughts on how they are answering the question before the question is answered (Forsyth and Lessler 1991). Interviewers will be provided with a series of probes and questions that will further explore the quality of responses and whether these responses are meeting the researchers’ goals. Interviewers are instructed to use the probes as a guideline, but are not required to use all of them, be limited by them, or to read them exactly as written (Beatty 2004). Interviewers may probe based upon the content of the interview and participant responses. All pre-scripted probes are found in the cognitive interviewing protocols in Attachment E. However, the interviewer requires the latitude to deviate from and/or make changes to these protocols “on the fly” during the interviewing process in order to follow up on new information that is gained from respondents during the course of the interviews. After the interview, participants will be thanked, paid for their time with a $40 gift card, and will be asked to sign a participation receipt form [Attachment F].
To assure the participation of the recruited cognitive interviewing participants, each selected person will receive a reminder telephone call the day before the interview with directions to the facility and an opportunity to ask any questions about the purpose or logistics of the study. The $40 incentive will also help ensure participation.
The activities to be conducted under this approval are in themselves tests of procedures.
Arthur Hughes (240) 276-1261 Project Officer/Mathematical Statistician Division of Population Surveys OAS, SAMHSA
|
Dicy Painter (240) 276-1264 Alternate Project Officer Division of Population Surveys OAS, SAMHSA
|
Joseph Gfroerer (240) 276-1262 Division Director/Supervisory Mathematical Statistician Division of Population Surveys OAS, SAMHSA
Peggy Barker (240) 276-1258 Statistician Division of Population Surveys OAS, SAMHSA
|
Joel Kennet (240) 276-1265 Statistician Division of Population Surveys OAS, SAMHSA
|
ATTACHMENTS
Attachment A - Recruitment Flyers
Attachment B - Recruitment Scripts
Attachment C - Parental Informed Consent Form
Attachment D - Participant Informed Consent Forms
Attachment E ‑ Protocols for Cognitive Interviews
Attachment F ‑ Receipt for Participation
REFERENCES
Beatty, P. (2004). The dynamics of cognitive interviewing. In S. Presser, J. Rothgeb, M. Couper, J. Lessler, E. Martin, J. Martin, and E. Singer (Eds.), Methods for Testing and Evaluating Survey Questionnaires (pp. 45-66). New York: John Wiley & Sons.
Forsyth, B.H. and Lessler, J.T. (1991) Cognitive Laboratory Methods: A Taxonomy. In Measurement Error in Surveys. P.P Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowitz and S. Sudman (eds). New York, Wiley, 393-418.
Willis, G. B. (2005). Cognitive Interviewing: A Tool for Improving Questionnaire Design (p. 144). Sage Publications, Inc.: Thousand Oaks, CA.
| File Type | application/msword |
| File Title | 2003 National Survey on Drug Use and Health |
| Author | rebecca |
| Last Modified By | Arthur Hughes |
| File Modified | 2007-08-03 |
| File Created | 2007-08-03 |