10-Day Letter Teen Alcoholand Marijuana Use Questions to be Cognitively Tested

DEPARTMENT OF HEALTH & HUMAN SERVICES Public Health Service

Centers for Disease Control and Prevention

National Center for Health Statistics

3311 Toledo Road

Hyattsville, Maryland 20782

September 7, 2017

Margo Schwab, Ph.D.

Office of Management and Budget

725 17th Street, N.W.

Washington, DC 20503

Dear Dr. Schwab:

The staff of the NCHS Collaborating Center for Questionnaire Design and Evaluation Research (CCQDER) (OMB No. 0920-0222, Exp. Date 07/31/2018) has entered into a receivable reimbursable agreement with CDC to evaluate questions on teen alcohol and marijuana consumption for the Youth Risk Behavior Survey (YRBS).

We propose to start recruiting for volunteer participants as soon as we receive clearance and to start testing as soon as possible after that.

Background Information about Cognitive Testing of Questionnaires

The methodological design of this proposed study is consistent with the design of typical cognitive testing research. The purpose of cognitive testing is to obtain information about the processes people use to answer survey questions as well as to identify any potential problems in the questions. The analysis will be qualitative.

Proposed project: Evaluation of Teen Alcohol and Marijuana Questions for the YRBS

The Youth Risk Behavior Surveillance System (YRBSS) monitors high priority health-risk behaviors among high school students, including alcohol use. The system includes a national survey that is conducted by CDC, as well as state and local surveys that are coordinated by state and local education and public health agencies in collaboration with other partners.¹

State and local Youth Risk Behavior Surveys (YRBS) include core questions on alcohol and other topics as well as state/local optional questions. The Collaborating Center for Questionnaire Design and Evaluation Research at NCHS cognitively and field tested the original YRBS questionnaire with high school students, including the questions on alcohol use.² However, there have been a number of changes to the core alcohol questions since that time, including the addition of new questions (e.g., a question on the source of alcohol, which was introduced in 2007), and changes to the wording of alcohol questions that have been used since the surveys inception (e.g., the introduction of a gender-specific question on binge drinking in 2017). In addition, several states have included in their state YRBS optional questions on the usual type of alcohol consumed by high school students who drink, as well as other topics (e.g., the mixing of alcohol and energy drinks) in other state youth risk behavior surveys (e.g., the Youth Tobacco Survey), which have not been cognitively tested. It is therefore timely to cognitively test the core alcohol questions in the YRBS, as well as some optional questions on alcohol consumption that are of particular interest to state public health agencies.

The proposed questions we are evaluating on teen use of alcohol and marijuana asked of teens aged 15-18 are included in Attachment 1. The testing procedure conforms to the cognitive interviewing techniques that have been described in CQDER’s generic OMB clearance package (No. 0920-0222, exp. 07/31/2018).

CCQDER will conduct one round of cognitive testing. The goal is to determine whether the tested items accurately measure teen alcohol and marijuana use. This is accomplished by examining the types of experiences respondents include in their answers and any difficulties they experience when answering. Difficulty answering questions may include trouble with comprehension, recall, phrases, and terminology. Systematic analysis will identify constructs captured by the questions as well as the potential for (and types of) response error.

We propose to recruit 30 English speaking teens for one-on-one cognitive interviews through a combination of newspaper advertisements, flyers and word-of-mouth. The newspaper advertisement/flyer will be used to recruit teens (aged 15-18) who (1) use alcohol and/or marijuana and (2) those who don’t use either. The newspaper advertisements/flyers used to recruit respondents are shown in Attachments 2a&b. The 5 minute screener used to determine eligibility of individuals responding to the newspaper advertisements/flyers is shown in Attachment 3. Note that wording of the template has been approved and is contained within our umbrella package. Only project specific information has been added to the document. It is anticipated that as many as 48 individuals may need to be screened in order to recruit 30 participants. Recruitment of individuals will be guided first by their experience with alcohol and/or marijuana use and second by demographic diversity, including gender, age, and race.  The initial goal is to recruit groups in equal measure.  However, because qualitative sampling is based on theoretical relevance, on-going analysis may reveal the need to recruit more from one group than others. During the screening process, ALL parents of minors determined eligible for the study will be informed of the purpose of the study, as well as the paperwork which needs to be read and signed before their children will be allowed to participate in the study. Arrangements will be made to get the paperwork to the parent and to have them return the completed paperwork. Our preference is that completed forms be returned by the parent via U.S. Mail or in person. However, if those options are not possible due to time constraints, completed forms may be returned through the minor in a sealed envelope provided by NCHS. In that case, the Laboratory Manager/Staff person will call the parent to verify that they have given their permission for their child to participate in the study (See Attachment 3, script for parents of minors once minor has been scheduled.) Within these constraints, we hope to recruit participants with some demographic variety (particularly in terms of gender, race/ethnicity, and education and income of parent).

Interviews averaging 60 minutes will be conducted by CCQDER staff members with English speaking respondents. Interviews will be conducted in the Collaborating Center for Questionnaire Design and Evaluation Research as well as at off-site locations. All interviews will be audio recorded. These recordings will allow researchers to ensure the quality of their interview notes and will serve as the raw data.

When a respondent arrives for the one-one-one interview they will be greeted by the Laboratory Manager/Staff person. After respondents have been briefed on the purpose of the study and the procedures that CCQDER routinely takes to protect human subjects, respondents (aged 18) and parents of minors (aged 15-17), will be asked to read and sign an Informed Consent and minors aged 15-17 will read and sign the Assent Form (Attachments 4a, 4b, and 4c). Only project specific information has been added to the documents. Signed parental consent forms for minors aged 15-17 will be received prior to the scheduled interview. Alternatively, the forms may be hand carried by the minor respondent in a sealed enveloped provided by NCHS, the Laboratory Manager/Staff person will check the forms for completeness and will call the parent to verify that they have given their permission for the minor child to participate in the one-on-one interview. If any of the required forms are missing or incomplete, the minor will not be allowed to participate in the interview and will not receive remuneration. Parents of minors will be informed of this during the recruiting and screening process (see Attachment 3, “For parents of minors once minor has been scheduled”). Respondents (18 year olds only) and parents of minors will also be asked to fill in their demographic characteristics on the Respondent Data Collection Sheet (Attachment 5). This document is contained in our umbrella package. The burden for completion of this form is captured in the interview.

As stated in the Informed Consent, so that their child feels comfortable answering the survey questions, parents of minors will be reminded that they will not be allowed to listen to the one-on-one interview. However, parents are encouraged to wait in the designated waiting area/room while the interview is going on.

The interviewer will then ask the respondent to confirm that he/she understands the information in the Informed Consent/Assent, and then state that we would like to record the interview. The recorder will be turned on once it is clear that the procedures are understood and agreed upon.

The interviewer will then orient the respondent to the cognitive interview with the following introduction:

[fill staff name] may have told you that we will be working on some questions that will eventually be added to national surveys. Before that happens, we like to test them out on a variety of people. The questions we are testing today are about teen alcohol and marijuana use. We are interested in your answers, but also in how you go about making them. I may also ask you questions about the questions—whether they make sense, what you think about when you hear certain words, and so on.

I will read each question to you, and I’d like you to answer as best you can. Please try to tell me what you are thinking as you figure out how to answer. Also, please tell me if:

there are words you don’t understand,

the question doesn’t make sense to you,

you could interpret it more than one way,

it seems out of order,

or if the answer you are looking for is not provided.

The more you can tell us, the more useful it will be to us as we try to develop better questions. Okay? Do you have any questions before we start? If yes, answer questions. If not, let’s get started.

After the interview, respondents will be given the thank-you letter (document contained in umbrella package) signed by the Charles J. Rothwell, Director of NCHS (Attachment 6), a copy of the informed consent/assent document, and $40.

Extreme care will be taken with all recordings and paperwork from the interviews conducted off-site. Recordings and identifying paperwork will be stored in a secured travel case until returned to NCHS, at which point they will be transferred to the usual secured locked storage cabinets.

We propose giving participants $40 incentives, which is our standard incentive. In total, for this project, the maximum respondent burden will be 34 hours. A burden table for this project is shown below:

Form Name	Number of Participants	Number of Responses/ Participant	Average hours per response	Response Burden (in hours)
Screener	48	1	5/60	4
Questionnaire	30	1	60/60	30

Attachments (6)

cc:

V. Buie

T. Richardson

DHHS RCO

1 Centers for Disease Control and Prevention. Methodology of the Youth Risk Behavior Surveillance System — 2013. MMWR 2013;62(No. RR-1):1-20.

2 Blanken AJ. Measuring use of alcohol and other drugs among adolescents. Public Health Reports. 1993;108(Suppl 1):25-30.

3 | Page