Attachment X_Cognitive Interview Report Final
Attachment X_Cognitive Interview Report Final.docx
Pilot Implementation of the Violence Against Children and Youth Survey (VACS) in the US
Attachment X_Cognitive Interview Report Final
OMB: 0920-1356
FINAL REPORT D-VACS Cognitive Interviewing Study in Baltimore |
May 25, 2022 |
Presented to: Jeffrey Ratto, DrPH Centers for Disease Control and Prevention (CDC), Division of Violence Prevention |
Presented by: NORC at the University of Chicago 55 East Monroe Street Chicago, IL 60603 (312) 759-4000 (Main) (312) 759-4004 (Fax)
|
Table of Contents
Introduction
CDC has contracted NORC at the University of Chicago to pilot the Domestic Violence Against Children (D-VACS) survey in a United States site. The D-VACS program includes in-person household surveys to collect information and generate estimates of physical, emotional, and sexual violence against children and youth aged 13-24. Previously fielded globally, this present project seeks to test the usability of the VACS in the United States, piloting in the City of Baltimore, Maryland and later in a rural county in Maryland (Garrett County). Prior to fielding the survey, a key research step has been to collect target population feedback on the survey instrument through cognitive interviews, allowing the CDC to make any necessary corrections or improvements to the questionnaire for accurate data collection in the United States setting.
The summary and suggestions that follow reflect the cognitive interview testing work implemented by NORC in February – April 2022. The objectives of the cognitive interviews were to evaluate key D-VACS survey questions for correct interpretation by respondents and to assess the ability of respondents to accurately answer the survey questions. More generally, the cognitive interviews informed the research team on levels of comprehension and the thought process respondents used to answer the D-VACS survey questions. The results of this research will aid in refining the D-VACS questionnaire items, item order, and inform the data collection methods.
The material presented in this report reflect our methodologies and procedures, key observations, and suggestions for consideration moving into the pilot study. Ultimately, 30 interviews were completed with Baltimore youth.
NORC used both remote and in-person recruitment methods. Eight youth were recruited and scheduled remotely (i.e., via email, text, and phone) and completed their interviews via Zoom. The rest of the youth were recruited in-person and completed their interviews immediately. When recruited in person, youth were assigned to one of four interviewers, three of which were on Zoom and one of which was in person (the lead interviewer). About half of all interviews were completed via Zoom, and half were face-to-face.
Methodology
Approvals. Prior to beginning recruitment for the cognitive interview study, NORC and CDC secured OMB approval (OMB# 0920-1356). Additionally, the NORC Institutional Review Board (IRB) reviewed and approved the study protocol. NORC followed the approved cognitive interviewing protocol on record with the NORC IRB and OMB, with the survey questions presented to interviews as they would be during survey implementation.
Site of Baltimore for Cognitive Testing. Located in the Eastern Seaboard of the US (the Mid-Atlantic), about 40 miles northeast of Washington, DC, Baltimore is the most populous city in the State of Maryland and 30th most populous city in the US, with a long history as an important seaport. With a population of 585,708 in 2020, the population of the larger Baltimore metropolitan area was estimated to be around 2.8 million.1 Baltimore was a compelling option for the D-VACS study because of its high violent crime rates and the Baltimore City Health Department’s experience in both youth violence data collection and prevention program implementation, as well as their deep experience in leading stakeholder coalitions to address local public health challenges.
Purpose. The purpose of the cognitive testing has been to test portions of the current D-VACS instrument for intended comprehension of key questions, new questions, and to better understand any potential data collection concerns around recruitment and privacy. The findings from these cognitive interviews will allow CDC to field the full pilot study more confidently in the City of Baltimore in 2022 and use this information to plan for expanded fielding of D-VACS in the United States, as has occurred in other countries. A series of 30 cognitive interviews were conducted to test comprehension and acceptability of the questions to inform revisions to the existing instrument. After about half of the cognitive interviews were completed (n= 17), the CDC and NORC research teams met to reflect on findings to date. The team also discussed areas for focus in the remaining interviews that needed additional exploration (e.g., the issue of gaining greater trust from respondents to promote more truthful reporting). The cognitive interview results have helped the research team to identify problems with items, the organization and order of the instrument, revealing potential sources of response error in the D-VACS that the research team will draw on these findings to inform any re-design efforts.
The cognitive interviews assessed issues such as:
Respondent understanding of terms in the survey.
Respondent confidence in responses.
Recall techniques.
When answering factual questions, such as when an injury occurred and what happened, we reminded respondents that we did not want actual answers to the survey items, but rather the thought process respondents would use to recall such events. In these instances, we asked the participant to think about an injury or event that might have occurred in the past. As part of our instructions to the participant, we introduced this recall pattern as “helpful to us to understand how you might recall a situation like that to answer questions on the survey…specifically, would you first think about WHEN it happened first, or WHERE it happened, or WHO the person was and if they were alone or other people were there?”
Suitability of response options.
Ease of answering questions.
Issues with sensitive questions.
General Approach. Our cognitive testing was designed to provide a description of the meanings and processes used by study participants to answer questions and explore concepts, processes and patterns of interpretation by youth of the D-VACS survey items. Cognitive interviewing, particularly with a youth population and around potentially sensitive topics, requires trained interviewers and an understanding of potential interviewer effects. NORC used one main lead trained interviewer for the testing, a NORC methodologist with over 10 years of cognitive interviewing experience. NORC also had three additional trained interviewers to expand the interviewing capacity of the team when multiple interviews needed to be completed at the same time. Interviewers followed a semi-structured in-depth guide, allowing for guided probes as well as free responses.
The NORC team used ‘verbal probing’ cognitive interview techniques and asked participants specific questions to gauge their understanding of the D-VACS survey content. As a qualitative method, the primary benefit of cognitive interviewing is that it provides rich, contextual insight into the ways in which respondents 1) interpret a question, 2) consider and weigh out relevant aspects of their lives and, finally, 3) formulate a response approach based on that consideration (note: we will not solicit actual responses to the questions). As such, cognitive interviewing provides in-depth understanding of the ways in which a question operates, the kind of phenomena that it captures, and how it ultimately serves (or fails) the scientific goal.
Recruitment. Recruitment was completed using a descriptive flyer and emails about the cognitive testing effort and distributing it to known networks through referrals. Primarily, this included steering committee members (e.g., members of the Baltimore City Health Department) making the opportunity to participate known to youth. NORC developed flyers and FAQ sheets for both physical and digital distribution that were used in recruitment. NORC used a benign topic for recruitment (i.e., not violence; but maintaining good health during the pandemic) to protect the study participants and limit concerns around participation. Recruitment materials directed individuals interested in participating in an interview to contact NORC (via telephone or email). A trained NORC staff member explained the study and the cognitive interview process, answered any questions the respondent may have had, and screened the respondent into the study (e.g., meeting age and residency requirements). Near the end of data collection and interviewing, we also turned to known networks of NORC employees to attempt capture all of the neighborhoods in Baltimore. Youth were eligible if they were aged 13-24, resided in the city of Baltimore, and were willing to speak with us. If the youth was aged 13-17, an additional requirement was the ability to obtain permission from a parent/guardian.
Ultimately, we were able to successfully complete 30 interviews representing geographic diversity and diversity of experience across Baltimore. Participants who completed the interview were thanked with a $50 Visa gift card.
Overall, recruitment posed a challenge similar to other recruitment efforts the industry has seen in the past several years, particularly with youth. We experienced significant rates of ‘no-shows’ and reschedules, making the recruitment process challenging. A key challenge was scheduling, as many youth had competing activities that would arise making it difficult for them to keep a set interview appointment. Additionally, while the youth received 3-5 reminders (largely by text, as email was not commonly used by the youth), including right before the interview, they were not always able to be communicated with in time to complete the interviews. In-person recruitment proved the most successful, as we could recruit, screen, and complete the interview without separation. Additionally, in-person recruitment allowed for community partners and organizations to make introductions and speak on our behalf, earning trust and cooperation with the youth.
We obtained a diverse group of participants on gender (50% male, 40% female, 10% non-binary), age (almost an even split between those under 18 years old and those over 18 years old), and race (African American (86%), as two or more races (7%) or as ‘other’ race (7%)).
Materials. NORC prepared a protocol from the D-VACS Core Participant Questionnaire which included items and definitions selected from across all D-VACS topic areas. NORC determined questions for inclusion in collaboration with CDC. Most D-VACS items selected for the cognitive interview protocol were those that had been significantly adapted for the U.S. context or not previously fielded in global VACS. The procedure for the cognitive interviewing focused on instructing respondents on how to think about their answers—rather than providing us actual answers. NORC’s cognitive interviewers were trained on identifying and responding if a participant displayed signs of distress or reported abuse and equipped in advance with a written response plan should concerns arise.
Interviewer Training Procedures. Training included several components, beginning with in-depth review of the interview protocols and read-throughs. Interviewers were then trained on the response plan and how to properly respond if concerns about a youth participant’s welfare arose. To continue training, each interviewer spent time practicing with the protocols by serving as interviewer, respondent, and observer. Interviewer training concluded by a review of the response protocol and review of documentation procedures. Interviewers also met several times throughout the data collection to review, discuss, and reiterate the processes.
Interview Procedures. Interviews took approximately 60 minutes to complete. All interviews were conducted in English (as will the full Baltimore D-VACS study). Our goal through the recruitment and completion of these interviews was saturation of information around ability, comfort, and ease of responding to these types of questions, an assessment of comprehension of the questions, participation in the study, and identifying key obstacles or opportunities we can apply to the full study. As with the full study we anticipate many of the questions will be of a highly sensitive nature. We will prompt participants to share their process in interpreting and coming to an answer for these questions—but we will not seek actual answers during these cognitive interviews.
Interviews were all conducted in private settings to be sure the interview could be conducted confidentially. Interviews were completed over Zoom with a NORC interviewer, or in person by a NORC interviewer in public settings in Baltimore (e.g., a partner school or youth program). In-person recruitment and collection was coordinated and hosted by the Baltimore City Health Department (BCHD). At the outset of the cognitive interviews, ongoing health concerns related to the COVID-19 Pandemic, we started with Zoom interviewing. As local, state, and federal public health guidance became less restrictive and given the limited success we had with Zoom only interviewing (n= 7 completed interviews at the outset when we only offered a Zoom option), we introduced some in-person interviewing (with face masks) into the study.
Prior to participating in the study, respondents were first informed of their rights and the study confidentiality. The interviewers also addressed any questions and concerns from the respondents. Each remote and in-person interview also included a detailed informed consent process for the youth, and for their parent/guardian for minor youth. Throughout the interview, the interviewer repeatedly reminded the participant to report only their comprehension of the question and to talk about their thought process, but not to answer the actual question. Youth were reminded that they could take a break or skip questions if they were uncomfortable.
When youth were recruited, there were three possible scheduling outcomes. The first was to schedule a remote Zoom interview for a future date (n=8). The remaining interviews were recruited in person, and either completed immediately via Zoom with a NORC interviewer (n=12), or immediately face-to-face with a NORC interviewer in Baltimore (n=10).
Analysis. The 30 cognitive interviews were analyzed using the transcripts of each cognitive interview and notes maintained by each interviewer. The team analyzed the transcripts and notes using Excel and MS Word to collate the question items so that comparisons could be made systematically across all respondents. We also used the collated results to develop item summaries and check patterns in the data. The item summaries were used to make recommendations about the questionnaire items to retain, remove or revise. In general, we were looking for broad agreement among the respondents to make a recommendation for a change to a D-VACS survey item. We also examined the results to identify specific cases in which respondents had trouble or comprehension issues in answering specific questions. We compared each respective respondent’s interpretation of specific question items to assess consistency and the degree of variation in interpretation among the respondents.
As noted above, the cognitive interviewing protocol included a representative set of questions from the D-VACS questionnaire that could be completed within one hour, as it would have been too onerous on respondents to review — and discuss with the NORC interviewer — every single item in the D-VACS questionnaire. Our analyses include a question-by-question review of all the items covered in cognitive interviewing protocol. We present the D-VACS item first, followed by our summary comments based on participant feedback. The cognitive investigation of each D-VACS item included three components: (1) the NORC interviewer’s impressions of the item performance, (2) a flag for sensitivity concerns, and (3) as appropriate, suggestions for changes to improve comprehension of the D-VACS item. We have taken a conservative approach, suggesting the fewest deviations from the original question as possible and only recommending changes we believe to be absolutely necessary for data quality. D-VACS item numbers in red are items we highly recommend changes to, while other suggestions would be beneficial but are less crucial as compared to the red labeled items.
Sensitivity. CDC indicated strong interest in the sensitivity of D-VACS items or topics, to be able to determine the best mode of collection for asking these questions. Topics are considered “sensitive” if the item wording is triggering to the respondent. For each item, we note in the Results tables below the sensitive or non-sensitive nature of the question using a 4-category classification system:
Not sensitive—these questions are largely viewed as not sensitive and could be asked and reported orally to an interviewer.
Minimally sensitive—these questions are largely able to be asked and reported to an interviewer orally, although certain youth (age, experiences) may find them triggering.
Moderately sensitive—these are sensitive questions that youth are likely to only answer through ACASI so that the interviewer or others in the building cannot hear or see their responses. Even under this method, they will need question-by-question reassurances of privacy and confidentiality explained to them simply and directly (e.g., reassurances that there is no way a teacher/parent/police could ever learn of their response to this question).
Highly sensitive—these questions were reported by many youth to be very sensitive to answer, especially to an interviewer., . These primarily encompass reporting weapons or gang activity in their communities and disclosing child abuse by a family member. These questions were originally planned to be interviewer administered.
Findings and Recommendations
Interviewer Training Suggestions. The sensitivity of questions was largely influenced by trust—whether the youth trusted interviewers to truly protect their information and responses, ensuring that their survey responses would never be made known to teachers, parents, or law enforcement. Concerns around trust stemming from the interviewer being non-judgmental were not explored. While earning trust is not something we can easily accomplish in this work, we can ‘borrow it’ from individuals and offices in Baltimore with whom these youth have a good relationship or opinion of. This can be accomplished in recruitment materials, by hosting interviews in these neutral community settings, and training interviewers to distinguish themselves from these trusted organizations and individuals while also highlighting their partnership. Many youth, when probed on why they were participating in the cognitive interviews, reported it was because their referrer was a trusted source, and if that trusted source vetted us, the youth wouldn’t question it. We would also ask our partners at BCHD and/or the Baltimore Steering Committee with strong community relationships in Baltimore to join us at the interviewer training to help impress upon the interviewers the importance of trust issues for the respondents.
Additionally, interviewer training will need to emphasize reiteration of privacy and confidentiality regularly throughout the interview/survey administration. These repeated assurances will help to mitigate some of the challenges youth will have with reporting and can possibly help to convert those uncertain into completing the questions. However, we also confirmed from the cognitive interviews that some of the D-VACS items are quite sensitive and disclosure could be challenging. Thus, intensive interviewer training will be necessary to ensure interviewers do not push youth too much, recognize participant needs if they arise, and are able to quickly get help as needed. During the cognitive interviewing, there were two instances after the completion of the cognitive interview made known to us where a youth was upset after the interview but was able to quickly receive support from trusted adults located in the public setting in which we were interviewing.
General Observations. Overall, use of private reporting methods (e.g., self-administered iPad/web or ACASI) will be key—along with very direct and repeated assurances of complete privacy and confidentiality and reminding youth that names have not been collected and cannot be connected to their data. The youth were able to largely understand the D-VACS questionnaire but struggled with the weight of the topics and number of topics covered. They reported wanting to share positive information on their communities, schools, and healthy habits, not just negative behavior. Including protective factors and other positive questions (e.g., mentoring or being a good friend) within each topic may help to lessen the cognitive burden and burnout, while also making youth feel more comfortable talking about challenges. Also, despite the 60-minute length of the interview, the participants generally stayed engaged throughout the interview. This suggests that many of the youth for the main pilot study will be okay with the length of the interview if it takes less than one hour.
D-VACS Survey Item-by-Item Review.
For each question, participants were asked whether they understood the question and if they could repeat the question in their own words. The interviewer took notes on the participant's response and observations of non-verbal cues, such as length of time to respond. The notes for each question were analyzed across participants to identify comprehension issues.
The following item-by-item summary lists all D-VACS items covered in the cognitive interviewing protocol and reviews findings from the interviews. Observations and notes were analyzed item by item using key metrics of whether a specific item was correctly or incorrectly understood based on respondents’ assertion of understanding and ability to paraphrase correctly. If more than five respondents struggled on a particular item, it was flagged for additional consideration. Below, NORC only suggests modifications to an item if an entire group (e.g., all younger respondents) or half of all respondents struggled uniformly. When more individualized issues arose, we took note and when it could be applicable beyond the individual, included it in the notes for the relevant items below. NORCs recommendations are derived from suggestions made by the youth, paraphrases offered by youth, and an expert best practices review by NORC interviewers working on this project.
Findings for each question are summarized in the following sections:
Opening Questions
Instructions to Respondent: For these first few questions, I am asking for an answer. After that, we’re going to switch and I’m only going to ask you about your thoughts on the question.
Question |
Response Options |
How old are you? |
(open ended) |
What is the highest year of school/degree you have completed?
Suggested Modification: [19 or younger] What grade were you in last year?
[20 or older] Do you have a high school diploma or GED?
[IF YES] Are you currently, or have you ever taken college or technical school classes?
[IF YES] Do you have a technical, 2-year, or bachelor’s degree?
|
(open ended) |
What is your gender?
Suggested Modification: What words to you use to describe your gender identity? Male, female, non-binary, or do you use another term? |
Male Female Non-binary Something else |
Are you Hispanic or Latino? |
Yes No I don’t know |
What is your race?
Suggested Modification: Do you think of yourself as: Black/African-American, White, Asian, American Indian, or something else? |
American Indian or Alaskan Native Asian Black or African American Native Hawaiian or other Pacific Islander White Other Don’t know Declined |
Interviewer Observation and Suggestions: Overall, these questions performed adequately and are similar to US Census items, with a few notable challenges. Items covering gender, class year, and race all presented some difficulties for youth with more challenges (e.g., non-traditional educational paths, known mental health/trauma experiences etc.). A few quick fixes are possible including building response options into questions and offering more categories. However, none of these suggested edits are critical.
Sensitivity: Not sensitive (except gender—as noted in SO/GI sections below).
|
|
No. |
Question |
Response Options |
|
|||||
Q9 |
Are you deaf or do you have serious difficulty hearing?
|
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed well. No revisions suggested.
Sensitivity: Not sensitive
|
||||||||
Q10 |
Are you blind or do you have serious difficulty seeing, even when wearing glasses? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed well. No revisions suggested.
Sensitivity: Not sensitive. |
||||||||
Q11 |
Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions?
|
YES NO don’t know DECLINED
|
1 2 -98 -99
|
|
||||
Interviewer Observation and Suggestions: This question was challenging due to the complexity of measuring these forms of difficulties but the item is consistent with other nationally representative surveys such as BRFSS.
Sensitivity: Minimally sensitive.
|
||||||||
Q12
|
Do you have serious difficulty walking or climbing stairs?
|
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed well, but with some challenges in uniform understanding.
Sensitivity: Not sensitive.
|
||||||||
Q13 |
Do you have difficulty dressing or bathing?
|
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed well, but with some challenges in uniform understanding.
Sensitivity: Not sensitive.
|
||||||||
Q14 |
Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a doctor’s office or shopping? (15 years old or older)
|
YES NO don’t know DECLINED
|
1 2 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question was challenging due to the complexity of measuring this construct. The doctor’s office is not an age-appropriate example, as minor youth reported being unable to go to the doctor alone simply because they were minors.
Sensitivity: Minimally sensitive.
|
||||||||
Q16
|
What is the highest grade or year of school you completed?
|
1. Elementary (grades 1-4)
2. Middle School/JUNIOR HIGH (GRADES 5-8)
3 high school (Grades 9-11)
4. High school graduate (Grade 12 or ged)
DON’T KNOW...... DECLINED....
|
-98 -99 |
|
||||
Q18 |
(For 12th grade and lower from Q17) When you were in {Q17-1 grade level} grade, how many times did you skip a full day of school (count only non-excused absences)?
|
0 DAYS.............................. 1-5 DAYS ............................... 6-10 DAYS ............... 10 OR MORE DAYS DON’T KNOW .......... DECLINED .............. |
0 1 2 3 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed adequately, but with some challenges in uniform understanding—particularly for youth without traditional educational experiences and given that challenges of schooling last year. No youth could accurately estimate their missed ‘Zoom school days’, for example. In-person school was easily estimated by the majority and is expected to perform adequately moving forward.
Sensitivity: Minimally sensitive, will need assurances that parents and teachers will not be made aware of responses.
|
||||||||
Q1506 |
In the last 30 days, would you say that in general your health has been excellent, very good, good, fair, or poor?
|
EXCEllent very good good fair poor Don’t know declined |
1 2 3 4 5 -98 -99 |
|
||||
Interviewer Observation and Suggestions: This question performed well. Youth made the distinction between mental and physical health.
Sensitivity: Not sensitive.
|
||||||||
Instructions to Respondent: We’re going to continue asking you to think through some more questions about yourself. Remember, I don’t want answers to the questions from you, but for you to think about how you’d think about your answer for them, what you think they’re asking, and anything that seems confusing or we could be clearer about.
No. |
Question |
Response Options |
|
|
Q5
|
Which of the following best represents how you think of yourself? Lesbian or gay? Straight, that is, not lesbian or gay? Bisexual? Or something else? |
don’t know.............................................. declined..................................................... |
-97 -98 -99 |
|
Interviewer Observation and Suggestions: This question performed well, and youth were able to understand it. One minor suggestion is to add don’t know/questioning response options.
Sensitivity: Moderately sensitive. Many youth reported that this could be difficult to report as youth are still figuring out their identities, but also because of fear of parents/classmates finding out if they are not in safe spaces. This needs to be asked privately.
|
||||
Q6
|
What sex were you assigned at birth, on your original birth certificate? |
Male Female (Suggestion) I don’t know what this question is asking (Suggestion) I don’t know how to answer this question/I’m still figuring it out don’t know declined |
1 2 -98 -99
|
|
Interviewer Observation and Suggestions: This question performed well, and youth were able to understand it. One minor suggestion is to add don’t know/questioning response options.
Sensitivity: Moderately sensitive. This needs to be asked privately.
|
||||
Q7 |
Do you currently describe yourself as male, female or transgender?
Suggested Modification (similar to above): What words to you use to describe your gender identity? Male, female, non-binary, or do you use another term?
|
1. Male 2. Female 3. I AM TRANSGENDER AND IDENTIFY AS A BOY OR MAN 4. I AM TRANSGENDER AND IDENTIFY AS A GIRL OR WOMAN 5. I AM TRANSGENDER AND IDENTIFY IN SOME OTHER WAY (Suggestion) I don’t know what this question is asking (Suggestion) I don’t know how to answer this question/I’m still figuring it out 6. NONE OF THESE DON’T KNOW DECLINED |
-98 -99 |
|
Interviewer Observation and Suggestions: See above suggestion plus add don’t know/questioning response options. Youth reported the diversity of identities their peers may report, and the need to have space to use terms that are inclusive is key.
Sensitivity: Moderately sensitive. Many youth reported that this could be difficult to report as youth are still figuring out their identities, but also because of fear of parents/classmates finding out if they are not in safe spaces. This needs to be asked privately using ACASI.
|
||||
Q8 |
Just to confirm, you were assigned [FILL TEXT] at birth and now you describe yourself as [FILL TEXT], is that correct? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
Interviewer Observation and Suggestions: This question seems unnecessary, and youth didn’t understand the confirmation. Some youth who did understand this confirmation question reported that it could come off as offensive and repetitive.
Sensitivity: Moderately sensitive. This needs to be asked privately. |
||||
SEXUAL BEHAVIOR
Instructions to respondent: The next questions are about sexual activity. By sex we mean vaginal, oral, or anal sex or the insertion of hands, fingers, or other objects into your vagina or anus by someone else. Again, please don’t answer these questions, but think about how you would think through your answers and tell me about any difficulty you have in understanding what they’re asking, or the words they use.
No. |
Question |
Response Options |
|||||||||||||||||||||||||||||
|
Q400
|
Have you ever had sex? |
YES NO Don’t know declined................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Q401 |
During your life, with whom have you had sexual contact, males, females, or both? |
MALES....................................................................... FEMALES.................................................................. BOTH MALES and females.............................. don’t know.......................................................... declined................................................................ |
1 2 3 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: These are difficult questions to ask, and the definition is difficult for the youth to hear. That said, nearly all understood what it meant. However, when read together or quickly, it is likely to be missed. Two key suggestions for the definition: (1) include terms like ‘private parts’ or ‘intimate experience’ for youth that are not familiar with anatomy yet (this came up multiple times); and (2) separate the definition to be read as individual acts so respondents are sure to hear that all three (vaginal, oral, anal) are meant to be included in their response.
Sensitivity: Moderately sensitive. This needs to be asked privately, especially for younger respondents who might worry their parents are able to hear them or friends could find out. |
|
|||||||||||||||||||||||||||||
|
Q403
|
The first time you had sex, would you say that you wanted to have sex, you did not want to have sex but it happened anyway, or were you forced to have sex?
|
wanted to............................................................. DID NOT WANT TO BUT IT HAPPENED ANYWAY.................................................................. forced to ................................................ Don’t know declined.................................................................
I wanted to do it ............................................................. i didn’t want to do it............................................ I was forced to do it ................................................ Don’t know declined................................................................. |
2 3 -98 -99 |
Q405
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question, and nearly all in this section, were challenging for the youth to fully understand. Key suggestions provided by the youth were to speak more plainly and directly.
Sensitivity: Moderately sensitive. It is good that this question is asked in ACASI as this question will work best if asked privately, especially for younger respondents who might worry their parents are able to hear them or friends could find out. |
|
|||||||||||||||||||||||||||||
|
Q404 |
The first time you had sex, were you verbally persuaded, pressured (through harassment or threats), too drunk or drugged to say no, or physically forced?
|
VERBALLY PERSUADED ... pressured/THREATENED too drunk to say no PHYSICALLY FORCED OTHER....................................................................... don’t know declined
I wanted to do it……… I was VERBALLY PERSUADED ... I was pressured/THREATENED I was too drunk to say no I was PHYSICALLY FORCED OTHER....................................................................... don’t know declined |
1 2 3 4 97 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question, and nearly all in this section, were challenging for the youth to fully understand. Key suggestions provided by the youth were to speak more plainly and directly.
Sensitivity: Moderately sensitive. This needs to be asked privately, especially for younger respondents who might worry their parents are able to hear them or friends could find out. |
|
|||||||||||||||||||||||||||||
|
Q410 |
Have you had sex within the past 12 months? |
YES NO DON’T KNOW DECLINED |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: Most youth skipped this question; they couldn’t figure out how to talk through the answer without answering it. Those that did, said they couldn’t remember 12 months back, but three months was a commonly acceptable length of time.
Sensitivity: Moderately sensitive. This needs to be asked privately, especially for younger respondents who might worry their parents are able to hear them or friends could find out. |
|
|||||||||||||||||||||||||||||
|
Q415 |
Has a current or ex-partner ever tried to get you pregnant when you did not want to become pregnant (F) / tried to get pregnant when you did not want them to get pregnant (M), or tried to stop you from using birth control? |
YES............................................................................. NO............................................................................... DON’T KNOW........................................................... DECLINED................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was not asked frequently, given the limited ability for youth to think through some of these difficult questions. When it was, it was reportedly too complex.
Sensitivity: Moderately sensitive. This needs to be asked privately, especially for younger respondents who might worry their parents are able to hear them or friends could find out. |
|
|||||||||||||||||||||||||||||
|
[INTERVIEWER]: The next questions are about sexual things that might have taken place on the internet, on social media apps, through email, or through text messaging.
|
|
|||||||||||||||||||||||||||||
|
Q1400 |
In the past 12 months did you have access to the internet, social media apps, email, or text messaging?
|
|
|
|||||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1401 |
In the past 12 months did you talk about sexual acts with someone on the internet, on social media, through email or through text messaging because you were pressured to do so against your will?
|
YES NO DON’T KNOW DECLINED |
2 98 99 |
Q1403 |
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward.
Sensitivity: Not sensitive. Youth reported these sorts of events factually and as a frequent occurrence. It was only sensitive if it wasn’t something youth felt they could deal with (and all seem to feel like they could deal with it or get help as needed). |
|
|||||||||||||||||||||||||||||
|
Q1402 |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER............................................................................... SOMEONE ELSE...................................................................... DON’T KNOW DECLINED |
1 2 3 97 98 99 |
|
|
||||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood but not likely to be reported accurately.
Sensitivity: Highly sensitive, if it happened. Reporting on others was a key theme throughout the cognitive interviews. Youth were willing to report their activities and experiences but were not willing to implicate others in most situations. |
|
|||||||||||||||||||||||||||||
|
Q1403 |
In the past 12 months did you send a photo or video showing your private parts on the internet, on social media, through email or through text messaging because you were pressured to do so against your will? |
YES NO DON’T KNOW DECLINED................................................................................ |
1
98 99 |
Q1405
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was understood and reported, although again youth suggested making it more straightforward. The youth had some trouble distinguishing this item compared to Q1401.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1404 |
What was this person’s relationship to you? |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER.............................................................................. SOMEONE ELSE..................................................................... DON’T KNOW DECLINED................................................................................. |
1 2 3 97 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood but not likely to be reported accurately.
Sensitivity: Highly sensitive, if it happened. Reporting on others was a key theme throughout the cognitive interviews. Youth were willing to report their activities and experiences but were not willing to implicate others in most situations. |
|
|||||||||||||||||||||||||||||
|
Q1405 |
In the past 12 months, did you do anything else sexual on the internet, on social media, through email or through text messaging because you were pressured to do so against your will? |
YES NO DON’T KNOW DECLINED................................................................................. |
2 98 99 |
Q1407
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1406 |
What was this person’s relationship to you? |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER............................................................................... SOMEONE ELSE...................................................................... DON’T KNOW DECLINED................................................................................ |
1 2 3 97 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood but not likely to be reported accurately.
Sensitivity: Highly sensitive, if it happened. Reporting on others was a key theme throughout the cognitive interviews. Youth were willing to report their activities and experiences but were not willing to implicate others in most situations. |
|
|||||||||||||||||||||||||||||
|
Q1407 |
In the past 12 months did you pressure a partner or ex-partner against their will to talk about sexual acts on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1408 |
In the past 12 months did you pressure anyone [else] against their will to talk about sexual acts on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1409 |
In the past 12 months did you pressure a partner or ex-partner against their will to send you a photo or video showing their private parts on the internet, on social media, through email or through text message?
|
YES NO DON’T KNOW DECLINED................................................................................ |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1410 |
In the past 12 months did you pressure anyone [else] against their will to send you a photo or video showing their private parts on the internet, on social media, through email or through text message?
|
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1411 |
In the past 12 months did you pressure a partner or ex-partner against their will to do anything else sexual on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1412 |
In the past 12 months did you pressure anyone [else] against their will to do anything else sexual on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99
|
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: This question was easily understood and reported, although again youth suggested making it more straightforward. There was no distinction between Q1401 and this question.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
|
Q1413 |
Did you tell anyone about any of these unwanted sexual experiences online? |
YES NO (suggestion: i didn’t need help) (Suggestion: No, I didn’t know how to get help) DON’t KNOW DECLINED................................................................................. |
1 2 98 99 |
Q1500 |
|
|||||||||||||||||||||||||
|
Q1414 |
Whom did you tell? SELECT ALL THAT APPLY
|
CURRENT OR EX DATING PARTNER RELATIVE FRIEND OR NEIGHBOR SERVICE PROVIDER (like a doctor or counselor) AUTHORITY FIGURE (like a teacher or police officer) oTHER DON’T KNOW DECLINED |
1 2 3
4
5 97 98 99 |
|
|
|||||||||||||||||||||||||
|
Interviewer Observation and Suggestions: These questions were easily understood and reported. However, the vast majority reported that they wouldn’t think to tell anyone because they can handle it themselves. So a ‘no’ to Q1413 might be “no—I didn’t need help” or “no—I didn’t know how to get help”, two very different outcomes for analysis. A suggestion to the response options might resolve this.
Sensitivity: Not sensitive. |
|
|||||||||||||||||||||||||||||
1
WEAPONS
Instructions to Respondent: Now I am going to ask questions about weapons. For these questions, please do not include times you carried a gun or knife only for hunting or for a sport, such as target shooting. Remember you can skip questions if you don’t want to answer, and I don’t want your actual answer to any of these questions, but just to understand how you would think about and come to your answer if you were taking this survey on your own.
No. |
Question |
Response Options |
|
|
|
Q53 |
Have you ever carried a weapon such as a gun, knife, or club? |
YES No DON’T KNOW DECLINED |
1 2 -98 -99 |
|
|
Q54 |
What type of weapon have you carried? (Select all that apply) |
GUN KNIFE Club OTHER WEAPON DON’T KNOW DECLINED |
1 2 3 -97 -98 -99 |
|
|
Q55 |
During the past 12 months, how often did you carry a weapon such as a gun, knife, or club? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
1 2 3 4 5 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview with this type of question. Some were able to proxy report “Nearly everyone carries a weapon, most people carry a weapon, half of people carry a weapon, most don’t carry a weapon, nobody carries a weapon”, but not directly. For these questions, and the items that follow, proxy reports in addition to or replacement of these core questions is an option to consider.
Sensitivity: Highly sensitive. |
|||||
Q56 |
What is the main reason that you carried a weapon such as a gun, knife, or club, during the past 12 months? |
FOR PROTECTION AGAINST ATTACKS BY OTHER PEOPLE... BECAUSE MY FRIENDS CARRY WEAPONS................................ BECAUSE IT MAKES ME FEEL IMPORTANT............................... TO “SHOW OFF” AND IMPRESS MY FRIENDS............................ BECAUSE I WANTED TO HURT SOMEONE................................. OTHER DON’T KNOW...... DECLINED....... |
1 2 3 4 5 -97 -98 -99 |
|
|
Q57 |
During the past 12 months, did you use a weapon to threaten or injure another person? |
YES............... NO.................. DON’T KNOW............. DECLINED........... |
1 2 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report “Nearly everyone carries a weapon, most people carry a weapon, half of people carry a weapon, most don’t carry a weapon, nobody carries a weapon”, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions are suggested.
Sensitivity: Highly sensitive. |
|||||
Q58 |
During the past 12 months, how many times did you use a weapon to threaten or injure another person? |
0 TIMES.................... 1 TIMES..................... 2 – 4 TIMES................ 5 OR MORE TIMES.... DON’T KNOW............. DECLINED................... |
0 1 2 3 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report “Nearly everyone carries a weapon, most people carry a weapon, half of people carry a weapon, most don’t carry a weapon, nobody carries a weapon”, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions are suggested.
Sensitivity: Highly sensitive. |
|||||
Q59 |
[INTERVIEWER]:
Now I would like to specifically ask you about guns. For these questions, do not count any times when you may have carried a gun only for hunting or for a sport, such as target shooting.
During the past 12 months, how often did you carry a gun onto school property? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
2 3 4 5 -98 -99 |
Q62 |
|
Q60 |
During the past 12 months, how often did you carry a gun in your neighborhood? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
2 3 4 5 -98 -99 |
Q62 |
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report “Nearly everyone carries a weapon, most people carry a weapon, half of people carry a weapon, most don’t carry a weapon, nobody carries a weapon”, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions are suggested.
Of note, carrying a firearm in school was reported less likely but more difficult to report, whereas ‘everyone knows everyone is doing it in neighborhoods’ was a common theme, but reporting on neighborhood activity was viewed as dangerous. Reporting on either truthfully was reported to be impossible or highly unlikely.
Sensitivity: Highly sensitive. |
|||||
Q61 |
Where did you get the gun(s) that you carried during the past 12 months? SELECT ALL THAT APPLY
|
1.PARENT OR OTHER FAMILY MEMBER’S GUN TAKEN FROM HOME 2. I PURCHASED FROM A STORE THAT SELLS GUNS 3. ADULT PURCHASED FOR ME 4. PURCHASED “ON THE STREET” 5. BORROWED FROM A FRIEND OR SOMEONE I KNOW 6. STOLEN 7. PURCHASED FROM A GUN SHOW DON’T KNOW DECLINED |
-98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
Sensitivity: Highly sensitive. |
|||||
Q62 |
[INTERVIEWER]: (For 13-17) How long would it take you to get and be ready to fire a loaded gun without a parent’s or adult’s permission? This gun could be yours or someone else’s and it could be located anywhere.
(For 18-24) How long would it take you to get and be ready to fire a loaded gun? This gun could be yours or someone else’s and it could be located anywhere. |
1.I COULD NOT GET A LOADED GUN LESS THAN 10 MINUTES 2. 10 OR MORE MINUTES, BUT LESS THAN 1 HOUR 3. 1 OR MORE HOURS, BUT LESS THAN 4 HOURS 4. 4 HOURS OR MORE, BUT LESS THAN 24 HOURS 5. 24 HOURS OR MORE don’t know declined |
-98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
Sensitivity: Highly sensitive. |
|||||
Q63 |
Now I have a few questions about gangs.
Is there gang activity in your school? |
YES NO DON’T KNOW DECLINED.. |
1 2 -98 -99 |
|
|
Q64 |
Is there gang activity in your neighborhood? |
YES NO DON’T KNOW DECLINED |
1 2 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
Sensitivity: Highly sensitive. |
|||||
Q65 |
During the past 12 months, how many times did you feel threatened as a result of gang activity? |
0 TIMES 1 TIME 2 – 4 TIMES 5 OR MORE TIMES DON’T KNOW DECLINED |
0 1 2 3 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
Sensitivity: Highly sensitive. |
|||||
Q66 |
During the past 12 months, have you ever been a member of a gang? |
YES NO DON’T KNOW DECLINED |
1 2 -98 -99 |
|
|
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no reliable way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
Sensitivity: Highly sensitive. |
|||||
1
1
WITNESSING VIOLENCE
Physical Violence and Emotional Violence
Q124
|
pV3: PARENTS, adult caregivers AND OTHER ADULT RELATIVES
The next set of questions are about parents, adult caregivers, or other adult relatives. This means only adults related to you, responsible for your care or that you have lived with. We will ask about other adults in the next section, so please do not include them here. Remember, you can skip any question you do not want to answer.
|
|||||||||||||||||||||||||||||
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth reported there was no easy way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
A key challenge with these questions is that they call for reports of ‘family business’ as we are directly asking them to report family violence. A common theme among youth was a willingness to report things that happened to them, but not that happened to/by someone else—particularly a family member.
These questions, particularly witnessing violence and being a secondary victim, are highly triggering and a very common experience (many youth said “all youth have experienced this”, community partners estimate it to be 80% or more). Having immediate access to resources and having these interviews conducted, or available to be conducted, in safe public spaces and using privacy enhancing methods such as ACASI may help. Additionally, we will need to ‘borrow’ the trust of community leaders and organizations, as these youth have all reported experiences of mistrust and specifically mistrust of outsiders.
Sensitivity: Highly sensitive. |
||||||||||||||||||||||||||||||
Q304 |
What was this person’s relationship to you? |
Parent ... stepparent OTHER adult RELATIVE oR CAREGIVER….. don’t know DECLINED |
1 2 -97 -98 -99 |
|
||||||||||||||||||||||||||
Interviewer Observation and Suggestions: These questions were highly sensitive. Youth understood the question but reported there was no easy way to get a truthful answer during a survey or interview. Some were able to proxy report, but not directly. For these questions, and the ones that follow, proxy reports in addition to or replacement of these core questions should be considered.
A key challenge with these questions is that they call for reports of ‘family business’ as we are directly asking them to report family violence. A common theme among youth was a willingness to report things that happened to them, but not that happened to/by someone else—particularly a family member.
These questions, particularly witnessing violence and being a secondary victim, were reported to be highly sensitive and a very common experience (many youth said “all youth have experienced this”, community partners estimate it to be 80% or more). Having immediate access to resources and having these interviews conducted, or available to be conducted, in safe public spaces and using privacy enhancing methods such as ACASI may help. Additionally, we will need to ‘borrow’ the trust of community leaders and organizations, as these youth have all reported experiences of mistrust and specifically mistrust of outsiders.
Sensitivity: Highly sensitive. |
||||||||||||||||||||||||||||||
Q319
|
Note these topics, and as they come up during internet/social media conversations (e.g., bullying).
EV3: PEER EMOTIONAL VIOLENCE The next questions are about people your own age not including a current or ex-partner. This could be a schoolmate, a neighbor, or a stranger. It could have happened in person or virtually through things like text messages, instant/direct messages, calls, videos, emails, social media, apps, online gaming etc.
|
|||||||||||||||||||||||||||||
Q328 |
Did this happen in person or virtually through things like text messages, instant/direct messages, calls, videos, emails, social media, apps, etc.? |
in person…...…..... virtually................................... DON’t KNOW……………………... DECLINED……………............... |
1 2 -98 -99 |
|
||||||||||||||||||||||||||
Interviewer Observation and Suggestions: The peer emotional violence questions were easier for the respondents to answer. The location question was easily interpreted and responded to, although the core part of the questioning was not. These less invasive questions may perform better.
Sensitivity: Minimally sensitive. |
||||||||||||||||||||||||||||||
SERVICES
Q150
|
Did you know of a place to go for help when you had been hurt by anyone we just asked about, including parents, other adults, dating partners, people about your own age, or others in the community?
|
YES NO DON’t KNOW DECLINED |
1 2 -98 -99 |
|
Interviewer Observation and Suggestions: Youth nearly uniformly reported not knowing, and others like them not knowing, where to go to get help.
|
||||
Conclusion
NORC’s cognitive testing suggests that a survey on child abuse, violence and related constructs can be done with youth 13-24 years old in Baltimore. However, the truthfulness of the reports on these issues is a real concern, as in similar studies fielded in the US. There will always be underreporting when surveying on such topics due to some of the stigma surrounding these topics. However, we can reduce this underreporting by making sure the respondents understand the questions. Our cognitive test results suggest that most respondents should have few problems with comprehension and willingness to respond provided we make a few changes.. Sensitive questions, as noted above, should be moved to the ACASI portion of the survey. The script should emphasize the private and confidential nature of the survey. Finally, for those respondents that understand the questions but are hesitant to report on violent behavior, we can address this issue by trying to create a comfortable and safe place for the interviewing with trusted adults nearby (who might not be their parents or guardians but other trusted adults in their community).
Overall, youth expressed considerable concern around trust and confidentiality around many questions, but also had good comprehension for most of the D-VACS questions. Many expressed concern around question complexity, , and can also be addressed with interviewer training to slow the reading of questions and allow ample processing time.
Additional key considerations:
Delivery of incentives should be in cash or via physical gift cards. Visa gift cards were a big challenge for most youth because they were not familiar with how to use them and reported preferring services like CashApp over traditional gift cards (which cannot be transferred to the service). Several youth who expressed interest in participating in-person were dissuaded by the incentive process. It also added great burden in the moment to manage, and as instant incentives are key to continuing trust, considering more simplistic solutions is strongly recommended.
Chronological and developmental age were often incongruent in these interviews, with older youth (18+) struggling with common concepts such as race or education levels. Being mindful of a lower literacy rate coupled with a different baseline for 'common concepts’ will assist interviewers in collecting the highest quality data during the pilot.
Trust is a key challenge in this population, particularly as institutions and other entities have broken trust frequently in the past with Baltimore youth, according to youth reports. However certain offices or individuals have earned the trust of these youth. We can ‘borrow’ this trust through partnerships with community leaders on recruitment of youth to the study, conducting interviews in common/public spaces or with trusted adults nearby, and working with these individuals and organization for simultaneous recruitment and conducting of interviews. For example, we could have after-school sessions where NORC has a team of interviewers ready to go, where the key part of recruitment is done in advance to get them there, but any final recruitment or consenting, and conducting, happened in that community setting. If we mailed postcards for example, and then used community partners to help reinforce any recruitment efforts, we could then have invite-only sessions in these more public spaces. This approach can help with recruitment and trust, but also aid the team with converting refusals among youth who are not comfortable completing a survey at home.
Youth seemed to like the idea of the ACASI interviewing mode and felt it would at least somewhat help protect their privacy. Youth were able to articulate key situations in which all youth would benefit from privacy, such as where engaging in a behavior or having an identity may result in difficulties with peers or family (e.g., gender identity/sexual orientation, skipping school).
The cognitive testing process also revealed the importance of keeping a flexible interview schedule that could accommodate the needs of a variety of different situations for Baltimore youth (e.g., children of parents and caregivers who are working multiple jobs, busy after school schedules, etc.).
The topics covered by the D-VACS survey are challenging and there are many questions for youth respondents to consider. During the cognitive interviews, youth frequently inserted positive experiences into the discussion (e.g., what they do to stay healthy; the ability of maintaining healthy relationships) which could be examples to include in the final instrument. The full survey does include questions on protective factors. .
Attachments
Attachment A: Protocol
D-VACS Cognitive Testing Protocol & Interviewer Guide
MATERIALS NEEDED FOR INTERVIEW
INTERVIEWER PROTOCOL BOOKLET (THIS BOOKLET)
CONSENT FORM (PARENT/CHILD; PARTICIPANT)
Respondent phone number, in case reminder is needed.
$50 GIFT CARD CODE, PREPARED TO DELIVER VIA EMAIL OR IN PERSON
ZOOM LOGIN AND TROUBLESHOOTING INFO
All recordings saved to NORC computer and moved to PDrive SensitiveData.
ATTACHMENT L: RESPONSE PLAN USE IF YOU OBSERVE CONCERN
RESOURCES LIST; SEND UPON COMPLETION
FAQs (In this document)
NOTE PAPER, PENS AND PENCILS
TO-DO: 30 minutes prior: Send Zoom link & reminder
Background, Approach, & Purpose: For interviewer context only
The purpose is to test portions of the current D-VACS instrument for intended comprehension of key questions, new questions, and to better understand any potential data collection concerns around recruitment and privacy. The findings from these cognitive interviews will allow CDC to field the full pilot study more confidently in the City of Baltimore in 2022 and use this information to plan for expanded fielding of D-VACS in the United States, as has occurred in other countries.
A series of 30 cognitive interviews will be conducted to test comprehension and acceptability of the questions to inform revisions to the existing instrument. The cognitive interviews will enable the research team to identify problems with items, the organization and order of the instrument, revealing potential sources of response error in the D-VACS that the research team will draw on to inform any re-design efforts.
The cognitive interviews will assess issues such as:
Respondent understanding of terms in the survey,
Respondent confidence in responses
Recall techniques
When answering factual questions, such as when an injury occurred and what happened, we will remind respondents that we do not want actual answers to the survey items, but rather the thought process respondents would use to recall such events. In these instances, we would ask the participant to think about an injury or event that might have occurred in the past. As part of our instructions to the participant, we would introduce this recall pattern as “helpful to us to understand how you might recall a situation like that to answer questions on the survey…specifically, would you first think about WHEN it happened first, or WHERE it happened, or WHO the person was and if they were alone or other people were there?”. We do not want the WHAT. For example, if we asked “What did you have for lunch yesterday?” we would want the participant to think about when lunchtime was, where they were, who else was there, and how they would think of ‘lunch’. But we do not want them to actually tell us what they had for lunch.
In a more related example, if we ask them about experiences with violence, we do not want them to report demonstrating or being victimized by violence—rather the WHEN, WHERE, WHO, and definitions they would think about to help them answer the question. We do not what the WHAT happened. If any immediate harm is reported, or there is distress, refer to the flow chart below and Attachment L.
KEY INFO: Response Plan Flow Chart
KEY INFO Frequently Asked Questions
*Read only if asked/necessary
What is the study about?
This study is being conducted on behalf of Centers for Disease Control and Prevention (CDC) to try to learn how many youth and young people have experienced health concerns and protecting their health.
Who is Centers for Disease Control and Prevention (CDC)?
The Centers for Disease Control and Prevention (CDC) is the national public health agency of the United States. It is a United States federal agency, under the Department of Health and Human Services. CDC works 24/7 to protect America from health, safety and security threats, both foreign and in the U.S. Whether diseases start at home or abroad, are chronic or acute, curable or preventable, human error or deliberate attack, CDC fights disease and supports communities and citizens to do the same.
Who is NORC?
NORC is a not-for-profit social science research organization affiliated with the University of Chicago. NORC is conducting this study on behalf of Centers for Disease Control and Prevention (CDC). You can learn more about NORC at its website, www.norc.org, or by contacting the Study Director, Bruce Taylor 301-634-9512.
Do I have to participate?
No. You do not have to participate. If you do choose to participate, you can choose not to answer or stop participating at any time.
How much will I be paid?
You will be given $50 for participating in the interview.
How long will the study take?
The interview will take less than 60 minutes.
Why should I participate?
Your input on how the questionnaire is working will help improve the data CDC collects. By participating in this study you can help make sure that the CDC collects the most complete and accurate data possible on youth health in Baltimore and in the United States, information that is used to inform policies designed to keep youth healthy.
Who do I contact if I have questions about my rights as a respondent?
If you have any questions regarding your rights as a study participant, you may call the NORC IRB Manager, toll-free, at 866-309-0542.
How is my privacy protected?
Your answers will always be kept private, and none of the information that you provide will be used for any purpose other than research. Your name or any information that could identify you will never be used.
STEP 1: INTRODUCTION & CONSENT
Step 1: Introduction and Agreement to Participate, Part A & B (Attachment F.1)
PART A: Interviewer: READ THIS INFORMATION FORM TO THE RESPONDENT AND OBTAIN CONSENT.
Hello, my name is ______________. I am a survey interviewer from NORC at the University of Chicago for a research study supported by the Baltimore City Health Department in partnership with the U.S. Centers for Disease Control and Prevention. We are conducting interviews with young people to improve the way information is collected in surveys.
We are only interviewing youth in your area who are between 13 and 24 years old. We are doing this interview with you to help us improve the way we ask young people about their health, educational, and life experiences. This study is intended to suggest ways to improve the procedures the CDC uses to collect survey data.
I want to assure you that we do not plan to share your information or your answers with anyone. All of the answers will be kept strictly confidential, and your name and address will not be connected with your survey responses. You have the right to stop the interview at any time, or to skip any questions that you don’t want to answer. There are no ‘Right’ or ‘Wrong’ answers. Your participation is completely voluntary, but your experiences could be very helpful to understand how to improve the way we collect information in surveys. As a token of appreciation for your time, you will receive a $50 gift card after completing the interview.
Do you have any questions about what will happen to the information that you share?
The interview takes about 60 minutes to complete.
INTERVIEWER: I HAVE CONFIRMED THAT RESPONDENT UNDERSTANDS THEIR ANSWERS ARE CONFIDENTIAL. INTERVIEWER INITIALS______
Do you have any other questions before I tell you a bit more about the survey, and our conversation tonight?
Interviewer: Answer any quetsions, and confrim continuation.
May i continue?
NO Does not agree to DISCUSS Survey FURTHER. (THANK RESPONDENT FOR THEIR TIME AND END)
Yes Continue. agrees to DISCUSS Survey FURTHER.
It is very important that we find a place to talk where others cannot hear our conversation. This is common practice in surveys to protect your information and provides you with a level of comfort while you answer my questions. It also ensures that you have the ability to ask me any questions that you may have.
Are you in a good place to talk during the interview or is there somewhere else that you would like to go?
No Wait until they relocate.
YesàContinue.
PART
B: Informed Assent/Consent
Interviewer: This information will be read to the respondent as the second component to consent.
.
Thank you for participating in this interview. There are a few things you should know before agreeing to participate:
You get to decide if you want to participate and whatever you decide is OK. It is also OK to say ‘Yes’, start the interview, and change your mind later. You have the right to stop the interview at any time or decline to respond to any questions you do not want to answer. The interview will take about 60 minutes.
We are doing this study to learn how to improve the ways we collect information using surveys. The survey asks a lot of different types of questions. I will not ask you to answer the questions – what we want to know is how well you think people your age would understand the questions. What do you think the question is asking? Are there any words, terms, or phrases that are confusing to you?
The information you provide is confidential. Your survey responses will never be connected to your name or any other personal information.
During this research you may be audio and/or videotaped, or you may be observed. The recording allows us to more carefully study and improve the questions. If you agree to have the interview recorded, you may still ask to stop the recording at any time, and we will turn it off. If you decide to stop recording, we will ask your consent to retain the portion already recorded. When the interview is finished, you may [watch/listen to] the recording.
If you agree to record the interview, we will keep it in a locked room either in a secure storage cabinet or on a password-secured computer that is not connected to the internet. Only researchers from NORC and CDC working on the project will be allowed to [watch/listen to] the recording in a secured room. When in use all recordings will be in the safe keeping of a staff person from NORC.
You might be wondering whether bad things could happen to you or your family if you take this survey.
The chance that bad things would happen as a result of taking this survey is really low. As I said, some of the questions are sensitive and responding to them may be difficult or uncomfortable, but you can choose not to respond to certain questions or to end your participation at any time. And remember, we are not asking you to answer the questions but rather to tell us whether you understand the questions.
You might also be wondering if good things could happen to you or your family if you participate.
There are no direct benefits for participating. However, as a thank you and in appreciation of your time, you will receive a $50 gift card. You may also benefit by learning more about different programs available for young people in your area.
There are a few more things I need to tell you due to the content of the survey and the responses you may provide.
The only people who know the questions asked during the survey are the young people selected to take the survey and the people who work with me on the study. No one else, including the other people in your home or community, knows the exact questions on the survey so you can feel safe in responding honestly.
Your participation is completely voluntary. You may decline to answer any questions you wish and you are free to take a break or end the survey at any time. If you do end the survey early or do not answer certain questions you will still receive the $50 gift card.
At the end of the survey, I will offer to connect you with the project’s social worker. You will get to decide if you want to be connected with her. I will never give her any of your information without your permission.
Some questions may be upsetting, difficult, or uncomfortable, so you will also have the option of being connected to the social worker at any time during the survey.
At the end of the survey, you will also receive the Youth Resources Handout and QR code. It has contact information for organizations that you can use if you want to talk about any feelings or emotions you experienced. It also contains information on a range of free programs and services that you may find helpful.
You should also be aware that a Certificate of Confidentiality has been obtained from the Federal Government for this study to further help ensure your privacy. This Certificate means that the researchers cannot be forced to disclose information that may identify you, even by a court subpoena, in any federal, state, or local civil, criminal, administrative, legislative, or other proceedings. But, if you request disclosure, we can release the information. The Certificate of Confidentiality does not prevent the researchers from disclosing, without your consent, information that would identify you as a respondent in the research project if during the interview you reveal intent to hurt yourself or someone else.
Also, if you voluntarily share information with me directly at any point during our time together that makes it known that you are in immediate danger OR if you tell me that you may hurt yourself or someone else, OR if you share that someone is hurting you now or in the past, I will have to tell a social worker so you can get help and may have to file a report.
Do you have any questions at this time?
You have the right to contact people working on this project with any questions, complaints or concerns about the survey or your participation, either:
Bruce Taylor at NORC at the University of Chicago at 301-351-7167.
Here is this contact information if you would like to call at any point. Please know that if you decide at any point after completing this survey that you would like your data destroyed and removed from analysis, we will honor that request.
I am going to read a final statement. When I am done, please respond either “Yes” if you agree to participate or “No” if you do not agree to participate.
You understand the purpose of the survey, that participating is your choice and no one will be upset if you do not want to participate or if you change your mind, and that your responses entered in the tablet will be confidential. Do you agree to participate in the survey?
YES, THE RESPONDENT AGREES TO PARTICIPATE IN THE SURVEY
NO, THE RESPONDENT DOES NOT AGREE TO PARTICIPATE IN THE SURVEY
IF YES [DO NOT READ TO RESPONDENT]: Interviewer Verbal Consent Certification
In completing the certification information below and conducting the survey, you verify that you have read the informed consent to the respondent, answered any questions to the best of your ability and in-line with your training, and that all project protocols will be followed.
Interviewer Name _____________________________________________
Interviewer ID _____________________________________________
Date _____________________________________________
IF NO: Thank you for taking the time to hear more about the survey. I will not contact you again regarding the survey or for any other reason. You are welcome to have a Youth Resource Handout, I will email it to you.
Interviewer Starting Checklist:
☐ CONSENT/PARENT PERMISSION AND ASSENT COMPLEATED
☐ IF THE RESPONDENT(S) HAS AGREED TO RECORDING, START THE RECORDER.
ONCE THE RECORDING IS STARTED, SAY: “Now that the recording is started, I just need your confirmation once more. Do you agree to participate and have this interview recorded?”
AND CONTINUE WITH RECORDING WITH RECORDED PERMISSION.
STEP TWO: Cognitive Interview
Interviewer: If respondent is struggling with how to process the questions without answering them, it might be helpful to read the question (and as applicable response options) followed by “What do you think this question is asking?”.
INTRODUCTION--INTERVIEWER READ ALOUD:
Interviewer: Paraphrase as needed, based on information and discussion in introduction/consent and any sharing of FAQs.
This interview is about improving the ways we ask youth about their health, and how youth experience their health and stay healthy. Before health surveys are conducted, the questions are tested with people of different backgrounds. It is important that the questions make sense, are easy to answer, and that everyone understands the questions the same way. I will read you the survey questions. Then, I will ask you to explain what you think the questions are asking about and how you or someone like you would come up with your answers.
As I mentioned before, I don’t want you to answer the questions, but rather think about how you would answer them if you were taking this survey on your own. What do you think the question is asking? Are there any words, terms, or phrases that are confusing to you?
This type of study will teach us about the different kinds of problems people have answering survey questions. The study will help us write better questions in the future.
Sometimes I’ll ask you a question and ask you to think about it, and tell me what you think it’s asking about. Other times, I’ll have you listen to questions that only you can hear, and I’ll ask you to tell me what you think the question is asking, and whether it would be better asked by me, or privately so only you can hear. I would be happy to answer any questions you may have about the survey. [ANSWER RESPONDENT QUESTIONS] Let’s begin.
I will read each question to you, and I’d like you to tell me what you are thinking as you figure out how to answer. Also, please tell me if:
there are words you don’t understand,
the question doesn’t make sense to you,
you could interpret it more than one way,
it seems out of order,
or if the answer you are looking for is not provided.
The more you can tell us, the more useful it will be to us as we try to develop better questions.
INTERVIEWER NOTES AND PROBE BANK:
The discussion will be qualitative in nature to include a variety of probes to understand respondent comprehension, question clarity, wording, etc.
Follow the interview protocol, and ask emergent probes to better understand the question-response process. Use the Interviewer Probe Bank below.
How would you come up with that answer?
Can you tell me in your own words what you think the question is asking?
Can you tell me more about that?
Is this question easy or hard to answer? Why?
When thinking about answering questions like this, is it easier for you to do it with audio prompts like this, me asking you directly, or does it not matter?
If you pick up on a visual cue that suggests an issue or confusion:
Tell me what you are thinking.
What does the word [term] in this question mean to you?
You said [answer]. Can you tell me more about that?
If respondent is uncertain and asking for confirmation:
There is not a right or wrong answer for this question. I am interested in hearing your thoughts on what the question is asking, and not how you’d answer it—but how you’d think about it, and what you think it’s asking.
You may use some or all of these probes, depending upon the content of the interview and how much information the respondent reveals without being prompted. In some interviews, probes will be administered throughout the interview, and for others they will be administered after completing the questionnaire. Proceed through the respondent questionnaire, probing for information on comprehension, clarity, and other respondent reactions to the questions.
Opening Questions
Interviewer Note: Each question is highlighted in green, yellow, or red. Green questions are ’must ask’, yellow are largely dependent on answers to green questions, and red questions should be asked as appropriate. Overall, we would ideally ask all questions, but should not exceed about 75 minutes (we promised 60 minutes) inclusive of the 10-15 minute consent process.
Open with demographics: For these first few questions, I am asking for an answer. After that, we’re going to switch and I’m only going to ask you about your thoughts on the question.
How old are you?
What is the highest year of school/degree you have completed?
What is your gender?
Male
Female
Non-binary
Something else
Are you Hispanic or Latino?
What is your race?
Interviewer A bit of a lead in or warm up question might help respondents to start thinking about the process of thinking about questions rather than the answers. Here’s one to consider using as an example: Now we’re going to turn to some more questions. While I asked your real answers on these first few, let’s think about that last question for a moment, like we’ll think of the rest of these questions. If I had asked “what is your race?”, but didn’t want the answer, I would ask “what do you think this question is asking?” or “how do you think other youth your age would think about this question”. How might you answer these questions? What do you think this question “What is your race?” is asking?
Q9 |
Are you deaf or do you have serious difficulty hearing?
|
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank:
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q10 |
Are you blind or do you have serious difficulty seeing, even when wearing glasses? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q11 |
Because of a physical, mental, or emotional condition, do you have serious difficulty concentrating, remembering, or making decisions? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q12 |
Do you have serious difficulty walking or climbing stairs? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q13 |
Do you have difficulty dressing or bathing? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q14 |
Because of a physical, mental, or emotional condition, do you have difficulty doing errands alone such as visiting a doctor’s office or shopping? (15 years old or older) |
YES NO don’t know DECLINED |
1 2 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q16
|
What is the highest grade or year of school you completed?
*See recruitment/scheduling info for answer. Confirm conversationally. |
1. Elementary (grades 1-4)
2. Middle School/JUNIOR HIGH (GRADES 5-8)
3. some high school (Grades 9-11)
4. High school graduate (Grade 12 or ged)
DON’T KNOW...... DECLINED.... |
-98 -99 |
|
|||
Q18 |
(For 12th grade and lower from Q17) When you were in {Q17-1 grade level} grade, how many times did you skip a full day of school (count only non-excused absences)? |
0 DAYS.............................. 1-5 DAYS ............................... 6-10 DAYS ............... 10 OR MORE DAYS DON’T KNOW .......... DECLINED .............. |
0 1 2 3 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
Q1506 |
In the last 30 days, would you say that in general your health has been excellent, very good, good, fair, or poor? |
EXCEllent very good good fair poor Don’t know declined |
1 2 3 4 5 -98 -99 |
|
|||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||
[INTERVIEWER]: We’re going to continue asking you to think through some more questions about yourself. Remember, I don’t want answers to the questions from you, but for you to think about how you’d think about your answer for them, what you think they’re asking, and anything that seems confusing or we could be clearer about.
Q5
|
Which of the following best represents how you think of yourself? Lesbian or gay? Straight, that is, not lesbian or gay? Bisexual? Or something else? |
don’t know.............................................. declined..................................................... |
-97 -98 -99 |
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||
Q6
|
What sex were you assigned at birth, on your original birth certificate? |
Male Female don’t know declined |
1 2 -98 -99
|
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||
Q7 |
Do you currently describe yourself as male, female or transgender? |
1. Male 2. Female 3. I AM TRANSGENDER AND IDENTIFY AS A BOY OR MAN 4. I AM TRANSGENDER AND IDENTIFY AS A GIRL OR WOMAN 5. I AM TRANSGENDER AND IDENTIFY IN SOME OTHER WAY 6. NONE OF THESE DON’T KNOW DECLINED |
-98 -99 |
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||
Q8 |
Just to confirm, you were assigned [FILL TEXT] at birth and now you describe yourself as [FILL TEXT], is that correct? |
YES NO don’t know DECLINED |
1 2 -98 -99 |
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||
SEXUAL BEHAVIOR
*These questions ask about sexual behavior and seek to test whether these questions/topics would be viewed as sensitive by the average participant.
[INTRODUCTION]: The next questions are about sexual activity. By sex we mean vaginal, oral, or anal sex or the insertion of hands, fingers, or other objects into your vagina or anus by someone else. Again, please don’t answer these questions, but think about how you would think through your answers and tell me about any difficulty you have in understanding what they’re asking, or the words they use.
Q400
|
Have you ever had sex? |
YES NO Don’t know declined................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Q401 |
During your life, with whom have you had sexual contact, males, females, or both? |
MALES....................................................................... FEMALES.................................................................. BOTH MALES and females.............................. don’t know.......................................................... declined................................................................ |
1 2 3 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q403
|
The first time you had sex, would you say that you wanted to have sex, you did not want to have sex but it happened anyway, or were you forced to have sex? |
wanted to............................................................. DID NOT WANT TO BUT IT HAPPENED ANYWAY.................................................................. forced to ................................................ Don’t know declined................................................................. |
2 3 -98 -99 |
Q405
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q404 |
The first time you had sex, were you verbally persuaded, pressured (through harassment or threats), too drunk or drugged to say no, or physically forced? |
VERBALLY PERSUADED ... pressured/THREATENED too drunk to say no PHYSICALLY FORCED OTHER....................................................................... don’t know declined |
1 2 3 4 97 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q410 |
Have you had sex within the past 12 months? |
YES NO DON’T KNOW DECLINED |
1 2 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q415 |
Has a current or ex-partner ever tried to get you pregnant when you did not want to become pregnant (F) / tried to get pregnant when you did not want them to get pregnant (M), or tried to stop you from using birth control? |
YES............................................................................. NO............................................................................... DON’T KNOW........................................................... DECLINED................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
[INTERVIEWER]: The next questions are about sexual things that might have taken place on the internet, on social media apps, through email, or through text messaging.
|
|||||||||||||||||||||||||||||
Q1400 |
In the past 12 months did you have access to the internet, social media apps, email, or text messaging?
|
|
|||||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1401 |
In the past 12 months did you talk about sexual acts with someone on the internet, on social media, through email or through text messaging because you were pressured to do so against your will? |
YES NO DON’T KNOW DECLINED |
2 98 99 |
Q1403 |
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1402 |
What was this person’s relationship to you? |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER............................................................................... SOMEONE ELSE...................................................................... DON’T KNOW DECLINED |
1 2 3 97 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1403 |
In the past 12 months did you send a photo or video showing your private parts on the internet, on social media, through email or through text messaging because you were pressured to do so against your will? |
YES NO DON’T KNOW DECLINED................................................................................ |
1
98 99 |
Q1405
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1404 |
What was this person’s relationship to you? |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER.............................................................................. SOMEONE ELSE..................................................................... DON’T KNOW DECLINED................................................................................. |
1 2 3 97 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1405 |
In the past 12 months, did you do anything else sexual on the internet, on social media, through email or through text messaging because you were pressured to do so against your will? |
YES NO DON’T KNOW DECLINED................................................................................. |
2 98 99 |
Q1407
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1406 |
What was this person’s relationship to you? |
CURRENT OR EX DATING PARTNER................................. FRIEND OR CLASSMATE...................................................... STRANGER............................................................................... SOMEONE ELSE...................................................................... DON’T KNOW DECLINED................................................................................ |
1 2 3 97 98 99 |
|
|||||||||||||||||||||||||
Q1407 |
In the past 12 months did you pressure a partner or ex-partner against their will to talk about sexual acts on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1408 |
In the past 12 months did you pressure anyone [else] against their will to talk about sexual acts on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Q1409 |
In the past 12 months did you pressure a partner or ex-partner against their will to send you a photo or video showing their private parts on the internet, on social media, through email or through text message?
|
YES NO DON’T KNOW DECLINED................................................................................ |
1 2 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
Q1410 |
In the past 12 months did you pressure anyone [else] against their will to send you a photo or video showing their private parts on the internet, on social media, through email or through text message?
|
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Q1411 |
In the past 12 months did you pressure a partner or ex-partner against their will to do anything else sexual on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99 |
|
|||||||||||||||||||||||||
Q1412 |
In the past 12 months did you pressure anyone [else] against their will to do anything else sexual on the internet, on social media, through email or through text message? |
YES NO DON’T KNOW DECLINED................................................................................. |
1 2 98 99
|
|
|||||||||||||||||||||||||
Q1413 |
Did you tell anyone about any of these unwanted sexual experiences online? |
YES NO DON’t KNOW DECLINED................................................................................. |
1 2 98 99 |
Q1500 |
|||||||||||||||||||||||||
Q1414 |
Whom did you tell? SELECT ALL THAT APPLY
|
CURRENT OR EX DATING PARTNER RELATIVE FRIEND OR NEIGHBOR SERVICE PROVIDER (like a doctor or counselor) AUTHORITY FIGURE (like a teacher or police officer) oTHER DON’T KNOW DECLINED |
1 2 3
4
5 97 98 99 |
|
|||||||||||||||||||||||||
Interviewer probe bank:
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
|||||||||||||||||||||||||||||
WEAPONS
*This section focuses on experiences possessing and using weapons. We are not seeking experiences but interpretation of questions around experiences with weapons.
[INTERVIEWER]: Now I am going to ask questions about weapons. For these questions, please do not include times you carried a gun or knife only for hunting or for a sport, such as target shooting. Remember you can skip questions if you don’t want to answer, and I don’t want your actual answer to any of these questions, but just to understand how you would think about and come to your answer if you were taking this survey on your own.
Q53 |
Have you ever carried a weapon such as a gun, knife, or club? |
YES No DON’T KNOW DECLINED |
1 2 -98 -99 |
Q54 Q62
|
Q54 |
What type of weapon have you carried? (Select all that apply) |
GUN KNIFE Club OTHER WEAPON DON’T KNOW DECLINED |
1 2 3 -97 -98 -99 |
|
Q55 |
During the past 12 months, how often did you carry a weapon such as a gun, knife, or club? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
1 2 3 4 5 -98 -99 |
Q62 |
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q56 |
What is the main reason that you carried a weapon such as a gun, knife, or club, during the past 12 months? |
FOR PROTECTION AGAINST ATTACKS BY OTHER PEOPLE... BECAUSE MY FRIENDS CARRY WEAPONS................................ BECAUSE IT MAKES ME FEEL IMPORTANT............................... TO “SHOW OFF” AND IMPRESS MY FRIENDS............................ BECAUSE I WANTED TO HURT SOMEONE................................. OTHER DON’T KNOW...... DECLINED....... |
1 2 3 4 5 -97 -98 -99 |
|
Q57 |
During the past 12 months, did you use a weapon to threaten or injure another person? |
YES............... NO.................. DON’T KNOW............. DECLINED........... |
1 2 -98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q58 |
During the past 12 months, how many times did you use a weapon to threaten or injure another person? |
0 TIMES.................... 1 TIMES..................... 2 – 4 TIMES................ 5 OR MORE TIMES.... DON’T KNOW............. DECLINED................... |
0 1 2 3 -98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q59 |
[INTERVIEWER]:
Now I would like to specifically ask you about guns For these questions, do not count any times when you may have carried a gun only for hunting or for a sport, such as target shooting
During the past 12 months, how often did you carry a gun onto school property? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
2 3 4 5 -98 -99 |
Q62 |
Q60 |
During the past 12 months, how often did you carry a gun in your neighborhood? |
NEVER RARELY SOMETIMES MOST OF THE TIME ALL OF THE TIME DON’T KNOW DECLINED |
2 3 4 5 -98 -99 |
Q62 |
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q61 |
Where did you get the gun(s) that you carried during the past 12 months? SELECT ALL THAT APPLY
|
1.PARENT OR OTHER FAMILY MEMBER’S GUN TAKEN FROM HOME 2. I PURCHASED FROM A STORE THAT SELLS GUNS 3. ADULT PURCHASED FOR ME 4. PURCHASED “ON THE STREET” 5. BORROWED FROM A FRIEND OR SOMEONE I KNOW 6. STOLEN 7. PURCHASED FROM A GUN SHOW DON’T KNOW DECLINED |
-98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q62 |
[INTERVIEWER]: (For 13-17) How long would it take you to get and be ready to fire a loaded gun without a parent’s or adult’s permission? This gun could be yours or someone else’s and it could be located anywhere.
(For 18-24) How long would it take you to get and be ready to fire a loaded gun? This gun could be yours or someone else’s and it could be located anywhere. |
1.I COULD NOT GET A LOADED GUN LESS THAN 10 MINUTES 2. 10 OR MORE MINUTES, BUT LESS THAN 1 HOUR 3. 1 OR MORE HOURS, BUT LESS THAN 4 HOURS 4. 4 HOURS OR MORE, BUT LESS THAN 24 HOURS 5. 24 HOURS OR MORE don’t know declined |
-98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q63 |
Now I have a few questions about gangs.
Is there gang activity in your school? |
YES NO DON’T KNOW DECLINED.. |
1 2 -98 -99 |
|
Q64 |
Is there gang activity in your neighborhood? |
YES NO DON’T KNOW DECLINED |
1 2 -98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q65 |
During the past 12 months, how many times did you feel threatened as a result of gang activity? |
0 TIMES 1 TIME 2 – 4 TIMES 5 OR MORE TIMES DON’T KNOW DECLINED |
0 1 2 3 -98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Q66 |
During the past 12 months, have you ever been a member of a gang? |
YES NO DON’T KNOW DECLINED |
1 2 -98 -99 |
|
WITNESSING VIOLENCE
Q69
|
WITNESSING PHYSICAL VIOLENCE: The following questions are about witnessing physical violence by strangers or people you know well in your home or neighborhood.
For participants 13-17: How many times in your life have you seen or heard your parent or guardian being hit, punched, kicked, or beaten by her or his husband, wife, or dating partner? Would you say: never, once, or more than once?
For participants 18-24: Before the age of 18:
How many times did you see or hear your parent or guardian being hit, punched, kicked, or beaten by her or his husband, wife, or dating partner? Would you say: never, once, or more than once?
|
NEVER................................................................................................. ONCE................................................................................................... MORE THAN ONCE.......................................................................... DON’T KNOW..................................................................................... DECLINED........................................................................................... |
2 3
|
Q100
Q100 |
||||||||||||||||||||||||
Q71
|
How many times did you see or hear a parent or guardian or their husband, wife, or dating partner punch, kick, or beat your brothers or sisters? Never, once, more than once, or I have no brothers or sisters? |
Never Once more than once .......................................................................... I HAVE NO BROTHERS OR SISTERS .............................................. DON’T KNOW ..................................................................................... DECLINED ........................................................................................... |
2
4 -98 -99 |
Q73
Q73
|
||||||||||||||||||||||||
Q73
|
Outside of your home, in real life, how many times did you see anyone: a. physically attacked or beaten? b. stabbed with a knife or sharp weapon? c. shot with gun, BB gun or air rifle?
Would you say never, once, or more than once? |
Never Once more than once DON’T KNOW DECLINED
|
2
-98 -99 |
Q100
Q100 |
Physical Violence and Emotional Violence
Q124
|
pV3: PARENTS, adult caregivers AND OTHER ADULT RELATIVES
The next set of questions are about parents, adult caregivers, or other adult relatives. This means only adults related to you, responsible for your care or that you have lived with. We will ask about other adults in the next section, so please do not include them here. Remember, you can skip any question you do not want to answer.
|
|||||||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||||||||||||||||||||||||||||
Q304 |
What was this person’s relationship to you? |
Parent ... stepparent OTHER adult RELATIVE oR CAREGIVER….. don’t know DECLINED |
1 2 -97 -98 -99 |
|
||||||||||||||||||||||||||
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||||||||||||||||||||||||||||
Q319
|
Note these topics, and as they come up during internet/social media conversations (e.g., bullying).
EV3: PEER EMOTIONAL VIOLENCE The next questions are about people your own age not including a current or ex-partner. This could be a schoolmate, a neighbor, or a stranger. It could have happened in person or virtually through things like text messages, instant/direct messages, calls, videos, emails, social media, apps, online gaming etc.
|
|||||||||||||||||||||||||||||
Q328 |
Did this happen in person or virtually through things like text messages, instant/direct messages, calls, videos, emails, social media, apps, etc.? |
in person….....................….......….......…............. virtually.............................................................. DON’t KNOW……………………………………... DECLINED……..…………………………............... |
1 2 -98 -99 |
|
||||||||||||||||||||||||||
Interviewer probe bank (required):
|
||||||||||||||||||||||||||||||
SERVICES
Q150
|
Did you know of a place to go for help when you had been hurt by anyone we just asked about, including parents, other adults, dating partners, people about your own age, or others in the community? |
YES NO DON’t KNOW DECLINED |
1 2 -98 -99 |
|
Interviewer probe bank (use as appropriate):
If you pick up on a visual cue that suggests an issue or confusion:
|
||||
Debriefing questions:
Post-Interview Debriefing: INTERVIEWER ASK AS APPROPRIATE & TIME ALLOWS—try to ask 1-2 questions from each of the three categories below.
Recall and reference period
Sensitivity
General debriefing
What questions are the most and least difficult for Rs to answer? What answers are they uncertain about? Do they have suggestions for improving the survey?
Return to any questions that need further exploration.
|
||||
Thank you for taking part in this survey.
STOP THE RECORDER.
Thank them again.
ANSWER ANY QUESTIONS THEY MAY HAVE.
Have them type their email address in the chat box, so you can copy it into the email with their $50.
Within 24 hours, add your notes to the summary sheet, organized by respondent.
This concludes the interview. I would be happy to answer any questions that you have. Thank you for your help with this study. When we hang up, I’ll email you your gift card. Can you please type your email address into the chat, just to make sure I have the right one?
If you have any issues or concerns, my email will have my name, email, and phone number at the bottom. Don’t hesitate to reach out to me or the project director, Bruce Taylor, at [email protected].
SEND INCENTIVE CODE AND RESOURCE DOCUMENT VIA EMAIL IN 5-10 MINUTES.
MATERIALS TO TAKE AWAY FROM INTERVIEW
| File Type | application/vnd.openxmlformats-officedocument.wordprocessingml.document |
| Author | Windows User |
| File Modified | 0000-00-00 |
| File Created | 2022-08-12 |
© 2024 OMB.report | Privacy Policy