Part C Item Justification

At multiple points in the history of the School Survey on Crime and Safety (SSOCS), the survey items have been examined for the quality of both their content and data, and, when necessary, the questionnaire has been adjusted. In order to maintain consistent benchmarks over time, few changes have been made to the questionnaire over the most recent survey iterations. For SSOCS:2016, some items were revised, based on the results of the SSOCS:2010 collection, to clarify their meaning. Additionally, a few items were removed (based on historically low response rates) and several items were added (to reflect emerging issues). Information on specific editorial changes, content modifications, additions, and deletions is included in the following section.

Presented below is a complete description of the sections and the corresponding items in the SSOCS:2016 questionnaire (see Appendix B for the questionnaire). The SSOCS:2018 questionnaire and procedures are expected to be the same as in SSOCS:2016. The SSOCS:2016 questionnaire consists of the following sections:

School practices and programs;
Parent and community involvement at school;
School security staff;
School mental health services;
Staff training;
Limitations on crime prevention;
Frequency of crime and violence at school;
Number of incidents;
Disciplinary problems and actions; and
School characteristics: 2015–16 school year.

1.1 School Practices and Programs

This section collects data pertaining to the nature of current school policies and programs relating to crime and discipline. These data are important in helping schools know where they stand in relation to other schools and in helping policymakers know what actions are already being taken in schools and what actions schools might be encouraged to take in the future. These data can also benefit researchers interested in evaluating the success of school policies. Although SSOCS is not designed as an evaluation, the presence of school policies can be correlated with the rates of crime provided elsewhere in the questionnaire, with appropriate controls for school characteristics.

Question 1 specifically asks about the various school policies and practices that are in place, including those that restrict access to school grounds, monitor student behavior to prevent crime, impact the school’s ability to recognize an outsider, and enable communication in the event of a school-wide emergency. These policies and practices are important because they influence the control that administrators have over the school environment as well as the potential for students to bring weapons or drugs onto school grounds. Such actions can directly affect crime because students may be more reluctant to engage in inappropriate activities for fear of being caught. The school climate may also be affected because students may feel more secure knowing that violators of school policies are likely to be caught.

Question 2 asks about the existence of written plans for dealing with various crisis scenarios, and Question 3 asks whether schools drill students on the use of specific emergency procedures. When emergencies occur, there may not be time or an appropriate environment for making critical decisions, and key school leaders may not be available to immediately provide guidance. Thus, having a written plan for crises and drilling students on emergency procedures are important in preparing schools to deal with crises effectively.

Question 4 is a general question designed to provide an initial measure of the type of preventative programs that schools have in place. The presence of such programs is a sign that schools are being proactive by seeking to prevent violence before it occurs rather than reacting to it.

Questions 5 and 6 ask whether schools have threat assessment teams, and, if so, how often the threat assessment team meets. Threat assessment teams are an emerging practice in schools to identify and interrupt students who may be on a path to violent behavior.

Question 7 asks about the presence of recognized student groups that promote inclusiveness and acceptance in schools. The presence of these groups is important in creating a positive environment where students are respectful of different backgrounds and may reduce conflict and violence.

1.2 Parent and Community Involvement at School

This section asks about the involvement of parents and community groups in the school. Parent and community involvement in schools can affect the school culture and may impact the level of crime in a school.

Questions 8 and 9 ask about formal policies implemented to involve parents in school programs and the percentage of parents participating in specific events.

Question 10 asks if specific community agencies are involved in promoting a safe school environment to determine the extent to which the school involves outside groups.

1.3 School Security Staff

Questions 11 through 18 ask about the use and activities of sworn law enforcement on the school grounds or at school events, and Question 19 asks about the presence of additional security personnel. In addition to directly affecting school crime, the use of security staff can also affect the school environment. Security staff may help prevent illegal actions, reduce the amount of crime, and contribute to feelings of security or freedom on school grounds. Thus, the times that law enforcement personnel are present, their visibility, their roles and responsibilities, and their carrying of weapons are all important.

1.4 School Mental Health Services

Question 20 asks whether diagnostic assessments and treatment for mental health disorders were available to students under the official responsibilities of a licensed mental health professional and funded by the school or district. Assessing the presence of mental health services in schools can demonstrate how well equipped schools are to deal with students with mental disorders, which may influence the frequency and severity of delinquency and behavioral problems within the school.

Question 21 asks for principals’ perceptions of the factors that limit their schools’ efforts to provide mental health services to students. The question asks about limiting factors, such as inadequate access to licensed mental health professionals, inadequate funding, lack of parental and community support, and the legal responsibilities of the school. Schools that face issues relating to inadequate resources or support may have limited effectiveness in providing mental health services to students. Schools’ financial obligation to pay for mental health services may also make them reluctant to identify students who require these services.

1.5 Staff Training

Question 22 asks about training provided by schools or districts for classroom teachers or aides, including classroom management; school-wide policies and practices related to violence; bullying and cyberbullying; alcohol and/or drug use; and safety procedures. Other types of training include recognizing potentially violent students; recognizing signs of substance abuse; intervention and referral strategies for student displaying signs of mental health disorders; recognizing physical, social, and verbal bullying; positive behavioral intervention strategies, and crisis prevention and intervention.

Schools can now obtain early warning signs to identify such potentially violent students, and their use of such profiles may affect both general levels of discipline and the potential for crises (such as multiple shootings). The type of training provided to teachers is important because teachers collectively spend the most time with students and observe them closely. Moreover, there is evidence in recent research that a substantial discrepancy exists in the percentage of schools that have these types of policies and the percentage of teachers that are trained in them. Collecting these data on teacher training will inform efforts to combat violence and discipline problems in schools.

1.6 Limitations on Crime Prevention

This section asks for principals’ perceptions of the factors that limit their schools’ efforts to reduce or prevent crime. Question 23 asks about limiting factors, such as lack of training for teachers, lack of support from parents or teachers, inadequate funding, and federal, state, or district policies on disciplining students. Although principals are not trained evaluators, they are the people who are the most knowledgeable about the situations at their schools and whether their own actions have been constrained by the factors listed.

Schools that face issues relating to inadequate resources or support may have limited effectiveness in responding to disciplinary issues and reducing or preventing crime. Identifying principals’ perceptions of the factors that limit their ability to prevent crime in school can inform efforts to minimize obstructions to schools’ crime prevention measures.

1.7 Frequency of Crime and Violence at School

This section asks about violent deaths, specifically homicides and shootings at school (Questions 24 and 25). Violent deaths get substantial attention by the media but are actually relatively rare, and there is evidence that, in general, schools are much safer than students’ neighboring communities. Based on analyses of such previous SSOCS data, these crimes are such rare events that the National Center for Education Statistics (NCES) is unable to report estimates per its statistical standards. Nonetheless, it is important to include these items in the questionnaire as they are significant incidents of crime that, at the very least, independent researchers can evaluate. Furthermore, the survey represents a comprehensive picture of the types of violence that can occur in schools, and the omission of violent deaths and shootings would be questioned by respondents who may have experienced such violence.

1.8 Number of Incidents

The questions in this section ask about the frequency of various kinds of crime and disruptions at school (other than violent deaths). Question 26 specifically asks principals to provide counts of the number of incidents that occurred at school and the number of crimes that were reported to the police or other law enforcement. Question 26 will assist in identifying which crimes in schools are being underreported to the police and will provide justification for further investigation as to why this is the case. Question 27 asks about the number of arrests that occurred at school, and Questions 28 and 29 ask about the frequency of hate crimes and the biases that may have motivated these hate crimes. Questions 30 and 31 are designed to gather data on the number of unplanned disruptions that occurred during the school year. The data gained from this section can be used directly as an indicator of the degree of safety in U.S. public schools and indirectly to rank schools in terms of the number of problems they face.

1.9 Disciplinary Problems and Actions

There is evidence that schools’ ability to control crime is affected by their control of lesser violations, since lesser violations are an indication of the state of discipline in the school. When lesser violations are controlled, students do not progress to more serious disciplinary problems. This section asks about the degree to which schools face such disciplinary problems and how schools respond to them. The data gathered in this section will be helpful in confirming or denying the importance of schools’ control of lesser violations and provide another measure of the disciplinary situation in U.S. schools. The data may also be helpful in multivariate models of school crime by providing a way of grouping schools that are similar in their general disciplinary situation but different in their school policies or programs.

Question 32 asks principals to report, to the best of their knowledge, how often certain disciplinary problems occur at school. Problems of interest include student racial/ethnic tensions, bullying, sexual harassment, harassment based on sexual orientation, harassment based on gender identity, widespread disorder in classrooms, student disrespect of teachers, and gang activities. This question provides a general measure of the degree to which there are disciplinary problems at each school.

Question 33 asks about the frequency of three aspects of cyberbullying, providing a general measure of the degree to which cyberbullying is an issue for students and how often staff resources are used to deal with cyberbullying.

Question 34 asks what kinds of disciplinary actions were available to each school and whether they were actually used. The item is not intended to be comprehensive; instead, it focuses on some of the most important strategies. The data will help policymakers to know what options and what constraints principals face; for example, if an action is allowed in principle but not used in practice, then policymakers would need to act in a different way than if the action is not allowed.

Question 35 asks about the number of various types of offenses committed by students, and the resulting disciplinary actions taken by schools. Question 36 asks how many students were removed or transferred from school for disciplinary reasons. These items provide valuable information about how school policies are actually implemented (rather than simply what policies are in place), with a particular emphasis on how many different kinds of actions are taken with regard to a particular offense as well as how many times no actions are taken. For example, many schools claim to have zero-tolerance policies, but some schools have extremely strong policies, while other schools’ zero-tolerance policies allow so many options that there is little or no constraint on what disciplinary action is imposed.

1.10 School Characteristics: 2015–16 School Year

This section asks for a variety of information about the characteristics of the schools responding to the survey. The information provided in this section is necessary in order to be able to understand the degree to which different schools face different situations. For example, one school might have highly effective programs and policies, yet still have high crime rates because of the large number of disadvantaged students at the school; another school might appear to have effective policies based on its crime rates but actually have higher crime rates than similar schools.

Question 37 asks for the school’s total enrollment.

Question 38 requests information on the school’s student population, including the percentage of students receiving free or reduced-price lunches (a measure of poverty), with limited English proficiency (a measure of the cultural environment), in special education (a measure of the academic environment), and who are male (most crimes are committed by males, so the percentage who are male can affect the overall crime rate).

Question 39 addresses various levels of academic proficiency and interest, which are factors that have been associated with crime rates.

Question 40 asks for the number of classroom changes made in a typical day. This is important because it affects schools’ ability to control the student environment. When students are in hallways, there are more opportunities for problems. Also, a school with fewer classroom changes is likely to be more personal and to have closer relationships between the students and teachers.

Questions 41 and 42 specifically ask about the crime levels in the neighborhoods where students live and in the area where the school is located. This is an important distinction, since some students may travel a great distance to their school, and their home community may have a significantly different level of crime than their school community.

Question 43 asks for the school type. Schools that target particular groups of students (such as magnet schools) have more control over who is in the student body and may have better motivated students (because the students have chosen a particular program). Charter schools have more freedom than regular schools in their school policies, may have more control over who is admitted into the student body, and may have better motivated students (because the students chose the school).

Question 44 asks for the school’s average daily attendance. This is a measure of truancy and thus a measure of the level of disciplinary problems at the school. It also is a measure of the academic environment.

Question 45 asks for the number of transfers. When students transfer after the school year has started, schools have less control over whether and how the students are acculturated to the school. These students are likely to have less attachment to the school and to the other students, thus increasing the risk of disciplinary problems.

Questions 46 requests the start and end date of the school’s school year as well as the date that the principal completed the questionnaire. This question could be used to examine whether schools that respond to the survey before the school year is completed report fewer crimes than schools reporting for the entire year.

C2. Changes to the Questionnaire and Rationale: 2015–16 SSOCS

The following details the editorial changes, deletions, and additions made since the 2011–12 SSOCS instrument. The result is the proposed instrument for the 2015–16 SSOCS, which is located in Appendix B. For additional information on the rationales for item revisions, please see the findings from cognitive testing, which is located in Part C3.

1. Changes to Definitions

Several definitions have been added to the questionnaire to clarify the terms used in new survey items for SSOCS:2016. Additionally, six definitions have been added or modified to clarify terms already contained in the questionnaire (bullying, cyberbullying, gender identity, hate crime, rape, and sexual orientation were terms used on SSOCS 2010), and one definition has been removed as the corresponding survey item has been dropped for SSOCS:2016.

Active shooter – A formal definition has been provided to clarify the phrase “active shooter,” using the same language as the Department of Homeland Security’s (DHS) definition that is used in the Department of Education’s emergency recommendations.

Bullying – A formal definition for bullying has been added to the survey using language from the Centers for Disease Control (CDC) as the agency recently revised the definition.

Cult or extremist group – This definition has been removed from the questionnaire in accordance with the deletion of item 20, subitem i, in the 2011–12 questionnaire.

Cyberbullying – The definition for cyberbullying has been removed from the stem of item 30 and relocated to the definition section as additional questions now require this definition.

Diagnostic assessment – A formal definition for diagnostic assessment has been added to the survey in accordance with the addition of a new section on school mental health services.

Evacuation – A formal definition for evacuation has been added to the survey to clarify the emergency procedures discussed in item 3.

Gender identity – Per feedback received from advocacy groups during the 30-day public comment period, a formal definition for gender identity has been added to the survey to clarify the terms used in item 7, subitem a; item 29, subitem g; and item 32, subitem e.

Hate crime – This definition has been revised to match the Federal Bureau of Investigation (FBI)’s updated definition for a hate crime and, per feedback received from advocacy groups during the 30-day public comment period, to specifically include gender identity as a bias.

Lockdown – A formal definition for lockdown has been added to the survey to clarify the emergency procedures discussed in item 3.

Mental health disorder – A formal definition for mental health disorder has been added to the survey in accordance with the addition of a new section on school mental health services.

Mental health professional – A formal definition for mental health professionals has been added to the survey from the School Health Policies and Practices Survey (SHPPS), which is administered by the CDC.

Rape – The definition of rape has been modified per suggestions from our federal partners to note that rape includes sodomy and to instruct respondents to report attempted rapes with rapes in the questionnaire.

Restorative circle – A formal definition has been provided to clarify the processes and participants involved in a restorative circle.

Sexual assault – Per suggestions made from our federal partners, an editorial change was made to revise the definition of “sexual battery” to an updated definition of “sexual assault.” This definition mirrors the Office of Civil Rights’ definition in its key elements.

Sexual orientation – Per feedback received from advocacy groups during the 30-day public comment period, a formal definition for sexual orientation has been added to the survey to clarify the terms used in item 7, subitem a; item 29, subitem f; and item 32, subitem d.

Shelter-in-place – A formal definition for shelter-in-place has been added to the survey to clarify the emergency procedures discussed in item 3.

Threat assessment team – A formal definition for threat assessment team has been added to the survey in accordance with the addition of two new survey items that ask about formal groups of persons who meet to identify students who might be a potential risk for violent behavior.

Treatment – A formal definition for treatment has been added to the survey in accordance with the addition of a new section on school mental health services.

2.2 Editorial Changes

Throughout the questionnaire, the school year has been updated to reflect the most recent 2015–16 school year.

Item 1, subitem a. “Require visitors to sign or check in” has been changed to “Require visitors to sign or check in and wear badges.” This revision was based on this variable having shown little variance and having limited analytic use, as policies now generally require all schools to sign in visitors.

Item 1, subitem d. The phrase “pass through” has been removed from this item. This revision was based on this variable having shown little variance and having limited analytic use. The “pass through” language assumed checks using only stationary metal detectors and may not have captured schools that use handheld metal detectors on a daily basis.

Item 1, subitem x. This item has been modified to replace outdated examples of social media networking sites. As Myspace is no longer widely used, the examples have been updated to instead include Instagram and YouTube.

Item 2. The stem of this item has been rephrased to “Does your school have a written plan that describes procedures to be performed in the following scenarios?” This item is being retained to maintain trend from previous SSOCS surveys, but modified to incorporate the inclusion of item 3. As such, this question no longer asks about drilling students on written plans, as the drilling of students on emergency procedures is now captured in item 3.

Item 2, subitem a. Based on a recommendation from our federal partners, this item has been changed from “shootings” to “active shooter” as the term “active shooter” is now the language widely used by the government, law enforcement, media, and schools.

Item 4, subitem a. “Conflict resolution,” “anti-bullying,” and “dating violence prevention” have been added as examples to this item in a parenthetical notation.

Item 4, subitem b. This item has been modified to include a parenthetical notation that specifies that behavioral or behavior modification intervention for students can include positive reinforcement. The additional phrase will place an emphasis on schools rewarding good behavior as a positive intervention strategy to reduce bad behavior.

Item 26, subitem b. Per a suggestion made from our federal partners, an editorial change was made to revise “sexual battery” to “sexual assault.”

Item 33. As a new item about cyberbullying has been added to an earlier part of the questionnaire, the definition for cyberbullying has been removed from the stem of this item and relocated to the definition section of the questionnaire.

2.3 Item deletions and rationale

2011–12 Questionnaire, Item 1, subitem k. This item was deleted. This variable was shown to have little variance and seems of little analytic use.

2011–12 Questionnaire , Item 2, subitem g. This item was deleted as the DHS no longer uses a color-coded terrorism alert system, and our federal partners indicated that schools no longer widely use these plans.

2011–12 Questionnaire , Item 20, subitem i. This item was deleted. This variable was shown to have little variance and is an issue of declining relevancy in schools.

2.4 Content modifications, item additions, and rationale

Item 1, subitem f. This item was added as an indicator of whether teachers and staff members have the ability to stop visitors from physically entering a classroom. In January 2012, the DHS recommended that all classrooms have doors that can be locked from either side to prevent entry from the corridor side.¹ According to DHS reports, interior locks on classroom doors saved lives during the 1999 attack at Columbine High School in Colorado, but such locks were not available in classrooms in Norris Hall during the 2007 Virginia Tech campus shooting.

Item 1, subitem p. This addition was based on a review of literature that suggests that staff accessibility to panic buttons or silent alarms can reduce the response time of emergency personnel to arrive on site and can help to minimize damage.²

Item 2, subitem h. This item was added to determine the percentage of schools that have written plans for reuniting students with their families in the aftermath of a crisis. In the chaos of an emergency, it may be difficult for schools to coordinate the reunification of students with their families; this item will help to identify schools that have taken precautionary measures to address the aftermath of an emergency.

Item 3. This item was added to look at whether schools drill students on particular emergency procedures. While SSOCS already collects information on written plans for various crisis scenarios, there were no items that collected information on what procedures schools implemented based on the type of emergency. This addition was based on the literature and state-level legislation, both of which emphasize the need for schools to drill students on different types of procedures, depending on the emergency, such as by making provisions both for the evacuation and lockdown of classrooms and the school building.³

Item 4, subitem g. This item has been added to separately identify what percentage of schools have students involved in peer mediation as a form of addressing student conflict.

Item 4, subitem h. This item has been added to separately identify what percentage of schools have a formalized process to address student conduct problems or minor offenses.

Item 4, subitem i. This item has been added to collect information on student involvement in restorative justice circles. Restorative circles are a more formalized process and generally involve an adult facilitator (unlike student court), and their purpose is to restore relationships rather than adjudicate an incident.

Item 4, subitem j. This item has been added to identify the percentage of schools that have social emotional learning (SEL) training for students. The presence of SEL training in schools is important to measure as the literature indicates its use can boost academic performance by increasing student interest in learning; improve student behavior by preventing a variety of problems such as truancy, alcohol and drug use, bullying, and violence; and improve school climate overall.⁴ Additionally, “social skills training” has been moved from subitem a to subitem j as this example better aligns with SEL.

Item 5. This item has been added to assess the percentage of schools that have threat assessment teams to identify and evaluate at-risk students. Threat assessment teams have been recommended by the U.S. Secret Service, FBI, and Department of Education as a means to “interrupt” students who may be on the path to violent behavior.⁵

Item 6. Asked as a follow-up to item 5, this item has been added to obtain a measure of how frequently a school’s threat assessment team meets, as more active threat assessment teams may be better equipped to prevent or address school violence.

Item 7. As the Department of Education and outside organizations are interested in collecting information on the presence of Gay-Straight Alliances and other inclusion groups in schools, this item was added to look at specific groups present in schools that promote inclusiveness. These groups can help create a positive environment where students are respectful of different backgrounds and may help to reduce conflict and violence.

Item 11. This item has been modified to no longer collect data on security guards and security personnel; the revised item asks only about the presence of sworn law enforcement officers. Information on security personnel is now collected separately in item 19 as the roles and responsibilities of security personnel can vary widely across schools and are often not comparable to those of sworn law enforcement.

Item 12. This item has been modified to no longer collect data on security guards and security personnel; the revised item asks only about the presence of sworn law enforcement officers.

Item 13. This item has been modified to no longer collect data on security guards and security personnel; the revised item asks only about the presence of sworn law enforcement officers.

Item 13, sub-item d. At the request of federal partners in the Office of Safe and Healthy Students, this item has been added to collect data on whether sworn law enforcement officers routinely wear a body camera to record their interactions on the job.

Item 14. This item has been modified to no longer collect data on security guards and security personnel; the revised item asks only about the presence of sworn law enforcement officers.

Item 14, subitem a. Feedback from our federal partners indicated that this item would identify schools that have sworn law enforcement officers present on campus who participate in motor vehicle traffic control—either as their only responsibility or in addition to other activities identified in item 14.

Item 14, subitem i. Feedback from our TRP members stated that law enforcement involvement within schools may help schools keep better records and report discipline problems more accurately.

Item 14, subitem j. Feedback from our TRP members indicated that law enforcement expertise may help schools categorize and, therefore, report, incidents more accurately.

Item 15. This item has been added based on feedback from our federal partners who emphasized that it is important to know whether a sworn law enforcement officer is actually in the school building 5 days per week for the entire school day. It is possible that a school may report that they have a full-time sworn law enforcement officer assigned to their school, but the officer may spend a limited amount of time actually in the school building due to court appearances, sick leave, etc.

Item 16. Feedback provided by both our federal partners and TRP members suggested that there is a strong interest in whether schools have formal policies or written agreements with law enforcement that define the role of sworn law enforcement officers in the school.

Item 17. Asked as a follow-up to item 16, this item asks schools to report on what specific activities are defined by the school’s formal policies or written agreements with sworn law enforcement officers.

Item 18. This item has been modified to no longer collect data on security guards and security personnel and to only request counts of School Resource Officers and other sworn law enforcement officers. Information on security personnel is now collected separately in item 19, as the roles and responsibilities of security personnel can vary widely across schools and are often not comparable to those of sworn law enforcement.

Item 19. This item has been added to collect separate counts of security guards and personnel that schools have other than sworn law enforcement officers.

Item 20. This item asks schools to report on the types of mental health services provided to students (i.e., diagnostic assessments and treatment) as well as the location and provider of these services. Adequate assessment and treatment of mental health issues in students may help to prevent future violent acts, and research supports that school mental health programs can have an impact on reducing behavioral problems.^⁶ The information gained from this item will address issues of access for students and coordination problems that schools may need to resolve in order to provide these services to students.

Item 21. As noted by a mental health expert, some schools may hesitate to recommend students who they suspect as having a mental health disorder for fear that the school may become liable to pay for these services. Upon further discussion with the TRP and our federal partners, this question has been expanded to include several factors that may limit a school’s efforts to provide students with mental health services.

Item 22, subitem c. This item has been added to separately identify staff training related to cyberbullying.

Item 22, subitem d. This item has been added to separately identify staff training related to bullying (excluding cyberbullying).

Item 22, subitem h. This item has been added to gather information on training in the intervention and referral of students for mental health disorders. Proposed by our federal partners and supported by our TRP members, this item aims to measure whether teachers/aides have been trained in what steps to take once they have recognized the signs of a mental health disorder in students.

Item 27. Proposed by our federal partners and supported by our TRP members, this item will measure how many arrests of both students and non-students occurred on school grounds. This information could indicate whether or not crimes that have been reported to police result in an arrest on school grounds.

Item 28. Proposed by our federal partners and supported by our TRP members, this item has been modified to ask only about the number of hate crimes at school while removing “gang-related” crimes and “gang-related hate crimes,” which generally have had low counts and little variance in previous SSOCS administrations.

Item 29. Asked as a follow-up to item 28, this item asks schools to report on the types of biases (including race or color, national origin or ethnicity, gender, religion, disability, sexual orientation, and gender identity) that may have motivated the hate crimes reported in the previous item.

Item 32, subitem d. Per feedback received from advocacy groups during the 30-day public comment period, this item has been modified to separately identify the frequency of student harassment of other students based on sexual orientation.

Item 32, subitem e. Per feedback from advocacy groups during the 30-day public comment period, this item has been modified to separately identify the frequency of student harassment of other students based on gender identity.

C3. Cognitive Testing Findings and Final Recommendations

Introduction and Background

The School Survey on Crime and Safety (SSOCS), a nationally representative survey of elementary and secondary public schools, is one of the nation’s primary sources of school-level data on crime and safety. Managed by the U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics (NCES), SSOCS has been administered five times—covering the 1999–2000, 2003–04, 2005–06, 2007–08, and 2009–10 school years. SSOCS is unique in that it is the only recurring federal survey collecting detailed information on the incidence, frequency, seriousness, and nature of violence affecting students and school personnel, as well as other indices of school safety from the schools’ perspective. As such, SSOCS fills an important gap in data collected by NCES and other agencies.

An updated SSOCS questionnaire, including two new items⁷ on school-wide discipline policies and staff training related to bullying, received OMB approval for a spring 2012 administration, but due to funding issues the collection was never fielded. With new funding available through the National Institute of Justice (NIJ), SSOCS will be conducted again in the spring of the 2015–16 school year. To the greatest extent possible, NCES would like to retain items contained in earlier collections to preserve trend lines. That said, some items have been modified to stay relevant, and new items have been added to address new and emerging issues and interests.

NCES and its contractor, the American Institutes for Research (AIR), held a series of meetings in the late summer and fall of 2014 to discuss the proposed content of the 2015–16 SSOCS questionnaire. NCES and AIR met twice with federal partners from NIJ and the Bureau of Justice Statistics (BJS), who identified two priority areas recommended for SSOCS item development: (1) the expansion of a current section on school security staff and (2) a new section on mental health services in schools. NCES and AIR held two additional meetings with a Technical Review Panel (TRP) consisting of some of the nation’s top experts in school crime and safety. Several panel members provided insight from the perspective of researchers who use the SSOCS data. The TRP also included a school mental health expert who provided information specifically about mental health services in schools. Over the course of the four meetings, modifications to historic SSOCS items were proposed, several new items were recommended for addition, and three items were removed from the questionnaire. Following these meetings, NCES and AIR agreed upon a proposed list of new and modified items.

As part of the SSOCS 2015–16 item development process, the new and modified survey items agreed upon by NCES and AIR were tested on target participants through cognitive interviews in late 2014 and early 2015 to uncover comprehension issues and to measure overall understanding of the survey content. This document outlines the cognitive interview process, including the recruitment and data collection for these interviews, and provides findings from the cognitive interviews and recommendations for revisions that have been incorporated into the final items for the 2016 questionnaire.

Cognitive Interviews

In a cognitive interview, an interviewer uses a structured protocol in a one-on-one interview, drawing on methods from cognitive science. In particular, the cognitive interviews for SSOCS investigated the cognitive processes that principals use to answer survey questions. These interviews were intended to identify problems of ambiguity or misunderstanding in question wording, with the goal of ensuring that all items included in the final survey would be easily understood, with interpretations consistently aligned with the concepts being measured.

Cognitive interviewing methods consisted of two key components: think-aloud interviewing and verbal probing techniques (these two methods are also known as concurrent and retrospective recall probing, respectively). With think-aloud interviewing, participants are explicitly instructed to think aloud (i.e., describe what they are thinking) as they work through items. With verbal probing techniques, the interviewer asks probing questions, as necessary, to clarify points that are not evident from the “think-aloud” process or to explore additional issues that have been identified a priori as being of particular interest. Cognitive interview studies produce qualitative data in the form of verbalizations made by participants during the think-aloud phase and in response to the interviewer probes. Both the think-aloud approach and probing techniques were applied to all participants during the SSOCS cognitive interviews.

SSOCS is a paper-based survey; for the cognitive testing, participants were provided with a copy of the survey. Participants were asked to complete items in sets (broken out by the topical sections of the survey). They were asked to read the questions out loud and use the think-aloud process to describe how they understood the question and chose their response. Following the think-aloud portion for each section, the interviewer followed up with a set of pre-established probing questions, as necessary, to gather additional feedback and clarification.⁸

Survey Items for Testing

Based on a review of previous cognitive interviews conducted for SSOCS, the number of survey items to be tested, and consultations with staff experienced with cognitive interviewing, NCES and AIR concluded that the ideal length of the cognitive interview should not exceed 90 minutes. This would minimize the burden on the participating principals while ensuring the quality of the feedback. Because approximately 40 items (including sub-items) required cognitive testing, and given the degree of item modification anticipated, as well as the length of sections, it was decided that the cognitive interviews would focus on the School Security Staff and School Mental Health Services sections.

New items from these two sections (called “core items”) were tested across all participants, while additional new and modified items from other sections, such as School Practices and Programs, Staff Training, and Number of Incidents, were each tested across half of the participants. Two versions of the questionnaire—Version A and Version B—were created for use in the cognitive testing. Specific items that were tested in each version can be found in table C3.1 below. The core items (tested in both versions) are highlighted, while those tested with a partial sample of principals are not highlighted.

After cognitive interviews had been conducted with nine participants, NCES and AIR reviewed the initial findings and found that a number of the participants had issues with the terms, phrasing, and structure of two items in the mental health section and found them difficult to answer. Because of this, revisions were made to these items, and they were tested in a second wave of interviews with eight participants. These two items are noted with an asterisk in table C3.1. Although multiple versions of the questionnaire were used during the actual cognitive interviews, they have been combined into one questionnaire for ease of presentation in this report. The complete version of the questionnaire that contains all of the items that underwent cognitive testing, as well as the survey instructions and definitions, can be found in attachment 1 of this report.

Table C3.1 Split of items for SSOCS cognitive interview instruments, by version

Version A		Version B
Item number	Item topical area	Item number	Item topical area
Item 1	School practices and programs	Item 4	Formal programs to reduce violence
Item 2	Emergency plan items	Item 2	Emergency plan items
Item 3	Emergency plan items	Item 3	Emergency plan items
Item 6	School security items	Item 6	School security items
Item 7		Item 7
Item 8		Item 8
Item 9		Item 9
Item 10		Item 10
Item 11		Item 11
Item 12		Item 12
Item 13		Item 13
Item 18*	Mental health items	Item 18*	Mental health items
Item 19*		Item 19*
Item 20		Item 20
Item 21		Item 21
Item 22		Item 22
Item 5	Student groups to promote acceptance	Item 15	Stalking incidents
Item 14	Staff training	Item 16	Hate-crime incidents
		Item 17	Hate-crime biases
18 items		19 items

*Based on a discussion between NCES and AIR on January 20, 2015, after nine cognitive interviews had been conducted, it was agreed that these items would be revised for the remaining eight interviews. A number of principals had issues with the terms, phrasing, and structure of these items and found them difficult to answer, so revisions were implemented and tested in a second wave of interviews with eight participants.

Data Collection Process

NCES contracted with the American Institutes for Research (AIR) to conduct the SSOCS cognitive laboratory testing. Seventeen principals were recruited from across the nation to participate in the testing. The interviews took place between December 15, 2014, and February 5, 2015, and principals were asked to report items for the school year to date.

Recruitment

AIR used multiple outreach methods and resources to recruit participants, including contacts with schools and community organizations, newspaper/internet ads, social media, and direct e-mail and phone outreach to eligible schools. Eligible schools included regular public schools, charter schools, and schools that have partial or total magnet programs with students in any of grades prekindergarten through 12. E-mails and phone calls were used to contact potential participants and schedule interviews. Interested participants were screened to ensure they were eligible for participation in the interviews. Participants were asked whether they preferred in-person interviews or interviews conducted remotely through video conference or teleconference. In-person interviews were conducted in the Washington, DC, metro area to maximize scheduling and interviewing efficiency. All participants authorized their consent at the time of the interview, and as a thank-you for their time and participation, received a $25 gift card for remote interviews or a $40 gift card for in-person interviews. AIR recruited a total of 17 participants, one from each of 17 schools. Table C3.2 below shows the distribution of these participants across questionnaire versions, waves, and interview modes. Note that the school names have been removed and replaced with “School A, B, C, etc.” to ensure participant confidentiality.

Table C3.2 Distribution of cognitive interview participants, by school and interview characteristics

School

Participant

Distribution

Questionnaire

Version A

Wave 1

Wave 2

Version B

Wave 1

Wave 2

Mode of interview

In-person

Remote teleconference

Remote video conference

The participants represented schools with a range of characteristics (by grade level, urbanicity, enrollment size, and percent White enrollment). The participants included elementary, middle, and high school principals. Table C3.3 below shows descriptive statistics of the participants’ schools. While the sample included a variety of characteristics, the results of these interviews do not explicitly measure differences by these characteristics. Additional details on recruitment can be found in attachment 2 of this report.

Table C3.3 Distribution of cognitive interview participants, by school and school characteristics

School

Participant

Distribution

Level

Primary

Middle

High school

Combined

Enrollment

Less than 300

300–499

500–999

1,000 or more

Urbanicity

City

Suburb

Town

Rural

Percent White enrollment

More than 95 percent

More than 80 to 95 percent

More than 50 to 80 percent

50 percent or less

Cognitive Testing Findings and Recommendations

The following section summarizes the findings from these interviews by providing a synopsis of principal feedback for each tested item; the initial recommendation by AIR; and subsequent discussions between NCES, AIR, and the U.S. Census Bureau (Census)⁹ detailing the final recommendation for each item. Note that the item numbering in this report does not directly relate to the item numbering in the final 2016 SSOCS questionnaire, but instead corresponds with the numbering in the cognitive testing questionnaire found in attachment 1 of this report. Note that the multiple versions used in testing have been combined into a single questionnaire for ease of presentation in this report.

Note that definitions for many terms were provided to respondents at the front of the questionnaire. These defined terms are set in bold type and marked with an asterisk (*) throughout this report; a complete list of the definitions provided can be found on page 2 of the cognitive testing questionnaire, attached to this report. Although definitions were provided, interviewers generally probed respondents to define terms in their own words as they worked through questions in order to identify whether the definitions provided were accurate and comprehensive.

Section: School Practices and Programs

School Practices, General Findings (Item 1)

Cognitive Testing Findings: Note that additional sub-items are included in this item beyond those tested. Only sub-items in this question which were revised or new were included in the cognitive testing.

Nine principals received this item. Only one principal showed minor confusion in answering item 1a.

Please see the sub-item findings and recommendations below.

Item 1, Sub-item Findings

a. Require visitors to sign or check in and wear badges

Cognitive Testing Findings: This sub-item underwent minor revisions from SSOCS:2010 to specify that visitors are required to wear badges.

One principal expressed confusion in answering the question because at the principal’s school, they do require visitors to sign in, but they do not always require visitors to wear badges.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

b. Require metal detector checks on students every day

Cognitive Testing Findings: This sub-item underwent minor revisions from SSOCS:2010 to remove language specifying that students had to “pass through” metal detectors every day.

No principals had difficulty answering this question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

c. Equip classrooms with locks so that doors can be locked from the inside

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

No principals had difficulty answering this question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

d. Have “panic buttons” or silent alarms that directly connect to law enforcement in the event of an incident

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

No principals had difficulty answering this question; however, one principal noted that the question is plural (“panic buttons”) and that their school has only one button.

Initial Recommendation: No change.

Discussion: Further discussion suggested that many schools may have only a single panic button or silent alarm. NCES, AIR, and Census were in agreement that the item should be rewritten so as to not exclude schools that may only have one button.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Have “panic button(s)” or silent alarm(s) that directly connect to law enforcement in the event of an incident.

Emergency Plans, General Findings (Items 2 and 3)

Cognitive Testing Findings: Items 2 and 3 are a major revision of a SSOCS:2010 item; the previous item asked schools about both emergency plans and drills related to crises. For SSOCS:2016, item 2 asks about plans and drills based on procedures while item 3 asks about what crises are addressed in emergency plans.

All 17 principals received items 2 and 3. Seven principals indicated that item 2 (emergency plans by type of procedure) more accurately describes their school’s experience, while two principals indicated that item 3 (emergency plans by type of crisis scenario) more accurately describes their school’s experience.

Furthermore, six principals indicated that drills are typically designated by the type of procedures performed; five principals indicated that drills are typically designated by the type of crises they are intended to address; and six principals indicated that drills are typically designated by both the type of procedures performed and by the type of crises they are intended to address.

Initial Recommendation: No change; both items should be retained in the questionnaire since principals identified that both are relevant to emergency planning and drills in schools.

Discussion: As principals were able to distinguish between the type of procedure and crisis scenarios, NCES, AIR, and Census were in agreement that both items 2 and 3 should be included in the final 2016 SSOCS questionnaire. However, it was agreed that the order of these questions should be reversed in the final questionnaire so that the item on emergency plans by crisis scenarios precedes the item on emergency plan by type of procedure.

The item on emergency plans by crisis scenarios should be revised to ask only if schools have a written plan that addresses each scenario, while the item on emergency plans by type of procedure should be revised to ask only if schools have drilled on the listed procedures. These modifications should allow this question series to flow more smoothly and should allow the stem of the item on crisis scenarios to more closely resemble the item used in prior SSOCS administrations.

FINAL RECOMMENDATION: Revise; move item 3 to precede item 2.

The order of items 2 and 3, as tested in the cognitive questionnaire, will be switched in the final questionnaire. Tested item 3 will be renumbered as final item 2 and will appear prior to tested item 2, which will be renumbered as final item 3.

FINAL ITEM 2: Does your school have a written plan that describes procedures to be performed in the following scenarios?

FINAL ITEM 3: During the 2015–16 school year, has your school drilled students on the use of the following emergency procedures?

Please respond to each of these according to the definitions provided on pages 2 and 3.¹⁰

Please see findings and recommendations specific to tested items 2 and 3 below.

Item 2, Sub-item Findings

a. Evacuation*

Cognitive Testing Findings: No principals had difficulty answering this question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

b. Lockdown*

Cognitive Testing Findings: One principal had difficulty answering this question, and two principals mentioned that there are two types of lockdown: (1) “soft/modified,” where there is no access into the building except for those with clearance; and (2) “hard/full,” where a threat is potentially inside the building and everyone locks themselves into classrooms and offices.

Initial Recommendation: No change.

Discussion: Since the feedback from principals on this item was minor and they indicated they did not have difficulty in answering it, NCES, AIR, and Census were in agreement that no change was necessary.

FINAL RECOMMENDATION: No change.

c. Shelter-in-place*

Cognitive Testing Findings: Seven principals had trouble understanding the meaning of “shelter-in-place” in this question. Five principals identified these plans/drills as primarily having to do with natural disasters, which is outside the scope of the definition that was provided. Two principals indicated that the difference between “shelter‐in‐place” and “lockdown” was whether the threat is inside (lockdown) or outside (shelter‐in‐place) the building.

Initial Recommendation: Since principals were unable to correctly identify these plans/drills and instead identified them as primarily having to do with natural disasters, it was recommended that the item either be removed or revised to “Shelter-in-place (hazardous materials).”

Discussion: Further discussion suggested incorporating the definition of “shelter-in-place” into the question itself; however, it would appear to take up too much space with the additional amount of text. NCES, AIR, and Census instead agreed to add a note after the question referring the principal to the definitions on pages 2 and 3 of the final 2016 SSOCS questionnaire.

FINAL RECOMMENDATION: Add a note to the question.

FINAL ITEM: During the 2015-16 school year, has your school drilled students on the use of the following emergency procedures?

• Please respond to each of these according to the definitions provided on pages 2 and 3.¹¹

d. Reunification plan*

Cognitive Testing Findings: Seven principals had trouble understanding the term “reunification plan.” Many principals identified these plans/drills as primarily having to do with bringing students and staff back together during or after a drill, such as meeting at an alternative site. Furthermore, two principals were unfamiliar with the term “reunification plan.”

Initial Recommendation: Due to the difficulty expressed by principals with this term, it was recommended that the item be revised to “Family reunification plan” to emphasize students being reunited with their family.

Discussion: Further discussion suggested that “reunification plan” should be removed from this question and instead be revised as “Post-crisis reunification of students with their families” as a new sub-item in the item on emergency plans by crisis scenario (see Item 3 below). This change was also recommended because schools may have reunification plans but typically do not perform drills of these plans. NCES, AIR, and Census were in agreement regarding this change.

FINAL RECOMMENDATION: Revise and relocate item.

This item was removed to be addressed in the next item (tested item 3 below) as a new sub-item: “Post-crisis reunification of students with their families.” The order of items 2 and 3, as tested in the cognitive questionnaire, will be switched in the final version. Tested item 3 will be renumbered as final item 2 and will appear prior to tested item 2, which will be renumbered as final item 3. This sub-item will appear as part of final item 2.

FINAL ITEM: Post crisis reunification of students with their families.

Item 3, Sub-item Findings

a. Active shooter*

Note that active shooter is only one of several sub-items included in this item. Only sub-items in this question which were revised or new were included in the cognitive testing findings. Sub-item a underwent minor revisions from SSOCS:2010 to update the language to that currently used predominantly in schools (the previous sub-item language was “shootings”).

Cognitive Testing Findings: One principal had trouble understanding the term “active shooter” in this question, noting that “active shooter” was not a term that his school used; however, the principal’s school did have a drill that essentially amounted to the same thing (i.e., a person in the building with intent to do harm to others).

Initial Recommendation: Since the feedback from principals on this item was minor and they indicated they did not have difficulty in answering it, no change was recommended.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

Violence Prevention Programs, General Findings (Item 4)

Nine principals received this question. Two principals had trouble understanding the question. One principal said that she wasn’t sure if the question was referring to programs that are school-wide or available on an individual basis, and the other principal thought this question was asking what specific programs the school offered for different items (e.g., “What does my school offer for anti‐bullying?”).

Please see the sub-item findings and recommendations below.

Item 4, Sub-item Findings

a. Prevention curriculum, instruction, or training for students (e.g., conflict resolution, anti-bullying*, dating violence prevention)

Cognitive Testing Findings: This sub-item underwent minor revisions from SSOCS:2010 to add additional examples.

One principal initially hesitated over the word “violence.” The principal noted that the examples in the question were helpful and helped to clarify the item.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

b. Behavioral or behavior modification intervention for students (including the use of positive reinforcements)

Cognitive Testing Findings: This sub-item underwent minor revisions from SSOCS:2010 to add an example.

One principal interpreted “positive reinforcement” to mean physical incentives (i.e., objects/something that someone can put their hands on) rather than comments or verbal feedback.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

c. Student involvement in peer mediation

Cognitive Testing Findings: Sub-items c, d, and e are new items proposed to replace a prior sub-item that encompassed all these programs.

No principals had difficulty understanding the question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

d. Student court to address student conduct problems or minor offenses

Cognitive Testing Findings: Sub-items c, d, and e are new items proposed to replace a prior sub-item that encompassed all these programs.

One principal had trouble understanding the term “student court.” The principal indicated that this term sounded like “public shaming.”

Initial Recommendation: Since the feedback from principals on this item was minor and they indicated they did not have difficulty in answering it, no change was recommended.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

e. Student involvement in restorative circles* (e.g., “peace circles,” “talking circles,” “conflict circles”)

Cognitive Testing Findings: Sub-items c, d, and e are new items proposed to replace a prior sub-item that encompassed all these programs.

Three principals had trouble understanding the term “restorative circle” in the context of the question. However, with the exception of a single principal who did not reference the definition, all principals were able to deduce the meaning of the term from the context and the examples.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

f. Social emotional learning (SEL) training for students (e.g., social skills, anger management, mindfulness)

Cognitive Testing Findings: One principal had trouble with the term “training” in the context of the question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

Student Groups to Promote Acceptance, General Findings (Item 5)

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Nine principals received this question. Only one principal had trouble understanding a term in sub-item a.

Please see the sub-item findings and recommendations below.

Item 5, Sub-item Findings

a. Acceptance of LGBTQ students (e.g., Gay-Straight Alliance)

Cognitive Testing Findings: One principal had trouble understanding the meaning of “Acceptance of LGBTQ students.” The principal did not know what LGBTQ was, but assumed that it had something to do with sexual preference.

Initial Recommendation: The term LGBTQ should be spelled out in this item.

Discussion: Further discussion and feedback received from advocacy groups during the 30-day public comment period led to the agreement to remove references to LGBTQ in this item and throughout the survey in order to better distinguish between sexual orientation and gender identity. It was also agreed to add definitions for sexual orientation and gender identity to the survey to ensure that respondents could appropriately differentiate between these terms.

FINAL RECOMMENDATION: Revise item to spell out the acronym.

Replace LGBTQ with sexual orientation and gender identify and add definitions for these two terms..

FINAL ITEM: Acceptance of sexual orientation and gender identity of students (e.g., Gay-Straight Alliance)

FINAL DEFINITION: Sexual orientation* – means one’s emotional or physical attraction to the same and/or opposite sex.

FINAL DEFINITION: Gender identity* – means one’s inner sense of one’s own gender, which may or may not match the sex assigned at birth. Different people choose to express their gender identity differently. For some, gender may be expressed through, for example, dress, grooming, mannerisms, speech patterns, and social interactions. Gender expression usually ranges between masculine and feminine, and some transgender people express their gender consistent with how they identify internally, rather than in accordance with the sex they were assigned at birth.

b. Acceptance of students with disabilities (e.g., Best Buddies)

Cognitive Testing Findings: No principals had trouble with this item.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

c. Acceptance of cultural diversity

Cognitive Testing Findings: One principal had trouble finding a response that matched their school’s experience since the principal’s school has several events, seminars, and an international day to help children understand cultural diversity, but the school does not have a student group.

Initial Recommendation: For clarification of the type of groups in this category, an example should be added to the sub-item.

Discussion: NCES, AIR, and Census were in agreement that an example should be added to the item for clarification.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Acceptance of cultural diversity (e.g., Cultural Awareness Club).

Section: School Security Staff

School Security Staff, General Findings (Items 6, 7, 8, 9, and 10)

Cognitive Testing Findings: All 17 principals received the first question in this section (item 6). In accordance with the skip logic that begins in this question, 12 principals received additional questions in this section (items 7, 8, 9, and 10).

Three principals had trouble understanding the meaning of “School Resource Officer” in this section, while one principal had trouble understanding the meaning of “sworn law enforcement officer.” Two principals thought that a School Resource Officer (SRO) was not distinguishable from a sworn law enforcement officer.

Initial Recommendation: No change. The terms “School Resource Officer” and “sworn law enforcement officer” have been used successfully in prior administrations of SSOCS, and detailed definitions are provided in the questionnaire.

Discussion: Further discussion led to the agreement that it is not necessary for schools to report on the responsibilities and activities of School Resource Officers and sworn law enforcement officers separately. Analysis of prior SSOCS data indicates that schools tend to use one type of security personnel (SRO or other sworn law enforcement officer) rather than both; therefore, attempting to gather separate information on each type of officer may be confusing and burdensome for the majority of schools. Schools report on the number of full-time and part-time SROs and other sworn law enforcement officers separately in a SSOCS item that did not undergo cognitive testing.

FINAL RECOMMENDATION: Revise items in this section.

Revise the stem and sub-items of questions 6, 7, 8, 9, and 10 so that they do not request information on School Resource Officers and other sworn law enforcement officers separately. Furthermore, the definition of a School Resource Officer should be removed from the stem of this item. The definition should be moved into item 18¹² of the final questionnaire (an item that requests full-time and part-time counts of SROs and other sworn law enforcement officers). This is consistent with the placement of this definition in prior SSOCS administrations, where it appears in the first item in the questionnaire where respondents are requested to delineate SROs from other law enforcement officers for reporting purposes.

Please see findings and recommendations specific to items 6, 7, 8, 9, and 10 below.

Item 6, Findings

All 17 principals received this question. Three principals missed the skip pattern that begins in this question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census agreed that no change was necessary to this item based on the results of the cognitive testing; however, in accordance with the decision to revise this section to not report on School Resource Officers and sworn law enforcement separately, the stem of this item should be modified slightly to correctly introduce the skip logic that begins in this item.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: During the 2015–16 school year, did you have any sworn law enforcement officers (including School Resource Officers) present at your school* at least once a week?

Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in item 19.¹³

Item 7, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Twelve principals received this question. Several principals had trouble understanding it. Two principals initially selected “Yes” to the question, but when probed further, both changed their answer to “No.” Additionally, principals had different interpretations of the term “all school hours”:

• Two principals interpreted this phrase to mean contract hours for their teachers.

• Two principals interpreted this phrase to mean “from the time in the morning that doors are unlocked until the time in the evening that they are closed.”

• Four principals indicated that “all school hours” can vary depending on if there are after-school or off-campus events. (Alternatively, one principal said that this would exclude extracurricular events.)

• Three principals interpreted this phrase to mean “the general time that school is open” or “the hours that class is in session.”

Initial Recommendation: Suggest revising this item to ask about the presence of officers “from the beginning of the school day until students are dismissed.”

Discussion: During further discussion, it was suggested that this item use the phrase “all instructional hours” to better align with the language commonly used at the school level and in other surveys to indicate the time in which students are present for the school day.

FINAL RECOMMENDATION: Revise item.

In accordance with the revision made to this entire section, the stem of this item should be revised to ask jointly about coverage of sworn law enforcement officers (including School Resource Officers) and to replace “all school hours” with “all instructional hours.”

FINAL ITEM: During the 2015–16 school year, did your school have a sworn law enforcement officer (including School Resource Officers) present for all instructional hours every day that school was in session?

Include officers who are used as temporary coverage while regularly assigned officers are performing duties external to the school (such as attending court) or during these officers’ personal leave time.
Check “No” if your school does not have officer coverage while regularly assigned officers are performing duties external to the school (such as attending court) or during these officers’ personal leave time.
Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in item 19.¹⁴

Item 8, Findings

Cognitive Testing Findings: This item underwent minor revisions from SSOCS:2010 to remove references to security guards or security personnel other than SROs or other sworn law enforcement officers. Note that additional sub-items are included in this item beyond those tested. Only sub-items in this question which were revised or new were included in the cognitive testing.

Twelve principals received this question. Three principals did not respond in the “Other sworn law enforcement” column for this question. One of these principals thought that a principal should never respond in both columns since the columns presented an “either/or” situation (i.e., that a school would respond for only SROs or sworn law enforcement officers).

Initial Recommendation: Remove the second column that asks principals to provide information for “other sworn law enforcement officers” as several principals were confused by the two columns, and some principals did not record answers under the second column or selected “No.”

Discussion: Further discussion led to the agreement that it is not necessary for schools to respond separately for School Resource Officers and other sworn law enforcement officers in this item (see the School Security Staff, General Findings section above).

FINAL RECOMMENDATION: Revise and relocate item.

Revise item to remove the second column and modify the stem to ask jointly about the activities of School Resource Officers and other sworn law enforcement officers. Additionally, to be consistent with the ordering used in prior SSOCS administrations, this item should be moved to precede the item on sworn law enforcement coverage during all instructional hours (tested item 7 above) in the final questionnaire.

FINAL ITEM: Did these sworn law enforcement officers (including School Resource Officers) participate in the following activities at your school*?

Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in item 19.¹⁵
Check “Yes” or “No” on each line.

Please see the sub-item findings and recommendations below.

Item 8, Sub-item Findings

a. Motor vehicle traffic control

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

No principals had difficulty answering this item.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

b. Recording or reporting discipline problems

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

Two principals had trouble answering the item because they could not find a response that matched their school’s experience. Both principals indicated that they were unsure how to answer the item because law enforcement is not involved in minor issues, but can become involved if an issue rises to a higher level. Overall, four principals thought that this item was difficult to answer.

Initial Recommendation: In order to clarify the item, it was recommended that the item be revised to read “Recording or reporting discipline problems to school authorities.”

Discussion: NCES, AIR, and Census were in agreement that the item should be revised as suggested

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Recording or reporting discipline problems to school authorities.

c. Providing information to school authorities about the legal definitions of behavior for recording or reporting purposes

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

Two principals had trouble understanding the question; one of these principals suggested that it would be nice to include an example. One principal had trouble finding a response that matched their school’s experience. This principal indicated that she first thought about the question in terms of school policies, on which the SRO does not provide information. She noted that the school may ask for clarification on definitions for terms necessary for the officer to write a ticket, but this would not be for the school’s reporting purposes, only to assist in the SRO’s legal purposes of writing the ticket. Overall, five principals thought that this item was difficult to answer.

Initial Recommendation: To clarify this item we suggest adding an example, such as “(e.g., defining assault for school authorities).”

Discussion: NCES, AIR, and Census were in agreement that this item needed to be revised for clarity.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Providing information to school authorities about the legal definitions of behavior for recording or reporting purposes (e.g., defining assault for school authorities).

Formal Policies/Written Documents for Sworn Law Enforcement Officers,

General Findings (Items 9 and 10)

Cognitive Testing Findings: Twelve principals received item 9, and, based on their response to this item, eight principals received the follow-up item (item 10). Note that one principal missed the skip logic that began in this question and responded to the follow-up item.

Item 9, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Twelve principals received this question. Two principals had difficulty finding a response option that matched their school’s experience; specifically, one principal noted that she believes that such a document exists but said that principals would not need to see it. Additionally, two principals indicated that while their school did not have formal policies or documents, school staff had met with the SROs to define roles and expectations.

Initial Recommendation: No change.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: During the 2015–16 school year, did your school or school district have any formalized policies or written documents (e.g., Memorandum of Use, Memorandum of Agreement) that outlined the roles, responsibilities, and expectations of sworn law enforcement officers (including School Resource Officers) at school?

Item 10, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Based on their responses to item 9, eight principals received this item. One principal indicated that it was difficult to answer item 10 since the sub-items are so specific, and the principal was not sure if his school’s policies mentioned these specific things (this prompted the principal to select “Don’t know” for some items). One principal had trouble finding a response that matched their school’s experience, as the school does have policies on these items but the Memorandum of Agreement does not address them.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item based on the results of the cognitive testing; however, in accordance with the decision to revise this section to not report on SROs and sworn law enforcement separately, the stem of this item should be modified slightly. Additionally, it was agreed that “written documents or policies” would be changed to “formalized policies or written documents” to be consistent with the language in the previous item.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Did these formalized policies or written documents include language defining the role of sworn law enforcement officers (including School Resource Officers) at school in the following areas?

Please see the sub-item findings and recommendations below.

Item 10, Sub-item Findings

a. Student discipline

Cognitive Testing Findings: No principals had difficulty answering the question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

b. Use of physical restraints (e.g. handcuffs, Tasers, Mace, pepper spray, or other physical or chemical restraints)

Cognitive Testing Findings: One principal indicated that this question was difficult to answer because the school very rarely uses restraints, and the principal was not sure if the school had a written policy.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

c. Use of firearms*

Cognitive Testing Findings: One principal had trouble understanding the question; the principal was not sure if this item meant policies on the SRO using firearms or if it referred to what the SRO should do if people had firearms in the building.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

d. Making arrests on school grounds

Cognitive Testing Findings: One principal had difficulty answering the question because the principal was sure that there was documentation about this, but said that [making arrests] is the police’s prerogative and that there is no procedure.

Initial Recommendation: No change necessary to this item; however, it may be helpful to principals to include a definition for “arrests” based on issues identified with the item on arrests.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item. This is consistent with the use of the term “arrest” in other federal surveys, such as the Civil Rights Data Collection (CRDC), which do not provide a definition for the term “arrest.”

FINAL RECOMMENDATION: No change.

e. Reporting of criminal offenses to a law enforcement agency

Cognitive Testing Findings: No principals had difficulty answering the question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

Item 11, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

All 17 principals received this item. Several principals had trouble with it. One principal indicated that the placement of this question within a list of questions about SROs and sworn law enforcement officers made the principal wonder if this question is meant to focus on whether probation or parole officers are involved with student discipline.

Additionally, eight principals noted that probation officers are attached to individual students at the school and not assigned to the school itself. Several principals noted that parole or probation officers did come to their school, but less frequently than once a week, based on the requirements of students in the school needing these officers.

Initial Recommendation: One option would be to revise the time frame in the question to “at least once per month” or “at least every other month.” Another option would be to revise the question to a “Yes/No” format asking if schools had probation/parole officers assigned to students in their school (or alternatively ask for a count of probation/parole officers).

Discussion: Further discussion suggested that the findings indicated a need to better understand the relationship between the parole/probation officers and schools as well as how accurately principals are able to gauge when students have a parole officer and when that officer is present on campus. The findings indicate that we do not know enough to gather reliable data on the presence and frequency of parole/probation officers at this time.

FINAL RECOMMENDATION: Further development needed.

Remove item from the 2015–16 questionnaire and continue development on it, specifically to clarify how schools work with parole/probation officers, for possible inclusion in SSOCS:2018.

Item 12, Findings

Cognitive Testing Findings: This item is a minor revision to a SSOCS:2010 item; the previous item asked for counts of security guards or security personnel in the same question as counts of SROs and other sworn law enforcement officers. For SSOCS:2016 these will appear as two separate questions.

All 17 principals received this item. One principal had trouble understanding it. The principal considered a juvenile court officer as another type of security staff or personnel and counted this officer in this item. This principal was also unsure if the staff member should be counted as “full-time” or “part-time” since nobody replaced the staff member when he was not in the building.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that item should not be changed in order to retain the trend from previous SSOCS administrations.

FINAL RECOMMENDATION: No change.

Item 13, Findings

Cognitive Testing Findings: This was a new item proposed for SSOCS:2016.

All 17 principals received this item. Three principals had trouble understanding this question overall. One principal indicated that the question was confusing because it does not clarify between “being charged for something” and “actually being arrested,” while another principal was confused by the phrase “regardless of whether a student or non-student was arrested.”

Furthermore, several principals had trouble understanding the meaning of “arrest” in this question:

• One principal defined an arrest as “being read your Miranda rights and being handcuffed” and indicated that being taken to the police station is not enough to constitute an arrest.

• Six principals said that an arrest involves “being taken away by the police” or “being taken into custody.”

• Three principals defined an arrest as involving being handcuffed.

• Two principals defined an arrest as involving being charged with something and taken away.

• One principal defined an arrest as “to take control of one’s body and person and have them under your authority or control.”

• One principal defined an arrest as “being taken to juvenile detention.”

Six principals indicated that they knew their answer (number of arrests) from memory, while two principals did not know the number of arrests but indicated that they would be able to look this information up if they were actually taking the survey.

Initial Recommendation: As there was a wide variety in the way principals interpreted the term “arrest” in this item, it was recommended that a definition of arrest (that specifies being detained by the police) be included in the final 2016 SSOCS questionnaire.

Discussion: While further discussion questioned whether schools would be able to accurately respond to this item given their varying definitions of arrest, it was decided to retain the item as written for comparability to other federal surveys (e.g., the CRDC). Other federal surveys do not provide a definition for the term “arrest,” so it was recommended that the SSOCS item not include a definition since there is not a commonly used definition and this is consistent with other surveys.

FINAL RECOMMENDATION: No change, but relocate item.

Move this item to the “Number of Incidents” section of the questionnaire as this item more directly relates to the other items in this section as opposed to school security staff.

Section: Staff Training

Staff Training, General Findings (Item 14)

Nine principals received this question, and two principals had difficulty with a term in sub-item a.

Please see the sub-item findings and recommendations below.

Item 14, Sub-item Findings

a. Training in school-wide discipline policies and practices related to bullying*

Cognitive Testing Findings: This sub-item underwent minor revisions from SSOCS:2010 to add a definition for bullying.

Two principals had trouble with the term “bullying” in the context of this question. One principal indicated that he strongly dislikes the word “bullying” because it is so misconstrued by parents, while another principal seemed to define bullying as harassment.

Initial Recommendation: No change.

Discussion: Further discussion led to the agreement that this item should be placed after the item related to cyberbullying to ensure that principals do not include cyberbullying in their interpretation of bullying. For clarification, “other than cyberbullying” should be added to the end of the sub-item.

FINAL RECOMMENDATION: Revise and relocate item.

FINAL ITEM: Training in school-wide discipline policies and practices related to bullying* other than cyberbullying*

b. Training in school-wide discipline policies and practices specifically related to cyberbullying*

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

No principals had difficulty answering the question.

Initial Recommendation: No change.

Discussion: Relating to the previous item, further discussion led to the agreement that this item should be placed before the item related to bullying to ensure that principals do not include cyberbullying in their interpretation of bullying. Due to the change in order and the clarification added to the bullying sub-item, it was agreed that “specifically” could be removed from the sub-item text.

FINAL RECOMMENDATION: Revise and relocate item.

FINAL ITEM: Training in school-wide discipline policies and practices related to cyberbullying*

c. Training in intervention and referral strategies for students displaying signs of mental health issues (e.g. depression, mood disorders, ADHD)

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

Two principals had difficulty answering this question, and one principal had trouble understanding the term “mental health issues” in the context of this question. The principal pointed out that there was a discrepancy between this item using “mental health issues” and other items that used the term “mental health disorders.” The principal was confused as to whether these terms were supposed to be the same thing or if there was a differentiation.

Initial Recommendation: Revise item to replace “mental health issues” with “mental health disorders.”

Discussion: NCES, AIR, and Census were in agreement that consistent terminology should be used throughout the survey, and that the item should be revised accordingly.

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: Training in intervention and referral strategies for students displaying signs of mental health disorders* (e.g., depression, mood disorders, ADHD)

Section: Number of Incidents

Number of Incidents, General Findings (Item 15)

Cognitive Testing Findings: Seven principals received this question.

Note that stalking is only one of several sub-items included in this item. Only sub-items in this question which were revised or new were included in the cognitive testing. Sub-item a is a new item.

Please see the sub-item findings and recommendations below.

Item 15, Sub-item Findings

a. Stalking*

Cognitive Testing Findings: This is a new sub-item proposed for SSOCS:2016.

No principals had any difficulty answering this question. Two principals, who did not know the answer to the question, indicated that they could look up incidents of stalking in their school’s records. Two other principals, who responded from memory that there had been no incidents of stalking, indicated that they would not be able to query this information from a database since stalking is not reported on separately.

Initial Recommendation: No change.

Discussion: Further discussion led to the agreement that schools may not be able to accurately provide this information as cognitive testing indicated that many schools do not keep sufficient records of incidents of stalking.

FINAL RECOMMENDATION: Further development needed.

Remove the item from the 2015–16 questionnaire and continue development on it (specifically, clarifying how incidents of stalking are counted for recording and reporting purposes) for possible inclusion in SSOCS:2018.

Hate Crimes, General Findings (Items 16 and 17)

Cognitive Testing Findings: Six principals received item 16, and, based on their answers to this item, two principals received the follow-up item (item 17). Note that if principals could not recall hate crimes that had occurred at school during the current school year, principals were asked to respond to item 17 based on hate crimes that they could recall from previous school years.

No principals had difficultly responding to the items in this section.

Item 16, Findings

Cognitive Testing Findings: This item underwent minor revisions from SSOCS:2010; the previous item asked about the occurrence of gang-related crimes and gang-related hate crimes in addition to hate-crimes.

No principals had difficulty answering the question.

Initial Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item based on cognitive testing. However, it was agreed that the definition for hate crime should be revised to match the FBI’s updated definition for a hate crime and, per feedback received from advocacy groups during the 30-day public comment period to specifically include gender identity as a bias.

FINAL RECOMMENDATION: No change to item, but revise definition of hate crime.

FINAL DEFINITION: Hate Crime* – A committed criminal offense that is motivated, in whole or in part, by the offender’s bias(es) against a race, religion, disability, sexual orientation, ethnicity, gender, or gender identity; also known as Bias Crime.

Item 17, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Two principals received this question based on hate crimes that they could recall from previous school years. Both principals indicated that it was easy to decide which answers to give and did not have any difficulty in answering the question.

Recommendation: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item based on cognitive testing. However, further discussion led to the suggestion that the item start with “To the best of your knowledge…” to align with language used in similar perception questions in the SSOCS questionnaire. Additionally, per feedback received from advocacy groups, it was agreed to separate sub-item f into two sub-items: (f) sexual orientation and (g) sexual identity. It was also decided to add definitions for these items (see item 5, sub-item a).

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: To the best of your knowledge, were any of these hate crimes* motivated by the offender’s bias against the following characteristics?

Check “Yes” or “No” on each line
If a hate-crime was motivated by multiple characteristics, answer “Yes” for each that applies.

Section: School Mental Health Services¹⁶

School Mental Health Services, General Findings (Items 18 and 19)

Cognitive Testing Findings: This is a new section proposed for SSOCS:2016.

All 17 principals received the items in this section. However, after approximately half of the interviews had been conducted, principals showed a high level of confusion with the terms and item structures in this section. Based on these initial findings, NCES, AIR, and Census revised items 18 and 19 for further testing, resulting in Wave 1 and Wave 2 versions of these items.

Items 18 and 19 were revised on January 20, 2015. Nine principals received the Wave 1 version of these items, and eight principals received the Wave 2 version.

Item 18 (Wave 1), Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Of the nine principals who received this item, four principals had trouble with the structure of this question. Three principals indicated that they were confused about how to provide answers within the matrix, specifically within the “Mark all that apply” section. Three principals initially only responded to the Yes/No column of the matrix.

Additionally, two principals had trouble finding a response that matched their school’s experience. One principal indicated that they have some services (screenings) that are done at school using a collaboration between district and community personnel, and, if the student is identified as a risk, they are referred to an off-site psychiatrist within 24 hours. This principal expressed uncertainty about where to record this in the matrix. Another principal was unsure of where to categorize a mental health professional from a co-op from the larger school district who has an office in their school and tests students.

Furthermore, three principals had trouble understanding the meaning of “community provider” in this question, while one principal had trouble understanding the meaning of “at school by school or district employee” in this question.

Finally, two principals had trouble understanding the meaning of “mental health professional” in this question. For example, one principal defined this as someone licensed in counseling; however, he only gave the example of a special education teacher.

Initial Recommendation Following Wave 1: As a number of principals had issues with the terms, phrasing, and structure of this question and found the item difficult to answer, following a discussion with NCES on January 20, 2015, it was recommended this item be revised for the remaining cognitive interviews. A revision was implemented and tested in Wave 2 of the cognitive labs with eight participants.

See below for the results from the revised Wave 2 item.

Item 18 (Wave 1), Sub-item Findings

a. Pre-diagnostic counseling*

Cognitive Testing Findings: Six principals had trouble understanding the meaning of “pre-diagnostic counseling” in this question.

Initial Recommendation Following Wave 1: Following a discussion with NCES on January 20, 2015, it was recommended this sub-item be changed to “mental health counseling* prior to or without a diagnosis.” A revision was implemented and tested in Wave 2 of the cognitive labs with eight participants.

See below for the results from the revised Wave 2 item.

b. Diagnostic assessment* for mental health disorders*

Cognitive Testing Findings: Three principals had trouble understanding the term “diagnostic assessment” in this question. One principal noted that the terminology of this question made it difficult to answer because this is not her area and because the school does refer students for mental health services but they do not use these phrases (e.g., pre-diagnostic counseling, diagnostic assessment). This principal expressed particular confusion regarding the term diagnostic assessment.

Another principal had trouble finding a response that matched their school’s experience as the principal’s school does not do diagnostic assessments unless they are required under an individualized education program (IEP).

Initial Recommendation Following Wave 1: Following a discussion with NCES on January 20, 2015, this item was not revised before testing in Wave 2 of the cognitive labs with eight participants.

See below for the results from the revised Wave 2 item.

c. Treatment* for mental health disorders*

Cognitive Testing Findings: Two principals had trouble understanding the meaning of the term “treatment.” One of these principals noted this part of the question was confusing because he defined treatment as counseling, which had previously been asked about.

Initial Recommendation Following Wave 1: Following a discussion with NCES on January 20, 2015, this item was not revised before testing in Wave 2 of the cognitive labs with eight participants.

See below for the results from the revised Wave 2 item.

Item 18 (Wave 2), Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Eight principals received this item in Wave 2. Five principals had trouble with the structure of this question. Specifically, three principals found the matrix format confusing; one principal noted that it was hard to get what the question was asking because of its format; and one principal said that a “Don’t know” or “N/A” would be more appropriate.

Additionally, two principals had trouble understanding the term “mental health professional” and two principals had trouble understanding the meaning of “at school…employed by the school or district” in this question. Five principals had trouble understanding the term “formal relationship” in the context of this question, while one principal had trouble understanding the term “outside of school” in the context of this question.

Initial Recommendation Following Wave 2: Suggest changing “formal relationship” to “contracted by the school or district” and specifying that, for the last column, these professionals do not have a contract with the school or district.

Discussion: As many principals in Wave 2 were confused by the “formal relationship” language, it was agreed that this language would be removed from the column headers and replaced with “other than a school or district employee” to differentiate whether services are being provided by school personnel.

Additionally, the last column (outside of school by a mental health professional* as a referral of the school or district) should be dropped to limit the question to services available to students and funded by schools. The matrix for the final questionnaire should contain the following three columns:

AT SCHOOL* by a mental health professional* employed by the school or district
AT SCHOOL* by a mental health professional* other than a school or district employee, funded by the school or district
OUTSIDE OF SCHOOL by a mental health professional* other than a school or district employee, funded by the school or district

FINAL RECOMMENDATION: Revise item.

FINAL ITEM: During the 2015–16 school year, were the following mental health services available to students under the official responsibilities of a licensed mental health professional*?

Item 18 (Wave 2), Sub-item Findings

a. Mental health counseling* prior to or without a diagnosis

Cognitive Testing Findings: Three principals in Wave 2 had trouble understanding the meaning of “mental health counseling” in this question. For example, one of these principals noted that it was difficult to define counseling because it is used to address a wide range of issues—from student conflict resolution, to problems at home, to student cutting or self-harm—and because it can also include families in counseling that are referred to an agency. Another principal said that she didn’t see the difference between counseling in sub-item a and treatment in sub-item c since treatment includes counseling according to the definition.

Initial Recommendation Following Wave 2: In both iterations of this sub-item (Wave 1 and Wave 2), principals expressed difficulty with the term “counseling.” Principals indicated that the term was broad and difficult to define and were often not able to differentiate between whether it referred to mental health counseling or academic or other counseling.

Discussion: NCES, AIR, and Census consulted with the mental health expert who served on the SSOCS TRP regarding this issue. The expert indicated that he did not think this counseling item belonged in this matrix because pre-diagnostic counseling was not typically done by a mental health professional in or through schools. At the point where a mental health professional is counseling a student, it would be considered the treatment stage, which is captured in sub-item c of this question.

Given the difficulty in principals’ interpretation of “counseling” and the input of the TRP member, it was recommended that this sub-item be removed from this item.

FINAL RECOMMENDATION: Remove item.

b. Diagnostic assessment* for mental health disorders*

Cognitive Testing Findings: Two principals had trouble understanding the meaning of the term “diagnostic assessment” in this question. One of these principals described this as going through the formal process of assessment for an IEP for an emotional disturbance. Additionally, two principals had trouble understanding the meaning of the term “mental health disorders” in this question.

Initial Recommendation Following Wave 2: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item.

FINAL RECOMMENDATION: No change.

c. Treatment* for mental health disorders*

Cognitive Testing Findings: No principals had trouble understanding the meaning of the term “treatment” in this question, but one principal had trouble understanding the meaning of the term “mental health disorders” in this question.

Initial Recommendation Following Wave 2: No change.

Discussion: NCES, AIR, and Census were in agreement that no change was necessary to this item based on the cognitive interview findings and the removal of sub-item a.

FINAL RECOMMENDATION: No change.

Item 19 (Wave 1), Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Of the nine principals who received this question, one principal had trouble understanding the phrase “mental health disorders,” and two principals had trouble understanding the phrase “conduct disorders” in the context of this question.

Furthermore, three principals indicated that it was difficult to answer this question because “the question was blending together a lot of things” and because “there are a lot of students who act out or behave inappropriately.”

Initial Recommendation Following Wave 1: As a number of principals in Wave 1 had issues with the terms, phrasing, and structure of this question and found the item difficult to answer, following a discussion with NCES on January 20, 2015, it was recommended this item be revised for the remaining cognitive interviews. A revision was implemented and tested in Wave 2 of the cognitive labs with eight participants.

See below for the results from the revised Wave 2 item.

Item 19 (Wave 2), Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

Of the eight principals who received this item in Wave 2, four had trouble finding a response that matched their school’s experience, and three principals had trouble understanding the question. One principal had to read the question twice before answering and changed one of her answers after her initial response, while another principal indicated that she was having difficulty with the multiple parts of the question that needed to be addressed. One principal also had trouble understanding the phrase “conduct disorders” in the context of this question.

Finally, one principal felt that this question had a negative connotation and does not adequately address the point at which a student would be evaluated. She felt that not all students who display even somewhat repetitive problem behavior (e.g., disrupts class three times) should be assumed to have a disorder and tested.

Initial Recommendation Following Wave 2: Given the issues demonstrated during cognitive testing, this item may not be ready to field in the 2016 SSOCS questionnaire. It was recommended that this item be refined through further research and the input of our mental health TRP consultant and that a revised version of this question undergo additional testing for possible inclusion in the next iteration of the SSOCS questionnaire.

Discussion: During further discussion, it became clear that the revised item was still confusing to principals and that further revisions to the item would require additional testing.

FINAL RECOMMENDATION: Further development needed.

Remove this item from the 2015–16 questionnaire and continue development on it for possible inclusion in SSOCS:2018.

Item 20, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

All 17 principals received this item. Due to the placement of this item within the mental health framework, one principal was confused as to whether “threat assessment team” was referring to issues of mental health or issues of violence. Another principal said that of all the terms in this survey, “threat assessment team” was the term most foreign to her.

Initial Recommendation: To alleviate confusion as to whether “threat assessment team” addresses issues of mental health or violence, it was recommended that the items on threat assessment teams be moved to the “School Practices and Programs” section for the 2016 SSOCS questionnaire.

Discussion: NCES, AIR, and Census were in agreement that this item should be moved to the “School Practices and Programs” section for the 2016 SSOCS questionnaire and that no additional changes to the item are necessary.

FINAL RECOMMENDATION: No change, but relocate item.

Move this item (and the subsequent item) to the “School Practices and Programs” section to follow the item on formal programs intended to prevent or reduce violence.

Item 21, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

All 17 principals received this item. Only one principal had difficult answering it, because the principal’s school has a threat assessment team that meets bi-weekly.

Initial Recommendation: It was recommended that this item be moved to the “School Practices and Programs” section for the 2016 SSOCS questionnaire.

Discussion: Further discussion led to the conclusion that the item should be moved (along with the above item) to the “School Practices and Programs” section and that the response options should be revised to address schools that may meet at intervals different from those listed. The response options were crafted to mirror those used in item 35¹⁷ in the final 2016 SSOCS questionnaire.

FINAL RECOMMENDATION: Revise and relocate item.

Move this item to the “School Practices and Programs” section to follow the above item on the existence of a threat assessment team. Additionally, revise the response options.

FINAL ITEM: During the 2015–16 school year, how often did your school’s threat assessment team* formally meet?

At least once a week.
At least once a month.
On occasion.
Never.

Item 22, Findings

Cognitive Testing Findings: This is a new item proposed for SSOCS:2016.

All 17 principals received this question. One principal had trouble understanding this question and thought that the question was driving at whether it would be helpful to have a mental health professional at school. Furthermore, six principals indicated that it was difficult to answer this question: four principals noted that they had difficulty in answering the parts regarding legal issues, and two had difficulty answering the item on policies about schools’ liability to pay. Several principals indicated that it was difficult to distinguish between the three response choices for some items, including one principal who said that she could not gauge if things were a chronic or a minor need.

Additionally, four principals had trouble finding a response that matched their school’s experience: two principals felt that “support from the school community” was a missing option from this list; one principal felt that the location of the school administration office might also be a factor, and one principal suggested access to formal relationships/outside providers was a limiting factor.

Initial Recommendation: Given the interpretive/subjective nature of this item, suggest adding “To the best of your knowledge” to the item stem to mirror similar items included in the questionnaire. Additionally, propose to add a sub-item on “support from the wider school community” since several principals noted this was a limiting factor.

Discussion: It was decided that “refer” should be removed from the stem of the item in order to focus the question solely on factors that limit schools’ ability to provide these mental health services. In order to clarify the distinction between sub-item c on legal issues and sub-item e on schools’ liability to pay for mental health services, it was also decided that sub-item e should be revised to avoid using legal-sounding language. Furthermore, NCES, AIR, and Census agreed that an additional sub-item on lack of community support should be added to this question as several principals identified this as a limiting factor. The recommendation to include “To the best of your knowledge” was not implemented after further review of the language used in a similar historic SSOCS item; the current wording maintained consistency with the similar item.

FINAL RECOMMENDATION: Revise item.

Revise the item stem of tested sub-item e, and add a new item on lack of community support.

FINAL ITEM: During the 2015–16 school year, to what extent did the following factors limit your school’s efforts to provide mental health services to students?

Inadequate access to licensed mental health professionals*
Inadequate funding
Potential legal issues for school or districts (e.g., malpractice, insufficient supervision)
Lack of parental support in addressing their children’s mental health disorders*
Lack of community support for providing mental health services to students in your school
Written or unwritten policies regarding the school’s requirement to pay for the diagnostic assessment or treatment of students
Reluctance to label students with mental health disorders* to avoid stigmatizing the child

Attachment 1: SSOCS:2016 Cognitive Testing Questionnaire

U.S. DEPARTMENT OF EDUCATION

NATIONAL CENTER FOR EDUCATION STATISTICS

SCHOOL SURVEY ON CRIME AND SAFETY

PRINCIPAL QUESTIONNAIRE

2014–15

NCES is authorized to conduct the School Survey on Crime and Safety by the Education Sciences Reform Act of 2002 (ESRA; 20 U.S.C. §9543). Your answers may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573). Reports of the findings from the survey will not identify participating districts, schools, or staff. Individual responses will be combined with those from other participants to produce summary statistics and reports.

DRAFT FORM SSOCS-2016-COGLAB

(10-30-14)

DEFINITIONS
*The following words are bolded and marked by an asterisk () wherever they appear in the questionnaire. Please use these definitions as you respond.**
Active shooter – an individual actively engaged in killing or attempting to kill people in a confined and populated area; in most cases, active shooters use firearm(s) and there is no pattern or method to their selection of victims. At school/at your school – activities happening in school buildings, on school grounds, on school buses, and at places that hold school-sponsored events or activities. Unless otherwise specified, this refers to normal school hours or to times when school activities/events were in session. Bullying – any unwanted aggressive behavior(s) by another youth or group of youths who are not siblings or current dating partners that involves an observed or perceived power imbalance and is repeated multiple times or is highly likely to be repeated. Community provider – a mental health professional who is not a school district employee. This may include professionals employed by local mental health or social service agencies. Conduct disorder – refers to a group of behavioral and emotional problems that can occur in children and teens. A child with this disorder may display a pattern of disruptive and violent behavior and have problems following rules. Children with conduct disorder may exhibit some of the following behaviors: aggression to people and animals; destruction of property; deceitfulness, lying, or stealing; and serious violations of rules. Counseling – the provision of information, assistance, and guidance to students with behavioral or mental health issues. This is in contrast to counselors who provide only academic counseling or college or career placement. Cyberbullying – occurs when willful and repeated harm is inflicted through the use of computers, cell phones, or other electronic devices. Diagnostic assessment – an evaluation conducted by a medical or mental health professional that identifies whether an individual has one or more medical and/or mental health diagnoses. This is in contrast to an educational assessment, which does not focus on clarifying a student’s diagnosis. Evacuation – a procedure that requires all students and staff to leave the building. While evacuating to the school’s field makes sense for a fire drill that only lasts a few minutes, it may not be an appropriate location for a longer period of time. The evacuation plan should encompass relocation procedures and include backup buildings to serve as emergency shelters, such as nearby community centers, religious institutions, businesses, or other schools. Evacuation also includes “reverse evacuation,” a procedure for schools to return students to the building quickly if an incident occurs while students are outside. Firearm/explosive device – any weapon that is designed to (or may readily be converted to) expel a projectile by the action of an explosive. This includes guns, bombs, grenades, mines, rockets, missiles, pipe bombs, or similar devices designed to explode and capable of causing bodily harm or property damage.		Hate crime – a criminal offense or threat against a person, property, or society that is motivated, in whole or in part, by the offender’s bias against a race, color, national origin, ethnicity, gender, religion, disability, or sexual orientation. Lockdown – a procedure that involves occupants of a school building being directed to remain confined to a room or area within a building with specific procedures to follow. A lockdown may be used for when a crisis occurs outside of the school and an evacuation would be dangerous. A lockdown may also be called for when there is a crisis inside and movement within the school will put students in jeopardy. All exterior doors are locked and students and staff stay in their classrooms. Mental health disorders – collectively all diagnosable mental disorders or health conditions that are characterized by alterations in thinking, mood, or behavior (or some combination thereof) associated with distress and/or impaired functioning. Mental health professionals – mental health services are provided by several different professions, each of which has its own training and areas of expertise. Types of professionals who may provide mental health services include: psychiatrists, psychologists, psychiatric/mental health nurse practitioners, psychiatric/mental health nurses, clinical social workers, and professional counselors. Restorative circle – a formal mediation process led by a facilitator that brings affected parties of a problem together to explore what happened, reflect on their roles, find a solution, and ultimately restore harmony to individual relationships and larger community. Reunification plan – a procedure to return students to their homes and family as rapidly as possible. Schools need to have current plans that include names and telephone numbers of family and designated surrogates. Stalking - a course of conduct directed at a specific person that involves repeated visual or physical proximity; nonconsensual communication; verbal, written, or implied threats; or a combination of these actions that would cause a reasonable person to feel fear. This includes cyber-stalking (threatening communication or unwanted advances directed at another person using the Internet and other forms of online and computer communications). Shelter-in-place – a procedure similar to a lockdown in that the occupants are to remain on the premises; however, shelter-in-place is designed to use a facility and its indoor atmosphere to temporarily separate people from a hazardous outdoor environment. Everyone would be brought indoors and building personnel would close all windows and doors and shut down the heating, ventilation, and air conditioning system (HVAC). This would create a neutral pressure in the building, meaning the contaminated air would not be drawn into the building. Threat assessment team – a formalized group of persons who meet on a regular basis with the common purpose of identifying, assessing, and managing students who may pose a threat of targeted violence in schools. Treatment – a clinical service addressed at lessening or eliminating the symptoms of a disorder. In mental health, this may include psychotherapy, medication treatment, and/or counseling.
	SURVEY INSTRUCTIONS: For most questions, please mark the box that best reflects your school’s circumstances. Please mark your response with an "X". For questions that ask for counts or percents, please place an “X” in the None box, rather than leaving the item blank. Definitions are available for many terms on page 2. Defined terms are bolded and marked with an asterisk (*) throughout the survey. Some questions refer to the 2015–16 school year. Please report for the school year to date. Please have this questionnaire filled out by the person most knowledgeable about school crime and policies to provide a safe environment. Paperwork Burden Statement According to the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it displays a valid OMB control number. The valid OMB control number for this voluntary survey is 1850–0803. The time required to complete this survey is estimated to average 45 minutes, including the time to review instructions, gather the data needed, and complete and review the survey. If you have any comments concerning the accuracy of the time estimate, suggestions for improving this survey, or any comments or concerns regarding the status of your individual submission of this survey, please write to: School Survey on Crime and Safety, National Center for Education Statistics, 1990 K Street, N.W., Room 9027, Washington, D.C. 20006.

School Practices and Programs
1. During the 2014–15 school year, was it a practice of your school to do the following? If your school changed its practices during the school year, please answer regarding your most recent practice. Check “Yes” or “No” on each line.
				YES	NO
	a.	Require visitors to sign or check in and wear badges	xxx	1	2
	b.	Require metal detector checks on students every day	xxx	1	2
c.		Equip classrooms with locks so that doors can be locked from the inside	xxx	1	2
d.		Have “panic buttons” or silent alarms that directly connect to law enforcement in the event of an incident	xxx	1	2

2. Does your school have a written plan on actions to be performed in the event of an emergency that describes the following procedures? If yes, has your school drilled students on the use of this procedure during the 2014–15 school year?
				Have a written plan?		If “Yes,” has your school drilled students on the plan during the 2015–16
				YES	NO		YES	NO
	a.	Evacuation*	xxx	1	2	xxx	1	2
	b.	Lockdown*	xxx	1	2	xxx	1	2
	c.	Shelter-in-place*	xxx	1	2	xxx	1	2
	d.	Reunification plan*	xxx	1	2	xxx	1	2

3. Do your school’s emergency plans (as identified item 2) address the following crisis scenarios?
				YES	NO
	a.	Active shooter*	xxx	1	2
	b.	Natural disasters (e.g. earthquakes or tornadoes)	xxx	1	2
	c.	Hostages	xxx	1	2
	d.	Bomb threats or incidents	xxx	1	2
	e.	Chemical, biological, or radiological threats or incidents (e.g., release of mustard gas, anthrax, smallpox, or radioactive materials)	xxx	1	2
	f.	Suicide threat or incident	xxx	1	2
	h.	Pandemic flu	xxx	1	2

*Please use the definitions as provided on page 2.

4. During the 2014–15 school year, did your school have any formal programs intended to prevent or reduce violence* that included the following components for students? If a program has multiple components, answer "Yes" for each that applies. Check "Yes" or "No" on each line.
				YES	NO
	a.	Prevention curriculum, instruction, or training for students (e.g., conflict resolution, anti-bullying*, dating violence prevention)	xxx	1	2
	b.	Behavioral or behavior modification intervention for students (including the use of positive reinforcements)	xxx	1	2
c.		Student involvement in peer mediation	xxx	1	2
d.		Student court to address student conduct problems or minor offenses	xxx	1	2
e.		Student involvement in restorative circles* (e.g., “peace circles,” “talking circles,” “conflict circles”)	xxx	1	2
f.		Social emotional learning (SEL) training for students (e.g., social skills, anger management, mindfulness)	xxx	1	2

5. During the 2014–15 school year, did your school have any recognized student groups with the following purposes? Check "Yes" or "No" on each line.
				YES	NO
	a.	Acceptance of LGBTQ students (e.g., Gay-Straight Alliance)	xxx	1	2
	b.	Acceptance of students with disabilities (e.g., Best Buddies)	xxx	1	2
	c.	Acceptance of cultural diversity	xxx	1	2

*Please use the definitions as provided on page 2.

School Security Staff

6. During the 2014–15 school year, did you have any School Resource Officers (career law enforcement officers with arrest authority, who have specialized training and are assigned to work in collaboration with school organizations) or other sworn law enforcement officers present at your school* at least once a week?

Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in NEW ITEM 12.

Shape57 xxx 1 Yes

2 No – GO TO ITEM 11 ON PAGE 7.

7. During the 2014–15 school year, did your school have a School Resource Officer or other sworn law enforcement officer present for all school hours every day that school was in session?

Include officers who are used as temporary coverage while regularly assigned officers are performing duties external to the school (such as attending court) or during these officers’ personal leave time.
Check “No” if your school does not have officer coverage while regularly assigned officers are performing duties external to the school (such as attending court) or during these officers’ personal leave time.
Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in item 12.

Shape58 xxx 1 Yes

2 No

8. Did these School Resource Officers or other sworn law enforcement officers participate in the following activities at your school*?

Do not include security guards or other security personnel who are not sworn law enforcement in your response to this item; information on additional security staff is gathered in item 12.
Check “Yes” or “No” on each line.

School Resource officers

Other sworn law enforcement officers

YES

Motor vehicle traffic control

xxx

Shape59 1

Shape60 2

xxx

Shape61 1

Shape62 2

Recording or reporting discipline problems

xxx

Shape63 1

Shape64 2

xxx

Shape65 1

Shape66 2

Providing information to school authorities about the legal definitions of behavior for recording or reporting purposes

xxx

Shape67 1

Shape68 2

xxx

Shape69 1

Shape70 2

9. During the 2014–15 school year, did your school or school district have any formalized policies or written documents (e.g., Memorandum of Use, Memorandum of Agreement) that outlined the roles, responsibilities, and expectations of School Resource Officers or other sworn law enforcement officers at school?

Shape71 xxx 1 Yes

2 No - GO TO ITEM 11 ON PAGE 7.

*Please use the definitions as provided on page 2.

10. Did these written documents or policies include language defining the role of School Resource Officers or other sworn law enforcement officers at school in the following areas? Check “Yes,” “No,” or “Don’t know” on each line.
				YES		NO		Don’t know
	a.	Student discipline	xxx		1		2		3
	b.	Use of physical restraints (e.g. handcuffs, Tasers, Mace, pepper spray, or other physical or chemical restraints)	xxx		1		2		3
	c.	Use of firearms*	xxx		1		2		3
	d.	Making arrests on school grounds	xxx		1		2		3
	e.	Reporting of criminal offenses to a law enforcement agency	xxx		1		2		3

11. During the 2014–15 school year, were there any probation or parole officers present at your school* at least once a week?

Shape87 xxx 1 Yes

2 No

12. Aside from School Resource Officers or other sworn law enforcement officers, how many additional security guards or security personnel were present in your school at least once a week during the 2014–15 school year? If a security guard or other security personnel works full-time across various schools in the district, please count this person as “part-time” for your school. If none, please place an “X” in the None box.
			Number at your school*
	Security guards or security personnel i. Full-Time	xxx		None
	ii. Part-Time	xxx		None

13. Please record the number of arrests that occurred at your school during the 2014–15 school year. Please include all arrests that occurred at school*, regardless of whether a student or non-student was arrested.

If none, please place an “X” in the None box.

Shape92

xxx Number of arrests

Shape93

None

*Please use the definitions as provided on page 2.

Staff Training
14. During the 2014–15 school year, did your school or school district provide any of the following for classroom teachers or aides? Check “Yes” or “No” on each line.
				YES	NO
a.		Training in school-wide discipline policies and practices related to bullying*	xxx	1	2
b.		Training in school wide discipline policies and practices specifically related to cyberbullying*	xxx	1	2
	c.	Training in intervention and referral strategies for students displaying signs of mental health issues (e.g. depression, mood disorders, ADHD)	xxx	1	2

Number of Incidents
15. Please record the number of incidents that occurred at school* during the 2014–15 school year for the offenses listed below. (NOTE: The number in column 1 should be greater than or equal to the number in column 2). If none, please place an “X” in the None box. Please provide information on: The number of incidents, not the number of victims or offenders. Recorded incidents, regardless of whether any disciplinary action was taken. Recorded incidents, regardless of whether students or non-students were involved. Incidents occurring before, during, or after normal school hours.
				Column 1		Column 2
				Total number of recorded incidents		Number reported to police or other law enforcement
	a.	Stalking*	xxx	None	xxx	None

16. During the 2014–15 school year, how many hate-crimes* occurred at your school*?

If none, please place an “X” in the None box.

xxx

Number of hate-crimes*

Shape104

Shape105

None – GO TO ITEM 18 ON PAGE 9.

17. Were any of these hate-crimes* motivated by the offender’s bias against the following characteristics? Check “Yes” or “No” on each line Check “Yes” or “No” on each line.
			YES		NO
a.	Race or color	xxx		1		2
b.	National origin or ethnicity	xxx		1		2
c.	Gender	xxx		1		2
d.	Religion	xxx		1		2
e.	Disability	xxx		1		2
f.	Sexual orientation or gender identity (i.e., lesbian, gay, bisexual, transgender, questioning)	xxx		1		2

School Mental Health Services

(WAVE 1) 18. During the 2014–15 school year, were the following mental health services available to students under the official responsibilities of a licensed mental health professional*? If so, please indicate the location and provider for each mental health service that was available to students.

If the service was available to students, mark “yes,” regardless of whether the service was used this school year.
Check “Yes” or “No” for each type of service.

MARK ALL THAT APPLY

Service was available

At school* by school or district employee

At school* by community provider* as part of a relationship with the school/school district

Outside of the school by a community provider* as part of a relationship with the school/school district

Outside of the school with a provider as a referral of the school/school district

YES

Pre-diagnostic counseling*

xxx

Shape118

Shape119

xxx

Shape120

xxx

xxx

xxx

Diagnostic assessment* for mental health disorders*

xxx

Shape124 1

Shape125

xxx

Shape126 1

xxx

xxx

xxx

Treatment* for mental health disorders*

xxx

Shape130 1

Shape131 2

xxx

Shape132 1

xxx

xxx

xxx

(WAVE 2) 18. During the 2014–15 school year, were the following mental health services available to students under the official responsibilities of a licensed mental health professional*?

If the service was available to students, mark “yes,” regardless of whether the service was used this school year.
Check “Yes” or “No” for each type of service available to students.

Service was available to students…

AT SCHOOL*

by a mental health professional* employed by the school or district

AT SCHOOL*

by a mental health professional* as part of a formal relationship with the school or district

OUTSIDE OF SCHOOL

by a mental health professional* as part of a formal relationship with the school or district

OUTSIDE OF SCHOOL

by a mental health professional* as a referral of the school or district

Mental health counseling* prior to or without a diagnosis

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

Diagnostic assessment* for mental health disorders*

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

Treatment* for mental health disorders*

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

xxx

1 Yes

2 No

*Please use the definitions as provided on page 2.

(WAVE 1) 19. During the 2014–15 school year, were all students who exhibited conduct disorder* behaviors evaluated by a mental health professional* to identify coexisting mental health disorders*?

Shape136 xxx 1 Yes

2 No

22. During the 2014–15 school year, to what extent did the following factors limit your school’s efforts to provide or refer students for mental health services? Check one response on each line.
				Limits in major way	Limits in minor way	Does not limit
	a.	Inadequate access to licensed mental health professionals*	xxx	1	2	3
	b.	Inadequate funding	xxx	1	2	3
	c.	Potential legal issues for school or districts (e.g., malpractice, insufficient supervision)	xxx	1	2	3
	d.	Lack of parental support in addressing their children’s mental health issues	xxx	1	2	3
	e.	Written or unwritten policies regarding the school’s potential liability to pay for these services	xxx	1	2	3
	f.	Reluctance to label students with mental health disorders* to avoid stigmatizing the child	xxx	1	2	3

(WAVE 2) 19. To the best of your knowledge, during the 2014–15 school year, was it the practice of your school to recommend that students exhibiting problem behaviors (e.g., stealing, assault, lying, destruction of property, etc.) be evaluated by a mental health professional* for both conduct disorder* and coexisting mental health disorders*?

Shape155 1 Yes, all students exhibiting conduct problem behaviors were recommended for both evaluations

2 That recommendation was made in some situations

Shape156 3 No, our school did not make these recommendations

4 Not applicable, no students exhibited problem disorder behaviors during this school year

20. During the 2014–15 school year, did your school have a threat assessment team* or any other formal group of persons to identify students who might be a potential risk for violent or harmful behavior (towards themselves or others)?

Shape157 xxx 1 Yes

2 No – GO TO ITEM 22 ON PAGE 10.

21. During the 2014–15 school year, how often did your school’s threat assessment team* formally meet?

Check one response.

Shape158 xxx 1 Weekly

2 Monthly

3 Occasionally

4 Never

*Please use the definitions as provided on page 2.

Attachment 2: SSOCS:2016 Cognitive Testing Tracking

SSOCS Cognitive Testing Tracking
Program Name	Sample Surveys Division: Cross-Sectional Surveys Branch, NCES
Study Name	2015–16 School Survey on Crime and Safety (SSOCS) Cognitive Interviews
OMB Clearance Date	11/7/2014
Recruitment Efforts Start Date	11/20/2014
Data Collection Start Date	12/15/2014
Data Collection End Date	02/04/2015
Target Principal Number	20
Type of research (cognitive interview, usability study, focus group)	Cognitive interview
Target participants* (key demographic characteristics)	5 participants from primary schools 3 participants from middle schools 8 participants from high schools 1 participant from a combined school 2 participants from schools with a total enrollment of less than 300 2 participants from schools with a total enrollment between 300 and 499 9 participants from schools with a total enrollment between 500 and 999 4 participants from schools with a total enrollment of 1,000 or more 9 participants from schools located in urban areas 4 participants from schools located in suburban areas 3 participants from schools located in towns 1 participant from schools located in rural areas 2 participants from schools with more than 95 percent White enrollment 2 participants from schools with more than 80 to 95 percent White enrollment 5 participants from schools with more than 50 to 80 percent White enrollment 8 participants from schools with 50 percent or less White enrollment
Expected interview length (in minutes)	90 minutes
Mode of the interview (e.g., in person, WebEx, telephone)	3 WebEx, 11 phone, 3 in-person (17 total)
Location where interviews were conducted (participant’s home, participant’s business, focus group facility, or location convenient for participant [e.g., a coffee shop])	14 interviews conducted remotely; 3 in-person interviews conducted at participants’ schools
Source of participants (pre-recruited database, advertisements, etc.)	ONLINE ADVERTISING (resulting in approximately 2 interviews) (e.g., Craigslist ads; Young Education Professionals Listserv in DC and Boston; Reddit advertisements; social media) POINT-OF-CONTACT (resulting in approximately 8 interviews) (e.g., professional and personal contacts of contractor [AIR] staff in DC, Chicago, and Boston; NCES staff forum) TARGETED E-MAILS/CALLS (resulting in approximately 7 interviews) (e.g., e-mails to state administrator associations; e-mails and reminder e-mails to over 600 principals in targeted school districts; direct calls to over 100 principals in targeted school districts)
Number of eligible participants screened	Outreached to 700+ potential participants; contact was made with approximately 60 people: 18 people were screened and were not eligible to participate 25 screened and eligible, but did not participate 17 screened and eligible, and participated in an interview
Final number of interviews conducted	17 interviews
Incentive offered	$40 for in-person interviews; $25 for remote interviews
Participant travel allowance, parking reimbursement in addition to honorarium, etc. (if applicable)	none
Any other funds paid to principal (e.g., bonus for arriving at all or early, referral bonus, child care allowance)	none
Any nonmonetary gratuities/gifts to participants (e.g., food provided, free parking, child care, gift cards)	none
Urbanicity of area where interview took place	9 urban; 4 suburb; 3 town; and 1 rural
Time of day interviews were scheduled	9 a.m.–7 p.m.
Days of the week interviews were scheduled	Monday–Friday
*Participants will fit into more than one category.

1 Dejka, J. (2013, February 26). After Sandy Hook, Metro-Area Schools Look at Safer Classroom Door Locks. Omaha News. Retrieved from http://www.omaha.com/news/after-sandy-hook-metro-area-schools-look-at-safer-classroom/article_6e7ffe69-9a71-52ac-9385-04ca976ff0e4.html.

2 Schneider, T. (2002). Ensuring Quality School Facilities and Security Technology. Portland, OR: Northwest Regional Educational Laboratory. Retrieved from https://www.ncjrs.gov/pdffiles1/ojjdp/book4.pdf.

3 Graham, J., Shirm, S., Liggin, R., Aitken, M., and Dick, R. (January 2006). “Mass-Casualty Events at Schools: A National Preparedness Survey,” Pediatrics, 117(1): e8–e15. Retrieved from http://pediatrics.aappublications.org/content/117/1/e8.full.html.

4 Civic Enterprises, Bridgeland, J., Bruce, M., and Hariharan, A. (2013). The Missing Piece: A National Teacher Survey on How Social and Emotional Learning Can Empower Children and Transform Schools. Chicago: Collaborative for Academic, Social, and Emotional Learning.

5 Miller, A. (February 2014). Threat Assessment in Action. Monitor on Psychology, 45(2): 37. Retrieved from http://www.apa.org/monitor/2014/02/cover-threat.aspx.

6 Hussey, D.L., and Guo, S. (2003). Measuring Behavior Change in Young Children Receiving Intensive School-Based Mental Health Services, Journal of Community Psychology, 31(6): 629–639. doi: 10.1002/jcop.10074. Retrieved from http://onlinelibrary.wiley.com/doi/10.1002/jcop.10074/pdf.

7 Items 22d and 22i on the final 2016 SSOCS questionnaire had received approval for the spring 2012 administration but were never fielded. For the 2016 administration, item 22d was slightly modified to make room for a similar item (22c) on cyberbullying. Items 22c and 22d underwent cognitive testing and correspond to items 14a and 14b in the cognitive testing questionnaire; the findings from the cognitive testing of these items can be found on pages 33–34 of this report.

8 Due to the nature of the paper-based administration of SSOCS, principals would have time to check records and consult with other school staff before answering items during the operational survey; however, the structure of the cognitive testing did not allow for this consultation.

9 NCES has entered into an interagency agreement with the Census Bureau to conduct the 2016 collection of SSOCS.

10 Refers to the page numbering of the final 2016 SSOCS questionnaire.

11 Refers to the page numbering of the final 2016 SSOCS questionnaire.

12 This item number is a reference to the numbering in the final 2016 SSOCS questionnaire; this item did not undergo cognitive testing and therefore is not included in this report.

13 This item number is a reference to the numbering in the final 2016 SSOCS questionnaire; this item reference also corresponds with item 12 in this report.

14 This item number is a reference to the numbering in the final 2016 SSOCS questionnaire; this item reference also corresponds with item 12 in this report.

15 This item number is a reference to the numbering in the final 2016 SSOCS questionnaire; this item reference also corresponds with item 12 in this report.

16 Note that this section was moved to follow the School Security Staff section in the final version of the questionnaire to improve the flow of the questionnaire. During cognitive testing, the School Mental Health Services section was the final section tested.

17 This item number is a reference to the numbering in the final 2016 SSOCS questionnaire; this item did not undergo cognitive testing and therefore is not included in this report.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
Author	Kathryn.Chandler
File Modified	0000-00-00
File Created	2021-01-23