OMB83C Memo

MEMORANDUM OMB # 1850-0923 v.2

DATE: January 13, 2016

TO: Robert Siviniski

Office of Information and Regulatory Affairs, Office of Management and Budget

FROM: Isaiah O'Rear

National Center for Education Statistics

THROUGH: Kashka Kubzdela

National Center for Education Statistics

SUBJECT: ED School Climate Surveys (EDSCLS) Benchmark Study 2016 Items Update Change Request

As part of October 2015 clearance for the ED School Climate Surveys (EDSCLS) Benchmark Study 2016 data collection (OMB# 1850-0923 v.1), OMB stipulated the following terms of clearance (TOC)[also item 2 in the EDSCLS Benchmarking Study 2016 Responses to Passback document]: “OMB recommends cognitive testing for the sexual assault items which were not part of the initial cognitive testing.” NCES subsequently conducted small scale cognitive testing of the sexual assault items (OMB# 1850-0803 v.145).

Forty-one cognitive interviews were conducted in Illinois, in the Chicago Standard Metropolitan Statistical Area between November 7 and 30, 2015. Ten interviews were conducted with teachers, either working in high schools or in middle schools; eleven interviews were conducted with non-instructional staff; ten interviews were conducted with middle school students; and ten with high school students. A summary of the cognitive interview results is provided in Appendix F, and NCES conclusions regarding the three items tested are provided below. NCES discussed the cognitive test results and plan of action with OSHS and OSHS agreed to the proposed plan of action. The revisions described below are reflected in the revised Questionnaires (Appendix B), and in the reference to these items in the Supporting Statements Part A and Part C (part of Part B & C EDSCLS Benchmarking Study 2016 document).

1. The following item was tested for teachers and non-instructional staff:

“The following types of problems occur at this school often: sexual assault or violence.”

The cognitive testing revealed that respondents found the question confusing because it was unclear if the question was asking about all violence or ‘sexual violence.’ To make the question clearer, NCES recommends changing the question to:

“The following types of problems occur at this school often: sexual assault or dating violence.”

The phrase ‘sexual assault or dating violence’ is used in the existing student item, so NCES recommends maintaining consistency between items.

2. The following item was tested for middle school and high school students:

“Students at this school feel unsafe because of sexual assault or dating violence.”

The cognitive testing showed that students do not have consistent interpretation of the intent of the item. NCES recommends not including this item on the EDSCLS survey.

3. Additionally, the following item was tested for middle school and high school students (per item 3 in the EDSCLS Benchmarking Study 2016 Responses to Passback document):

“At this school, there is a teacher or some other adult who students can go to if they need help because of sexual assault or dating violence.”

According to the cognitive lab report, some middle school students found the concept of dating violence difficult. NCES recommends continuing to exclude this item from Middle School Student Survey.

Lastly, during the 30-day public comment period, the New America Foundation asked NCES to “ensure a nationally representative sampling of schools serving students with disabilities.” The following is a summary of school-level CWD enrollment in the EDSCLS Benchmark Study 2016 sample showing that that the sample matches closely the national distribution of children with disability.

The original school-level children with disability (CWD) enrollment counts data files contain multiple records for up to 14 different disability categories (including “Missing”) and the grand total (“No Category Codes”) for 92,920 schools. The total number of records contained in the data files is 677,155, which means many schools do not have rows for all categories. School enrollment counts for the categories that are not present in the data were assumed to be zero.

The EDSCLS frame includes regular schools with students enrolled in the target grades 5–12 in the 50 states and the District of Columbia, excluding Department of Defense schools. The number of schools in the EDSCLS frame totaled 79,225. Among the 79,225 schools in the frame, 2,362 of them could not be matched with the CWD data. These 2,362 schools were assumed to have no CWD enrollment in the analysis.

As shown in table 1, the average school disability enrollment is 60.8. Because the EDSCLS sample was designed to give larger schools with higher chance to be selected, estimates for the sample need to be weighted. As we can see in table 1, the weighted averages are similar to those from the EDSCLS frame and the CWD universe.

Table 1. Average CWD enrollment by disability category for the CWD universe, EDSCLS frame, EDSCLS sample, and EDSCLS weighted sample

	CWD missing categories assumed to be zero	EDSCLS frame	EDSCLS sample	EDSCLS weighted sample
Autism	5.1	5.3	6.0	5.1
Deaf-blindness	0.0	0.0	0.0	0.0
Developmental delay	1.5	1.4	0.6	1.2
Emotional disturbance	3.3	3.4	5.1	3.5
Hearing impairment	0.7	0.7	0.8	0.7
Intellectual disability	4.1	4.3	5.9	4.4
Multiple disabilities	1.1	1.1	1.5	1.1
Orthopedic impairment	0.5	0.5	0.6	0.5
Other health impairment	8.9	9.4	12.5	9.7
Specific learning disability	23.9	25.8	35.7	26.3
Speech or language impairment	10.7	10.5	8.3	11.1
Traumatic brain injury	0.3	0.3	0.4	0.3
Visual impairment	0.3	0.3	0.3	0.3
Missing	0.6	0.6	0.4	0.3
No Category Codes	60.8	63.4	78.2	64.4

SOURCE: U.S. Department of Education Civil Rights Data Collection and the EDSCLS frame

Table 2 shows the percent of student enrollment that were CWD enrollment. The statistic does not strongly correlate with the school enrollment size that was used to determine the selection probability during the EDSCLS sampling process. As we can see, this statistic looks very similar for the EDSCLS frame, EDSCLS sample, and EDSCLS weighted sample, and the numbers are close to those for the CWD universe.

Table 2. Percentage of CWD enrollment by disability category for the CWD universe, EDSCLS frame, EDSCLS sample, and EDSCLS weighted sample

	CWD missing categories assumed to be zero	EDSCLS frame	EDSCLS sample	EDSCLS weighted sample
Autism	1.1%	0.9%	0.8%	0.8%
Deaf-blindness	0.0%	0.0%	0.0%	0.0%
Developmental delay	0.4%	0.3%	0.2%	0.3%
Emotional disturbance	1.1%	0.6%	0.8%	0.7%
Hearing impairment	0.2%	0.1%	0.1%	0.1%
Intellectual disability	1.0%	0.7%	0.8%	0.7%
Multiple disabilities	0.4%	0.2%	0.2%	0.2%
Orthopedic impairment	0.1%	0.1%	0.1%	0.1%
Other health impairment	2.0%	1.7%	1.8%	1.7%
Specific learning disability	4.9%	4.5%	5.5%	4.6%
Speech or language impairment	2.4%	2.2%	1.5%	2.3%
Traumatic brain injury	0.1%	0.0%	0.1%	0.0%
Visual impairment	0.1%	0.0%	0.0%	0.0%
Missing	0.2%	0.2%	0.1%	0.1%
No Category Codes	13.9%	11.6%	12.1%	11.6%

Note: Among the 92,920 schools in the CWD universe, the total enrollment is not available for 1,981 of them because either the schools are not in the 2013-14 CCD or the total enrollment is missing in the 2013-14 CCD for those schools.

SOURCE: U.S. Department of Education Civil Rights Data Collection, the EDSCLS frame, and the 2013-14 CCD.