Revised Sampling Plan for SCS 103008 (3)

Revised Sampling Design for School Climate Survey

The sampling plan for conducting the proposed School Climate Survey outlined below presents our sampling methodology, including: 1) target population, 2) sampling design, 3) sample sizes, and 4) anticipated power for statistical tests of study hypotheses.

Target population. The target population of the SS/HS Initiative includes 46 grantees funded in the 2006 and 2007 cohorts (excluding 40 from the 2005 cohort), and 1,203 individual schools participating in the SS/HS Initiative, yielding a total (based on an estimated 50 staff per participating school) of 60,150 instructional staff, school administrators, school counselors, and other school personnel from which to sample. Grantees and their participating schools will implement a variety of programs and activities across key areas designed to impact school safety, violence, substance abuse, access to mental health services, and school attendance.

Sample design. For the purpose of the School Climate Survey, we propose a stratified systematic sampling design in which we first stratify all participating schools into four strata by school type—Elementary, Middle, High, and Other Schools and then draw the sample by selecting one school administrator and one school counselor from each participating school, yielding a total of 2,406 school administrators and school counselors (1,203 school administrators and 1,203 school counselors, respectively). Then, a random sample of instructional staff will be selected through a systematic sampling with an equal sampling rate across all participating schools within each type of school. The actual sampling rate for schools of each type will be determined later when information on the exact number of instructional staff is available from each participating school. Due to the fact that some of the school districts have extremely small number of schools, ranging from one to two schools per school district, we are prepared to oversample school administrators, school counselors, and instructional staff to ensure all schools are represented in the sample.

Sample sizes. As indicated in the preceding section, the proposed stratified systematic sampling strategy will include all 1,203 participating schools, but only sample 7,218 school administrators, school counselors, and instructional staff out of a total of 60,150 school personnel from the schools that participate in the survey SS/HS Initiative (1,203 participating schools x 6 school administrators, school counselors, and instructional staff). Of the total sample of 7,218 school personnel, one school administrator and one school counselor will be selected from each of the 1,203 participating schools, yielding a total of 2,406 school administrators and school counselors (1,203 school administrators and 1,203 school counselors, respectively). The remaining sample of 4,812 instructional staff will be selected through the use of a systematic sampling approach with an equal sampling rate used within each of the four types of schools. Thus, this stratified systematic sampling strategy will yield a total sample of 7,218 school personnel (1,203 school administrators + 1,203 school counselors + 4,812 instructional staff), with 1,805 school personnel sampled for each of the four types of schools. We believe this sample will not only minimize the burden on the participating school personnel but also ensure the representativeness of the sample. Since schools participating in the SS/HS Initiative are required to participate in this survey, and since we have achieved an overall response rate of 99 percent for our ongoing School-level Survey, overall response to the School Climate Survey is anticipated to be high.

Statistical power. To establish the proposed sample size, we examined minimum detectable differences and the required sample sizes per school type (see Table 1). Table 1 can be used to determine the minimum detectable difference for comparisons of percentages from school personnel for reporting domains with sample sizes ranging from 300 to 7,200. The table assumes a power of 0.80 and a significance level of 0.05. Because minimum detectable differences vary based upon the size of the percentages involved, this table shows minimum detectable differences based upon various percentage values assumed for domain 1. For example, our proposed sample of 4,812 instructional staff will provide us with adequate power to obtain minimum detectable differences of three percent or less overall for comparisons between year one and baseline. This sample size will also provide minimum detectable differences of five percent or less for breakdowns into smaller domains defined by type of school (about 1,800 staff sampled).

Table 1. Minimal Detectable Differences for Various Percentage Values and Various Sample Sizes for a Two-Sided Test with α=0.05 and β=0.80
	School Personnel Sample Size ( n )
Percentage
	300	600	900	1,200	1,800	2,400	4,800	7,200
10	5.8	4.3	3.6	3.2	2.6	2.3	1.6	1.4
15	7.2	5.3	4.4	3.8	3.2	2.8	2.0	1.6
20	8.3	6.0	5.0	4.4	3.6	3.1	2.2	1.8
25	9.2	6.6	5.5	4.8	3.9	3.4	2.4	2.0
30	9.9	7.1	5.9	5.1	4.2	3.6	2.6	2.1
40	10.8	7.8	6.4	5.5	4.5	3.9	2.8	2.3
50	11.3	8.0	6.6	5.7	4.7	4.0	2.9	2.3

3

Battelle draft October 30, 2008