FRSS Arts Educ OMB Section B

Request for OMB Clearance

Fast Response Survey System (FRSS)

Arts Education in Public Elementary and Secondary Schools

Supporting Statement for Paperwork Reduction Act Submissions

Section B. Description of Statistical Methodology

B.1. Respondent Universe and Sample Design

Respondent Universe

The respondents for the proposed surveys on arts education will include principals, classroom teachers, and visual arts and music specialists in a stratified sample of regular public elementary and secondary schools in the United States. The school sample will be selected from the 2006-07 Common Core of Data (CCD) Public School Universe File maintained by the National Center for Education Statistics (NCES). Vocational education, special education, alternative/other non-regular schools, and schools operated by the Department of Defense or Bureau of Indian Affairs are ineligible for the survey, as are schools with a high grade of kindergarten or lower, ungraded schools, and schools in the outlying U.S. territories. For the purposes of this study, elementary schools are defined to be those schools with (a) grade 4 or lower and no grade higher than 8 or (b) only grades 5 or 6. Secondary schools are defined to be those with (a) no grade lower than 6 or (b) a low grade of 5 and a high grade of 7 or 8. All of the remaining schools have both elementary and secondary grades and are referred to as “combined” schools (e.g., schools with grades K to 12). Table 1 summarizes the definitions of the three instructional level categories used in this study. Table 2 gives the distribution of regular schools, students, and full-time equivalent teachers in the 2006-07 Public School Universe File by level and enrollment size class.

Since the administration of arts education in schools generally depends on instructional level, separate sets of questionnaires will be developed for the proposed surveys: one set applicable to elementary schools and another set applicable to secondary schools (see Section A.1.). As described below, separate stratified samples of elementary and secondary schools will be selected to receive the appropriate survey instrument. However, schools with combined grades can respond to either of the survey instruments. In order to give the combined schools appropriate representation in the study, they will be included in both the elementary and secondary school sampling frames. A combined school that is selected from the elementary school frame will be requested to complete only the elementary school questionnaire, while those selected from the secondary school frame will be requested to complete the secondary school questionnaire. Thus, the respondent universe for the elementary school questionnaire includes both elementary and combined schools, and similarly the respondent universe for the secondary school questionnaire includes both secondary and combined schools. Since the responses from the two sets of questionnaires will be analyzed separately, no bias results from including the combined schools in both frames.

Table 1. Definition of instructional level categories based on grade span of school

High grade

Low grade

1

2

3

4

5

6

7

8

9

10

11

12

PK

E

E

E

E

E

E

E

E

C

C

C

C

K

E

E

E

E

E

E

E

E

C

C

C

C

1

E

E

E

E

E

E

E

E

C

C

C

C

2

E

E

E

E

E

E

E

C

C

C

C

3

E

E

E

E

E

E

C

C

C

C

4

E

E

E

E

E

C

C

C

C

5

E

E

S

S

C

C

C

C

6

E

S

S

S

S

S

S

7

S

S

S

S

S

S

8

S

S

S

S

S

9

S

S

S

S

10

S

S

S

11

S

S

12

S

E: Elementary; S: Secondary; C: Combined

Table 2. Number of schools, students, and teachers in the 2006-07 CCD public school universe file by instructional level and enrollment size class

Instructional level	Enrollment size class	Number of schools	Enrollment	Number of teachers (full-time equivalent)
Elementary	Under 300	14,089	2,595,357	190,057
	300 to 499	19,009	7,606,800	494,879
	500 or more	19,777	13,616,194	823,305
Secondary	Under 500	13,630	3,476,951	254,456
	500 to 999	10,222	7,364,711	474,902
	1,000 or more	7,556	12,316,771	691,182
Combined	Under 500	1,580	316,965	24,438
	500 to 999	398	272,449	16,533
	1,000 or more	211	368,298	18,874
Total		86,472	47,934,496	2,988,625

Statistical Methodology

The sample design for the proposed surveys on arts education will be a stratified two-stage sample consisting of the selection of schools at the first stage of sampling and the selection of teachers within the sampled schools at the second stage. The primary strata for selecting schools will defined by instructional level (e.g., see Table 1) and enrollment size class. Within the primary sampling strata defined by level and size class, schools in the sampling frame will be sorted geographically by region, type of locale (central city, suburban, town, rural) within region, and other variables (e.g., categories defined by percent of students eligible for free/reduced price lunch or by minority status) to the extent feasible. When used in conjunction with systematic sampling, the sorting will induce additional implicit substratification within the primary strata which will help to ensure that relevant subcategories of schools are appropriately represented in the sample. As mentioned earlier, schools with combined grades will be included in both the elementary and secondary school sampling frames and thus will have chances of selection for both instructional levels.

Selection of Schools

In order to ensure that sufficient numbers of visual arts and music teachers are included in the sample, 1,800 elementary schools and 1,600 secondary schools will be selected for the study. The number of elementary schools to be selected is slightly larger than the number of secondary schools to be selected because the availability of arts specialists in elementary schools is expected to be somewhat less common. However, while the entire sample of 1,800 elementary schools and 1,600 secondary schools will be used to identify and select music and visual arts specialists, a subsample of 1,200 from each level will be retained for the administration of the principals’ survey. For elementary schools, the same subsample of 1,200 will be used to select classroom teachers. It should also be noted that a portion of the 1,800 elementary schools and the 1,600 secondary schools will be combined schools for reasons explained earlier.

Initially, the sample of 1,800 elementary schools and 1,600 secondary schools will be allocated to the primary strata in rough proportion to the aggregate square root of the enrollment of the schools in the stratum. Under the proposed allocation, larger schools will be sampled at relatively higher rates than smaller ones. The use of the square root of enrollment to determine the sample allocation is expected to be reasonably efficient for estimating both school-level characteristics (e.g., the number or percent of schools that provide more than two hours of arts instruction) and quantitative measures correlated with enrollment (e.g., the number of arts instructors/specialists or arts classes). Further, the proposed sample sizes are expected to be large enough to permit limited analysis of the various questionnaires for subgroups defined by instructional level, region, type-of-locale, and three or four enrollment size classes (e.g., small, medium, or large). Within each sampling stratum, the specified number of sampled schools will be selected systematically and with equal probabilities. The full sample of 1,800 elementary schools and 1,600 secondary schools will be used to select the samples of teachers/specialists as described below. Systematic subsamples of 1,200 schools from each level will be selected for the principals’ survey. The numbers of schools to be selected for the principals’ survey under the proposed design are summarized in Table 3 for selected subgroups.

Selection of Teachers

At the second stage of sampling, classroom teachers and arts education specialists will be subsampled from the 1,800 elementary schools and the 1,600 secondary schools selected for the study. Participating schools will be requested to provide lists of their classroom teachers and music/visual arts specialists. Up to three teachers will be randomly selected from each elementary school, including at most one teacher from the following three groups: (a) self-contained classroom teachers, (b) full- or part-time music teachers or specialists, and (c) full- or part-time visual arts specialists. Note, however, that the selection of self-contained classroom teachers will be restricted to the subsample of 1,200 elementary schools that are included in the principals’ survey. Based on the previous FRSS study on arts education in public elementary schools and additional information obtained during recent pretest calls, an estimated 79 percent of elementary schools have a music teacher/specialist and about 60 percent have a visual arts specialist. Thus, under the proposed design, an average of about 2 teachers/specialists will be sampled per elementary school.

Similarly, up to two teachers will be sampled from each of the 1,600 secondary schools, including one full- or part-time music teacher/specialist and one full- or part-time visual arts teacher/specialists. Based on the previous FRSS study and additional information obtained during recent pretest calls, an estimated 81 percent of secondary schools have a music specialist and about the same proportion have a visual arts specialist; thus, an average of 1.6 teachers will be selected per secondary school. Assuming that 85 percent of the sampled schools provide teacher lists and that 85 percent of the selected teachers respond to the teacher survey, completed teacher questionnaires will be obtained for approximately 2,675 elementary teachers/specialists and 1,873 secondary teachers/specialists. The numbers of teachers to be selected under the proposed design are summarized in Table 4.

Expected Levels of Precision

Tables 3 and 4 summarize the approximate sample sizes and standard errors to be expected under the proposed design for the principal and teacher surveys, respectively. The standard errors in these tables reflect design effects ranging from 1.10 to 1.30 to reflect the use of variable sampling fractions across size strata. The standard errors can be converted to 95 percent confidence bounds by multiplying the entries by 2. For example, from Table 3, an estimated proportion of the order of 20 percent (P = 0.20) based on a sample of 1,020 responding elementary schools would be subject to a margin of error of ±2.8 percent at the 95 percent confidence level. An estimated proportion of the order of 50 percent (P = 0.50) based on a sample of 287 small (less than 500 students) secondary schools would be subject to a margin of error of ±6.2 percent at the 95 percent confidence level. Similarly, from Table 4, the margin of error for estimates of a 50 percent characteristic (P = 0.50) for music and visual arts specialists in elementary schools would be ±3.2 percent and ±3.8 percent, respectively. The corresponding margins of error for music and visual arts specialists in secondary schools will be approximately ±3.4 percent.

Table 3. Expected sample sizes for the principals’ survey and estimated standard errors for selected subgroups

			Standard error for proportion (P) =
Category of schools	Sample schools	Resp. schools†	0.20	0.33	0.50
ELEMENTARY	1,200*	1,020	0.014	0.017	0.018
SIZE CLASS
Less than 300	218	185	0.031	0.036	0.039
300 to 599	616	523	0.018	0.022	0.023
600+	367	312	0.024	0.028	0.030
TYPE OF LOCALE
City	344	292	0.027	0.031	0.033
Suburban	386	328	0.025	0.030	0.031
Town	150	128	0.040	0.047	0.050
Rural	320	272	0.028	0.033	0.035
REGION
Northeast	220	187	0.033	0.039	0.042
Southeast	274	233	0.030	0.035	0.037
Central	291	248	0.029	0.034	0.036
West	414	352	0.024	0.029	0.030
SECONDARY	1,200**	1,020	0.014	0.017	0.018
SIZE CLASS
Less than 500	338	287	0.025	0.029	0.031
500 to 999	412	350	0.022	0.026	0.028
1000+	450	382	0.021	0.025	0.027
TYPE OF LOCALE
City	302	256	0.028	0.033	0.036
Suburban	364	310	0.026	0.030	0.032
Town	176	150	0.037	0.044	0.047
Rural	358	304	0.026	0.031	0.033
REGION
Northeast	226	192	0.033	0.039	0.041
Southeast	289	246	0.029	0.034	0.036
Central	302	257	0.028	0.033	0.036
West	383	326	0.025	0.030	0.032

* This is a subsample of the 1,800 schools to be selected for the elementary school teachers survey.

† Assumes 85 percent response rate for principals’ survey.

** This is a subsample of the 1,600 schools to be selected for the secondary school teachers survey.

Table 4. Expected sample sizes for the teachers’ survey and estimated standard errors for selected subgroups

			Standard error for proportion (P) =
Category of teachers	Sample teachers	Resp. teachers	0.20	0.33	0.50
ELEMENTARY	3,147*	2,675	0.009	0.010	0.011
TYPE OF TEACHER
Music	1,209	1,028	0.013	0.015	0.016
Visual arts	918	780	0.015	0.018	0.019
Classroom	1,020	867	0.014	0.017	0.018
TYPE OF LOCALE
City	901	766	0.016	0.019	0.021
Suburban	1,012	860	0.016	0.018	0.019
Town	394	335	0.025	0.029	0.031
Rural	839	713	0.017	0.020	0.021
REGION
Northeast	577	490	0.021	0.024	0.026
Southeast	719	611	0.018	0.022	0.023
Central	764	649	0.018	0.021	0.022
West	1,086	923	0.015	0.018	0.019
SECONDARY	2,204†	1,873	0.011	0.012	0.013
TYPE OF TEACHER
Music	1,102	937	0.014	0.016	0.017
Visual arts	1,102	937	0.014	0.016	0.017
TYPE OF LOCALE
City	554	471	0.021	0.025	0.026
Suburban	669	569	0.019	0.022	0.024
Town	324	275	0.027	0.032	0.034
Rural	657	558	0.019	0.023	0.024
REGION
Northeast	415	353	0.024	0.029	0.030
Southeast	531	451	0.021	0.025	0.027
Central	554	471	0.021	0.025	0.026
West	703	598	0.019	0.022	0.023

* Music and visual arts specialists will be selected from the full sample of 1,800 elementary schools. Classroom teachers will be selected from a subsample of 1,200 elementary schools.

† Music and visual arts specialists will be selected from the full sample of 1,600 secondary schools.

Estimation and Calculation of Sampling Errors

For estimation purposes, sampling weights reflecting the overall probabilities of selection and adjustments for nonresponse will be attached to each data record. Separate weights will be constructed for the principal and teacher surveys. To properly reflect the complex features of the sample design, standard errors of the survey-based estimates will be calculated using jackknife replication. Under the jackknife replication approach, 50-100 subsamples or "replicates" will be formed in a way that preserves the basic features of the full sample design. A set of estimation weights (referred to as "replicate weights") will then be constructed for each jackknife replicate. Using the full sample weights and the replicate weights, estimates of any survey statistic can be calculated for the full sample and each of the jackknife replicates. The variability of the replicate estimates is used to obtain a measure of the variance (standard error) of the survey statistic. Previous surveys, using similar sample designs, have yielded relative standard errors (i.e., coefficients of variation) in the range of 2 to 10 percent for most national estimates. Similar results are expected for this survey.

B.2. Statistical Methodology

The statistical methodology is described in detail in Section B.1.

B.3. Methods for Maximizing the Response Rate

District superintendents will be informed of the survey and any special district requirements for surveys will be met. For collection of teacher lists that include classroom teachers, respondents will be encouraged to start with existing staff lists and cross off or delete ineligible staff; for lists of arts specialists only, schools will be asked to provide just those few names. Lists will be accepted in any format. For survey collection, principals and teachers have the option of completing the survey on paper or on the web. To minimize the burden on schools, the collection of teacher sampling lists will be coordinated with the collection of the school surveys. Collection and followup activities for the teacher lists and school surveys will be handled by the same staff to minimize the number of contacts to principals and other school staff. Telephone followup for nonresponse, which will be conducted by Westat staff, will begin about 3 weeks after mailout for each type of collection. Experienced telephone interviewers will be trained to conduct followup and will be monitored by Westat supervisory personnel during all interviewing hours. Collection procedures will follow standard FRSS methods developed on previous surveys. Sampling weights will include adjustments for nonresponse at both the school and teacher levels.

B.4. Tests of Procedures and Methods

According to the procedures for NCES quick-response surveys (PEQIS and FRSS), pretests of each survey are conducted with nine respondents to determine what problems respondents might have in providing the requested information and to make appropriate changes to the questionnaire. Responses and comments on the questionnaire are collected by fax and telephone during the pretest, and the results summarized as part of the documentation for the survey. Pretests of the two school-level surveys were conducted with elementary and secondary school principals in February and March 2009. Pretests of the five teacher surveys were conducted in April and May 2009. All tests of procedures involved nine or fewer respondents per survey.

B.5. Reviewing Statisticians

Statistician Adam Chu of Westat (301-251-4326) was consulted about the statistical aspects of the design.

FRSS surveys are sponsored by NCES. Westat is the contractor currently conducting the FRSS surveys for NCES. Westat will draw the samples; mail the questionnaires; collect data by web, mail, and telephone; edit, code, key, and verify the data; and produce tabulations and the survey report.

10