Supporting Statement Part B

Section B. Description of Statistical Methodology

B.1. Respondent Universe and Sample Design

Respondent Universe

The respondent universe for the proposed survey on teachers’ use of educational technology will include all full-time classroom teachers teaching at least one regularly scheduled class (other than physical education) in grades K through 12. Teachers that teach only physical education, substitute, itinerant, part-time, and preschool teachers are ineligible for the survey.

The proposed teacher survey is one of three related surveys to be conducted under a nested design involving a sample of districts, schools within districts, and teachers within schools. As described in the following section, the sample of teachers will be selected in two stages. At the first stage of selection, a stratified sample of 2,000 regular schools will be selected from the most recent NCES Common Core of Data (CCD) Public School Universe File. As indicated in Table 1, a total of 85,719 regular schools are included in the 2005-06 CCD universe file, of which 63,647 are elementary schools and 22,072 are secondary schools or schools with combined elementary/secondary grades. Vocational education, special education, alternative/other non-regular schools, and schools operated by the Department of Defense or Bureau of Indian Affairs are ineligible for the survey, as are schools with a high grade of kindergarten or lower, ungraded schools, and schools in the outlying U.S. territories. As described in the following section, the first-stage sample will include approximately 1,000 elementary schools and 1,000 secondary/combined schools.

For the sampled schools, lists of eligible teachers will be compiled and an average of slightly more than two teachers per school will be randomly selected for the survey. Table 1 summarizes the distribution of schools, students, and full-time equivalent (FTE) teachers in the 2005-06 CCD Public School Universe File by level and percent of students eligible for free/reduced price lunch. Note that while the counts in the table are based on 2005-06 CCD data, the more current 2006-07 CCD file will be used for sampling if it is available.

Table 1. Number of schools, students, and teachers in the 2005–06 CCD public school frame by instructional level and percent of students eligible for free/reduced-price lunch*

Instructional level†	Percent of students eligible for free/reduced lunch	Number of schools	Enrollment	Number of teachers (full-time equivalents)
Elementary	Unknown/missing	550	254,475	9,186
	Under 35 percent	23,685	13,197,800	731,974
	35 to 49 percent	10,906	5,734,600	316,059
	50 to 75 percent	15,584	8,277,700	466,797
	75+ percent	12,922	7,183,850	418,503
Secondary/ combined	Unknown/missing	468	234,200	14,235
	Under 35 percent	10,809	9,324,350	557,006
	35 to 49 percent	4,438	3,167,350	192,671
	50 to 75 percent	4,219	2,883,300	177,239
	75+ percent	2,138	1,161,050	73,094
Total		85,719	51,418,675	2,956,764

* The counts in this table are based on the 2005-06 CCD file. If available, the 2006–07 CCD public school universe file will be used to select the sample.

† For sampling purposes, schools with a low grade of 6 or less and a high grade of 8 or less are considered to be "elementary" schools. All other schools are considered to be "secondary/combined" schools.

Statistical Methodology

The sample design for the teachers’ survey will be a two-stage stratified sample in which schools are selected at the first stage, and teachers within the sampled schools are selected at the second stage. A sample of 2,000 schools and 4,000 teachers will be selected, with a 90 percent response rate expected for each stage. For sampling purposes, the primary strata for selecting the first-stage sample of schools will be defined by instructional level (elementary and secondary/other), enrollment size class, and percent of students eligible for free/reduced-price lunch. Table 2 summarizes the proposed allocation of the schools by instructional level and size class. The expected numbers of cooperating schools given in the last column of the table are based on the assumption that 90 percent of the schools will provide useable teacher lists. Note that the proposed school sample will be used for both the school-level and teacher surveys.

Within the primary strata defined by level, enrollment size class, and poverty status (based on the percent of students eligible for free/reduced price lunch), schools will be sorted by type of locale (city, urban fringe, town, rural) and OE region prior to sample selection to induce additional implicit stratification. A sample of 2,000 schools will then be selected from the sorted frame with probabilities proportionate to size (PPS), where the measure of size is the square root of the estimated number of full-time equivalent (FTE) teachers in the school. Ordinarily, the count of FTE teachers (rather than the square root of the FTE teacher count) would be the preferred sampling measure of size for selecting schools in a two-stage design. However, since data on educational technology will also be obtained at the school-level through the related school survey, the use of the square root will reduce the variation in the resulting school weights within a stratum, and thus be somewhat more efficient for estimating school characteristics. Note that the required measure of size will be imputed for schools with missing FTE data by applying the average enrollment-to-FTE teacher ratio for schools in the same level, type-of-locale, and size class category to the enrollment of the school with the missing FTE teacher count.

To select the teachers for the survey, each sampled school will be requested to provide a comprehensive list of their eligible teachers. Assuming that teacher lists are obtained for 90 percent of the sampled schools (prior surveys using a similar two-stage sampling approach have achieved cooperation rates of 87-91 percent), an initial sample of 2,000 schools will yield about 1,800 participating schools. An average of slightly more than two teachers will then be selected randomly from each participating school, yielding a total initial sample size of about 4,000 teachers (2,000 elementary teachers and 2,000 secondary/combined teachers). Assuming a teacher-level response rate of 90 percent, the initial sample of 4,000 teachers will yield about 3,600 completed teacher questionnaires.

Table 2. Proposed allocation of the first-stage school sample for the educational technology survey by level and enrollment size class

Instructional level	Enrollment size class	Number of schools to be sampled	Expected number of cooperating schools*
1. Elementary	1. <300	173	156
	2. 300 to 499	314	282
	3. 500 to 599	150	135
	4. 600 to 749	158	142
	5. 750+	205	185
2. Secondary/	1. <300	165	148
combined	2. 300 to 499	135	122
	3. 500 to 999	272	245
	4. 1,000 to 1,499	175	158
	5. 1,500+	253	227
Total		2,000	1,800

* Assumes 90 percent of schools provide teacher lists.

Expected Levels of Precision

Table 3 summarizes the approximate sample sizes and standard errors to be expected under the proposed design for the teacher survey. Note that in addition to unequal weighting design effects ranging from 1.05 to 1.17 (due to the disproportionate allocation of the sample to strata), the standard errors in Table 3 also reflect an overall “clustering design effect” of 1.10 resulting from the selection of more than one teacher per school. Since the sample sizes in Table 3 are based on preliminary tabulations of the CCD file, the actual sample sizes may differ from those shown. Also, note that the sample sizes represent the expected numbers of completed questionnaires, and not the initial numbers of schools and teachers to be selected. The standard errors in Tables 3 can be converted to 95 percent confidence bounds by multiplying the entries by 2. For example, as can be seen in Table 3, an estimated proportion of the order of 20 percent (P = 0.20) for elementary school teachers would be subject to a margin of error of ±2.0 percent at the 95 percent confidence level.

Table 3. Expected sample sizes (number of completed questionnaires) and corresponding standard errors for the teacher survey on educational technology, by selected analytic domains

Subset of sample	Expected sample size*	Standard error† of an estimated proportion equal to ...
		P = 0.20	P = .33	P = .50
Total sample (all teachers)	3,600	0.008	0.009	0.009
Teachers by type of locale
City	943	0.015	0.017	0.018
Urban fringe	1,282	0.013	0.015	0.016
Town	340	0.025	0.029	0.031
Rural	1,035	0.014	0.017	0.018
Teachers by OE region
Northeast	724	0.017	0.020	0.021
Southeast	829	0.016	0.019	0.020
Central	913	0.015	0.018	0.019
West	1,133	0.013	0.016	0.017
Elementary school teachers	1,800	0.010	0.012	0.013
Secondary/combined school teachers	1,800	0.010	0.012	0.013

*Number of completed teacher questionnaires.

^†Assumes a clustering design effect of 1.10 and an unequal weighting design effect ranging from 1.05 to 1.17 depending on subgroup.

Estimation and Calculation of Sampling Errors

For estimation purposes, sampling weights reflecting the overall probabilities of selection will be attached to each data record. These weights will include upward adjustments for nonresponse at both the school and teacher levels. To properly reflect the complex features of the sample design, standard errors of the survey-based estimates will be calculated using jackknife replication. Under the jackknife replication approach, 50 subsamples or “replicates” will be formed in a way that preserves the basic features of the full sample design. A set of estimation weights (referred to as “replicate weights”) will then be generated for each jackknife replicate. Using the full sample weights and the replicate weights, estimates of any survey statistic can be calculated for the full sample and each of the 50 jackknife replicates. The mean square error of the replicate estimates then provides a measure of the variance (standard error) of the survey statistic. Previous surveys, using similar sample designs, have yielded relative standard errors (i.e., coefficients of variation) in the range of 2 to 10 percent for most national estimates. Similar results are expected for this survey.

B.2. Statistical Methodology

The statistical methodology is described in detail in Section B.1.

B.3. Methods for Maximizing the Response Rate

District superintendents will be informed of the survey and any special district requirements for surveys will be met. For collection of teacher lists, respondents will be encouraged to start with existing staff lists and cross off or delete ineligible staff. Lists will be accepted in any format. For survey collection, teachers have the option of completing the survey on paper or on the web. Telephone followup for nonresponse, which will be conducted by Westat staff, will begin about 3 weeks after mailout for each type of collection. Experienced telephone interviewers will be trained to conduct followup and will be monitored by Westat supervisory personnel during all interviewing hours. Collection procedures will follow standard FRSS methods developed on previous surveys. For example, on the 1999 FRSS Survey on Public School Teachers’ Use of Computers and the Internet, the overall response rate was 83 percent (91 percent for the list collection multiplied by 91 percent for the teacher questionnaire). Sampling weights will include adjustments for nonresponse at both the school and teacher levels.

B.4. Tests of Procedures and Methods

According to the procedures for NCES quick-response surveys (PEQIS and FRSS), a pretest with nine institutions was conducted prior to OMB review to determine what problems respondents might have in providing the requested information and to make appropriate changes to the questionnaire. Responses and comments on the questionnaire were collected by telephone during the pretest, and the results summarized as part of the documentation for the survey. No tests of procedures involving ten or more respondents are planned for the survey.

B.5. Reviewing Statisticians

Statistician Adam Chu of Westat (301-251-4326) was consulted about the statistical aspects of the design.

FRSS surveys are sponsored by NCES. Westat is the contractor currently conducting the FRSS surveys for NCES. Westat will draw the samples; mail the questionnaires; collect data by web, mail, and telephone; edit, code, key, and verify the data; and produce tabulations and the survey report.