PEELS Supporting statement Part B revised 10-22-08

B. COLLECTION OF INFORMATION USING STATISTICAL METHODS

1. Sampling

PEELS was designed to include a nationally representative sample of 3- through 5-year-olds who were receiving special education services in 2003-04. The study used a two-stage sample design. The first-stage sample included local education agencies (LEAs) or school districts. The second-stage sample included preschoolers selected from lists of the names of eligible children provided by the participating LEAs.

This section describes the different samples used in PEELS and the ways they were selected. The sample selected following the original sample design is called the main sample. This sample was selected by a two-stage design, LEAs at the first stage and children at the second stage. To address nonresponse bias at the LEA level, a nonresponse bias study sample was selected from the nonparticipating LEAs to examine potential differences between the respondents and nonrespondents. The combined sample of the main and the nonresponse study sample is a three-phase sample, where the first phase is the same as the main sample, the second phase is a combined LEA sample comprising the main sample LEAs and the nonresponse study sample LEAs, and the third phase is the sample of children selected from the combined LEA sample. This combined sample was treated as one, as if it had been selected with the original sample design. It is called the amalgamated sample.

The child sample also includes two components. The first was selected using the initial design. The second component was a supplemental sample selected in Wave 2 (2004-05) from a state that was not covered in Wave 1. The amalgamated sample was augmented by adding the supplemental sample and is named the augmented sample. All these efforts were made to produce a truly nationally representative and efficient sample of preschoolers with disabilities. Further information on the sample design is described in Changes in the Characteristics, Services, and Performance of Preschoolers with Disabilities from 2003-04 to 2004-05: PEELS Wave 2 Overview Report <insert link once available>.

2. Procedures for Data Collection

In Waves 1-4, PEELS included several different data collection instruments and activities, including a series of computer-assisted parent interviews, direct one-on-one assessments of participating children, and several mail questionnaires. In Wave 5, only the Child Status Report and child assessment are planned, which are described below. Completion rates for all the Waves 1-4 data collections are provided in exhibit 5.

Exhibit 5. Total unweighted number of respondents and response rate for each PEELS instrument

	Wave 1		Wave 2			Wave 3		Wave 4
Instrument type	Frequency	Response rate (%)		Frequency	Response rate (%)	Frequency	Response rate (%)	Frequency	Response rate (%)
Parent interview	2,802	96		2,893	93	2,719	88	2,488	80
LEA questionnaire	207	84		†	†	†	†	†	†
SEA questionnaire	51	100		†	†	†	†	†	†
Principal/program director questionnaire	852	72		665	77	406	56	†	†
Teacher mail questionnaire	2,287	79		2,591	84	2,514	81	2,502	81
Early childhood teacher questionnaire	2,018	79		1,320	86	346	82	†	†
Kindergarten teacher questionnaire	269	73		957	79	992	81	419	79
Elementary teacher questionnaire	†	†		314	86	1176	81	2083	81
Child assessment	2,794	96		2,932	94	2,891	93	2,632	85
English/ Spanish direct assessment	2,463	97		2,704	96	2,726	93	2,507	85
Alternate assessment only	331	93		228	79	165	93	125	84
Note: Wave 1 frequencies do not include cases in the supplemental sample for which data were imputed. † Not applicable

Child Assessment

The direct one-on-one assessment is designed to obtain information on the knowledge and skills of preschoolers with disabilities in a number of domains, including emerging literacy and early math proficiency. An alternate assessment is available for children who cannot meaningfully participate in the direct assessment. Five direct assessments are planned for PEELS, one each in 2003-04, 2004-05, 2005-06, 2006-07, and 2008-09.

The Wave 5 direct assessment in English is expected to average about 40 minutes and included the following subtests:

• Peabody Picture Vocabulary Test (PPVT) (Dunn and Dunn 1997);

• Woodcock-Johnson III: Letter-Word Identification (Woodcock, McGrew, and Mather 2001);

• Woodcock-Johnson III: Applied Problems (Woodcock, McGrew, and Mather 2001);

• Woodcock-Johnson III: Calculation (Woodcock, McGrew, and Mather 2001);

• Woodcock-Johnson III: Passage Comprehension (Woodcock, McGrew, and Mather 2001); and

• Dynamic Indicators of Basic Early Literacy Skills (DIBELS) Oral Reading Fluency (Good and Kaminski 2002).

For children who cannot complete the direct assessment, the Adaptive Behavior Assessment System-II (ABAS-II) is used as an alternate assessment. The ABAS-II is a checklist of the child’s functional knowledge and skills and is completed by a teacher or other service provider. It assesses children’s functional performance in several areas: communication, community use, functional (pre) academics, school living, health and safety, leisure, self-care, self-direction, social, and work. It also can be used to produce composite scores in conceptual, social, and practical domains. The scaled scores for each of the skill areas are based on a mean of 10 and a standard deviation of 3.

Child status report

The Child Status Report (CSR) was sent to district Site Coordinators prior to the start of data collection in Waves 2, 3, and 4. It will be administered again in Wave 5. The CSR asks for information on the current school location of all the children originally recruited from the district.

Through the CSR, project staff identified children who were no longer living in their sampled districts and initiated procedures for tracking them to their new districts. Parent interviews and mail questionnaires were attempted for all locatable children. Assessments were attempted for children who lived within 50 miles of a PEELS assessor or when a traveling assessor could easily test a child.

3. Maximizing Response Rates

There are two key aspects to maximizing the number of sample members for whom data are collected: minimizing the number of sample members lost through attrition, and completing data collection with the maximum number of sample members who are retained in the sample.

To minimize sample attrition over the years of data collection, aggressive tracking mechanisms have been used to maintain accurate and up-to-date contact information for sample members. Prior to each wave of data collection, the districts are asked to complete a Child Status Report to indicate whether the children in the district participating in PEELS are attending the same school or program. If children are not attending the same school or program, the districts will be asked to provide as much information as they have about where the children transferred.

4. Testing of Instrumentation

No new instruments are being used in Wave 5, so no testing will be required.

5. Individuals Consulted on Statistical Issues

Persons involved in statistical aspects of the design include staff of the government’s design contractors, SRI International, Research Triangle Institute and Westat. Those consulted at these organizations are listed below.

SRI:

Dr. Harold Javitz, Senior Statistician

Center for Health Sciences

Westat

Dr. Hyunshik Lee

Dr. Frank Jenkins

Ms. Annie Lo

In addition, all aspects of the design, sampling plan, and instrumentation were reviewed by the PEELS TWG and Consultants listed in Exhibit 2 of Section A, Justification.

REFERENCES

Dunn, L.M., and Dunn, L.M. (1997). Peabody Picture Vocabulary Test-Third Edition. Circle Pines, MN: American Guidance Services.

Good, R.H., and Kaminski, R.A. (2002). Dynamic Indicators of Basic Early Literacy Skills (6th ed.). Eugene, OR: Institute for the Development of Educational Achievement.

MSPD Evaluation Support Center. (1995). Establishing a Research Agenda for Reauthorization of Idea. Research Triangle Park, NC: Research Triangle Institute and SRI International.

Woodcock, R.W., McGrew, K.S., and Mather, N. (2001). Woodcock-Johnson III Tests of Achievement. Itasca, IL: Riverside Publishing.