Justification

NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Volume I

SUPPORTING STATEMENT

FOR

WAVE 2 SUBMITTAL FOR NAEP 2010

(PART OF 2008–2010 SYSTEM CLEARANCE PROPOSAL

OMB# 1850-0790)

SD (Student with Disabilities) Worksheets/Questionnaires – Grades 4, 8, 12

ELL (English Language Learner) Worksheets/Questionnaires– Grades 4, 8, 12

June 23, 2009

Contents

Explanation and Burden Information for This Submittal 3

2010 Wave 2 Burden Information 4

Estimated Burden for NAEP 2010 Assessments Contained in This Submittal 5

Overview of NAEP 2008–2010 Assessments 6

How, By Whom, and For What Purpose the Data Will Be Used 6

Estimates of Costs to the Federal Government 7

Information for the 2010 Components Contained in Wave 2 Submittal 8

Explanation and Burden Information for This Submittal

This document contains supplemental information pertaining to the 2008–2010 National Assessment of Educational Progress (NAEP) System Clearance proposal. The System Clearance package was submitted in January 2007 and approved in May 2007 with OMB #1850-0790. The terms of clearance for OMB approvals state that each subsequent submittal activity under the System Clearance is to be submitted to OMB.

This submittal contains burden information and the actual background worksheets/questionnaires for the following components of the 2010 assessments:

SD (Student with Disabilities) Worksheets/Questionnaires–Grades 4, 8, 12

ELL (English Language Learner) Worksheets/Questionnaires–Grades 4, 8, 12

These specific worksheets/questionnaires are the second group (Wave 2) of questionnaires/data collection instruments submitted for approval for usage in 2010. The Wave 1 submittal contained the bulk of the questionnaires for the 2010 assessments which consisted of the following:

• Student (grades 4, 8, and 12), teacher (grades 4 and 8), and school (grades 4, 8, and 12) questionnaires for the operational assessments in civics, geography, and U.S. history.

• Student (grades 4, 8, and 12), teacher (grades 4 and 8), and school (grades 4, 8, and 12) questionnaires for the pilot assessments for writing. The student grade 4 questionnaires will be administered in a paper-and-pencil version; the grades 8 and 12 student questionnaires will be administered on computer.

• Student (grades 4 and 8) questionnaires for a mathematics accessible booklet study at grades 4 and 8.

2010 Wave 2 Burden Information

The Wave 2 submittal contains the worksheets/questionnaires (and related instructions) being used in the 2010 SD and ELL information collection process. SD and ELL worksheets/questionnaires are completed by school personnel who are most knowledgeable about students identified as SD or ELL. The response burden for these SD and ELL questionnaires is 0.33 hours (20 minutes) per student identified as SD or ELL. Since the estimated number of identified SD and ELL students per grade is different for each of the grades the burden estimates per grade are different. There are two sets of each instrument included—one for the paper-and-pencil assessments and one for the computer-delivered assessments (grades 8 and 12 writing).

The Wave 2 burden information (see chart on page 5) is listed by grade and includes the following:

• Grade 4

At grade 4, the estimated number of SD and ELL questionnaires to be completed is 1,050 resulting in a burden of 350 hours.

• Grade 8

At grade 8, the estimated number of SD and ELL questionnaires to be completed is 1,330 resulting in a burden of 443 hours.

• Grade 12

At grade 12, the estimated number of SD and ELL questionnaires to be completed is 1,225 resulting in a burden of 408 hours.

Wave 2 Questionnaires - Single Survey Burden Times

SD worksheets/questionnaire (grades 4, 8, 12)	20 minutes per identified student (completed by school personnel most knowledgeable about students identified as SD)
ELL worksheets/questionnaire (grades 4, 8, 12)	20 minutes per identified student (completed by school personnel most knowledgeable about students identified as ELL)

Wa Wave 2

Estimated Burden for NAEP 2010 Assessments Contained in This Submittal
by Grade Level

Components

SD/ELL (# of students for which school personnel will complete worksheets/questionnaires)

SD/ELL Burden

Total Burden

(in hours)

(in hours)

Grade 4

 

 

 

Civics

210

70

70

Geography

210

70

70

U.S. History

210

70

70

Writing (pilot)

315

105

105

Mathematics (Accessible Book Study)

105

35

35

Totals

1,050

350

350

 

 

 

 

Grade 8

 

 

 

Civics

280

93

93

Geography

280

93

93

U.S. History

350

117

117

Writing (pilot)

315

105

105

Mathematics (Accessible Book Study)

105

35

35

Totals

1,330

443

443

 

 

 

 

Grade 12

 

 

 

Civics

280

93

93

Geography

280

93

93

U.S. History

350

117

117

Writing (pilot)

315

105

105

Totals

1,225

408

408

 

 

 

 

Grand Totals

3,605

1,201

1,201

Overview of NAEP 2008–2010 Assessments

The following broad overview of the 2008–2010 NAEP assessments was included as part of the initial System Clearance submittal. The National Assessment Governing Board (the Governing Board) determines NAEP policy and the assessment schedule, and future Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology and assessment process will remain constant. In the 2010 assessment year, questionnaires will be administered to students at grades 4, 8, and 12; to teachers at grades 4 and 8; and to school administrators at grades 4, 8, and 12.

How, By Whom, and For What Purpose the Data Will Be Used

In the original request to the Office of Management and Budget (OMB) for system clearance, the National Center for Education Statistics (NCES) requested approval of the data-gathering instruments to be used in the 2008–2010 national and state assessments.

This submittal contains the SD and ELL worksheets/questionnaires which will be completed by school personnel pertaining to those students identified as SD and/or ELL and who were selected to participate in the 2010 assessments.

The purpose of NAEP is to gather data on student achievement in the subject areas assessed for use in monitoring education progress; combined with the program’s increasing visibility, the program must develop the most reliable and valid instruments possible. To do so, NAEP has employed the following strategies in recent assessment cycles:

1. Small-scale pilot testing of new materials and test administration techniques,

2. Pilot testing items to determine which items best measure the constructs under consideration, and

3. Full-scale operations.

Background item development follows the same pattern as that of cognitive item development, although fewer items with less duplication are piloted and the resulting data are used to refine the questions. General guidance for item development is provided by the Governing Board. NCES develops the questionnaires and requests input from the Governing Board prior to submission to OMB at two stages of development: (1) prior to pilot testing and (2) after NCES and its contractors make selections for the operational assessment based on pilot data.

The questions are designed to provide:

• the information for disaggregating data according to categories specified in the legislation,

• contextual information that is subject specific (e.g., reading, mathematics) and has an impact on and known relationship to achievement, and

• policy-relevant information specified by NCES and the Governing Board.

In all assessment years, questionnaires are generally administered to students at grades 4, 8, and 12; teachers at grades 4 and 8; and school administrators at grades 4, 8, and 12. SD and ELL worksheets/questionnaires are completed by the school personnel most knowledgeable about how the student is tested and/or most knowledgeable about the student’s disability, English language proficiency or grade-level performance (i.e., school testing coordinators, IEP coordinators, IEP/special education teachers, etc.).

Estimates of Costs to the Federal Government

Administration Cost Estimates

The Wave 1 estimate information included administration activities that would also encompass the SD and ELL components.

The following estimates apply to the 2010 administration costs:

• $3.0 million for the printing, packaging, and distribution phases of the administrations.

• $2.1 million for the cost of the 2010 field supervisors and assessment administrators to go into schools to administer the assessment, including travel expenses. This is not broken down by waves here because materials for the two waves of OMB submission are used concurrently during field administration.

• $450,000 for web operations and maintenance costs related to support of the 2010 assessments.

• Given that Wave 1 (the first submittal of 2010 questionnaires which includes all questionnaires except the SD and ELL questionnaires) and Wave 2 (for 2010 includes SD and ELL questionnaires) are in the field at the same time, the cost for administering Wave 1 is estimated to be about 95 percent of the total cost of administering the assessment, or about $5.275 million. The Wave 2 component’s costs are estimated to be $275,000.

Information for the 2010 Components Contained in Wave 2 Submittal

SD (Students with Disabilities) and ELL (English Language Learners)

NAEP encourages the inclusion of all students who can meaningfully participate in the assessment, including those with disabilities and English language learners. In order to obtain a complete picture of educational progress for all students, it is important to collect supplemental information on students in the sample who have been identified as having a disability or are English language learners. Students with disabilities include those who have an individual education plan (IEP), section 504 plan, or equivalent documentation.

In 2008 and 2009 the SD and ELL questionnaires received extensive review and revision by the NAEP State Coordinator Inclusion Special Interest Group (SIG), the NCES Accommodations and Inclusions Coordinating Council, and NAEP test developers and field staff. The 2009 versions of these questionnaires incorporated revisions to ensure a more efficient collection of valid student-level data. A flowchart format was used to reduce the number of questions required to complete the questionnaires. Since schools may have several students identified as SD and/or ELL, this format was aimed at reducing overall school burden.

However, field staff reported that school personnel still needed assistance in filling out the questionnaires completely and accurately and found the process burdensome. As a result, a new approach will be implemented for collecting the 2010 SD and ELL data. Instead of having a separate questionnaire booklet, including a fairly involved accommodations chart, for each SD- and/or ELL-identified student, there will now be a worksheet/questionnaire with one student per line. Since previously some school personnel could have more than 50 questionnaires, the new process is aimed at reducing the burden of completing so many individual questionnaire booklets. Student information, for as many as 10 students, can be collected on one worksheet/questionnaire with the new process. The NAEP field administrators who perform the pre-assessment visits will work with school personnel to have the schools accurately complete the SD and ELL worksheets/questionnaires. The new worksheets/questionnaires are designed to make the process less burdensome for determining if a student can participate in NAEP and with which accommodations¹.

Sample worksheets/questionnaires and instructions are in Volume II. The SD instructions are followed by the SD worksheet/questionnaire and the ELL instructions are followed by the ELL worksheet/questionnaire. Because the accommodations allowed are somewhat different for students taking the computer-delivered versions of the writing assessments at grades 8 and 12, there are separate worksheets/questionnaires and instructions for those in Volume II.

1 The estimated burden is based on previous years’ experience. Since this data collection will be using a new format, we will gather information from field staff to inform future burden estimates for SD and ELL data collection.

2010 Wave 2 Supporting Statement 2