NAEP 2011 Wave I Supporting Statement

Att_NAEP 2011 Wave 1 Vol I.docx

National Assessment of Education Progress (NAEP) 2011-13 System Clearance

NAEP 2011 Wave I Supporting Statement

OMB: 1850-0790

Document [docx]
Download: docx | pdf

NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Shape1


Volume I

SUPPORTING STATEMENT

FOR

WAVE 1 SUBMITTAL FOR 2011


(PART OF 2011–2013 SYSTEM CLEARANCE PROPOSAL

OMB# 1850-0790)


Student Grade 4 Core Questions

Student Grade 4 Reading Questions

Student Grade 4 Mathematics Questions

Teacher Grade 4 Questions (Background, Education, and Training (BET), Reading, Mathematics)

School Grade 4 Questions (School Characteristics and Policies (SCP), Reading, Mathematics, Charter School)



Student Grade 8 Core Questions

Student Grade 8 Reading Questions

Student Grade 8 Mathematics Questions

Student Grade 8 Science Questions

Teacher Grade 8 Reading Questions (Background, Education, and Training (BET), Classroom Organization and Instruction (COI) Reading)

Teacher Grade 8 Mathematics Questions (Background, Education, and Training (BET), Classroom Organization and Instruction (COI) Mathematics)

Teacher Grade 8 Science Questions (Background, Education, and Training (BET), Classroom Organization and Instruction (COI) Science)

School Grade 8 Questions (School Characteristics and Policies (SCP), Reading, Mathematics, Science, Charter School)



Student Grade 12 Core Questions

Student Grade 12 Economics Questions





March 31, 2010

Contents



1. Explanation and Burden Information for This Submittal

This document contains supplemental information pertaining to the 2011–2013 National Assessment of Educational Progress (NAEP) System Clearance proposal. The system clearance package was submitted in November 2009 and approved in April 2010 (OMB #1850-0790 v.26). The terms of clearance for OMB approvals state that each subsequent submittal activity under the system clearance is to be submitted to OMB with a 60-day federal register notice waver.

This submittal contains burden information and the background questionnaires (also referred to as noncognitive questions) for the following components of the 2011 NAEP assessments:

Grade 4

Student Grade 4 Core Questions

Student Grade 4 Reading Questions

Student Grade 4 Mathematics Questions

Teacher Grade 4 Questions: Background, Education, and Training (BET); Reading; Mathematics

School Grade 4 Questions: School Characteristics and Policies (SCP); Reading; Mathematics; Charter School


Grade 8

Student Grade 8 Core Questions

Student Grade 8 Reading Questions

Student Grade 8 Mathematics Questions

Student Grade 8 Science Questions

Teacher Grade 8 Reading Questions: Background, Education, and Training (BET); Classroom Organization and Instruction (COI) Reading

Teacher Grade 8 Mathematics Questions: Background, Education, and Training (BET); Classroom Organization and Instruction (COI) Mathematics

Teacher Grade 8 Science Questions: Background, Education, and Training (BET); Classroom Organization and Instruction (COI) Science

School Grade 8 Questions: School Characteristics and Policies (SCP); Reading; Mathematics; Science; Charter School

Grade 12

Student Grade 12 Core Questions

Student Grade 12 Economics Questions


These specific questionnaires are the initial group (Wave 1) of questionnaires submitted for approval for usage in 2011. A second group of 2011 questionnaires (Wave 2) will be submitted later this year. Wave 2 will contain student, teacher, and school questionnaires for the writing assessment; student, teacher, and school questions for the National Indian Education Study (NIES); special study questions; and students with disabilities (SD) and English language learner (ELL) worksheets.

2. 2011 Wave 1 Burden Information

The Wave 1 submittal contains the grades 4, 8, and 12 core questions; the reading, mathematics, science, and economics student questions; the grade 4 teacher and school questions; and the teacher and school grade 8 reading, mathematics, and science questions. Some special studies will also be administered in 2011 and their background questions will be included in Wave 2. Final decisions regarding the 2011 special studies are not available at this time, but will be included in the Wave 2 submittal.

The Wave 1 burden information is listed by grade (see chart on page 6) and includes the following categories of questions:

Students - Students in 4th, 8th and 12th grades complete assessment booklets that commonly contain two 25-minute cognitive blocks, followed by two background (noncognitive) question sections, which require a total of 15 minutes to complete. The first background question block contains core questions, many of which are related to demographic information. The second background block contains subject-specific questions.

Teachers - The teachers of 4th- and 8th-grade students participating in NAEP will be asked to complete questions about their teaching background, education, training, and classroom organization and instruction. Fourth-grade teacher burden is estimated to be 30 minutes while 8th- and 12th-grade teacher burden is 20 minutes. Fourth-grade teachers often have multiple subject-specific sections to complete, which results in the additional burden estimate.

Principals/Administrators - The school administrators in the sampled schools will be asked to complete a questionnaire. The school core questions are designed to measure school characteristics and policies that research has shown are highly correlated with student achievement. In addition, there is a section with subject-specific questions focusing on curriculum and instructional services. Finally, there is a section with charter school questions, to gather information for schools which identify themselves as charter schools.

In addition to the questionnaires, burden information is provided for the following:

E-Filing and Pre-Assessment Visit - Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically at the school, district, or state level. If done at the school or district level, some burden will be incurred by school personnel. The Pre-Assessment Visit is the opportunity for the NCES contractor field staff to meet with the school personnel to review procedures and logistics for the upcoming assessment. The pre-assessment visit requires one hour and the e-filing burden is also estimated at an hour. However, the e-filing process is only done in a subset of schools (approximately 38% in recent years), which is taken into account when computing the e-filing burden.

The total combined burdens for the Wave 1 materials are:

Respondent Category

Number of Respondents

Number of Hours of Burden

Student

897,540

224,386

Teacher

62,884

26,563

School

15,741

29,583

Totals

976,165

280,532


See the following tables for a summary of Wave 1 burden estimates.

Wave 1 Questions - Single Survey Burden Times

Grade 4


Student Grade 4 Core + Reading Questions

15 minutes

Student Grade 4 Core + Mathematics Questions

15 minutes

Teacher Grade 4 Questions

30 minutes

School Grade 4 Questions

30 minutes

Grade 8


Student Grade 8 Core + Reading Questions

15 minutes

Student Grade 8 Core + Mathematics Questions

15 minutes

Student Grade 8 Core + Science Questions

15 minutes

Teacher Grade 8 Reading Questions

20 minutes

Teacher Grade 8 Mathematics Questions

20 minutes

Teacher Grade 8 Science Questions

20 minutes

School Grade 8 Questions

30 minutes

Grade 12


Student Grade 12 Core + Economics Questions

15 minutes

Estimated Burden for NAEP 2011 Assessments Contained in This Submittal (Wave 1)

By Grade Level




3. Overview of NAEP 2011 Assessments

The following broad overview of the 2011 NAEP assessments was included as part of the 2011–2013 System Clearance submittal. The National Assessment Governing Board (the Governing Board) determines NAEP policy and assessment schedule, and future Governing Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology and assessment process will remain consistent. In the 2011 assessment year, questions will be administered to students at grades 4, 8, and 12; to teachers at grades 4 and 8; and to school administrators at grades 4, 8, and 12.

The 2011 data collection activities described in this Wave 1 submission consist of the following:

  • National, state, and urban district assessment in reading at grades 4 and 8;

  • National, state, and urban district assessment in mathematics at grades 4 and 8;

  • National and state in science at grade 8;

  • Pilot assessments for 2013 reading and mathematics at grades 4 and 8; and

  • Pilot assessment for 2012 economics at grade 12.


Committees that helped define the cognitive and background questions are listed in Appendix A.

4. How, By Whom, and For What Purpose the Data Will Be Used

The following broad overview of the use of the data from the 2011 NAEP assessments is abbreviated from the 2011–2013 System Clearance submittal. The purpose of NAEP is to collect and report assessment data on student achievement in the subject areas assessed for use in monitoring education progress. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. Guidance for what is asked in the questions is set by the Governing Board. NCES is responsible for developing the questions and for selecting the final set of questions. The questions are designed to (a) provide the information for disaggregating data according to categories specified in the legislation, (b) provide contextual information that is subject specific (e.g. reading, mathematics) and has an impact and known relationship to student achievement, and (c) provide policy-relevant information specified by the Governing Board.


In the original request to the Office of Management and Budget (OMB) for system clearance, NCES requested approval of the data-gathering instruments to be used in the 2011–2013 national and state assessments. This submittal applies to the first set of questions containing the questions for the students, teachers, and school administrators that will be submitted as part of the 2011 assessments. A second submittal, with the remainder of the questions for the 2011 assessments, will be submitted in approximately two months. An expanded description of the purposes of the data is provided in the System Clearance submittal.

5. Estimates of Costs to the Federal Government

Administration Cost Estimates

The following estimates apply to the 2011 administration costs:

  • $8.1 million for the printing, packaging, and distribution phases of the administrations.

  • $20.5 million for the cost of the 2011 field supervisors and assessment administrators to go into schools to administer the assessment, including travel expenses.

  • $470,000 for web support and maintenance related to the 2011 assessments.

Given that Wave 1 (this first submittal of 2011 questions which includes core, reading, mathematics, and science-related questions) and Wave 2 (including writing, NIES, special studies, and SD- and ELL-related worksheets) are in the field at the same time, the costs are broken out across the two waves in 2011 as follows:

  • the bulk of the printing, packaging, and distribution costs occur in Wave 1, which comprises 80–90% of the printing volume;

  • the field supervision and assessment costs are also spread over the entire administration, with the bulk associated with the Wave 1 materials; and

  • for web support and maintenance, the majority of the costs are associated with the computer-based writing assessment, which will be included in the Wave 2 material.

Thus, the approximate breakdown of the total $29.07 million for 2011 is allocated as follows:

  • Wave 1 – $23.80 million

  • Wave 2 – $5.27 million

6. Information Pertaining to the 2011 Assessments:

Student Questionnaires

Core

The core questions for 2011 are comprised of a subset of prior core questions.


Reading

The reading background questions will be a subset of the questions administered in the 2009 operational assessments. New frameworks for reading were implemented in 2009 and the questions will continue the trend for 2011. The reading pilot assessments administered in 2011 will also use the same 2009 reading background questions.


Mathematics

Like reading, the mathematics background questions will a subset of the questions administered in the 2009 operational assessments. Also, the 2011 pilot mathematics assessments will use the same 2009 mathematics background questions.


Science

The science background questions will be the same as the questions administered in the 2009 operational assessment. Note that no pilot assessment will be given in science in 2011.


Economics

The economics background questions (pilot assessment only) will be a subset of the questions administered in the 2006 assessment.

Teacher Questionnaires

Teachers of grades 4 and 8 students participating in the main NAEP assessments are each administered a questionnaire. No teacher questionnaire will be administered at grade 12. At grade 4, a single teacher questionnaire will be administered across all subjects assessed. At grade 8, teacher questionnaires will be administered for each subject-specific area. Teacher questionnaires are also available online for completion by teachers, if desired. Historically, approximately 10% of teachers complete the questionnaire online.

The NAEP 2011 teacher questionnaires are composed of multiple parts: background, education, and training (BET); and one or more subject-specific classroom organization and instruction (COI) components. Given the structure of elementary school, the grade 4 teacher questionnaire contains multiple subjects in the subject-specific section. The grade 8 teacher questionnaires focus on a single subject, resulting in multiple forms of questionnaires.

The following table depicts the layout for the 2011 teacher questionnaires included in the Wave 1 submittal:

Teacher Questionnaires for the NAEP 2011 Assessments

Grade

Type

Section 1

Section 2

Section 3

4

Reading, Mathematics

BET

COI (Reading)

COI (Mathematics)

8

Reading

BET

COI (Reading)


8

Mathematics

BET

COI (Mathematics)


8

Science

BET

COI (Science)


BET – Background, education, and training

COI – Classroom organization and instruction

School Questionnaires

A single school questionnaire will be administered at each grade. Each grade-specific questionnaire will include a section on school characteristics and policies (SCP), a section for each subject being assessed at that grade (at grade 12, pilot economics will not be included), and a section with charter school questions as the last section in the school questionnaire. Note that the charter school section is only completed if the school is a charter school. School questionnaires, similar to teacher questionnaires, are also offered in an online version. Historically, approximately 10% of school questionnaires are completed online.

The following table depicts the layout for the 2011 school questionnaires included in the Wave 1 submittal:

School Questionnaires for the NAEP 2011 Assessments

Grade

Section 1

Section 2

Section 3

Section 4

Section 4

Section 5

4

SCP

Reading

Mathematics

Charter school



8

SCP

Reading

Mathematics

Science

Writing*

Charter school

SCP – School characteristics and policies

* Writing will be included in Wave 2 of the 2011 NAEP submission.

APPENDIX A: Committee Lists


NAEP Background Variable Committee


Name Affiliation

Patricia Alexander University of Maryland


Arthur Applebee University at Albany


Claudia Buchman Ohio State University


Lizanne Destefano University of Illinois


Robert Hauser University of Wisconsin-Madison


Kathleen Heid Penn State University


Henry Levin Columbia University


Peter Levine University of Maryland


Linda Levstik University of Kentucky


Samuel Lucas University of California-Berkeley


Senta Raizen West-Ed


Andrew Sum Northeastern University


William Walstad University of Nebraska-Lincoln


NAEP Reading Committee


Name Affiliation

Patricia Alexander University of Maryland


Christine Carriere Carl Von Linne School


Mary Beth Curtis Lesley University


Patsy Dunton Maine Department of Education


Arzie Galvez Office of Curriculum, Instruction and School Support,

Los Angeles, CA


Evan Lefsky Lake County Schools


Pamela Mason Harvard Graduate School of Education


Joe McGonegal Catholic Memorial High School


Margaret McKeown University of Pittsburgh


Katie Oliver Morgan Park Middle School


Timothy Shanahan University of Illinois


Alfred Tatum University of Illinois at Chicago


Lisa White Plymouth Public Schools

Junko Yokota National-Louis University


NAEP Mathematics Committee


Name Affiliation

David J. Brancamp Nevada Department of Education


Sareeta Carter Benjamin Banneker Academic High School


Herb Clemens Ohio State University


Marcia Cole District of Columbia Public Schools 


Carl Cowen Indiana University-Purdue University, Indianapolis


Bradford R. Findell Ohio Department of Education


Diana Rivera Garcia Departamento De Educación Puerto Rico


Anne Gonzales South Gate Middle School


Kathleen Heid Penn State University


Judy Kinley Morris Elementary School


Carole Phillip Alice Deal Middle School


Penny Roberts Greenville Elementary School


Daren Starnes The Lawrenceville School


JT Sutcliffe St. Mark’s School of Texas


Elizabeth Sweeney Boston Public Schools

NAEP Science Committee



Name Affiliation

Alicia Cristina Alonzo Michigan State University


Charles W. Anderson Michigan State University

Susan Craft Hanahan High School


George Deboer American Association for the Advancement of Science


Alex Decaria Millersville University


Ibari Igwe Paul Public Charter School


Crystal Lovell Lawrence Township Public Schools


Ellen Mingione Delaware Department of Education


Amy Pearlmutter Littlebrook Elementary School


Senta Raizen West-Ed


Cary Sneider Portland State University

Mary Thandi Buthelezi Wheaton College


David White Vermont Department of Education


Gerald Wheeler National Science Teacher Association


NAEP Writing Committee


Name Affiliation

Arthur Applebee University at Albany, SUNY


Diane August Center for Applied Linguistics


Margretta Browne Montgomery County Public Schools

Elyse Eidman-Aadahl University of California


Nikki Elliot-Schuman Office of Superintendent of Public Instruction, WA


Rayna Goldfarb Abraham Lincoln High School


Carol Jago California Reading and Literature Project, UCLA


Charles Macarthur University of Delaware


Michael Mccloskey Johns Hopkins University


Norma Mota-Altman San Gabriel High School


Sandra Murphy University of California Davis


Drew Sterner Tamanend Middle School


Victoria Young Texas Education Agency


NAEP Economics Committee


Name Affiliation

Kris Bertelsen St. Charles High School


Stephen Buckles Vanderbilt University


Steven L. Cobb Center for Economic Education, Denton, TX


Jaime Festa-Daigle Lake Havasu High School


Julie Heath University of Memphis


Richard Macdonald St. Cloud State University


Andrea Morgan Oregon Department of Education


Kevin Smith Renaissance High School

William Walstad University of Nebraska-Lincoln


File Typeapplication/vnd.openxmlformats-officedocument.wordprocessingml.document
File TitleNATIONAL ASSESSMENT OF
Authorjoconnell
File Modified0000-00-00
File Created2021-02-02

© 2024 OMB.report | Privacy Policy