Justification

NATIONAL ASSESSMENT OF

EDUCATIONAL PROGRESS

Volume I

SUPPORTING STATEMENT

FOR

WAVE 1 SUBMITTAL FOR 2012

OMB# 1850–0790 v.30

Student Age 9 Long-Term Trend Core + Reading Questions

Student Age 9 Long-Term Trend Core + Mathematics Questions

Student Age 13 Long-Term Trend Core + Reading Questions

Student Age 13 Long-Term Trend Core + Mathematics Questions

Student Age 17 Long-Term Trend Core + Reading Questions

Student Age 17 Long-Term Trend Core + Mathematics Questions

March 9, 2011

1. Explanation for This Submittal

This document contains supplemental information pertaining to the 2011–2013 National Assessment of Educational Progress (NAEP) System Clearance proposal. NAEP is administered by the National Center for Education Statistics (NCES) in the Institute for Education Sciences of the U.S. Department of Education. The National Assessment Governing Board (henceforth referred to as the Governing Board) sets policy for NAEP and determines the content framework for each assessment. The system clearance package was submitted in November 2009 and approved in April 2010 (OMB #1850-0790 v.26). The terms of clearance for OMB approvals state that each subsequent submittal activity under the system clearance is to be submitted to OMB.

This Wave 1 submittal contains burden information and the actual background (also referred to as non-cognitive) questions for the following components of the 2012 assessments^1,2,3:

Age 9

Student Age 9 Long-Term Trend Core⁴ + Reading Questions

Student Age 9 Long-Term Trend Core + Mathematics Questions

Age 13

Student Age 13 Long-Term Trend Core + Reading Questions

Student Age 13 Long-Term Trend Core + Mathematics Questions

Age 17

Student Age 17 Long-Term Trend Core + Reading Questions

Student Age 17 Long-Term Trend Core + Mathematics Questions

These specific questions are the initial group (Wave 1) of questions submitted for approval for usage in 2012. A second group of questions (Wave 2) will be submitted as part of the overall 2012 OMB submission. Wave 2 will contain the student, teacher, school, and department chair questions to be administered as part of main NAEP⁵, as well as the students with disabilities (SD) and English language learner (ELL) worksheets to be used with both LTT and main NAEP.

2. Overview of Wave 1 NAEP 2012 Assessments

Much of the following broad overview of the 2012 NAEP assessments was included as part of the 2011–2013 System Clearance submittal. The Governing Board determines NAEP policy and assessment schedule, and future Governing Board decisions may result in changes to some aspects of an assessment (e.g., which subjects are assessed in which years). However, the overall methodology and assessment process will remain consistent. The Governing Board is responsible for selecting and approving all of NAEP’s background questions, as well as the cognitive items. These surveys are currently conducted by an alliance of organizations under contract with the U.S. Department of Education. The national surveys contain two kinds of questions – “cognitive” or assessment questions, which measure academic student knowledge of an academic subject; and “background” or survey questions which gather information on demographic as well as classroom instructional procedures.

NAEP consists of two assessment programs: the NAEP Long-Term Trend (LTT) assessment and the main NAEP assessment. In 2012, NAEP will administer LTT reading and mathematics assessments to students at ages 9, 13, and 17 (operational and pilot). The LTT assessments are designed to give information on the changes in academic performance since the 1970’s. Measuring trends of student achievement or change over time requires the precise replication of past procedures. LTT assessments are administered every four years to students at ages 9, 13, and 17, in mathematics and reading. Student performance results are reported for the nation, but not for states or districts. For the LTT assessments, students answer 45 minutes of cognitive questions (three consecutive 15-minute blocks of questions) and a single background question section containing core questions and a few subject-specific questions. The background question section follows the cognitive sections.

Committees that helped define the cognitive and background questions pertaining to the Wave 1 subjects are listed in Appendix A.

3. 2012 Wave 1 Burden Information

The total combined burden for the Wave 1 materials is as follows:

Table 1: Total burden for Wave 1

Respondent Category	Number of Respondents	Estimated Burden Time per Respondent	Total Burden Hours
Student	52,050	0.25	13,013
School pre-assessment visit and E-filing	1,475	2.255	3,326
Total	53,525	-	16,339

The specific per respondent and age category burden times are included in Tables 2 and 3 and include the following categories:

Students - The LTT background questions are comprised of a single section of questions which contain mostly demographic questions along with a few subject-specific questions in either reading or mathematics. A sample parental notification letter regarding NAEP is included in Appendix B below. The letter will be adopted for each grade/subject combination and the school principal may edit it.  However, the information regarding confidentiality and the appropriate law reference will remain unchanged.

Schools: E-filing - Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically (e-filing) at the school, district, or state level. If done at the school or district level, some burden will be incurred by school or district personnel. The e-filing burden is estimated at one hour per school. However, the e-filing process is only done in a subset of schools (approximately 38 percent in recent years), which is taken into account when computing the e-filing burden. In addition, given the time of year of the assessment (Fall 2011), the student sample for the age 13 assessments is completed at the school by the NCES contractor field staff, rather than through e-filing. As such, there is no e-filing burden at age 13.

Schools: Pre-Assessment Activities – Prior to the assessment, school personnel must prepare for the assessment. Part of the preparation is the pre-assessment visit, which is the opportunity for the NCES contractor field staff to meet with the school personnel to review procedures and logistics for the upcoming assessment. In addition, miscellaneous activity related to information gathering for the pre-assessment visit and communication with the teachers and students is required. Both the pre-assessment visit and the miscellaneous activities are estimated to require one hour of school personnel time. Thus, the total burden time for pre-assessment related activities is two hours. A sample brochure communication from the NAEP state coordinators to the participating schools describing these activities is included as Appendix C below. This brochure outlines what the school coordinator will be responsible for, including a description of e-filing and the pre-assessment visit.  The information in the brochure will be modified for each appropriate grade, subject area, mode of assessment, and timeline combination.

Table 2: Wave 1 Questions - Per Respondent Burden Time

Respondent Category	Time per Respondent
Age 9
Student Age 9 Core + Reading	15 minutes
Student Age 9 Core + Mathematics	15 minutes
Age 13
Student Age 13 Core + Reading	15 minutes
Student Age 13 Core + Mathematics	15 minutes
Age 17
Student Age 17 Core + Reading	15 minutes
Student Age 17 Core + Mathematics	15 minutes

Table 3: Estimated Burden for NAEP 2012 Assessments Contained in

This Submittal (Wave 1)

The nature of NAEP is that burden alternates from a relatively low burden in national-level administration years to a substantial burden increase in state-level administration years that include one or more assessments that support state-by-state and certain urban districts reporting. In state/district assessment years, NAEP samples approximate 1,000,000 students, while in national-only assessment years, NAEP samples approximate 100,000 students. In 2012, NAEP will conduct national-level assessments, so the estimated burden is much lower than in 2011 NAEP. Also, as in most years, the clearance for the NAEP 2012 collection is split into two submissions, Wave 1 and Wave 2. This is the first of the two, Wave 1, which presents the portion of the materials and burden hours estimated for LTT Reading and Mathematics. Wave 2 will present materials and burden for Grade 4 Writing; Grade 12 Reading, Math, Economics, and Charter Schools; Grades 4, 8, and 12 Pilot Questions; Main NAEP grades 4, 8, and 12 and LTT NAEP ages 9, 13, and 17 SD/ELL; and Special Studies.

4. How, By Whom, and For What Purpose the Data Will Be Used

The purpose of NAEP is to collect and report assessment data on student achievement in the subject areas assessed for use in monitoring education progress. In addition to reporting overall results of student performance and achievement, NAEP also reports student performance results for various subgroups of students and on various educational factors. Guidance for what is asked in the questions is set by the Governing Board. NCES is responsible for developing the questions and for selecting the final set of questions. The questions are designed to (a) provide the information for disaggregating data according to categories specified in the legislation⁶, (b) provide contextual information that is subject specific (e.g. reading, mathematics) and has an impact and known relationship to student achievement, and (c) provide policy-relevant information specified by the Governing Board.

In the original request to OMB for system clearance, NCES requested approval of the data-gathering instruments to be used in the 2011–2013 national and state assessments. This submittal applies to the first set of questions that will be submitted as part of the 2012 assessments. An expanded description of the purposes of the data is provided in the system clearance submittal.

5. Estimates of Costs to the Federal Government

Table 4: Administration Cost Estimates

The following estimates apply to the 2012 Wave 1 administration costs:

Activity	Provider	Estimated Costs
Printing, packaging, and distribution phases of the administration (related to LTT) including: prepare and package the assessment and all auxiliary materials; distribute assessment booklets and materials to the test administrators for each school	Pearson (the Materials, Distribution, Processing and Scoring (MDPS) contractor)	$1.5 million
Field administration (related to LTT) including: administration of assessments; pre-assessment contact/visits to school data collection from the field	Westat (the Data Collection contractor and NAEP State Service Center (NSSC) contractor)	$2.4 million
Totals		$3.9 million

6. Information Pertaining to the 2012 Questionnaires in This Submittal

Student Questionnaires

The LTT background questions are essentially the same questions that were administered in the 2008 LTT assessments.

Appendix A: Committee Lists

NAEP Background Variable Committee

Name Affiliation

Patricia Alexander University of Maryland

Arthur Applebee State University of New York at Albany

Claudia Buchman Ohio State University

Lizanne Destefano University of Illinois

Robert Hauser University of Wisconsin-Madison

Kathleen Heid Penn State University

Henry Levin Columbia University

Peter Levine Jonathan M. Tisch College of Citizenship and Public Service

Linda Levstik University of Kentucky

Samuel Lucas University of California-Berkeley

Senta Raizen West-Ed

Andrew Sum Northeastern University

William Walstad University of Nebraska-Lincoln

NAEP Long-Term Trend Reading Committee

Name Affiliation

Mary Beth Curtis Lesley University

Joe McGonegal Catholic Memorial High School, West Roxbury, MA

Katie Oliver Morgan Park Middle School, Duluth, MN

Timothy Shanahan University of Illinois

Lisa White Plymouth Public Schools, Plymouth, MA

NAEP Mathematics Committee

Name Affiliation

David J. Brancamp Northwestern RPDP (Regional Professional Development Program), Reno, NV

Sareeta Carter Benjamin Banneker Academic High School, Washington, DC

Herb Clemens Ohio State University

Marcia Cole District of Columbia Public Schools 

Carl Cowen Indiana University-Purdue University, Indianapolis

Bradford R. Findell Ohio Department of Education

Diana Rivera Garcia Departamento de Educación Puerto Rico

Anne Gonzales South Gate Middle School, South Gate, CA

Kathleen Heid Penn State University

Judy Kinley Morris Elementary School, Des Moines, IA

Carole Phillip Alice Deal Middle School, Washington, DC

Penny Roberts Greenville Elementary School, Greenville, KY

Daren Starnes The Lawrenceville School, Lawrenceville, NJ

JT Sutcliffe St. Mark’s School of Texas

Elizabeth Sweeney Boston Public Schools

Appendix B: Sample Parent Notification Letter

NAEP (NAEP Assessment Year) PARENT/GUARDIAN NOTIFICATION LETTER

(School Letterhead)

(Insert Date Here)

Dear Parent or Guardian:

We are pleased to tell you that (school name) has been selected to represent schools across our state and the nation by participating in the National Assessment of Educational Progress (NAEP). NAEP is given by the National Center for Education Statistics within the U.S. Department of Education and tells us what our nation’s students know and can do in key subject areas. It is the largest continuing and nationally representative assessment and has been providing valid and reliable data on student performance since 1969.

The results of NAEP are published in a report called The Nation's Report Card, which provides information about student achievement to educators, parents, policymakers, and the public.

In our school, the NAEP assessment will be given on (date) in (subject). Your child (may be/has been) selected to take the assessment. In addition to subject area questions, students will be asked some questions about themselves and their educational experience. You can access student and sample assessment questions at http://nces.ed.gov/nationsreportcard/parents/.

It will take about (assessment time) minutes for most students to complete NAEP. The results are completely confidential (in accordance with the Confidential Information Protection provisions of Title V, Subtitle A, Public Law 107-347), and your child’s grades will not be affected. Your child may be excused from participation for any reason, is not required to finish the assessment, and may skip any test question. While NAEP is voluntary, we depend on student participation to provide an accurate measure of student achievement that will inform improvements in education. Your child will represent many other students, so participation is very important. However, if you do not want your child to participate, please notify me in writing by (date).

There is no need to study in preparation for NAEP. We do ask parents to encourage their children to do their best and to have them get plenty of rest the night before the assessment.

If you would like to have additional information about NAEP, please visit http://nces.ed.gov/nationsreportcard. If you have questions or would like to review a booklet that includes sample subject area and student questions, please contact me at (telephone number) or via e-mail at (e-mail address).

We are excited that our school will be participating in NAEP, and we are pleased that your child (may be/has been) selected. We know that (school name)'s students will help us show what (state name)’s students know and can do.

Sincerely,

School Principal

1 In 2012, there will be both operational and pilot LTT assessments. However, the same background questions will be administered in all LTT assessments, whether operational or pilot.

2 The LTT assessment does not include either teacher or school questionnaires.

3 As noted in the system clearance, the assessment window for LTT is actually October 2011 through May 2012. However, for ease of discussion, all assessments will be referred to as the 2012 LTT assessments.

4 NAEP collects student question data that is required by law (P.L. 107-279, section 303 - i.e., race, ethnicity, socio-economic status) and that provides a context for reporting student performance. These questions are referred to as core questions.

5 Given that no grade 8 assessments will be administered in main NAEP in 2012, new core sections will be piloted with the age 13 LTT samples. Additional information regarding this pilot will be included in Wave 2.

6 Education Sciences Reform Act of 2002 (ESRA), National Assessment of Educational Progress (20 USC § 9622).