NAEP 2024 Part A v30

The National Assessment of Educational Progress (NAEP) is a federally authorized survey of student achievement at grades 4, 8, and 12 in various subject areas, such as mathematics, reading, writing, science, U.S. history, civics, and technology and engineering literacy (TEL).

NAEP is conducted by the National Center for Education Statistics (NCES) in the Institute of Education Sciences of the U.S. Department of Education. As such, NCES is responsible for designing and executing the assessment, including designing the assessment procedures and methodology, developing the assessment content, selecting the final assessment content, sampling schools and students, recruiting schools, administering the assessment, scoring student responses, determining the analysis procedures, analyzing the data, and reporting the results.¹

The National Assessment Governing Board (henceforth referred to as the Governing Board or NAGB), appointed by the Secretary of Education but independent of the Department, is a bipartisan group whose members include governors, state legislators, local and state school officials, educators, business representatives, and members of the general public. The Governing Board sets policy for NAEP and is responsible for developing the frameworks and test specifications that serve as the blueprint for the assessments.

The NAEP assessments contain diverse items such as “cognitive” assessment items, which measure what students know and can do in an academic subject, and “survey” or “non-cognitive” items, which gather information such as demographic variables, as well as construct-related information, such as courses taken. The survey portion includes a collection of data from students, teachers, and school administrators. Since NAEP assessments are administered uniformly using the same sets of test forms across the nation, NAEP results serve as a common metric for all states and select urban districts. The assessment stays essentially the same from year to year, with only carefully documented changes. This permits NAEP to provide a clear picture of student academic progress over time.

NAEP consists of two assessment programs: the NAEP long-term trend (LTT) assessment and the main NAEP assessment. The LTT assessments are given at the national level only and are administered to students at ages 9, 13, and 17 in a manner that is very different from that used for the main NAEP assessments. LTT reports mathematics and reading results that present trend data since the 1970s. NAEP provides results on subject-matter achievement, instructional experiences, and school environment for populations of students (e.g., all fourth-graders) and groups within those populations (e.g., female students, Hispanic students). NAEP does not provide scores for individual students or schools. The main NAEP assessments report current achievement levels and trends in student achievement at grades 4, 8, and 12 for the nation and, for certain assessments (e.g., reading and mathematics), states and select urban districts. The Trial Urban District Assessment (TUDA) is a special project developed to determine the feasibility of reporting district-level results for large urban districts. Currently, the following 27 districts participate in the TUDA program: Albuquerque, Atlanta, Austin, Baltimore City, Boston, Charlotte, Chicago, Clark County (NV), Cleveland, Dallas, Denver, Detroit, District of Columbia (DCPS), Duval County (FL), Fort Worth, Guilford County (NC), Hillsborough County (FL), Houston, Jefferson County (KY), Orange County (FL), Los Angeles, Miami-Dade, Milwaukee, New York City, Philadelphia, San Diego, and Shelby County (TN).

The possible universe of student respondents for NAEP 2024 is estimated to be 12 million at grades 4, 8, and 12, attending the approximately 154,000 public and private elementary and secondary schools in 50 states and the District of Columbia, including Bureau of Indian Education and Department of Defense Education Activity (DoDEA) Schools, and 4^th and 8^th grade public schools in Puerto Rico.

This request is to conduct NAEP in 2024, specifically:

Main NAEP operational assessments in 2024 for grade 4 (reading and mathematics), 8 (reading, mathematics and science), and 12 (reading and mathematics); In Puerto Rico, grades 4 and 8 mathematics will be the only subject assessed.
Pilot testing for new frameworks in mathematics (mainland U.S. and Puerto Rico) and reading for grades 4 and 8,
Pilot testing new Reading Router for grades 4 and 8,
High School Transcript Study (HSTS),
National Indian Education Study (NIES) for grades 4 and 8, and
2024 Field Trial (also known as the Dress Rehearsal) for grade 4 (reading and mathematics), 8 (reading and mathematics), and 12 (reading and mathematics) in U.S. mainland and grades 4 and 8 (mathematics) in Puerto Rico.

Beginning in 2024, NAEP will be transitioning to a Next-generation (Next-gen) version of the eNAEP test delivery software, the platform on which the assessment is delivered to students. NAEP will also be changing the operational assessment delivery model. While NAEP is currently administered by numerous NAEP field staff entering schools bringing NAEP-provided Surface Pros and Chromebooks, the program is planning to transition to a model that is ultimately less expensive and more aligned with the administration model used in state assessments. In particular, NAEP ultimately aims to administer the assessment using school-provided equipment and internet, with an alternate delivery model of cheaper, more streamlined NAEP-provided devices to be used in schools that do not meet the minimum specifications for school equipment.

In order to successfully transition to this ultimate plan, a staged approach is currently being undertaken so that trends can be measured across time. Namely, NAEP has conducted an Assessment Delivery study in 2022 (OMB# 1850-0803 v.305) as well as a Field Test in 2023 (OMB# 1850-0803 v.293) to provide more information about student interactions with the Next-gen eNAEP system and prepare for use of the updated system in operational NAEP assessments moving forward. In preparation for the 2024 NAEP administration, a Field Trial (also known as the “Dress Rehearsal”) will be conducted with students in a live classroom environment in November 2023 by NAEP field administration staff. The Field Trial will replicate the NAEP Operational Administration testing conditions to the fullest extent possible in a small number of schools. The NAEP program has utilized field trials prior to large-scale digitally based assessments since 2018, based on lessons learned and issues encountered by students in the field during the 2017 operational assessment. Results from this study will not be publicly released; rather they will be used to identify issues and generate solutions or workarounds in advance of the 2024 administration.

This Clearance package will have a new clearance request, with both a 60-day and 30-day consecutive public comment period notice published in the Federal Register. Three Amendments to this Clearance package are planned to be submitted in the coming months to update materials, like the Assessment Management System materials, survey questionnaires, communication materials, etc.

Some of the assessment, questionnaire, and recruitment materials are translated into Spanish. Specifically, Spanish versions of the student assessments and questionnaires are used for qualified English learner (EL) students when a bilingual accommodation is offered and for all students in Puerto Rico. Typically, this is done for all operational grade 4 and 8 assessments. In years in which a Puerto Rico NAEP assessment is given, such as in 2024, Spanish versions of communication materials for parents, teachers, and school coordinators as well as teacher and school questionnaires are created by translating their English equivalents into Spanish. In addition, every year, Spanish versions of parent communication materials are used nationwide for Spanish-speaking parents.

NAEP 2024 Amendment Schedule Table

Amendment #2

(this document)

Part A: Updates to the 2024 NAEP administration

Part B: Updates to the 2024 NAEP administration

Appendix A: External Advisory Committees

Appendix B: 2018 Weighting Procedures

Appendix C: 2024 Sampling Memo Draft

Appendix D (formerly D1 and D2): Updated communication materials (English, Spanish, and HSTS)

Appendix G: 2018 Sampling Design Final

Appendix I (formerly I and I2): New AMS System – Initial Access Components; HSTS AMS screens. Note: Appendix H is no longer included as a separate document since the AMS now includes the eFiling process in Appendix I. As a result, it is removed from the list of documents.

Appendix J1-J3 and J-S: All 2024 SQs, including NIES (formerly in Appendix K)

Amendment #3

(August 2023)

Part A: Updates to the 2024 NAEP administration (if needed)

Part B: Updates to the 2024 NAEP administration (if needed)

Appendix C: 2024 Sampling Memo Final

Appendix D: Final NAEP communication materials

Appendix E: Assessment Feedback Forms

Appendix I: New AMS System – Final Access Components

A.1.b. Legislative Authorization

In the current legislation that reauthorized NAEP, the National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622), Congress mandates the collection of national education survey data through a national assessment program:

ESTABLISHMENT- The Commissioner for Education Statistics shall, with the advice of the Assessment Board established under section 302, carry out, through grants, contracts, or cooperative agreements with one or more qualified organizations, or consortia thereof, a National Assessment of Educational Progress, which collectively refers to a national assessment, State assessments, and a long-term trend assessment in reading and mathematics.
PURPOSE; STATE ASSESSMENTS-

(1) PURPOSE- The purpose of this section is to provide, in a timely manner, a fair and accurate measurement of student academic achievement and reporting of trends in such achievement in reading, mathematics, and other subject matter as specified in this section.

The National Assessment of Educational Progress Authorization Act also requires the assessment to collect data on specified student groups and characteristics, including information organized by race/ethnicity, gender, socio-economic status, disability, and English learners. This allows for the fair and accurate presentation of achievement data and permits the collection of background, non-cognitive, or descriptive information that is related to academic achievement and aids in the fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and a variety of populations of students, and to monitor progress over time.

The statute and regulation mandating or authorizing the collection of this information can be found at https://www.law.cornell.edu/uscode/text/20/9622.

A.1.c. Overview of NAEP Assessments

This section provides a broad overview of main NAEP assessments, including information on the assessment frameworks, the cognitive and survey items, inclusion policies, the transition to digitally based assessments (DBA), and the assessment types.

A.1.c.1. NAEP Frameworks

NAEP assessments follow subject-area frameworks developed by the Governing Board and use the latest advances in assessment methodology. Frameworks capture a range of subject-specific content and thinking skills needed by students in order to deal with the complex issues they encounter inside and outside their classrooms. The NAEP frameworks are determined through a development process that ensures they are appropriate for current educational requirements. Because the assessments must remain flexible to mirror changes in educational objectives and curricula, the frameworks must be forward-looking and responsive, balancing current teaching practices with research findings.

NAEP frameworks can serve as guidelines for planning assessments or revising curricula. They also can provide information on skills appropriate to grades 4, 8, and 12 and can be models for measuring these skills in innovative ways. The subject-area frameworks evolve to match instructional practices. Developing a framework generally involves the following steps:

widespread participation and reviews by educators and state education officials;
reviews by steering committees whose members represent policymakers, practitioners, and members of the general public;
involvement of subject supervisors from education agencies;
public hearings; and
reviews by scholars in the field, by NCES staff, and by a policy advisory panel.

The frameworks can be found at https://www.nagb.gov/naep-frameworks/frameworks-overview.html.

A.1.c.2. Cognitive Item Development

As part of the item development process, NCES calls on many constituents to guide the process and review the assessment. Item development is guided by a multi-year design plan, which is guided by the framework and establishes the design principles, priorities, schedules, and reporting goals for each subject. Based on this plan, the NAEP contractor creates a development plan outlining the item inventory and objectives for new items and then begins the development process by developing more items than are needed. This item pool is then subjected to:

internal contractor review with content experts, teachers, and experts on political sensitivity and bias;
playtesting, tryouts, or cognitive interviews with small groups of students for select items (particularly those that have new item types, formats, or challenging content); and
refinement of items and scoring rubrics under NCES guidance.

Next, a standing committee of content experts, state and local education agency representatives, teachers, and representatives of professional associations reviews the items. The standing committee considers:

the appropriateness of the items for the particular grade;
the representative nature of the item set;
the compatibility of the items with the framework and test specifications; and
the quality of items and scoring rubrics.

For state-level assessments, this may be followed by a state item review where further feedback is received. Items are then revised and submitted to NCES and the Governing Board Assessment Development Committee for approval prior to pilot testing.

The pilot test is used to finalize the testing instrument. Items may be dropped from consideration or move forward to the operational assessment. The item set is once again subjected to review by the standing committee and NCES following generally the same procedure described above. A final set of test items is then assembled for NCES and the Governing Board’s review and approval. After the operational assessment, items are once again examined. In rare cases where item statistics indicate problems, the item may be dropped from the assessment. The remaining items are secured for reuse in future assessments, with a subset of those items publicly released.

A.1.c.3. Survey Items

In addition to assessing subject-area achievement, NAEP collects information that serves to fulfill the reporting requirements of the federal legislation and to provide context for the reporting of student performance. The legislation requires that, whenever feasible, NAEP includes information on special groups (e.g., information reported by race, ethnicity, socio-economic status, gender, disability, and limited English proficiency). As part of most NAEP assessments, three types of questionnaires are used to collect information: student, teacher, and school (including the COVID-19 Learning Recovery items for 2024 for each respondent type). An overview of the questionnaires is presented below.

Student Questionnaires

Each NAEP student assessment form includes non-cognitive items, also known as the student questionnaire. The questionnaires appear in separately timed blocks of items in the assessment forms. The items collect information on students’ demographic characteristics, classroom experiences, and educational support. Students’ responses provide data that give context to NAEP results and/or allow researchers to track factors associated with academic achievement. Students complete the questionnaires voluntarily (for confidentiality provisions see Section A.10 for more information). Student names are never reported with their responses or with the other information collected by NAEP.

Each student questionnaire includes three types of items:

General student information: Student responses to these items are used to collect information about factors such as race or ethnicity and parents’ education level. Answers on the questionnaires also provide information about factors associated with academic performance, including household composition, academic self-discipline, and the number of books in the home.
Other contextual/policy information: These items focus on students’ educational settings and experiences and collect information about students’ attendance (i.e., days absent), family discourse (i.e., talking about school at home), reading load (i.e., pages read per day), and technology use at school. There are also items that ask about students’ effort on the assessment and the difficulty of the assessment. In 2024, students will answer questions about their learning recovery experiences related to the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. Answers on the questionnaires provide information on how aspects of education and educational resources are distributed among different groups.
Subject-specific information: In most NAEP administrations, these items cover three categories of information: (1) time spent studying the subject; (2) instructional experiences in the subject; and (3) student factors (e.g., effort, confidence) related to the subject and the assessment.

Teacher Questionnaires

To provide supplemental information about the instructional experiences reported by students, teachers are asked to complete an online questionnaire using NAEPq about their instructional practices, classroom organization, teaching background and training, and the subject in which students are being assessed. NAEPq is an online platform used for the completion of online questionnaires. Printed versions of the survey questionnaires are also available, if requested. Approximately 3.8 percent of the responses received, were by hardcopy in 2022. Teacher responses are then matched to student data. While completion of the questionnaire is voluntary, NAEP encourages teachers’ participation since their responses improve the accuracy and completeness of the NAEP assessment.

Teacher questionnaires are typically only given to teachers at grades 4 and 8; NAEP typically does not collect teacher information for grade 12. By grade 12, there is such variation in student course-taking experiences that students cannot be matched to individual teachers for each tested subject. For example, a student may not be taking a mathematics class in grade 12, so he or she cannot be matched to a teacher. Conversely, a student could be taking two mathematics classes at grade 12 and have multiple teachers related to mathematics. Only an economics teacher questionnaire has been developed and administered at grade 12. However, these data were not released (with either the 2006 or the 2012 results) due to a student-teacher match rate below statistical standards.²

Teacher questionnaires are organized into different parts. The first part of the teacher questionnaire covers background and general training and includes items concerning years of teaching experience, certifications, degrees, major and minor fields of study, coursework in education, coursework in specific subject areas, the amount of in-service training, the extent of control over instructional issues, and the availability of resources for the classroom. Subsequent parts of the teacher questionnaire tend to cover training in the subject area, classroom instructional information, and teacher exposure to issues related to the subject and the teaching of the subject. In 2024, teachers will answer questions about their instructional design and learning recovery experiences for their students related to the COVID-19 and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. They also ask about pre- and in-service training, the ability level of the students in the class, the length of homework assignments, the use of particular resources, and how students are assigned to particular classes.

School Questionnaires

The school questionnaire provides supplemental information about school factors that may influence students’ achievement. It is given to the principal or another official of each school that participates in the NAEP assessment. While schools’ completion of the questionnaire is voluntary, NAEP encourages schools’ participation since it makes the NAEP assessment more accurate and complete.

The school questionnaire is accessed online through NAEPq and is organized into different parts. The first part tends to cover characteristics of the school, including the length of the school day and year, school enrollment, absenteeism, dropout rates, and the size and composition of the teaching staff. Subsequent parts of the school questionnaire tend to cover tracking policies, curricula, testing practices, special priorities, and schoolwide programs and problems. The questionnaire also collects information about the availability of resources, policies for parental involvement, special services, and community services. Printed versions of the survey questionnaires are also available, if requested. Approximately 4.7 percent of the responses received, were by hardcopy in 2022. In 2024, school administrators will answer questions about their school’s instructional organization and practices related to learning recovery following the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. The supplemental charter school questionnaire designed to collect information on charter school policies and characteristics is provided to administrators of charter schools who are sampled to participate in NAEP. The supplement covers organization and school governance, parental involvement, and curriculum and offerings.

Development of Survey Items

The Background Information Framework and the Governing Board’s Policy on the Collection and Reporting of Background Data (located at https://www.nagb.gov/content/nagb/assets/documents/policies/collection-report-backg-data.pdf), guide the collection and reporting of non-cognitive assessment information. In addition, subject-area frameworks provide guidance on subject-specific, non-cognitive assessment questions to be included in the questionnaires. The development process is very similar to the cognitive items, including review of the existing item pool; development of more items than are intended for use; review by experts (including the standing committee); and cognitive interviews with students, teachers, and schools. When developing the questionnaires, NAEP uses a pretesting process so that the final questions are minimally intrusive or sensitive, are grounded in educational research, and the answers can provide information relevant to the subject being assessed.

In the web-based NAEP Data Explorer,³ (located at https://www.nationsreportcard.gov/ndecore/landing) the results of the questionnaires are sorted into eight broad categories: Major Reporting Groups, Student Factors, Factors Beyond School, Instructional Content and Practice, Teacher Factors, School Factors, Community Factors, and Government Factors.

To minimize burden on the respondents and maximize the constructs addressed via the questionnaires, NAEP may spiral items across respondents and/or rotate some non-required items across assessment administrations. The possible “library” of items for the NAEP 2024 questionnaires, for each subject and respondent, are included in Appendix F. Approved versions of the Main NAEP questionnaires will be provided in Appendices J1-J3, and J-S in the upcoming Amendments.

A.1.c.4. Inclusion in NAEP

It is important for NAEP to assess as many students selected to participate as possible. Assessing representative samples of students, including students with disabilities (SD) and English learners (EL), helps to ensure that NAEP results accurately reflect the educational performance of all students in the target population and can continue to serve as a meaningful measure of U.S. students’ academic achievement over time.

The Governing Board, which sets policy for NAEP, has been exploring ways to ensure that NAEP continues to appropriately include as many students as possible and to do so in a consistent manner for all jurisdictions assessed and reported on. In March 2010 (updated August 2014), the Governing Board adopted a policy, NAEP Testing and Reporting on Students with Disabilities and English Language Learners (located at https://www.nagb.gov/content/nagb/assets/documents/policies/naep_testandreport_studentswithdisabilities.pdf). This policy was the culmination of work with experts in testing and curriculum and those who work with exceptional children and students learning to speak English. The policy aims to:

maximize participation of sampled students in NAEP;
reduce variation in exclusion rates for SD and EL students across states and districts;
develop uniform national rules for including students in NAEP; and
ensure that NAEP is fully representative of SD and EL students.

The policy defines specific inclusion goals for NAEP samples. At the national, state, and district levels, the goal is to include 95 percent of all students selected for the NAEP samples, and 85 percent of those in the NAEP sample who are identified as SD or EL.

Students are selected to participate in NAEP based on a sampling procedure⁴ designed to yield a sample of students that is representative of students in all schools nationwide and in public schools within each state. First, schools are selected, and then students are sampled from within those schools without regard to disability or English language proficiency. Once students are selected, those previously identified as SD or EL may be offered accommodations or excluded.

Accommodations in the testing environment or administration procedures are provided for SD and EL students. Some examples of accommodations permitted by NAEP are extra time, testing in small-group or one-on-one sessions, reading aloud to a student, and scribing a student’s responses. Some examples of testing accommodations not allowed are giving the reading assessment in a language other than English or reading the passages in the reading assessment aloud to the student.

States and jurisdictions vary in their proportions of students with disabilities and in their policies on inclusion and the use of accommodations. Despite the increasing identification of SD and EL students in some states, in particular of EL students at grade 4, NAEP inclusion rates have generally remained steady or increased since 2003. This reflects efforts on the part of states and jurisdictions to include all students who can meaningfully participate in the NAEP assessments. The NAEP inclusion policy is an effort to ensure that this trend continues.

A.1.c.5. Digitally Based Assessments (DBA)

Nearly all of our nation’s schools are equipped with computers, and an increasing number of schools are making digital tools an integral component of the learning environment, reflecting that the knowledge and skills needed for future post-secondary success involve the use of new technologies. NAEP is evolving to address the changing educational landscape through its transition to DBA.

NAEP currently administers the digital assessments on devices, which NAEP field staff bring into the schools.⁵ As mentioned, the 2024 NAEP operational administration model will be changing with the rollout of the Next-gen eNAEP delivery system, which will be administered on a mixture of NAEP-provided devices. In the future, the Next-gen system will also give NAEP the ability to transition to conducting assessments on school-based equipment.

DBA allow NAEP to:

more accurately reflect what is happening in today’s classrooms;
improve measurement of knowledge and skills; and
collect new types of data that provide depth in our understanding of what students know and can do, including how they engage with new technologies to approach problem solving.

Leveraging Technologies

NAEP DBA use testing methods and item types that reflect the growing use of technology in education. Examples of such item types include:

Multimedia elements, such as videos and audio clips: The NAEP computer-based writing assessment, administered in 2011 at grades 8 and 12, made use of multimedia. These elements have been incorporated into other NAEP DBA as well. The 2011 writing tasks were presented to students on computers in a variety of ways, including text, audio, photographs, video, and animation. Examples of these tasks are available at http://www.nationsreportcard.gov/writing_2011/sample_quest.aspx.
Interactive items and tools: Some questions may allow the use of embedded technological features to form a response. For example, students may use “drag and drop” functionality to place labels on a graphic or may tap an area or zone on the screen to make a selection. Other questions may involve the use of digital tools. In the mathematics DBA, an online calculator is available for students to use when responding to some items. An equation editor is also provided for the entry of mathematical expressions and equations, and we have incorporated some digital tools, such as rulers, data graph builders, and function graphers, and continue to explore more tools that can be used to gauge students’ mathematical skills. Students are shown how to use many of these interactive features and tools in the brief tutorials that are included at the beginning of each NAEP DBA. The 2022 tutorials are available at https://npd.naep.ed.gov/totw/2022/english.html.
Immersive scenario-based tasks: Scenario-based tasks use multimedia features and tools to engage students in rich, authentic problem-solving contexts. NAEP’s first scenario-based tasks were administered in 2009, when students at grades 4, 8, and 12 were assessed with interactive computer tasks in science. The science tasks asked students to solve scientific problems and perform experiments, often by simulation. They provide students more opportunities than a paper-based assessment (PBA) to demonstrate skills involved in doing science without many of the logistical constraints associated with a natural or laboratory setting. The science tasks administered in 2019 can be explored at https://www.nationsreportcard.gov/science/sample-questions/. NAEP also administered scenario-based tasks in the 2014 technology and engineering literacy (TEL) assessment, where students were challenged to work through computer simulations of real-world situations they might encounter in their everyday lives. A sample TEL task can be viewed at NAEP - 2014 Technology and Engineering Literacy - Sample TEL Tasks (nationsreportcard.gov). NAEP is continuing to expand the use of scenario-based tasks to measure knowledge and skills in other subject areas such as mathematics and reading. Scenario-based tasks have been part of the operational reading assessment since 2019.

In addition to new item types, the transition to DBA makes it possible for NAEP to employ an adaptive testing design, in which assessment content is targeted to students’ ability based on performance during the test administration. Thus, students see items that are tailored to their ability levels, and they may be more likely to be able to engage in the assessment and demonstrate what they know and can do. The goal of implementing adaptive testing is to achieve better measurement of student knowledge and skills across the wide range of student performance levels on which NAEP reports.

The type of adaptive testing being considered for NAEP is a multi-stage test (MST) design that uses two stages. Students take sections of cognitive items, just as in past NAEP administrations. Based on their performance on the first section of items, students receive one or more subsequent sections of items that is targeted to their ability level. For example, students who do not perform well on the first section of items receive a second section composed of somewhat easier items. The implementation of this two-stage MST design for NAEP has been informed by previous research on the benefits, applicability, and feasibility of adaptive testing for NAEP.

In order to support the transition from linear to MST in reading, a Reading Router is being developed, and will be piloted in 2024. This pilot of the reading router will be administered in a linear test and not as an MST. The reading router is a short block that will (when actually in MST) be scored in the system and a routing decision to targeted difficulty on subsequent blocks, based on students’ scores, will be made in real time.

The DBA technology allows NAEP to capture information about what students do while attempting to answer questions. While PBA only yields the final responses in the test form, DBA capture actions students perform while interacting with the assessment tasks, as well as the time at which students take these actions. These student interactions with the assessment interface are generally not used to assess students’ knowledge and skills, but rather this information might be used to provide context for student performance. For example, more proficient students may use digital tools such as the calculator in mathematics or the spell-checker in writing assessments, compared to less proficient students. As such, NAEP will potentially uncover more information about which actions students use when they successfully (or unsuccessfully) answer specific questions on the assessment. Unless specifically required by the scoring rubrics, process data are not scored; they are primarily used for improving assessment design and for providing contexts for interpreting reported scores.

NAEP will capture the following actions in the DBA, although not all actions will be captured for all assessments:

Student navigation (e.g., clicking back/next; clicking on the progress navigator; clicking to leave a section);
Student use of tools (e.g., zooming; using text-to-speech; opening and interactions with the scratchwork tool; opening and interactions with the calculator; using the equation editor; clicking the change language button; selecting the theme; opening the Help tool);
Student responses (e.g., clicking a choice; eliminating a choice; clearing an answer; keystroke log of student typed text);
Other student events (e.g., vertical and horizontal scrolling; media interaction such as playing an audio stimulus); and
Tutorial events (records student interactions with the tutorial practice item or not interacting with the tutorial when prompted).

Development of Digitally Based Assessments (DBA)

NAEP’s item and system development processes include several types of activities that help to ensure that our DBA measure the subject-area knowledge and skills outlined in the NAEP frameworks and not students’ ability to use the device or the particular software and digital tools included in the DBA.

During item development, new digitally based item types and tasks are studied and pretested with diverse groups of students. The purpose of these pretesting activities is to determine whether construct-irrelevant features, such as confusing wording, unfamiliar interactivity or contexts, or other factors, prevent students from demonstrating the targeted knowledge, skills, and abilities. Such activities help identify usability, design, and validity issues so that items and tasks may be further revised and refined prior to administration.

Development of the assessment delivery system, including the interface that students interact with when taking NAEP DBA, is informed by best practices in user experience design. Decisions about the availability, appearance, and functionality of system features and tools are also made based on the results of usability testing with students.

To help ensure that students know how to use the assessment system and tools, each administration of a NAEP DBA begins with a brief interactive tutorial that teaches students how to use the system features to take the assessment. Students actively engage with the tutorial, as they are asked to use specific tools and features. Help screens are also built into the system, and students can access them at any time while taking the assessment. The 2022 tutorials are available at https://npd.naep.ed.gov/totw/2022/english.html.

Accommodations and Universal Design Features with Digitally Based Assessments (DBA)

New technologies are improving NAEP’s ability to offer accommodations to increase participation and provide universal access to students of all learning backgrounds, including students with disabilities and English learners (EL). In a digital environment, what used to be an accommodation for PBA becomes a seamless part of universal design, available to all students. This means that things like adjusting font size, having test items read aloud in English (text-to-speech) is available for assessments other than the reading cognitive content, changing the appearance of the testing interface to have a higher contrast, using a highlighter tool, and eliminating answer choices can be accomplished by all students during the test administration. Please note, the highlighter tool is not available for scenario-based tasks (SBTs) as of the 2019 assessments.

In addition to these universal design features, NAEP also continues to offer accommodations to students with Individualized Education Programs (IEPs), Section 504 plans, and English learning plans requiring that they have them. Some accommodations are available in the testing system (such as additional time, a magnification tool, or a Spanish/English version of the test), while others are provided by the test administrator or the school (such as breaks during testing, sign language interpretation of the test, or a bilingual dictionary). Section B.2.b provides more information on the classification of students and the assignment of accommodations.

A.1.c.6. Assessment Types

NAEP uses three types of assessment activities, which may simultaneously be in the field during any given data collection effort. Each is described in more detail below.

Operational Assessments

Operational NAEP administrations, unlike pilot administrations, collect data to publicly report on the educational achievement of students as required by federal law. The NAEP results are reported in The Nation’s Report Card (http://nationsreportcard.gov/), which is used by policymakers, state and local educators, principals, teachers, and parents to inform educational policy decisions.

Pilot Assessments

Pilot testing (also known as field testing) of cognitive and non-cognitive items is carried out in all subject areas. Pilot assessments are usually conducted in conjunction with operational assessments and use the same procedures as the operational assessments. The purpose of pilot testing is to obtain information regarding clarity, difficulty levels, timing, and feasibility of items and conditions. In addition to ensuring that items measure what is intended, the data collected from pilot tests serve as the basis for selecting the most effective items and data collection procedures for the subsequent operational assessments. Pilot testing is a cost-effective means for revising and selecting items prior to an operational data collection because the items are administered to a small nationally representative sample of students, and data are gathered about performance that crosses the spectrum of student achievement. Items that do not work well can be dropped or modified before the operational administration.

Prior to pilot testing, many new items are pre-tested with small groups of sample participants (cleared under the NCES pretesting generic clearance agreement; OMB# 1850-0803). All non-cognitive items undergo one-on-one cognitive interviews, which are useful for identifying questionnaire and procedural problems before larger-scale pilot testing is undertaken. Select cognitive items also undergo pre-pilot testing, such as item tryouts or cognitive interviews, in order to test out new item types or formats, or challenging content. In addition, usability testing is conducted on new technologies and digitally-based platforms and instruments.

Special Studies

Special studies are an opportunity for NAEP to investigate particular aspects of the assessment without impacting the reporting of NAEP results. Previous special studies have focused on linking NAEP to other assessments or linking across NAEP same-subject frameworks, investigating the expansion of the item pool, evaluating specific accommodations, investigating administration modes (such as DBA alternatives), and providing targeted data on specific student populations.

In addition to the overarching goal of NAEP to provide data about student achievement at the national, state, and district levels, NAEP also provides specially targeted data on an as-needed basis. At times, this may only mean that a special analysis of the existing data is necessary. At other times, this may include the addition of a short, add-on questionnaire targeted at specified groups. For example, in the past, additional student, teacher, and school questionnaires were developed and administered as part of the National Indian Education Study (NIES) that NCES conducted on behalf of the Office of Indian Education. Through such targeted questionnaires, important information about the achievement of a specific group is gathered at minimal additional burden. These types of special studies are intentionally kept to a minimum and are designed to avoid jeopardizing the main purpose of the program.

Field Trial

The purpose of the 2024 Field Trial is to perform a “dress rehearsal” prior to the 2024 NAEP Operational Administration. The Field Trial will be conducted with students in a live classroom environment at a small number of schools, allowing the system to be tested in the manner in which it will be used in the national study to help identify platform system or operational issues prior to the administration.

A.1.d. Overview of 2024 NAEP Assessments

The Governing Board determines NAEP policy and the assessment schedule,⁶ and future Governing Board decisions may result in changes to the plans represented here. Any changes will be presented in subsequent clearance packages or revisions to the current package.

The 2024 data collection will consist of the following:

Operational national/state/TUDA DBA in mathematics and reading at grades 4 and 8, and Puerto Rico in mathematics at grades 4 and 8;
Operational national DBA in science at grade 8;
Operational national DBA in mathematics and reading at grade 12;
Pilot in mathematics (mainland U.S. and Puerto Rico) and reading (mainland U.S. only) at grades 4 and 8;
Pilot for Reading Router in grades 4 and 8;
National Indian Education Study (NIES);
High School Transcript Study (HSTS).

The operational assessments will include a comparability study between administration of the assessment on Chromebooks and Surface Pros.

National Indian Education Study (NIES)

NIES is designed to describe the condition of education for American Indian and Alaska Native (AI/AN) students in the United States. The study provides educators, policymakers, and the public with information about the academic performance in reading and mathematics of AI/AN fourth- and eighth-graders as well as their experiences with Native American culture and language.

Conducted in conjunction with the NAEP assessments in 2005, 2007, 2009, 2011, 2015 and 2019, NIES provides data on a nationally representative sample of American Indian and Alaska Native students in public, private, Department of Defense, and Bureau of Indian Education funded schools. It is an important source of data on American Indian and Alaska Native students, especially for educators, administrators, and policymakers who address the educational needs of these students. The study is funded by the Office of Indian Education and conducted by NCES for the U.S. Department of Education. A Technical Review Panel (see Appendix A3), whose members include American Indian and Alaska Native educators and researchers from across the country, provide guidance on multiple facets of the study, including questionnaire development and reporting.

This study was conducted through a survey to explore the educational experiences of the fourth- and eighth-grade American Indian and Alaska Native students based on responses to the NIES student, teacher, and school questionnaires. The survey focused on the integration of native language and culture into school and classroom activities.

The 2024 NIES study will use similar methods as those used in 2015 and 2019. All grade 4 and 8 students in the reading and mathematics operational sample identified as being AI/AN will be given an additional questionnaire. In addition, students’ reading and mathematics teachers will also respond to the NIES teacher questionnaire, as will administrators of the schools from which the AI/AN students were selected. Approximately 8,000 fourth grade and 6,500 eighth grade students will participate in the 2024 NIES study. Information related to the sampling, design, data collection methods, and analyses, as well as results from previous studies can be found at http://nces.ed.gov/nationsreportcard/nies/.

High School Transcript Study (HSTS)

The 2024 HSTS is designed to provide information about the coursetaking behavior of students graduating from high schools in the United States in 2024. The study is being conducted at schools participating in the twelfth-grade 2024 NAEP. Transcript information is linked to NAEP scores for those graduates who participated in the mathematics NAEP assessments, permitting analysis of the relationships between assessment scores and course taking behavior.

High school transcript studies have been conducted in 1987, 1990, 1994, 1998, 2000, 2005, 2009 and 2019. The 2024 HSTS will be the same as 2019 with no modifications.

HSTS will be conducted in high schools selected for NAEP 2024 and will collect transcripts for grade 12 graduates selected for participation in mathematics assessments. Districts and schools participating in HSTS will be notified about their selection at the same time as the regular NAEP schools. Initial notification will be done as part of the regular NAEP notification correspondence with districts and schools. Participating schools will be notified of their selection for the study and asked to identify an HSTS coordinator to manage the HSTS activities. HSTS will be conducted in several phases:

Collecting course catalogs: In October 2023, the HSTS school coordinator will be contacted via email to request the current year’s course catalog (2023–2024) and the three previous years’ catalogs (2015–2016, 2016–2017, and 2018–2019). If these course catalogs can be sent electronically, then the coordinator will be given the address of a secure File Transfer Protocol (FTP) website where the catalogs can be uploaded. If only physical copies of the catalogs are available, then prepaid and pre-addressed return mailers will be provided to the HSTS school coordinator. In some cases, the state or TUDA coordinator may provide both course catalogs and the student transcripts directly on behalf of all the schools in the state or TUDA district.
Collecting the School Information Form and sample transcripts: In January 2024, the participating schools will be asked to complete an online School Information Form (to be included in Amendment #2), provide a sample student transcript with the student name redacted, and to place the Family Educational Rights and Privacy Act (FERPA) disclosure notices with the files of students selected to take the NAEP 2024 mathematics assessments. These notices will include the provisions from FERPA, which explain the disclosure safeguards that grant NCES the authority to obtain transcript information. Normal procedures for the transcript study, as specified by FERPA, are to provide FERPA notices for the school, but not to notify parents of their child’s inclusion because no student time is involved, and all transcript information is collected with the student name redacted. However, parent information and notification letters are made available to schools (to be included in Amendment #2) to use if they wish to use them.
Collecting student transcripts: Between June and October 2024, NCES will collect transcripts and other student-level information (to be included in Amendment #2) using one of two collection methods: electronically or during in-person visits by NAEP field staff. It is expected that up to 50 percent of the 2024 transcripts will be electronic. Whenever possible, NAEP will obtain transcript data electronically from the states (or school districts, in case of TUDAs); otherwise, transcript data are collected directly from schools. If the transcripts cannot be transmitted electronically by a school, then a trained NAEP representative will return to the school to collect photocopies of the requested student transcripts. If the transcripts can be transmitted electronically, NAEP staff will provide information to either the NAEP state or TUDA coordinator or the HSTS school coordinator about the data the transcripts must include and directions on how to transmit the transcripts.

States, districts, and schools must meet the following minimum criteria for electronic transcript submissions: a) maintain electronic student course information system at the high school level; b) maintain electronic high school course catalogs; and c) have unique course identification numbers on both student course information and course catalog at the high school level. Files can be transmitted in a number of formats, including Microsoft Excel, Comma Separated Value (CSV), Microsoft Word, Microsoft Access, XML, or plain text files. Electronic transcripts will be transmitted via a secure FTP website where the electronic high school transcripts can be uploaded.

If the HSTS school coordinator submits the transcripts, a list of only the students sampled for grade 12 mathematics can be obtained from the secure AMS website and transcripts will be provided for those students only. If, however, the state or district (TUDA or non-TUDA) coordinator submits the transcripts, he or she will need to submit transcripts without realizing the student sample. Therefore, one of two methods can be used:

Download a data template from AMS containing a list of all grade 12 students.
Download a data template from AMS containing a list of students sampled for grade 12 mathematics and reading, plus an additional 10 percent of students who were not sampled.

Upon receiving the transcripts, the school and student information listed on the file will be used to link to the 2024 NAEP grade 12 assessments’ student records. Once the link is established, the school name and student directly identifying information will be removed.

Information related to the sampling, design, data collection methods, and analyses, as well as results from previous studies, can be found at http://nces.ed.gov/nationsreportcard/hsts/.

A.2. How, by Whom, and for What Purpose the Data Will Be Used

Results will be reported on the 2024 operational assessments in mathematics, reading, and science. In addition, results will also be reported from the 2024 HSTS and NIES special studies. Results from the special studies may be published as research reports. NAEP will use the results from the 2024 pilot testing to inform future assessments and procedures.

The NAEP operational results are reported in The Nation’s Report Card, which is used by policymakers, state and local educators, principals, teachers, and parents to help inform educational policy decisions. The main NAEP report cards provide national results, trends for different student groups, results on scale scores and achievement levels, and sample items. In reports with state or urban district results, there are sections that provide overview information on the performance of these jurisdictions. If NCES elects to release sample items, percentage correct statistics on those items will be provided in the report. NAEP does not provide scores for individual students or schools.

Results from each NAEP assessment are provided online in an interactive website (http://nationsreportcard.gov/) and in one-page summary reports, called snapshots, for each participating state or urban district. Additional data tools are available online for those interested in:

analyzing NAEP data and creating tables and graphics (https://www.nationsreportcard.gov/ndecore/landing);
viewing and comparing NAEP performance results overall and by various demographic groups across states and within each state (https://www.nationsreportcard.gov/profiles/stateprofile);
viewing and comparing NAEP performance results overall and by various demographic groups across and within each of the participating large urban districts (https://www.nationsreportcard.gov/profiles/districtprofile);
searching, sorting, and providing data for sample NAEP items (https://nces.ed.gov/nationsreportcard/nqt/); and
seeing the knowledge and skills demonstrated by students performing at different scale scores (https://www.nationsreportcard.gov/itemmaps).

In addition to contributing to the reporting tools mentioned above, data from the questionnaires are used as part of the marginal estimation procedures that produce the student achievement results. Questionnaire data are also used to perform quality control checks on school-reported data and in special reports, such as the Black–White Achievement Gap report (http://nces.ed.gov/nationsreportcard/studies/gaps/) and the Classroom Instruction Report in reading, mathematics, and science based on the 2015 Student Questionnaire Data (https://www.nationsreportcard.gov/sq_classroom/#mathematics).

Lastly, there are numerous opportunities for secondary data analysis because of NAEP’s large scale, the regularity of its administrations, and its stringent quality control processes for data collection and analysis. NAEP data are used by researchers and educators who have diverse interests and varying levels of analytical experience.

A.3. Improved Use of Technology

NAEP has continually moved to administration methods that include greater use of technology, as described below.

Online Teacher and School Questionnaires

The teacher and school questionnaires that accompany the NAEP assessment were traditionally available as paper-based questionnaires. Starting in 2001, NAEP offered teachers and school administrators an option of either completing the questionnaires on paper or online. In an effort to reduce costs and to streamline the data collection, starting in 2014 the NAEP program moved to the practice of having the teacher and school questionnaires available primarily online through a tool known as NAEPq. To support respondents who have limited internet connections, NAEP field staff have a limited number of printed copies of the questionnaires that can be distributed at the school’s request.

Electronic Pre-Assessment Activities

Each school participating in NAEP has a designated staff member to serve as its NAEP school coordinator. Pre-assessment and assessment activities include functions such as finalizing student samples, verifying student demographics, reviewing accommodations, and planning logistics for the assessment. NAEP is moving in the direction of paperless administrations. An electronic pre-assessment system (known as AMS) was developed so that school coordinators would provide requested administration information online, including logistical information, updates of student and teacher information, and the completion of inclusion and accommodation information.⁷

Digitally Based Assessments (DBA)

As described in Section A.1.c.5, NAEP has transitioned to DBA. The move to DBA allows NAEP to provide assessments consistent with other large-scale assessments (such as those given by the Partnership for Assessment of Readiness for College and Careers [PARCC] and the Smarter Balanced Assessment Consortium). In addition, the transition to DBA allows NAEP to more accurately reflect what is happening in today’s classrooms, improve measurement of knowledge and skills, and collect new types of data that provide depth in our understanding of what students know and can do.

Automated Scoring

NAEP administers a combination of selected-response items and open-ended or constructed-response items. NAEP currently uses human scorers to score the constructed-response items, using detailed scoring rubrics and proven scoring methodologies. With the increased use of technologies, the methodology and reliability of automated scoring (i.e., the scoring of constructed-response items using computer software) has advanced. While NAEP does not currently employ automated scoring methodologies operationally, these are being investigated for possible future use. In particular, NCES recently held a competition to examine a variety of automated scoring engines and methods for consideration in NAEP (see: https://nces.ed.gov/whatsnew/press_releases/1_21_2022.asp).

A.4. Efforts to Identify Duplication

The proposed assessments, including the questionnaires, do not exist in the same format or combination in the U.S. Department of Education or elsewhere. The non-cognitive data gathered by NAEP comprise the only comprehensive cross-sectional survey performed regularly on a large-scale basis that can be related to extensive achievement data in the United States. No other federally funded studies have been designed to collect data for the purpose of regularly assessing trends in educational progress and comparing these trends across states. None of the major non-federal studies of educational achievement were designed to measure changes in national achievement. In short, no existing data source in the public or private sector duplicates NAEP.

While the survey items in NAEP are unique, the items are not developed in a vacuum. Their development is informed by similar items in other assessments and survey programs. In addition, in future rounds of development, NCES will continue to better align the NAEP survey questions with other surveys (particularly, but not limited to, those from other NCES and federal survey programs).

Historically, NAEP has served as a critical national “audit” function, offering an extremely helpful reference point in the interpretation of score trends on “high-stakes” tests used for school accountability. The main NAEP scales have served this function well even though high-stake state assessments were not always closely aligned with the corresponding NAEP assessments. Given the significant changes currently underway in the American educational landscape, this “audit” function is even more important.

NAEP has provided the best available information about the academic achievement of the nation’s students in relation to consensus assessment frameworks, maintaining long-term trend lines for decades. In addition to reporting at the national level, NAEP has offered achievement comparisons among participating states for more than two decades, and since 2003, all states have participated in the NAEP mathematics and reading assessments at the fourth and eighth grades. More recently, NAEP has also reported achievement for selected large urban school districts. In addition to characterizing the achievement of fourth-, eighth-, and twelfth-grade students in a variety of subject areas, NAEP has also served to document the often substantial disparities in achievement across demographic groups, tracking both achievement and achievement gaps over time. In addition to describing educational achievement, NAEP has furthered deliberation as to the scope and meaning of achievement in mathematics, reading, and other subject areas. NAEP assessments are aligned to ambitious assessment frameworks developed by a thoughtful process to reflect the best thinking of educators and content specialists. These frameworks have served as models for the states and other organizations to follow. Finally, NAEP has also served as a laboratory for innovation, developing and demonstrating new item formats, as well as statistical methods and models now emulated by large-scale assessments worldwide.

NAEP has functioned well as a suite of complex survey modules conducted as assessments of student achievement in fixed testing windows. The complexity of NAEP evolved by necessity to address its legal and policy reporting requirements and the complex sampling of items and students needed to make reliable and valid inferences at the subgroup, district, state, and national level for stakeholders, ranging from policymakers to secondary analysts, and do so without creating an undue burden on students and schools.

A.5. Burden on Small Businesses or Other Small Entities

The school samples for NAEP contain small-, medium-, and large-size schools, including private schools. Schools are included in the sample proportional to their representation in the population, or as necessary to meet reporting goals. It is necessary to include small and private schools so that the students attending such schools are represented in the data collection and in the reports. The trained field staff work closely with all schools to ensure that the pre-assessment activities and the administration can be completed with minimal disruption.

A.6. Consequences of Collecting Information Less Frequently

Under the National Assessment of Educational Progress Authorization Act, Congress has mandated the on-going collection of NAEP data. Failure to collect the 2024 assessment data on the current schedule would affect the quality and schedule of the NAEP assessments and would result in assessments that would not fulfill the mandate of the legislation.

A.7. Consistency with 5 CFR 1320.5

No special circumstances are involved. This data collection observes all requirements of 5 CFR 1320.5.

A.8. Consultations Outside the Agency

The NAEP assessments are conducted by an alliance of organizations under contract with the U.S. Department of Education.⁸ The Alliance includes the following:

Management Strategies is responsible for managing the integration of multiple NAEP project schedules and providing data on timeliness, deliverables, and cost performance.
Educational Testing Service (ETS) is responsible for coordinating Alliance contractor activities, developing the assessment instruments, analyzing the data, preparing the reports, and platform development.
Huntington Ingalls Industries (HII) is responsible for NAEP web technology, development, operations, and maintenance including the Integrated Management System (IMS).
Pearson is responsible for printing and distributing the assessment materials, and for scanning and scoring students’ responses.
Westat is responsible for selecting the school and student samples, managing field operations and data collection, and coordinating with states and districts. Westat also provides ongoing support and training for full-time NAEP State and TUDA Coordinators in states across the nation through its NAEP Support and Service Center (NSSC).

In addition to the NAEP Alliance, other organizations support the NAEP program, all of which are under contract with the U.S. Department of Education. The current list of organizations include:⁹

American Institutes for Research (AIR) is responsible for providing technical support, conducting studies on state-level NAEP assessments, and running the NAEP Validity Studies Panel.
CRP, Inc. is responsible for providing logistical and programmatic support.
Hager Sharp is responsible for supporting the planning, development, and dissemination of NAEP publications and outreach activities.
Optimal Solutions Group is responsible for providing technical support.
State Education Agencies (SEAs) establish a liaison between the state education agency and NAEP, serve as the state’s representative to review NAEP assessment items and processes, coordinate the NAEP administration in the state, analyze and report NAEP data, and coordinate the use of NAEP results for policy and program planning.
Tribal Tech is responsible for providing support for the National Indian Education Study.

In addition to the contractors responsible for the development and administration of the NAEP assessments, the program involves many consultants and is also reviewed by specialists serving on various technical review panels. These consultants and special reviewers bring expertise concerning students of different ages, ethnic backgrounds, geographic regions, learning abilities, and socio-economic levels; the specific subject areas being assessed; the analysis methodologies employed; and large-scale assessment design and practices. Contractor staff and consultants have reviewed all items for bias and sensitivity issues, grade appropriateness, and appropriateness of content across states.

In particular, subject-area standing committees play a central role in the development of NAEP assessment instruments and have been essential in creating assessment content that is appropriate for the targeted populations, and that meets the expectations outlined in the Governing Board frameworks. One of the most important functions of the committees is to contribute to the validation of the assessments. Through detailed reviews of items, scoring guides, tasks, constructed-response item training sets for scorers, and other materials, the committees help establish that the assessments are accurate, accessible, fair, relevant, and grade-level appropriate, and that each item measures the knowledge and skills it was designed to measure. When appropriate, members of subject-area standing committees will also review the questionnaires with regards to appropriateness with existing curricular and instructional practices.

Appendix A lists the current members of the following NAEP advisory committees:

NAEP Design and Analysis Committee
NAEP Validity Studies Panel
NAEP National Indian Education Study Technical Review Panel
NAEP Mathematics Standing Committee
NAEP Reading Standing Committee
NAEP Survey Questionnaires Standing Committee
NAEP Mathematics Translation Review Committee
NAEP Grade 4 and 8 Survey Questionnaire and eNAEP DBA System Translation Review Committee
NAEP Principals’ Panel Standing Committee

As has been the practice for the past few years, OMB representatives will be invited to attend the technical review panel meetings that are most informative for OMB purposes.

In addition to the contractors and the external committees, NCES works with the NAEP State Coordinators, who serve as the liaison between each state education agency and NAEP, coordinating NAEP activities in his or her state. NAEP State Coordinators work directly with the schools sampled for NAEP.

A.9. Payments or Gifts to Respondents

In general, there will be no gifts or payments to respondents, although students do get to keep the NAEP earbuds used in DBA. On occasion, NAEP will leave educational materials at schools for their use (e.g., science kits from the science hands-on assessments). Some schools also offer recognition parties with pizza or other perks for students who participate; however, these are not reimbursed by NCES or the NAEP contractors. If any incentives are proposed as part of a future special study, they would be justified as part of that future clearance package. As appropriate, the amounts would be consistent with amounts approved in other studies with similar conditions.

A.10. Assurance of Confidentiality

Data security and confidentiality protection procedures have been put in place for NAEP to ensure that all NAEP contractors and agents (see section A.8 in this document) comply with all privacy requirements, including:

The Statements of Work of NAEP contracts;
National Assessment of Educational Progress Authorization Act (20 U.S.C. §9622);
Family Educational Rights and Privacy Act (FERPA) of 1974 (20 U.S.C. §1232(g));
Privacy Act of 1974 (5 U.S.C. §552a);
Privacy Act Regulations (34 CFR Part 5b);
Computer Security Act of 1987;
U.S.A. Patriot Act of 2001 (P.L. 107-56);
Education Sciences Reform Act of 2002 (ESRA 2002, 20 U.S.C. §9573);
Cybersecurity Enhancement Act of 2015 (6 U.S.C. §151);
Foundations of Evidence-Based Policymaking Act of 2018, Title III, Part B, Confidential Information Protection;
The U.S. Department of Education General Handbook for Information Technology Security General Support Systems and Major Applications Inventory Procedures (March 2005);
The U.S. Department of Education Incident Handling Procedures (February 2009);
The U.S. Department of Education, ACS Directive OM: 5-101, Contractor Employee Personnel Security Screenings;
NCES Statistical Standards; and
All new legislation that impacts the data collected through the contract for this study.

Furthermore, all NAEP contractors and agents will comply with the Department’s IT security policy requirements as set forth in the Handbook for Information Assurance Security Policy and related procedures and guidance, as well as IT security requirements in the Federal Information Security Management Act (FISMA), Federal Information Processing Standards (FIPS) publications, Office of Management and Budget (OMB) Circulars, and the National Institute of Standards and Technology (NIST) standards and guidance. All data products and publications will also adhere to the revised NCES Statistical Standards, as described at the website: http://nces.ed.gov/statprog/2012/. In addition, the Sampling and Data Collection (SDC) contractor has obtained an Authority to Operate (ATO) for the NCESLS System from the Education OCIO to operate at the FISMA moderate level through the Certification & Accreditation (C&A) process. Security controls include secure data processing centers and sites; properly vetted and cleared staff; and data sharing agreements.

An important privacy and confidentiality issue is the protection of the identity of assessed students, their teachers, and their schools. To assure this protection, NAEP has established security procedures, described below, that closely control access to potentially identifying information.

All assessment and questionnaire data are protected. This means that NAEP applications that handle assessment and questionnaire data:

enforce effective authentication password management policies;
limit authorization to individuals who truly need access to the data, only granting the minimum necessary access to individuals (i.e., least privilege user access);
keep data encrypted, both in storage and in transport, utilizing volume encryption and transport layer security protocols;
utilize SSL certificates and HTTPS protocols for web-based applications;
limit access to data via software and firewall configurations as well as not using well known ports for data connections; and
restrict access to the portable networks utilized to administer an assessment to only assessment devices.

Students’ names are submitted to the Sampling and Data Collection (SDC) contractor for selecting the student sample. This list also includes the month/year of birth, race/ethnicity, gender, and status codes for students with disabilities, English learners, and participation in the National School Lunch Program. This data request for NAEP fully conforms to the requirements of the Family Educational Rights and Privacy Act of 1974 (FERPA) [20 U.S.C. 1232g; 34 CFR Part 99]. FERPA is designed to protect the privacy rights of students and their families, by providing consistent standards for the release of personally identifiable student and family information. NCES and its agents are explicitly authorized under an exception to FERPA’s general consent rule to obtain student level data from institutions. For the purposes of this collection of data, FERPA permits educational agencies and institutions to disclose personally identifiable information from students’ education records, without consent, to authorized representatives of the Secretary of Education in connection with an evaluation of federally supported education programs (34 CFR §§ 99.31(a)(3)(iii) and 99.35).

After the student sample is selected, the data for selected students are submitted to the Materials Preparation, Distribution, Processing and Scoring (MDPS) contractor, who includes the data in the packaging and distribution system for the production of student-specific materials (such as labels to attach to the student forms or log-in ID cards), which are then forwarded to field staff and used to manage and facilitate the assessment. These data are also uploaded to the AMS online system for review by schools and used by field staff to print materials used by the schools. Student information is deleted from the packaging and distribution system before the assessment begins. Student information is securely deleted from the AMS typically two weeks after all quality control activities for the assessment are complete.

All paper-based student-specific materials linking personally identifiable information (PII) to assessment materials are destroyed at the schools upon completion of the assessment. The field staff remove names from forms and place the student names in the school storage envelope. The school storage envelope contains all of the forms and materials with student names and is kept at the school until the end of the school year and then destroyed by school personnel.^¹⁰

In addition to student information, teacher and principal names are collected and recorded in the AMS online system, which is used to keep track of the distribution and collection of NAEP teacher and school questionnaires. A paper copy of the questionnaire report is printed for use during the assessment, and this paper copy is left in the school storage envelope, which is destroyed at the end of the school year. The teacher and principal names are deleted from the AMS at the same time the student information is deleted.

For DBA, NAEP data are stored on systems in a locked-down environment at a secure hosting facility with strict measures in place to prevent unauthorized online access. The student names are not included on the assessment devices or stored by the same contractor or on the same database as the student responses. Shortly before, during, and after assessments, assessment data are transmitted through secure, encrypted channels (SSL, SSH) between NAEP systems, the NAEP assessment servers, and the assessment administration devices. Data on those devices are also encrypted—these data can be read only by the assessment software—and the devices are secured against unauthorized use.

Furthermore, to protect collected data, NAEP staff will use the following precautions:

Assessment and questionnaire data files will not identify individual respondents.
No personally identifiable information, either by schools or respondents, will be gathered or released by third parties. No permanent files of names or other direct identifiers of respondents will be maintained.
Student participation is voluntary.
NAEP data are perturbed. Data perturbation is a statistical data editing technique implemented to ensure privacy for student and school respondents to NAEP’s assessment questionnaires for assessments in which data are reported or attainable via restricted-use licensing arrangements with NCES. The process is coordinated in strict confidence with the IES Disclosure Review Board (DRB), with details of the process shared only with the DRB and a minimal number of contractor staff.

The following text appears on all student assessments, the AMS, and teacher and school questionnaires:

Paperwork Reduction Act (PRA) Statement

National Center for Education Statistics (NCES) conducts the National Assessment of Educational Progress to evaluate federally supported education programs. All of the information you provide may be used only for statistical purposes and may not be disclosed, or used, in identifiable form for any other purpose except as required by law (20 U.S.C. §9573 and 6 U.S.C. §151). By law, every NCES employee as well as every NCES agent, such as contractors and NAEP coordinators, has taken an oath and is subject to a jail term of up to 5 years, a fine of $250,000, or both if he or she willfully discloses ANY identifiable information about you. Electronic submission of your information will be monitored for viruses, malware, and other threats by Federal employees and contractors in accordance with the Cybersecurity Enhancement Act of 2015.

NCES estimates the time required to complete this information collection to average [XX] minutes, including the time to review instructions and complete and review the information collection. This voluntary information collection was reviewed and approved by OMB (Control No. 1850-0928). If you have any comments concerning the accuracy of the time estimate, suggestions for improving this collection, or any comments or concerns regarding the status of your individual submission, please write to: National Assessment of Educational Progress (NAEP), National Center for Education Statistics (NCES), Potomac Center Plaza, 550 12th St., SW, 4th floor, Washington, DC 20202, or send an email to: [email protected].

OMB No. 1850-0928 APPROVAL EXPIRES 4/30/2026

In addition, the following text appears on the log-in screen for the AMS system and NAEPq, the online system used for teacher and school administrator questionnaires.

AMS

When you have finished or if you need to stop before finishing, please LOG OUT of the survey system by clicking “Save and exit” and CLOSE ALL browser windows or screens to keep your responses secure. For example, if you used Chrome or Safari to open the survey, make sure no Chrome or Safari windows or screens are open after you end the survey. Not closing all browsers may allow someone else to see your responses.

NAEPq

When you have finished or if you need to stop before finishing, please LOG OUT of the survey system by clicking “Exit” and CLOSE ALL browser windows or screens to keep your responses secure. For example, if you used Chrome or Safari to open the survey, make sure no Chrome or Safari windows or screens are open after you end the survey. Not closing all browsers may allow someone else to see your responses.

More specific information about how NAEP handles PII is provided in the table below:

PII is created in the following ways	Public and non-public school samples are released by the SDC contractor to NAEP State Coordinators (public schools only), NAEP TUDA Coordinators (public schools only), and SDC Gaining Cooperation Field Staff (non-public schools only) using the secure Assessment Management System (AMS) for Schools website.
	Schools are recruited by SDC Gaining Cooperation field staff for participation in NAEP.
	Participating schools need to submit a current roster of students for the sampled grade for student sampling.
	Rosters of students can be created by NAEP State Coordinators, NAEP TUDA Coordinators, or NAEP School Coordinators.
	Rosters are submitted through the secure AMS website
	Rosters must be in Excel
	PII is contained in the roster files: state unique identifiers (optional), student names, month/year of birth, race/ethnicity, gender, and status codes for students with disabilities, English learners, and participation in the National School Lunch Program.
	PII is stored in the SDC contractor’s secure data environments.
PII is moved in the following ways	Student names (PII) are moved to the MDPS contractor via a secure FTP site. These names are used to print Student Login Cards.
	Student Login Cards are only created for students taking DBA, so the student names for the PBA students are not moved.
	Student PII data is available to the NAEP School Coordinators and the SDC contractor’s Field Staff through the secure AMS website.
	NAEP School Coordinators can view and update PII for their own schools
	NAEP School Coordinators can print materials containing PII for their own schools
	NAEP School Coordinators are instructed to destroy all materials containing PII at the end of the assessment cycle
	SDC contractor Field Staff can update PII for schools within their assignment
	SDC contractor Field Staff can print materials containing PII for schools within their assignment
	SDC contractor Field Staff store materials containing PII for schools within their assignment in their NAEP Provided storage
	At no point in time does any individual contractor have access to both the student name and student assessment and questionnaire responses. MDPS has access to both the student name and student assessment and questionnaire responses, but never at the same time. MDPS uses student PII to print Student Login Cards months in advance of the NAEP assessment window and destroys the student PII file after the assessment begins. SDC never has access to student responses, and no other contractor has access to Student PII.
PII is destroyed in the following ways	MDPS contractor destroys the PII after the assessment begins.
	School Coordinators destroy the materials containing PII on or before the end of the school year.
	SDC contractor Field Staff destroy the materials containing PII or leave materials containing PII at the school after the assessment has been completed. SDC contractor Field Staff return their NAEP School Folders to Westat Home Office for secure storage, and eventual secure destruction.
	SDC contractor destroys student names after all weighting quality control checks have been completed. This activity is completed approximately six months following the end of the administration.

In addition, parents are notified of the assessment. See Appendices D-10, D-11, D-17, D-21, and D-22 which provide a parental notification letter. The letter is adapted for each grade or age/subject combination and the school principal may edit it. However, the information regarding confidentiality and the appropriate law reference will remain unchanged. Please note that parents/guardians are required to receive notification of student participation but NAEP does not require explicit parental consent (by law, parents/guardians of students selected to participate in NAEP must be notified in writing of their child’s selection prior to the administration of the assessment).

A.11. Sensitive Questions

NAEP emphasizes voluntary respondent participation. Insensitive or offensive items are prohibited by the National Assessment of Educational Progress Authorization Act, and the Governing Board reviews all items for bias and sensitivity. The nature of the questions is guided by the reporting requirements in the legislation, the Governing Board’s Policy on the Collection and Reporting of Background Data, and the expertise and guidance of the NAEP Survey Questionnaire Standing Committee (see Appendix A-6). Throughout the item development process, NCES staff works with consultants, contractors, and internal reviewers to identify and eliminate potential bias in the items.

The NAEP student questionnaires include items that require students to provide responses on factual questions about their family’s socio-economic background, self-reported behaviors, and learning contexts, both in the school setting as well as more generally. In compliance with legislation, student questionnaires do not include items about family or personal beliefs (e.g., religious or political beliefs). The student questionnaires focus only on contextual factors that clearly relate to academic achievement.

Educators, psychologists, economists, and others have called for the collection of non-cognitive student information that can explain why some students do better in school than others. Similar questions have been included in other NCES administered assessments such as the Trends in International Mathematics and Science Study (TIMSS), the Program for International Student Assessment (PISA), the National School Climate Survey, and other federal questionnaires, including the U.S. Census. The insights achieved by the use of these well-established survey questions will help educators, policymakers, and other stakeholders make better informed decisions about how best to help students develop the knowledge and skills they need to succeed.

To provide additional context for NAEP performance results in 2024, the student, teacher, and school questionnaires include items that ask about students’ learning recovery experiences, teachers’ preparation and instructional practices, and schools’ instructional organization and practices related to COVID-19 learning recovery and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. During the development process, these COVID-19 learning recovery related items underwent a similar series of reviews for bias and sensitivity as the main questionnaire items. This included a sensitivity review conducted by the contractor’s independent group of reviewers who are not part of the NAEP program to identify potentially delicate, inflammatory, or inappropriate language, as well as small-scale cognitive interviews to determine whether students understood what the questions were asking and to identify and eliminate questions that students did not feel comfortable answering.

All questions proposed for piloting have gone through multiple rounds of reviews, including but not limited to reviews by NAEP subject-matter expert groups, organizational Internal Review Board (IRB), and the Governing Board, and have successfully passed extensive pre-testing via cognitive interviews with all respondent groups. Furthermore, NAEP does not report student responses at the individual or school level, but strictly in aggregate forms. To reduce the impact of any individual question on NAEP reporting, the program has shifted to a balanced reporting approach that includes multi-item indices, where possible, to maximize robustness and validity. In compliance with legislation and established practices through previous NAEP administrations, students may skip any question.

A.12. Estimation of Respondent Reporting Burden (2024)

The burden numbers for NAEP data collections fluctuate considerably, with the number of students sampled every other year being much larger than in the years in between.

Exhibit 1 provides the burden information per respondent group, by grade and by year, for the 2024 data collections.

Exhibit 2 summarizes the burden by respondent group.

A description of the respondents or study is provided below, as supporting information for Exhibit 1:

Students—Students in fourth, eighth, and twelfth grades will be assessed using 60-minutes of cognitive blocks in one subject followed by a non-cognitive block which requires up to a total of 15-minutes to complete. The core non-cognitive items are answered by students across subject areas and are related to demographic information. In addition, students answer subject-specific non-cognitive items. In 2024, students will also answer questions about their learning recovery experiences related to the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. Based on timing data collected from cognitive interviews and previous DBA, fourth-grade students can respond to approximately four non-cognitive items per minute, while eighth- and twelfth-grade students can respond to approximately six non-cognitive items per minute. Using this information, the non-cognitive blocks are assembled so that most students can complete all items in the allocated amount of time. Each cognitive and non-cognitive block is timed so that the burden listed above is the maximum burden time for each student. The administrators and/or test delivery system will move students to the next section once the maximum amount of time is reached. Additional student burden accounts for time to read directions, log on to the digital device, and view a tutorial. This additional burden is estimated at 15-minutes. The cognitive or assessment items are not included in the burden estimate because they are not subject to the Paperwork Reduction Act. Therefore, the total burden for students is 30-minutes. The assessments given in Puerto Rico are translated into Spanish. To account for the language complexities, additional time is provided for the cognitive blocks (for a total of 80-minutes). The burden for students in Puerto Rico is up to 15-minutes for the non-cognitive block, and an additional 15-minutes for directions, logging into the digital device, and the tutorial, for a total of 30-minutes.
Teachers—The teachers of fourth- and eighth-grade students participating in main NAEP are asked to complete questionnaires about their teaching background, education, training, and classroom organization. In 2024, teachers will also answer questions about their instructional design and learning recovery experiences for their students related to the COVID-19 and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. Average fourth-grade teacher burden is estimated to be 30-minutes because fourth-grade teachers often have multiple subject-specific sections to complete. Average eighth-grade teacher burden is 20-minutes if only one subject is taught and an additional 10-minutes for each additional subject taught. Based on timing data collected from cognitive interviews, adults can respond to approximately six non-cognitive items per minute. Using this information, the teacher questionnaires are assembled so that most teachers can complete the questionnaire in the estimated amount of time. For adult respondents, the burden listed is the estimated average burden.
Principals/Administrators—The school administrators in the sampled schools are asked to complete a questionnaire. The core items are designed to measure school characteristics and policies that research has shown are highly correlated with student achievement. Subject-specific items concentrate on curriculum and instructional services issues. In 2024, school administrators will also answer questions about their school’s instructional organization and practices related to learning recovery of students following the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. The burden for school administrators is determined in the same manner as burden for teachers (see above) and is estimated to average 30-minutes per principal/administrator, although burden may vary depending on the number of subject-specific sections included. The 30-minute burden estimate includes a supplemental charter school questionnaire designed to collect information on charter school policies and characteristics and is provided to administrators of charter schools who are sampled to participate in NAEP. The supplement covers organization and school governance, parental involvement, and curriculum and offerings.
SD and EL—SD and EL information is provided by school personnel concerning students identified as SD or EL. This information will be used by those personnel to determine the appropriate accommodations for students. The burden for school administrators is estimated at 15-minutes, on average, for each student identified as SD and/or EL.

Submission of Samples— Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically at the school, district, or state level. If done at the state level, some states require a data security agreement, which is customized based on the specific requests of the state and provides verbatim security and confidentiality information from Section A.10. If done at the school or district level, some burden will be incurred by school personnel. It is estimated that it will take two hours, on average, for school personnel to complete the submission process. Based on recent experience, it is estimated that approximately 17 percent of the schools will complete the submission process (based on the data from 2022).
School Coordinator Pre-Assessment and Assessment Activities—Each school participating in main NAEP has a designated staff member to serve as its NAEP school coordinator. Pre-assessment and assessment activities include functions such as finalizing student samples, verifying student demographics, reviewing accommodations, and planning logistics for the assessment. An electronic pre-assessment system (known as AMS) was developed so that school coordinators would provide requested administration information online, including logistical information, updates of student and teacher information, and the completion of inclusion and accommodation information. More information about the school coordinators’ responsibilities is included in Section B.2. Based on information collected from previous years’ use of the pre-assessment system, it is estimated that it will take four hours and 30 minutes, on average, for school personnel to complete these activities, including looking up information to enter into the system. We will use the AMS system data to learn more about participant response patterns and use this information to further refine the system to minimize school coordinator burden.
Technology Staff Pre-assessment and Assessment Activities—For each school in the sample, there will be up to 3 staff members designated to perform the technical preassessment and assessment activities at the school: up to two school technology coordinators and district technology coordinator. In preparation for the study, the school or district technology coordinators will take an online pre-assessment survey (which will be submitted in Amendment #3) to gather information about the schools’ ability to connect NAEP devices to the schools’ internet and bandwidth capacity at the schools. The schools’ technology coordinator or other appointed staff member will also run a NAEP online Internet Speed Test Tool during school hours to test the internet speeds at the school to determine if the school meets the minimum requirements as outlined in the Technical Fact Sheet (this will be included in Amendment #3). In addition, schools that meet the minimum technical requirements will run the Internet Speed Test Tool again at least once within two weeks prior to the scheduled assessment date to confirm the internet speeds at the school. It is estimated that for each school and district technology coordinator, it will take 20-minutes to complete the tasks. Additionally, technical staff assisting with school technology activities will be asked to take a brief (estimated at 5 minutes) survey, administered to inform future NAEP assessments administered on school-based equipment (SBE) (see draft in Appendix D-45). By utilizing these participants’ feedback about the technology from the 2024 operational assessment, NAEP will be better able to understand the available school-based equipment for the 2025 SBE Field Test.
School Staff Proctoring Accommodation Sessions—In 2024, NAEP will ask schools to assist in proctoring accommodation sessions that require a separate room. Based on recent assessments, about 40% of schools require a separate accommodation session on assessment day. One school staff member will be identified to proctor the assessment for the students requiring accommodations. The estimated burden for these staff will be 2.5 hours, which includes training to prepare for proctoring these separate sessions as well as proctoring the sessions themselves.

Assessment Feedback Survey—As part of the on-going quality control of the assessment process, schools will be asked to respond to an additional follow-up survey. Survey questions solicit pre-assessment feedback, assessment day feedback, and observer feedback. The sample post-assessment follow-up survey originally designed for use in 2022 is included in Appendix E and will be updated in Amendment #3. It is estimated that this interview will take on average 2-minutes.
HSTS—The NAEP HSTS periodically surveys the curricula being followed in our nation’s high schools and the course taking patterns of high school students through a collection of transcripts. To facilitate this study, school personnel submit the school information form and additional information, estimated at 1 hour. The course catalog and student transcripts can be submitted electronically or via paper copies. It is estimated that 40% of the states and most TUDAs will submit them electronically (estimated at 10 hours for these state or district personnel, which will account for the submissions for 30% of the schools) and 10% of the schools will submit them electronically (estimated at 2.5 hours for school personnel). For the remaining 60% of the schools, school personnel will support NAEP field staff collection of paper student transcripts (estimated at 2 hours).
NIES—NIES is designed to describe the condition of education for American Indian and Alaska Native (AI/AN) students in the United States. Additional questionnaires designed for NIES are given to students (estimated at 20 minutes), teachers (20 minutes), and school administrators (30 minutes).

Field Trial—The assessment and procedures are the same as those for the operational pilot assessment, with one exception: the teacher and school principal questionnaires are not administered in the field trial.

EXHIBIT 1

Estimated Burden for NAEP 2024 Assessments

(Note: all explanatory notes and footnotes are displayed following the table)

		4th Grade				8th Grade						12th Grade
	Subjects	OP and Pilot	Puerto Rico OP and Pilot	NIES⁴	Field Trial		OP and Pilot	Science	Puerto Rico OP and Pilot	NIES⁴	Field Trial	OP	HSTS⁵	Field Trial	Total
Students	# of Students	253,200	5,000	8,000	324		250,200	22,000	5,000	6,500	420	52,000	0	50	588,194
	Avg. min. per response	30	30	20	30		30	30	30	20	30	30	0	30	N/A
	Burden (in hours)	126,600	2,500	2,667	162		125,100	11,000	2,500	2,167	210	26,000	0	25	298,931
Teachers	# of Teachers	19,821	750	2,000	0		25,408	4,064	1,000	1,500	0	0	0	0	51,043
	Avg. minutes per response	30	30	20	0		20 and 10 minutes for each additional subject³	20	20	20	0	0	0	0	N/A
	Burden (in hours)	9,911	375	667	0		10,587	1,355	333	500	0	0	0	0	23,728
School Questionnaire (school principal)	# of Schools	6,607	250	2,200	0		6,352	1,016	250	2,000	0	1,545	0	0	16,020
	Avg. minutes per response	30	30	30	0		30	30	30	30	0	30	0	0	N/A
	Burden (in hours)	3,304	125	1,100	0		3,176	508	125	1,000	0	773	0	0	10,111
Pre-assessment Technology Activities⁶	# of Technical Staff	19,821	750	N/A	24		19,056	3,048	750	N/A	30	4,635	0	9	48,123
Pre-assessment Technology Activities⁶	Burden (in hours)	7,158	271	N/A	9		6,881	1,101	271	N/A	11	1,674	0	3	17,378
Pre-assessment, sample submission, & assessment feedback (school coordinator)	# of School Coord.	6,607	250	N/A	8		6,352	1,016	250	N/A	10	1,545	1,545	3	17,586
	Burden (in hours)¹	33,387	1,263	N/A	40		32,099	5,134	1,263	N/A	51	7,807	4,255	15	85,314
SD/EL (school personnel)	# of Schools	6,607	250	N/A	8		6,352	1,016	250	N/A	10	1,545	0	3	16,041
	# of SD/EL Students²	68,364	1,350	N/A	87		55,044	4,840	1,100	N/A	92	8,320	0	8	139,205
	Avg. minutes per response	15	15	N/A	15		15	15	15	N/A	15	15	0	15	135
	Burden (in hours)	17,091	338	N/A	22		13,761	1,210	275	N/A	23	2,080	0	2	34,802
Accommodation sessions proctored by school staff⁷	# of school staff	2,643	100	N/A	3		2,541	406	100	N/A	4	618	0	1	6,416
Accommodation sessions proctored by school staff⁷	Burden (in hours)	6,607	250	N/A	8		6,352	1,016	250	N/A	10	1,545	0	3	16,041
Total Burden (in hours)		196,900	4,851	4,434	232		191,075	20,223	4,746	3,667	294	38,205	4,255	45	486,305

Total number of respondents: 743,423

Total number of responses: 866,587

Notes for 2024 table in Exhibit 1

The burden for the school coordinator is as follows: Pre-assessment burden is 4.5 hours, sample submission burden is 2 hours (for 26% of schools in 2022 based on 2019 data), and the post-assessment follow-up survey is 2 minutes. For the purposes of the calculation of burden, we consider the performance of all of these tasks to constitute 1 response.
The estimated percent of SD/EL students is 27% and 22% at grades 4 and 8 (based on the NAEP 2022 sample), respectively, and 16% at grade 12 (based on the NAEP 2019 sample).
Grade 8 teachers who teach one subject have an estimated burden of 20-minutes, with an additional 10-minutes for each additional subject. The estimated number of teachers who teach 1 subject is 50% and 2 subjects is 50%. There is only one teacher questionnaire for science, which is assessed in a separate sample of schools from the reading and math assessments.
The burden for NIES is associated with the additional questionnaire that is given to the same students, teachers, and school administrators that respond to the main NAEP questionnaires. As such, the NIES questionnaire does not impact the total number of respondents. The estimated number of students, teachers, and school administrators that will respond to the NIES questionnaires is based on the 2019 sample.
The burden for HSTS is as follows: school personnel in all 12th grade schools will submit the school information form and additional information (1 hour); state or district personnel will submit the course catalog and student transcripts electronically (10 hours for 30% of the schools which come from 40% of the states and all TUDAs); school personnel in 10% of the schools will submit the course catalog and student transcript electronically (2.5 hours); and school personnel in 60% of the schools will support NAEP field staff collection of paper student transcripts (2 hours).
The Pre-assessment Technology Activities will be completed by the school technology coordinators (estimated at 2 per school) and the district technology coordinator (1 per school). The estimated burden for these staff is estimated to be approximately 20 minutes each, and an additional 5 minutes for one member to complete the SBE staff survey.
The burden for accommodation sessions proctored by school staff is 2.5 hours for 40% of the schools (based on recent assessments).

EXHIBIT 2

Total Annual Estimated Burden Time Cost for NAEP 2024 Assessments

Data Collection Year	Number of Respondents	Number of Responses	Total Burden (in hours)
2024	743,423	866,587	486,305

The estimated respondent burden across all these activities translates into an estimated total burden time cost 486,305 hours¹¹, broken out by respondent group in the table below.

	Students		Teachers and School Staff		Principals		Total
	Hours	Cost	Hours	Cost	Hours	Cost	Hours	Cost
2024	298,931	$2,167,250	177,263	$5,780,546	10,111	$498,978	486,305	$8,446,774

A.13. Cost to Respondents

There are no direct costs to respondents.

A.14. Estimates of Cost to the Federal Government

The total cost to the federal government for the administrations of the 2024 NAEP data collections (contract costs and NCES salaries and expenses) is estimated to be $149,999,426. The 2024 assessment cost estimate is shown in the table below.

NCES salaries and expenses	$1,400,300
Contract costs	$148,599,126
Printing, packaging, and distribution, and scoring	$16,732,431
Item Development	$20,602,000
Sampling, recruiting and training, data collection, and weighting	$88,641,881
Recruitment and State Support	$1,000,000
Design, analysis and reporting	$8,373,314
Securing and transferring DBA assessment data	$120,000
DBA system development	$13,129,500

A.15. Reasons for Changes in Burden and Budget (from last Clearance submittal)

Since the submission of Amendment #1, there have been some additional activities that have decreased burden and increased budget for this current Amendment #2. Changes to scope include the following:

1) Addition of Reading Router Pilot for grades 4 and 8, increasing costs;

2) Addition of School and District Technology Coordinator roles and SBE survey completion, increasing burden hours;

3) Addition of protocols for the health and safety of field staff, increasing costs;

4) Reduction in SQ burden time for students, teachers, and schools since COVID-19 learning recovery items are no longer adding additional time to the SQs; rather, other items were dropped to accommodate these items, reducing burden hours;

5) Addition of Field Trial for grades 4, 8 and 12, increasing burden hours and costs.

The combination of the new activities, while changing the scope of the administration between Amendments, equates to a 33,300 burden-hour decrease in 2024 Amendment #2 compared to Amendment #1 (486,305 burden hours in Amendment #2 compared to 519,605 burden hours in Amendment #1). The costs of added scope activities (#1, #2, #3, and #5 in the above listing), however, equate to an increase in $4,059,127 dollars in Amendment #2 compared to Amendment #1 ($149,999,426 in costs in Amendment #2 compared to $145,940,299 in costs in Amendment #1).

A.16. Time Schedule for Data Collection and Publications

The time schedule for the data collection for the 2024 assessments is shown below.

NAEP 2024 Field Trial	November 2023
NAEP 2024 Administration	January–March 2024
HSTS	October 2023–October 2024

The grades 4, 8, and 12 reading and mathematics national and state results are typically released to the public around October of the same year (i.e., about 6-7 months after the end of data collection). All other operational assessments are typically released 12-15 months after the end of data collection. However, given the comparability study comparing the administration using Chromebooks and Surface Pros, the analysis will require additional time and the results will be later.

The operational schedule for the NAEP assessments generally follows the same schedule for each assessment cycle. The dates below show the timeframe for the 2024 state-level assessments:

Spring–Summer 2023: Select the school sample and notify schools
October–November 2023: States, districts, or schools submit the list of students
November 2023: Administer Field Trial
December 2023: Select the student sample
December 2023–January 2024: Schools prepare for the assessments using the AMS system
January–March 2024: Administer the assessments
March–May 2024: Process the data, score constructed response items, and calculate sampling weights
March–September 2024: Analyze the data
September–December 2024: Prepare the reports, obtaining feedback from reviewers
January or February 2025 (Grades 4/8, Reading and Mathematics): Release the results
June or July 2025 (Grade 8 Science, Grade 12 Reading and Mathematics): Release the results

A.17. Approval for Not Displaying OMB Approval Expiration Date

No exception is requested.

A.18. Exceptions to Certification Statement

No exception is requested.

1 The role of NCES, led by the Commissioner for Education Statistics, is defined in 20 U.S.C. §9622 (https://www.law.cornell.edu/uscode/text/20/9622) and OMB Statistical Policy Directives No. 1 and 4 (https://obamawhitehouse.archives.gov/omb/inforeg_statpolicy).

2 The grade 12 economics teacher match rate was 56 percent in 2012. For comparison, the 2015 teacher match rates for grades 4 and 8 were approximately 94 percent and 86 percent, respectively.

3 See Section A.2 for more information about how NAEP results are reported.

4 See Section B.1.a for more information on the NAEP sampling procedures.

5 See Section B.2 regarding procedures for data collection.

6 The Governing Board assessment schedule can be found at https://www.nagb.gov/about-naep/assessment-schedule.html.

7 Additional information on the AMS site is included in the Section B.2.

8 The current contract expires on June 30, 2024.

9 The current contracts expire at varying times. As such, the specific contracting organizations may change during the course of the time period covered under this submittal.

10 In early May, schools receive an email from the AMS reminding them to securely destroy the contents of the NAEP storage envelope and confirm that they have done so. The confirmation is recorded in the system and tracked.

11 The average hourly earnings of teachers and principals derived from May 2021 Bureau of Labor Statistics (BLS) Occupation Employment Statistics is $32.61 for teachers and school staff and $49.35 for principals. If mean hourly wage was not provided, it was computed assuming 2,080 hours per year. The exception is the student wage, which is based on the federal minimum wage of $7.25 an hour. Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ datatype: Occupation codes: Elementary school teachers (25-2021); Middle school teachers (25-2022); High school teachers (25-2031); Principals (11-9032); last modified date May 2021.

File Type	application/vnd.openxmlformats-officedocument.wordprocessingml.document
File Modified	0000-00-00
File Created	0000-00-00

NAEP 2024 Part A v30