Change Memo

June 23, 2023

MEMORANDUM

To: Bev Pratt, OMB

From: Pat Etienne, NCES

Through: Carrie Clarady, NCES

Re: National Assessment of Educational Progress (NAEP) 2024 Materials Update #2
(OMB# 1850-0928 v.30)

The National Assessment of Educational Progress (NAEP), conducted by the National Center for Education Statistics (NCES), is a federally authorized survey of student achievement at grades 4, 8, and 12 in various subject areas, such as mathematics, reading, writing, science, U.S. history, civics, and technology and engineering literacy (TEL). The National Assessment of Educational Progress Authorization Act (Public Law 107-279 Title III, section 303) requires the assessment to collect data on specified student groups and characteristics, including information organized by race/ethnicity, gender, socio-economic status, disability, and limited English proficiency. It requires fair and accurate presentation of achievement data and permits the collection of background, noncognitive, or descriptive information that is related to academic achievement and aids in fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and subpopulations of students and to monitor progress over time. The nature of NAEP is that burden alternates from a relatively low burden in national-level administration years to a substantial burden increase in state-level administration years when the sample has to allow for estimates for individual states and some of the large urban districts.

The initial request for clearance of NAEP 2024 received OMB approval in April 2023 (OMB# 1850-0928 v.28). Amendment #1 to the NAEP 2024 clearance package received OMB approval in June 2023 (OMB#1850-0928 v.29). Since that package’s submission for public comment and OMB approval, changes have occurred to the scope of the 2024 NAEP administration, including the addition of: 1) Addition of Reading Router Pilot for grades 4 and 8, increasing costs, 2) Addition of School and District Technology Coordinator roles and SBE survey completion, increasing burden hours, 3) Addition of protocols for the health and safety of field staff, increasing costs, 4) Reduction in SQ burden time for students, teachers and schools since COVID-19 learning recovery items are no longer adding additional time to the SQs; rather, other items were dropped to accommodate these items, reducing burden hours; and 5) Addition of Field Trial for grades, 4, 8 and 12, increasing burden hours and costs. This revision updates Part A and Part B detailing the changes to scope and references to the communication materials and the amendment schedule, Appendix A, Appendix B, Appendix C, Appendix D (added communication materials), Appendix G, Appendix I, and Appendices J1, J2, J3, and J-S to include the operational survey questionnaires (SQs), COVID-19 Learning Recovery SQs, NIES SQs, and Pilot SQs.

The following table and pages below provide a summary of the changes that were made to the documents included in this submission as compared to the documents submitted in the 2024 NAEP Amendment #1 Package. There have been changes to the burden and costs to the Federal Government in this Amendment #2. The combination of the added activities listed above, while changing the scope of the administration between Amendments, does equate to a decrease in 33,300 burden hours in 2024 Amendment #2 compared to Amendment #1 (486,305 burden hours in Amendment #2 compared to 519,605 burden hours in Amendment #1). The costs of these added activities, however, equate to an increase in $4,059,127 dollars in Amendment #2 compared to Amendment #1 ($149,999,426 in costs in Amendment #2 compared to $145,940,299 in costs in Amendment #1).

Summary of All Changes

Document	Changes
Part A	Updates to scope: 1) Addition of Reading Router Pilot for grades 4 and 8, increasing costs; 2) Addition of School and District Technology Coordinator roles and SBE survey completion, increasing burden hours; 3) Addition of protocols for the health and safety of field staff, increasing costs; 4) Reduction in SQ burden time for students, teachers, and schools since COVID-19 learning recovery items are no longer adding additional time to the SQs; rather, other items were dropped to accommodate these items, reducing burden hours; 5) Addition of Field Trial for grades, 4, 8 and 12, increasing burden hours and costs. Updates to references to communication materials. Updates to the amendment schedule. Updates to the burden and costs to the Federal Government.
Part B	Updates to Appendix G Sample Design and Appendix C Design Memo years. Updates to B.2.a. (Recruitment of Schools) communication flow and Appendix D communication material references; small adjustments to communication material references B.2.b. (school coordinator responsibilities). Updates to references from Appendix H to Appendix I for Instructions for Entering Student Information as it is now incorporated into the AMS.
Appendix A	Updated committee members.
Appendix B	Updated Weighting Procedures from 2017 to 2018.
Appendix C (draft sampling memo)	Included; updated for 2024 NAEP.
Appendix D	Appendix D1 and D2 has been consolidated into Appendix D. Appendices D-6, D-7, D-10, D-11, D-16, D-21 to D36, D-38 to D-43, D-49, D-2-S-PR, D-5-S-PR to D-14-S-PR are new in this submission.
Appendix G (2018 sampling design)	Included in this submission.
Appendix H	Appendix H is no longer included as a separate document since the AMS now includes the eFiling process in Appendix I. As a result, it is removed from the list of documents.
Appendix I	New English and Spanish (PR) AMS screens added to previously approved AMS screens from Field Test OMB package (OMB# 1850-0803 v.332). The newly added PRA language on page 3 will be programmed into the log in screen for Amendment #3. Updated the Login screen, School Summary Page, Instructions for Preparing and Importing a Student Excel File. Updated Sex/Gender item available for schools for student roster upload. (pp. I-9-10) Appendix I2 is now fused into Appendix I, thus new HSTS screens are now included as part of the overall AMS Appendix I.
Appendix J1	Updates to the operational student core, mathematics, and reading survey questions are added. Some core, mathematics, and reading items were dropped or revised across grades 4, 8, 12 to allow for the addition of COVID-19 learning recovery items. Specific updates are detailed in the summary tables. COVID-19 learning recovery SQs, NIES, SQs, and Pilot Core, Mathematics, and Reading SQs were added. COVID-19 learning recovery items were added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
Appendix J2	“Teacher preparation” core items were replaced to reflect the subjects being administered in the 2024 assessments. The teacher gender items for grades 4 and 8 were revised to add a third option “another gender (please specify)” to be more inclusive and is now in line with an NCES-wide change. COVID-19 learning recovery SQs, NIES, SQs, and Pilot Core, Mathematics, and Reading SQs were added. COVID-19 learning recovery items were added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
Appendix J3	Updates to the operational school core, mathematics, and reading survey questions are added. Some core items were dropped across grades 4 and 8 to allow for the addition of COVID-19 learning recovery items. The “what is your sex” question was dropped from grade 12 core to align with this item being dropped in grades 4 and 8 before the 2022 admin. Specific updates are detailed in the summary tables. Grade 4, 8, 12 reading items were revised to update applicable terminology from English-language learners (ELLs) to English learners (ELs). COVID-19 learning recovery SQs, NIES, SQs, and Pilot Core, Mathematics, and Reading SQs were added. COVID-19 learning recovery items were added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
Appendix J-S	Updates to the operational student core, mathematics, and reading survey questions are added. Some core, mathematics, and reading items were dropped or revised across grades 4 and 8 to allow for the addition of COVID-19 learning recovery items. The race/ethnicity options were revised to align with the 2020 Census Bureau categories. Specific updates are detailed in the summary tables. COVID-19 learning recovery SQs and Pilot Core, Mathematics, Science, and Reading SQs were added. COVID-19 learning recovery items were added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. The teacher gender items for grades 4 and 8 were revised to add a third option “another gender (please specify)” to be more inclusive and is now in line with an NCES-wide change.

Changes from 2024 NAEP Part A Amendment #1 (v.29) to Amendment #2 (v.30) are shown below:

A.1.a. Purpose of Submission

NAEP consists of two assessment programs: the NAEP long-term trend (LTT) assessment and the main NAEP assessment. The LTT assessments are given at the national level only and are administered to students at ages 9, 13, and 17 in a manner that is very different from that used for the main NAEP assessments. LTT reports mathematics and reading results that present trend data since the 1970s. NAEP provides results on subject-matter achievement, instructional experiences, and school environment for populations of students (e.g., all fourth-graders) and groups within those populations (e.g., female students, Hispanic students). NAEP does not provide scores for individual students or schools. The main NAEP assessments report current achievement levels and trends in student achievement at grades 4, 8, and 12 for the nation and, for certain assessments (e.g., reading and mathematics), states and select urban districts. The Trial Urban District Assessment (TUDA) is a special project developed to determine the feasibility of reporting district-level results for large urban districts. Currently, the following 26 27 districts participate in the TUDA program: Albuquerque, Atlanta, Austin, Baltimore City, Boston, Charlotte, Chicago, Clark County (NV), Cleveland, Dallas, Denver, Detroit, District of Columbia (DCPS), Duval County (FL), Fort Worth, Guilford County (NC), Hillsborough County (FL), Houston, Jefferson County (KY), Orange County (FL), Los Angeles, Miami-Dade, Milwaukee, New York City, Philadelphia, San Diego, and Shelby County (TN).

This request is to conduct NAEP in 2024, specifically:

• Main NAEP operational assessments in 2024 for grade 4 (reading and mathematics), 8 (reading, mathematics and science), and 12 (reading and mathematics); In Puerto Rico, grades 4 and 8 mathematics will be the only subject assessed.

• Pilot testing for new frameworks in mathematics (mainland U.S. and Puerto Rico) and reading for grades 4 and 8,

• Pilot testing new Reading Router for grades 4 and 8,

• High School Transcript Study (HSTS), and

• National Indian Education Study (NIES) for grades 4 and 8, and

• 2024 Field Trial (also known as the Dress Rehearsal).

Beginning in 2024, NAEP will be transitioning to a Next-generation (Next-gen) version of the eNAEP test delivery software, the platform on which the assessment is delivered to students. NAEP will also be changing the operational assessment delivery model. While NAEP is currently administered by numerous NAEP field staff entering schools bringing NAEP-provided Surface Pros and Chromebooks, the program is planning to transition to a model that is ultimately less expensive and more aligned with the administration model used in state assessments. In particular, NAEP ultimately aims to administer the assessment using school-provided equipment and internet, with an alternate delivery model of cheaper, more streamlined NAEP-provided devices to be used in schools that do not meet the minimum specifications for school equipment.

In order to successfully transition to this ultimate plan, a staged approach is currently being undertaken so that trends can be measured across time. Namely, NAEP has conducted an Assessment Delivery study in 2022 (OMB# 1850-0803 v.305) and as well as is preparing for a Field Test in 2023 (OMB# 1850-0803 v.293) to provide more information about student interactions with the Next-gen eNAEP system and prepare for use of the updated system in operational NAEP assessments moving forward. In preparation for the 2024 NAEP administration, a Field Trial (also known as the “Dress Rehearsal”) will be conducted with students in a live classroom environment in November 2023 by NAEP field administration staff. The Field Trial will replicate the NAEP Operational Administration testing conditions to the fullest extent possible in a small number of schools in 15 schools selected throughout mainland U.S. and 6 schools in Puerto Rico. The NAEP program has utilized field trials prior to large-scale digitally based assessments since 2018, based on lessons learned and issues encountered by students in the field during the 2017 operational assessment. Results from this study will not be publicly released; rather they will be used to identify issues and generate solutions or workarounds in advance of the 2024 administration.

NAEP 2024 Amendment Schedule Table
Amendment #2 (May 2023 this document)	Part A: Updates to the 2024 NAEP administration Part B: Updates to the 2024 NAEP administration Appendix A: External Advisory Committees Appendix B: 2018 Weighting Procedures Appendix C: 2024 Sampling Memo Draft Appendixces D (formerly D1 and D2): Final Updated communication materials (English, Spanish, and HSTS) Appendix E: Assessment Feedback Form Appendix G: Sampling Design Final Appendix H: Instructions for Entering Student Information Appendix I (formerly I and I2): New AMS System – Initial Access Components; HSTS AMS screens. Note: Appendix H is no longer included as a separate document since the AMS now includes the eFiling process in Appendix I. As a result, it is removed from the list of documents. Appendix I2: HSTS AMS screens now merged into Appendix I Appendix J1-J3 and J-S: All 2024 SQs, including NIES (formerly in Appendix K): 2019 NIES SQs
Amendment #3 (July 2023)	Part A: Updates to the 2024 NAEP administration (if needed) Part B: Updates to the 2024 NAEP administration (if needed) Appendix C: Sampling Memo Final Appendix D: Updates to the 2024 Final NAEP communication materials Appendix E: Assessment Feedback Forms Appendix I: New AMS System – Final Access Components

A.1.c. Overview of NAEP Assessments

A.1.c.3. Survey Items

In addition to assessing subject-area achievement, NAEP collects information that serves to fulfill the reporting requirements of the federal legislation and to provide context for the reporting of student performance. The legislation requires that, whenever feasible, NAEP includes information on special groups (e.g., information reported by race, ethnicity, socio-economic status, gender, disability, and limited English proficiency). As part of most NAEP assessments, three types of questionnaires are used to collect information: student, teacher, and school (including the COVID-19 Learning Recovery items for 2024) for each respondent type). An overview of the questionnaires is presented below.

Each student questionnaire includes three types of items:

• General student information: Student responses to these items are used to collect information about factors such as race or ethnicity and parents’ education level. Answers on the questionnaires also provide information about factors associated with academic performance, including household composition, academic self-discipline, and the number of books in the home.

• Other contextual/policy information: These items focus on students’ educational settings and experiences and collect information about students’ attendance (i.e., days absent), family discourse (i.e., talking about school at home), reading load (i.e., pages read per day), and technology use at school. There are also items that ask about students’ effort on the assessment and the difficulty of the assessment. In 2024, students will answer questions about their learning recovery experiences related to the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. As of 2021, student questionnaires have been expanded to include COVID-19 outbreak questions which ask about the impact of the crisis on instruction. Answers on the questionnaires provide information on how aspects of education and educational resources are distributed among different groups.

• Subject-specific information: In most NAEP administrations, these items cover three categories of information: (1) time spent studying the subject; (2) instructional experiences in the subject; and (3) student factors (e.g., effort, confidence) related to the subject and the assessment.

Teacher Questionnaires

Teacher questionnaires are organized into different parts. The first part of the teacher questionnaire covers background and general training and includes items concerning years of teaching experience, certifications, degrees, major and minor fields of study, coursework in education, coursework in specific subject areas, the amount of in-service training, the extent of control over instructional issues, and the availability of resources for the classroom. Subsequent parts of the teacher questionnaire tend to cover training in the subject area, classroom instructional information, and teacher exposure to issues related to the subject and the teaching of the subject. In 2024, teachers will answer questions about their instructional design and learning recovery experiences related to the COVID-19 and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. As of 2021, teacher questionnaires have been expanded to include COVID-19 outbreak questions which ask about the impact of the crisis on their teaching. They also ask about pre- and in-service training, the ability level of the students in the class, the length of homework assignments, the use of particular resources, and how students are assigned to particular classes.

School Questionnaires

The school questionnaire is accessed online through NAEPq and is organized into different parts. The first part tends to cover characteristics of the school, including the length of the school day and year, school enrollment, absenteeism, dropout rates, and the size and composition of the teaching staff. Subsequent parts of the school questionnaire tend to cover tracking policies, curricula, testing practices, special priorities, and schoolwide programs and problems. The questionnaire also collects information about the availability of resources, policies for parental involvement, special services, and community services. Printed versions of the survey questionnaires are also available, if requested. Approximately 4.7 percent of the responses received, were by hardcopy in 2022. In 2024, school administrators will answer questions about their school’s instructional organization and practices related to learning recovery following the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. As of 2021, school questionnaires have been expanded to include COVID-19 outbreak questions which ask about the impact of the crisis on instruction.

A.1.c.5. Digitally Based Assessments (DBA)

In addition to new item types, the transition to DBA makes it possible for NAEP to employ an adaptive testing design, in which assessment content is targeted to students’ ability based on performance during the test administration. Thus, students see items that are tailored to their ability levels, and they may be more likely to be able to engage in the assessment and demonstrate what they know and can do. The goal of implementing adaptive testing is to achieve better measurement of student knowledge and skills across the wide range of student performance levels on which NAEP reports.

The type of adaptive testing being considered for NAEP is a multi-stage test (MST) design that uses two stages. Students take sections of cognitive items, just as in past NAEP administrations. Based on their performance on the first section of items, students receive one or more subsequent sections of items that is targeted to their ability level. For example, students who do not perform well on the first section of items receive a second section composed of somewhat easier items. The implementation of this two-stage MST design for NAEP has been informed by previous research on the benefits, applicability, and feasibility of adaptive testing for NAEP.

In order to support the transition from linear to MST in reading, a Reading Router is being developed, and will be piloted in 2024. This pilot of the reading router will be administered in a linear test and not as an MST. The reading router is a short block that will (when actually in MST) be scored in the system and a routing decision to targeted difficulty on subsequent blocks, based on students’ scores, will be made in real time. 

A.1.c.6. Assessment Types

Field Trial

The purpose of the 2024 Field Trial is to perform a “dress rehearsal” prior to the 2024 NAEP Operational Administration. The Field Trial will be conducted with students in a live classroom environment at a small number of schools, allowing the system to be tested in the manner in which it will be used in the national study to help identify platform system or operational issues prior to the administration.

A.1.d. Overview of 2024 NAEP Assessments

The Governing Board determines NAEP policy and the assessment schedule,¹ and future Governing Board decisions may result in changes to the plans represented here. Any changes will be presented in subsequent clearance packages or revisions to the current package.

The 2024 data collection will consist of the following:

• Operational national/state/TUDA DBA in mathematics and reading at grades 4 and 8, and Puerto Rico in mathematics at grades 4 and 8;

• Operational national DBA in science at grade 8;

• Operational national DBA in mathematics and reading at grade 12;

• Pilot in mathematics (mainland U.S. and Puerto Rico) and reading (mainland U.S. only) at grades 4 and 8;

• Pilot for Reading Router in grades 4 and 8;

• National Indian Education Study (NIES);

• High School Transcript Study (HSTS).

The operational assessments will include a comparability study between administration of the assessment on Chromebooks and Surface Pros.

High School Transcript Study (HSTS)

If the HSTS school coordinator submits the transcripts, a list of only the students sampled for grade 12 mathematics can be obtained from the secure AMS website and transcripts will be provided for those students only. If, however, the state or TUDA district (TUDA or non-TUDA) coordinator submits the transcripts, he or she will need to submit transcripts without realizing the student sample. Therefore, one of two methods can be used:

• Download a data template from AMS containing a list of all grade 12 students.

• Download a data template from AMS containing a list of students sampled for grade 12 mathematics and reading, plus an additional 10 percent of students who were not sampled.

A.3. Improved Use of Technology

Automated Scoring

NAEP administers a combination of selected-response items and open-ended or constructed-response items. NAEP currently uses human scorers to score the constructed-response items, using detailed scoring rubrics and proven scoring methodologies. With the increased use of technologies, the methodology and reliability of automated scoring (i.e., the scoring of constructed-response items using computer software) has advanced. While NAEP does not currently employ automated scoring methodologies operationally, these are being investigated for possible future use. In particular, NCES recently held a competition to examine a variety of automated scoring engines and methods for consideration in NAEP (see: https://nces.ed.gov/whatsnew/press_releases/1_21_2022.asp).

A.10. Assurance of Confidentiality

In addition, parents are notified of the assessment. See Appendices D2-10 and D2-11 samples used in 2022 which provide a parental notification letter. The letter is adapted for each grade or age/subject combination and the school principal may edit it. However, the information regarding confidentiality and the appropriate law reference will remain unchanged. Please note that parents/guardians are required to receive notification of student participation but NAEP does not require explicit parental consent (by law, parents/guardians of students selected to participate in NAEP must be notified in writing of their child’s selection prior to the administration of the assessment).

A.11. Sensitive Questions

To provide additional context for NAEP performance results in 2024, the student, teacher, and school questionnaires include items that ask about students’ learning recovery experiences, teachers’ preparation and instructional practices, and schools’ preparation and instructional organization and practices related to the COVID-19 outbreaklearning recovery and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. During the development process, these COVID-19 learning recovery related items underwent a similar series of reviews for bias and sensitivity as the main questionnaire items. This included a sensitivity review conducted by the contractor’s independent group of reviewers who are not part of the NAEP program to identify potentially delicate, inflammatory, or inappropriate language, as well as small-scale cognitive interviews to determine whether students understood what the questions were asking and to identify and eliminate questions that students did not feel comfortable answering.

A.12. Estimation of Respondent Reporting Burden (2024)

A description of the respondents or study is provided below, as supporting information for Exhibit 1:

• Students—Students in fourth, eighth, and twelfth grades will be assessed using 60-minutes of cognitive blocks in one subject followed by a non-cognitive block which requires up to a total of 15-minutes to complete. The core non-cognitive items are answered by students across subject areas and are related to demographic information. In addition, students answer subject-specific non-cognitive items. In 2024, students will also answer questions about their learning recovery experiences related to the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic, adding an additional 5-minutes to the non-cognitive block, for a total of 20-minutes. Based on timing data collected from cognitive interviews and previous DBA, fourth-grade students can respond to approximately four non-cognitive items per minute, while eighth- and twelfth-grade students can respond to approximately six non-cognitive items per minute. Using this information, the non-cognitive blocks are assembled so that most students can complete all items in the allocated amount of time. Each cognitive and non-cognitive block is timed so that the burden listed above is the maximum burden time for each student. The administrators and/or test delivery system will move students to the next section once the maximum amount of time is reached. Additional student burden accounts for time to read directions, log on to the digital device, and view a tutorial. This additional burden is estimated at 15-minutes. The cognitive or assessment items are not included in the burden estimate because they are not subject to the Paperwork Reduction Act. Therefore, the total burden for students is 3530-minutes. The assessments given in Puerto Rico are translated into Spanish. To account for the language complexities, additional time is provided for the cognitive blocks (for a total of 80-minutes). The burden for students in Puerto Rico is up to 15-minutes for the non-cognitive block, an additional 5-minutes for the COVID-19 block, and an additional 15-minutes for directions, logging into the digital device, and the tutorial, for a total of 3530-minutes.

• Teachers—The teachers of fourth- and eighth-grade students participating in main NAEP are asked to complete questionnaires about their teaching background, education, training, and classroom organization. In 2024, teachers will also answer questions about their instructional design and learning recovery experiences for their students related to the COVID-19 and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic their teaching preparation and instructional practices related to the COVID-19 outbreak. Average fourth-grade teacher burden is estimated to be 3530-minutes because fourth-grade teachers often have multiple subject-specific sections to complete. Average eighth-grade teacher burden is 2520-minutes if only one subject is taught and an additional 10-minutes for each additional subject taught. Based on timing data collected from cognitive interviews, adults can respond to approximately six non-cognitive items per minute. Using this information, the teacher questionnaires are assembled so that most teachers can complete the questionnaire in the estimated amount of time. For adult respondents, the burden listed is the estimated average burden.

• Principals/Administrators—The school administrators in the sampled schools are asked to complete a questionnaire. The core items are designed to measure school characteristics and policies that research has shown are highly correlated with student achievement. Subject-specific items concentrate on curriculum and instructional services issues. In 2024, school administrators will also answer questions about their school’s preparation, instructional organization and practices related to learning recovery of students following the COVID-19 outbreak and gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic. The burden for school administrators is determined in the same manner as burden for teachers (see above) and is estimated to average 4030-minutes per principal/administrator, although burden may vary depending on the number of subject-specific sections included. The 4030-minute burden estimate includes a supplemental charter school questionnaire designed to collect information on charter school policies and characteristics and is provided to administrators of charter schools who are sampled to participate in NAEP. The supplement covers organization and school governance, parental involvement, and curriculum and offerings.

• Submission of Samples— Survey sample information is collected from schools in the form of lists of potential students who may participate in NAEP. This sample information can be gathered manually or electronically at the school, district, or state level. If done at the state level, some states require a data security agreement, which is customized based on the specific requests of the state and provides verbatim security and confidentiality information from Section A.10. If done at the school or district level, some burden will be incurred by school personnel. It is estimated that it will take two hours, on average, for school personnel to complete the submission process. Based on recent experience, it is estimated that 26 approximately 17 percent of the schools will or districts will complete the submission process (based on the data from 2019 2022).

• Technology Staff Pre-assessment and Assessment Activities—For each school in the sample, there will be up to 3 staff members designated to perform the technical preassessment and assessment activities at the school: up to two school technology coordinators and district technology coordinator. In preparation for the study, the school or district technology coordinators will take an online pre-assessment survey (which will be submitted in Amendment #3) to gather information about the schools’ ability to connect NAEP devices to the schools’ internet and bandwidth capacity at the schools. The schools’ technology coordinator or other appointed staff member will also run a NAEP online Internet Speed Test Tool during school hours to test the internet speeds at the school to determine if the school meets the minimum requirements as outlined in the Technical Fact Sheet (this will be included in Amendment #3). In addition, schools that meet the minimum technical requirements will run the Internet Speed Test Tool again at least once within two weeks prior to the scheduled assessment date to confirm the internet speeds at the school. It is estimated that for each school and district technology coordinator, it will take 20-minutes to complete the tasks. Additionally, technical staff assisting with school technology activities will be asked to take a brief (estimated at 5 minutes) survey, administered to inform future NAEP assessments administered on school-based equipment (see draft in Appendix D-45). By utilizing these participants’ feedback about the technology from the 2024 operational assessment, NAEP will be better able to understand the available school-based equipment for the 2025 SBE Field Test. 

• School Staff Proctoring Accommodation Sessions—In 2024, NAEP will ask schools to assist in proctoring accommodation sessions that require a separate room. Based on recent assessments, about 40% of schools require a separate accommodation session on assessment day. One school staff member will be identified to proctor the assessment for the students requiring accommodations. The estimated burden for these staff will be 2.5 hours, which includes training to prepare for proctoring these separate sessions as well as proctoring the sessions themselves.

• Assessment Feedback Survey—As part of the on-going quality control of the assessment process, schools will be asked to respond to an additional follow-up survey. Survey questions solicit pre-assessment feedback, assessment day feedback, and observer feedback. The sample post-assessment follow-up survey originally designed for use in 2022 is included in Appendix E and will be updated in Amendment #3. It is estimated that this interview will take on average 2-minutes.

• Field Trial—The assessment and procedures are the same as those for the operational pilot assessment, with one exception: the teacher and school principal questionnaires are not administered in the field trial.

EXHIBIT 1

Estimated Burden for NAEP 2024 Assessments (Note: This table’s layout has changed from Amendment #1 to accommodate more respondent groups)

(Note: all explanatory notes and footnotes are displayed following the table)

Total number of respondents: 743,423

Total number of responses: 866,587

		4th Grade					8th Grade							12th Grade
	Subjects	OP and Pilot	Puerto Rico OP and Pilot	NIES4	Field Trial			OP and Pilot	Science	Puerto Rico OP and Pilot	NIES4	Field Trial		OP	HSTS5	Field Trial	Total
Students	# of Students	253,200	5,000	8,000	324			250,200	22,000	5,000	6,500	420		52,000	0	50	588,194
	Avg. min. per response	30	30	20	30			30	30	30	20	30		30	0	30	N/A
	Burden (in hours)	126,600	2,500	2,667	162			125,100	11,000	2,500	2,167	210		26,000	0	25	298,931
Teachers	# of Teachers	19,821	750	2,000	0			25,408	4,064	1,000	1,500	0		0	0	0	51,043
	Avg. minutes per response	30	30	20	0			20 and 10 minutes for each additional subject3	20	20	20	0		0	0	0	N/A
	Burden (in hours)	9,911	375	667	0			10,587	1,355	333	500	0		0	0	0	23,728
School Questionnaire (school principal)	# of Schools	6,607	250	2,200	0			6,352	1,016	250	2,000	0		1,545	0	0	16,020
	Avg. minutes per response	30	30	30	0			30	30	30	30	0		30	0	0	N/A
	Burden (in hours)	3,304	125	1,100	0			3,176	508	125	1,000	0		773	0	0	10,111
Pre-assessment Technology Activities6	# of Technical Staff	19,821	750	N/A	24			19,056	3,048	750	N/A	30		4,635	0	9	48,123
	Burden (in hours)	7,158	271	N/A	9			6,881	1,101	271	N/A	11		1,674	0	3	17,378
Pre-assessment, sample submission, & assessment feedback (school coordinator)	# of School Coord.	6,607	250	N/A	8			6,352	1,016	250	N/A	10		1,545	1,545	3	17,586
	Burden (in hours)1	33,387	1,263	N/A	40			32,099	5,134	1,263	N/A	51		7,807	4,255	15	85,314
SD/EL (school personnel)	# of Schools	6,607	250	N/A	8			6,352	1,016	250	N/A	10		1,545	0	3	16,041
	# of SD/EL Students2	68,364	1,350	N/A	87			55,044	4,840	1,100	N/A	92		8,320	0	8	139,205
	Avg. minutes per response	15	15	N/A	15			15	15	15	N/A	15		15	0	15	135
	Burden (in hours)	17,091	338	N/A	22			13,761	1,210	275	N/A	23		2,080	0	2	34,802
Accommodation sessions proctored by school staff7	# of school staff	2,643	100	N/A	3			2,541	406	100	N/A	4		618	0	1	6,416
	Burden (in hours)	6,607	250	N/A	8			6,352	1,016	250	N/A	10		1,545	0	3	16,041
Total Burden (in hours)		196,900	4,851	4,434	232			191,075	20,223	4,746	3,667	294		38,205	4,255	45	486,305

Notes for 2024 table in Exhibit 1

3. Grade 8 teachers who teach one subject have an estimated burden of 25 20-minutes, with an additional 10-minutes for each additional subject. The estimated number of teachers who teach 1 subject is 50% and 2 subjects is 50%. There is only one teacher questionnaire for science, which is assessed in a separate sample of schools from the reading and math assessments.

6. The Pre-assessment Technology Activities will be completed by the school technology coordinators (estimated at 2 per school) and the district technology coordinator (1 per school). The estimated burden for these staff is estimated to be approximately 20 minutes each, and an additional 5 minutes for one member to complete the SBE staff survey.

7. The burden for accommodation sessions proctored by school staff is 2.5 hours for 40% of the schools (based on recent assessments).

EXHIBIT 2

Total Annual Estimated Burden Time Cost for NAEP 2024 Assessments

Data Collection Year	Number of Respondents	Number of Responses	Total Burden (in hours)
2024	743,423 710,141	866,587 833,139	486,305 519,605

The estimated respondent burden across all these activities translates into an estimated total burden time cost 519,605 486,305 hours², broken out by respondent group in the table below.

	Students		Teachers and School Staff		Principals		Total
	Hours	Cost	Hours	Cost	Hours	Cost	Hours	Cost
2024	347,484 298,931	$2,519,259 $2,167,250	159,340 177,263	$5,196,077 $5,780,546	12,781 10,111	$630,743 $498,978	519,605486,305	$8,346,079 $8,446,774

A.14. Estimates of Cost to the Federal Government

The total cost to the federal government for the administrations of the 2024 NAEP data collections (contract costs and NCES salaries and expenses) is estimated to be $145,540,299 $149,999,426. The 2024 assessment cost estimate is shown in the table below.

NCES salaries and expenses	$1,400,300
Contract costs	$144,139,999 $148,599,126
Printing, packaging, and distribution, and scoring	$15,669,285 $16,732,431
Item Development	$20,000,000 $20,602,000
Sampling, recruiting and training, data collection, and weighting	$86,505,214 $88,641,881
Recruitment and State Support	$1,000,000
Design, analysis and reporting	$7,900,000 $8,373,314
Securing and transferring DBA assessment data	$120,000
DBA system development	$12,945,500 $13,129,500

A.15. Reasons for Changes in Burden and Budget (from last Clearance submittal)

Since the submission of Amendment #1, there have been some additional activities that have decreased burden and increased budget for this current Amendment #2. Changes to scope include the following:

1) Addition of Reading Router Pilot for grades 4 and 8, increasing costs;

2) Addition of School and District Technology Coordinator roles and SBE survey completion, increasing burden hours;

3) Addition of protocols for the health and safety of field staff, increasing costs;

4) Reduction in SQ burden time for students, teachers, and schools since COVID-19 learning recovery items are no longer adding additional time to the SQs; rather, other items were dropped to accommodate these items, reducing burden hours;

5) Addition of Field Trial for grades, 4, 8 and 12, increasing burden hours and costs.

The combination of the new activities, while changing the scope of the administration between Amendments, equates to a 33,300 burden-hour decrease in 2024 Amendment #2 compared to Amendment #1 (486,305 burden hours in Amendment #2 compared to 519,605 burden hours in Amendment #1). The costs of added scope activities (#1, #2, #3, and #5 in the above listing), however, equate to an increase in $4,059,127 dollars in Amendment #2 compared to Amendment #1($149,999,426 in costs in Amendment #2 compared to $145,940,299 in costs in Amendment #1).

A.16. Time Schedule for Data Collection and Publications

The time schedule for the data collection for the 2024 assessments is shown below.

NAEP 2024 Field Trial	November 2023
NAEP 2024 Administration	January–March 2024
HSTS	October 2023–October 2024

The grades 4, 8, and 12 reading and mathematics national and state results are typically released to the public around October of the same year (i.e., about 6-7 months after the end of data collection). All other operational assessments are typically released 12-15 months after the end of data collection. However, given the comparability study comparing the administration using Chromebooks and Surface Pros, the analysis will require additional time and the results will be later.

The operational schedule for the NAEP assessments generally follows the same schedule for each assessment cycle. The dates below show the timeframe for the 2024 state-level assessments:

• Spring–Summer 2023: Select the school sample and notify schools

• October–November 2023: States, districts, or schools submit the list of students

• November 2023: Administer Field Trial

• December 2023: Select the student sample

• December 2023–January 2024: Schools prepare for the assessments using the AMS system

• January–March 2024: Administer the assessments

• March–May 2024: Process the data, score constructed response items, and calculate sampling weights

• JuneMarch–July September 2024: Analyze the data

• July September –September-November December 2024: Prepare the reports, obtaining feedback from reviewers

• January or February 2025 (Grades 4/8, Reading and Mathematics): Release the results

• June or July 2025 (Grade 8 Science, Grade 12 Reading and Mathematics): Release the results

• October 2024: Release the results

Changes from 2024 NAEP Part B Amendment #1 (v.29) to Amendment #2 (v.30) are shown below:

B.1.a. Sampling Procedures

Additional information about the sampling procedures used in NAEP can be found in the technical documentation at http://nces.ed.gov/nationsreportcard/tdw/sample_design/. Note, while the latest documentation for main NAEP that has been published (as of the drafting of this document) is from 20172018, the procedures have essentially remained the same. A summary of the sampling procedures is included on the following page. Additional details (taken from the main NAEP 20172018 procedures on the technical documentation website) can be found in Appendix G (NAEP 20172018 Sample Design).

5. Select the school sample.
   After schools are assigned a measure of size and grouped on an ordered list based on the characteristics that are referred to in previous steps, the sample is selected using stratified systematic sampling with probability proportional to the measure of size using a sampling interval. This procedure ensures that each school has the required selection probability. By proceeding systematically throughout the entire list, schools of different sizes and varying demographics are selected, and a representative sample of students will be chosen for the assessment. Additional details regarding the selection of the school sample are included in the technical documentation:

(20172018 Sample Design)

NAEP yearly sample design plans are not available until the spring of the year preceding the assessments. The purpose of the sample design memorandum is to detail the specific sampling procedures used for the 2024 assessments. Included in this package (Appendix C) is the final 2022 draft 2024 design memorandum only as a placeholder, which will be replaced in a later amendment with the final 2024 version. The 2024 Field Trial schools will be selected from five different Primary Sample Units (PSU)⁴. Three of the PSUs have sampled schools at grades 4 and 8, and two of the PSUs have been sampled schools at grades 8 and 12.

B.1.b. Weighting Procedures

Additional information about the 20172018 weighting procedures used in NAEP can be found in the technical documentation at 20172018 Weighting Procedures.

Note, while the latest documentation that has been published (as of the drafting of this document) is from 20172018, the procedures have essentially remained the same. A summary of the weighting procedures is included below.

B.2.a. Recruitment of Schools

Once the sample of schools is selected for the 2024 NAEP administration, the NAEP State Coordinator and NAEP field staff typically follow a standard set of procedures for securing the participation of public and private schools. The following process includes sample final documents to be used in for the 2022 2024 administration. The 2024 versions will be are included in thise upcoming amendments:

• sending initial contact letters to chief state school and testing officers (see Appendix D2-17 16 for the letter and Appendix D2-13 12 for the included information);

• sending a notice to the district superintendents of which and how many schools were selected for NAEP from their district (see Appendix D2-3 for the letter and Appendix D2-1 for the included information);

• sending a notice of each school’s selection for NAEP to the principal or other administrative official, along with an assessment information packet containing introductory information and materials (see Appendix D2-4 for the letter and Appendix D2-2 for the included information);

• sending a notice with each school’s NAEP assessment date to the principal or other administrative official (see Appendix D-5), along with additional assessment information (see Appendix D2-7 for the letter and see Appendices D2-98 [public schools] and D2-87 [private schools] for the included information);

• sending a letter to each school’s principal with instructions for assigning a school coordinator (see letter Appendix D2-56); andalong with additional information (for the included information see Appendix D-10, D-11 and D-35, D-46, D-47 and D-48);

• sending a letter to each school’s coordinator with instructions (see letter Appendix D-6); along with additional information (for the included information see Appendix D-10, D-11 and D-35, D-46, D-47 and D-48).

• sending information to each school coordinator regarding his/her role (see Appendix D-6 for the letter and Appendices D1-2 8[public schools] and D1- [private schools] for the brochure describing the role).

NOTEote: Below outlines the flow of technical logistics communications to be included in Amendment #3.

• NAEP State Coordinator sends letters to the District Assessment Coordinator, District Technical Director, and onsite Technology Coordinator (D-42), which will include thea revised Technology Coordinator Responsibilities Guide (D-43), Technical Requirements Factsheet, and URLs to be sSafelisted.

B.2.b. School Coordinator Responsibilities

The school coordinators are responsible for preparing for the NAEP assessment in the school using the new Assessment Management System (AMS), which is an online secure site that provides participating schools with a convenient way to prepare for the upcoming assessment. AMS serves as the primary resource and action center throughout the assessment process. The secure AMS may also be used for special studies. The site also offers school coordinators an electronic way to prepare for the assessment at their own pace. The NAEP field representative will schedule an initial contact in December to pre-review the major areas of the AMS with the school coordinator. The final content of the 2024 AMS will be provided in Amendment 1 3 to be submitted in February August 2023.

• Import Student List/Sample

• Tasks: NAEP collects a list of all students in the selected grade for each school. The school submits an Excel file with all students and their demographic data (see Appendix H for a sample from 2021 I). Note, as described in Section A.12, the school coordinator is only responsible for this task if the State Coordinator has not previously submitted the student list for sampling. As such, only a portion of the school coordinators are responsible for this task.

• Purpose: Draw a representative sample of students from the school to participate in the NAEP assessments. Ensure all students have an opportunity to be sampled.

• Timeline for 2024: October – November 2023

• Complete SD/EL Student Information

• Tasks: Determine how students participate in NAEP (i.e., without accommodations, with accommodations, or do not test). Provide the Individuals with Disabilities Education Act (IDEA) disability status, English proficiency, primary language, grade- or age-level performance, and accommodations (see Appendix I section B-3)., using the state-specific NAEP inclusion policies (see Appendices D-5 and D-6 for samples of the NAEP 2021 SD and EL inclusion policies, which are customized by the NAEP State Coordinators).

• Purpose: Make sure students have appropriate supports to access the NAEP assessment.

• Timeline for 2024: December 2023 – January 2024

• Notify Parents

• Tasks: Download and customize the parent notification letter (see Appendix D2-12 10 [private schools], and D2-11 [public schools]), upload the customized letter to the system, and certify the date parents were notified. A translation notice is available to accompany the parent notification letter in instances where parents do not speak English or Spanish (see Appendix D-44).

• Purpose: Ensure that parents/legal guardians are notified of their student’s selection to participate in NAEP, which is a requirement of the Reauthorized Elementary and Secondary Education Act (ESEA).³

• Timeline for 2024: December 2023 – January 2024

• Manage Questionnaires

• Tasks: For the main NAEP administration only, identify respondents for school and teacher questionnaires, send respondents links to online questionnaires, and monitor completion of questionnaires. Distribute information about NAEP to teachers (see Appendix D1-4 35) for a sample from 2021)

• Purpose: Results are used to provide contextual data from schools and teachers in The Nation’s Report Card.

• Timeline for 2024: December 2023 – January 2024

As part of the ongoing quality control of the assessment process, schools will be asked to complete an additional follow-up survey. Survey questions solicit feedback on pre-assessment, assessment, and procedural processes. The Assessment Feedback Survey from 2022 is included in Appendix E; the 2024 versions will be submitted in Amendment #3 in July 2024.

B.3. Methods to Secure Cooperation, Maximize Response Rates, and Deal with Nonresponse

Schools within each state will be selected and the chief state school officer and the NAEP State Coordinator will be asked to solicit their cooperation. Since states and school districts receiving Title I funds are required to participate in the main NAEP reading and mathematics assessments (grades 4 and 8) under the National Assessment of Educational Progress Authorization Act, NAEP response rates have improved for these assessments. An area that has typically had lower response rates in NAEP is these audiences, high schools and private schools. As such, NCES has created specialized materials targeted at this audience:

• Videos and additional information on the NAEP website for schools, students, parents, and teachers (see http://nces.ed.gov/nationsreportcard/about/schools.aspx).

• Additional brochure and resources targeting private schools, includes NAEP in Your Private School (see Appendix D2-8 7), and a webpage dedicated just to private schools (http://nces.ed.gov/nationsreportcard/about/nonpublicschools.aspx).

B.3.a. Methods to Maximize Response Rate

Early Distribution of Information and Materials

Over the years, feedback from schools and states indicated that notification of a school’s selection in the NAEP sample earlier rather than later is beneficial to the school for planning purposes and improves school response rate. NAEP generally notifies schools of selection in May or June of the year prior to the assessment. In addition, to facilitate the school coordinators’ completion of the tasks associated with the administration, the AMS is available to the school coordinators approximately 6–7 weeks before the administration window begins.

Effective Communication with School Staff

The participation of schools can be increased by effectively communicating information about NAEP, including what NAEP measures, the various assessment components, why it is important that schools, students, and teachers participate, and the role of the school staff. Effective communication materials from the State Coordinator and the field staff (as described in Section B.2.a.) will help maximize the participation of schools. In addition, an intuitive and easy-to-use AMS system (as described in Section B.2.b.) will help ensure that the school coordinator’s experience is positive.

In addition, NCES may thank school staff and the principal for their participation in NAEP (see Appendix D2-14).

Encouraging Student Participation

Previous feedback from school administrators has shown that students respond more positively to the assessment when they know the assessment has the support of the school administration. Therefore, the field staff will encourage the school coordinator to make efforts to encourage students to do their best, including having the principal introduce the assessment. In addition, field staff will suggest to the school coordinator that grade 8 and 12 schools may want to issue community service credits for participating. Given that grade 12 student participation can be particularly challenging, NAEP has developed a Best Practices Guide to encourage grade 12 participation (to be included in Amendment #32), which is shared with sampled high schools.

B.3.b. Statistical Approaches to Nonresponse

We are working to increase engagement of private school organization leaders in recruitment efforts and requesting customized endorsement letters from these organizations (see Appendix D2-9). We have also expanded outreach efforts to schools to promote the use of NAEP data tools to highlight the value of NAEP data to private schools. Furthermore, a customized dashboard for private schools is available on The Nation’s Report Card site.

B.4. Pilot Testing and Data Uses

Beginning in 2024, NAEP will be transitioning to a Next-generation (Next-gen) version of the eNAEP test delivery software, the platform on which the assessment is delivered to students. NAEP will also be changing the operational assessment delivery model. While NAEP is currently administered by numerous NAEP field staff entering schools bringing NAEP-provided Surface Pros and Chromebooks, the program is planning to transition to a model that is ultimately less expensive and more aligned with the administration model used in state assessments. In particular, NAEP ultimately aims to administer the assessment using school-provided equipment and internet, with an alternate delivery model of cheaper, more streamlined NAEP-provided devices to be used in schools that do not meet the minimum specifications for school equipment.

In order to successfully transition to this ultimate plan, a staged approach is currently being undertaken so that trends can be measured across time. Namely, NAEP has piloted an Assessment Delivery study in 2022 (OMB# 1850-0803 v.305) as well as a Field Test in 2023 (OMB# 1850-0803 v.293) to provide more information about student interactions with the Next-gen eNAEP system and prepare for use of the updated system in operational NAEP assessments moving forward. In preparation for the 2024 NAEP administration, a Field Trial (also known as the “Dress Rehearsal”) will be piloted with students in a live classroom environment in November 2023 by NAEP field administration staff. The Field Trial will replicate the NAEP Operational Administration testing conditions to the fullest extent possible in a small number of schools. The NAEP program has utilized pilot field trials prior to large-scale digitally based assessments since 2018, based on lessons learned and issues encountered by students in the field during the 2017 operational assessment. Results from this study will not be publicly released; rather they will be used to identify issues and generate solutions or workarounds in advance of the 2024 administration.

Changes from 2024 NAEP Appendix J1 – Student Questionnaires Amendment #1 (v.29) to Amendment #2 (v.30) are shown below:

2024 Operational Grade 4/8/12 Core Mathematics/Reading Student

This table is a collapsed version of J1-1, J1-3, J1-5, J1-7, J1-9, J1-11. These changes were made across the following instruments: Grade 4 (Core, Mathematics); Grade 4 (Core, Reading); Grade 8 (Core, Mathematics); Grade 8 (Core, Reading); Grade 12 (Core, Mathematics); Grade 12 (Core, Reading).

Previous Item	2024 Item	Rationale
(2022 Grades 4 and 8, 2019 Grade 12)	N/A Issue: Grit	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.

2024 Operational Grade 4 Mathematics/Reading Student

This table is a collapsed version of J1-13, J1-19. These changes were made across the following instruments: Grade 4 (Mathematics); Grade 4 (Reading).

Previous Item	2024 Item	Rationale
(2022 Grade 4)	N/A Issue: Resources for Learning and Instruction	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grade 8 Mathematics Student

This table is a collapsed version of J1-15. These changes were made across the following instruments: Grade 8 (Mathematics).

Previous Item	2024 Item	Rationale
(2022 Grade 8)	N/A Issue: Resources for Learning and Instruction	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.
(2022 Grade 8)	Issue: Resources for Learning and Instruction	This Item was selected to be revised to reduce burden due to the addition of the Learning Recovery Items.
(2022 Grade 8)	N/A Issue: Student Factors	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grade 8 Reading Student

This table is a collapsed version of J1-21. These changes were made across the following instruments: Grade 8 (Reading).

Previous Item	2024 Item	Rationale
(2022 Grade 8)	N/A Issue: Student Factors	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.
(2022 Grade 8)	N/A Issue: Resources for Learning and Instruction	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.
(2022 Grade 8)	N/A Issue: Student Factors	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grade 12 Mathematics Student

This table is a collapsed version of J1-17. These changes were made across the following instruments: Grade 12 (Mathematics).

Previous Item	2024 Item	Rationale
(2019 Grade 12)	Issue: Resources for Learning and Instruction	SubItem “b.” added to match the Grade 8 Item that was administered in 2022.
(2019 Grade 12)	N/A Issue: Resources for Learning and Instruction	Due to the addition of the Learning Recovery Items, and to further reduce the expected response time at Grade 12, this Item was selected to be dropped.
(2019 Grade 12)	N/A Issue: Student Factors	Due to the addition of the Learning Recovery Items, and to further reduce the expected response time at Grade 12, this Item was selected to be dropped.
(2019 Grade 12)	Issue: Resources for Learning and Instruction	Due to the addition of the Learning Recovery Items, and to further reduce the expected response time at Grade 12, this Item was revised.
(2019 Grade 12)	N/A Issue: Resources for Learning and Instruction	Due to the addition of the Learning Recovery Items, and to further reduce the expected response time at Grade 12, this Item was selected to be dropped.

2024 Operational Grade 12 Reading Student

This table is a collapsed version of J1-23. These changes were made across the following instruments: Grade 12 (Reading).

Previous Item	2024 Item	Rationale
(2022 Grade 12)	N/A Issue: Resources for Learning and Instruction	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grade 4 NIES Student

This table is a collapsed version of J1-28. These changes were made across the following instruments: Grade 4 (NIES).

Previous Item	2024 Item	Rationale
		Revised based on expert group feedback.
		Revised based on expert group feedback.
		Revised based on expert group feedback.
	N/A	Dropped due to expert group feedback.
		Revised based on expert group feedback.
	N/A	Dropped due to content redundancy.
	N/A	Dropped due to content redundancy.
	N/A	Dropped due to content redundancy.
	N/A	Dropped due to content redundancy.
	N/A	Dropped due to expert group feedback.
	N/A	Dropped due to expert group feedback.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of cultural activities.
N/A		Added to extend coverage of cultural activities.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of language use.
N/A		Added to cover remote learning experiences

2024 Operational Grade 8 NIES Student

This table is a collapsed version of J1-30. These changes were made across the following instruments: Grade 8 (NIES).

Previous Item	2024 Item	Rationale
		Revised based on expert group feedback.
		Revised based on expert group feedback.
		Revised based on expert group feedback.
	N/A	Dropped due to expert group feedback.
		Revised based on expert group feedback.
		Sub-item e only. Revised based on expert group feedback.
	N/A	Sub-item b only. Dropped due to expert group feedback.
	N/A	Sub-item c only. Dropped due to expert group feedback.
	N/A	Dropped due to expert group feedback.
		Sub-item e only. Revised due to expert group feedback.
	N/A	Dropped due to expert group feedback.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of cultural activities.
N/A		Added to extend coverage of cultural activities.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of language use.
N/A		Added to cover remote learning experiences

Changes from 2024 NAEP Appendix J2 – Teacher Questionnaires Amendment #1 (v.29) to Amendment #2 (v.30) are shown below:

2024 Operational Grade 4 (Core, Reading, and Mathematics) Teacher

This table is a version of J2-1.

Previous Item	2024 Item	Rationale
(2022 Grade 4)	Issue: Gender	This Item was revised to be more inclusive and is now in line with an NCES-wide change.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.

2024 Operational Grade 8 (Core, Reading, Mathematics, and Science) Teacher

This table is a version of J2-3.

Previous Item	2024 Item	Rationale
(2022 Grade 8)	Issue: Gender	This Item was revised to be more inclusive and is now in line with an NCES-wide change.
(2022 Grade 8)	Issue: Teacher Preparation	The 2022 version of the Item, which references Social Studies, was replaced with the Item that references Science. Social Studies is not in the 2024 NAEP Assessment. Science is being administered at Grade 8.
(2022 Grade 8)	Issue: Teacher Preparation	SubItems “g.” through “k.” in the 2022 version of the Item, which reference Social Studies, have been replaced with SubItems that reference Science. SubItem “l.” also references Science. Social Studies is not in the 2024 NAEP Assessment. Science is being administered at Grade 8.
(2022 Grade 8)	Issue: Teacher Preparation	SubItems “g.” through “k.” in the 2022 version of the Item, which reference Social Studies, have been replaced with SubItems that reference Science. SubItem “l.” also references Science. Social Studies is not in the 2024 NAEP Assessment. Science is being administered at Grade 8.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.

2024 Operational Grade 4 (NIES) Teacher

This table is a version of J2-10.

Previous Item	2024 Item	Rationale
		Revised due to updated NCES language conventions.
	N/A	Dropped due to expert group feedback.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of language use.
N/A		Added to cover social emotional learning.
N/A		Added to cover social emotional learning.
N/A		Added to cover instructional practices.
N/A		Added to cover new aspects of instructional practices.
N/A		Added to cover new aspects of professional development.
N/A		Added to cover social emotional learning.
N/A	VR616316	Added to cover American Indian/Alaska Native engagement.
N/A		Added to cover American Indian/Alaska Native engagement.
N/A		Added to cover remote learning experiences.
N/A		Added to cover use of technology.

2024 Operational Grade 8 (NIES) Teacher

This table is a version of J2-12.

Previous Item	2024 Item	Rationale
		Revised due to updated NCES language conventions.
	N/A	Dropped due to expert group feedback.
N/A		Added to extend coverage of language use.
N/A		Added to extend coverage of language use.
N/A		Added to cover social emotional learning.
N/A		Added to cover social emotional learning.
N/A		Added to cover instructional practices.
N/A		Added to cover instructional practices.
N/A		Added to cover professional development.
N/A		Added to cover social emotional learning.
N/A	VR616316	Added to cover American Indian/Alaska Native engagement.
N/A		Added to cover American Indian/Alaska Native engagement.
N/A		Added to cover remote learning experiences.
N/A		Added to cover use of technology.

Changes from 2024 NAEP Appendix J3 – School Questionnaires Amendment #1 (v.29) to Amendment #2 (v.30) are shown below:

2024 Operational Grades 4 and 8 (Core) School

This table is a version of J3-1.

Previous Item	2024 Item	Rationale
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
(2022 Grades 4 and 8)	N/A Issue: School Climate	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grade 12 (Core) School

This table is a version of J3-4.

Previous Item	2024 Item	Rationale
(2019 Grade 12)	N/A Issue: Gender	This Item was dropped so Grade 12 matches Grades 4 and 8, which were administered in 2022.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.
N/A	Issue: Learning Recovery	This Item was added to address gaps in learning that have developed due to the extended period of remote and hybrid learning that took place during the pandemic.

2024 Operational Grade 8 (Mathematics) School

This table is a version of J3-7.

Previous Item	2024 Item	Rationale
(2022 Grade 8)	N/A Issue: Resources for Learning and Instruction	This Item was selected to be dropped to reduce burden due to the addition of the Learning Recovery Items.

2024 Operational Grades 4, 8, and 12 (Reading) School

This table is a version of J3-10.

Previous Item	2024 Item	Rationale
(2022 Grades 4 and 8, 2019 Grade 12)	Issue: Resources for Learning and Instruction	This Item was revised to update applicable terminology: English-language learners (ELLs) updated to English learners (ELs).

2024 Operational Grades 4 (NIES) School

This table is a version of J3-19.

Previous Item	2024 Item	Rationale
		Sub-item c only. Revised due to updated NCES language conventions.
		Sub-item e only. Revised due to expert group feedback.
	N/A	Dropped due to expert group feedback.
N/A		Added to cover social emotional learning.
N/A		Added to cover social emotional learning.
N/A		Added to cover American Indian/ Alaska Native engagement.
N/A		Added to cover American Indian/ Alaska Native engagement.
N/A		Added to cover professional development.
N/A		Added to cover professional development.
N/A		Added to solicit additional feedback.

2024 Operational Grade 8 (NIES) School

This table is a version of J3-21.

Previous Item	2024 Item	Rationale
		Sub-item c only. Revised due to updated NCES language conventions.
		Sub-item e only. Revised due to expert group feedback.
	N/A	Dropped due to expert group feedback.
N/A		Added to cover social emotional learning.
N/A		Added to cover social emotional learning.
N/A		Added to cover American Indian/ Alaska Native engagement.
N/A		Added to cover American Indian/ Alaska Native engagement.
N/A		Added to cover professional development.
N/A		Added to cover professional development.
N/A		Added to solicit additional feedback.

1 The Governing Board assessment schedule can be found at https://www.nagb.gov/about-naep/assessment-schedule.html.

2 The average hourly earnings of teachers and principals derived from May 2021 Bureau of Labor Statistics (BLS) Occupation Employment Statistics is $32.61 for teachers and school staff and $49.35 for principals. If mean hourly wage was not provided, it was computed assuming 2,080 hours per year. The exception is the student wage, which is based on the federal minimum wage of $7.25 an hour. Source: BLS Occupation Employment Statistics, http://data.bls.gov/oes/ datatype: Occupation codes: Elementary school teachers (25-2021); Middle school teachers (25-2022); High school teachers (25-2031); Principals (11-9032); last modified date May 2021.  

4 Primary Sample Units https://nces.ed.gov/nationsreportcard/tdw/sample_design/2000_2001/2000_main_samprimarysampleunits_defpsustrata.aspx

3 Please note that parents/legal guardians are required to receive notification of student participation but NAEP does not require explicit parental consent (by law, parents/guardians of students selected to participate in NAEP must be notified in writing of their child’s selection prior to the administration of the assessment).