Emergency Request Change Memo

U NITED STATES DEPARTMENT OF EDUCATION

National Center for Education Statistics

March 18. 2021

MEMORANDUM

To: Dominic Mancini, Acting Administrator, Office of Information and Regulatory Affairs

From: James Lynn Woodworth, NCES

Through: Carrie Clarady, NCES

Re: National Assessment of Educational Progress (NAEP) 2021 School and Teacher Questionnaire Special Study Emergency Justification and Change Memo

The primary purpose of this memo is to demonstrate the need for an emergency review under 5 CFR 1320.13(a) to collect voluntary responses from individual teachers and schools via an online survey to better understand the impact of the COVID-19 pandemic on students and educators. The need for immediate clearance is due to the time sensitivity of this data collection, as the 2020-2021 school year will conclude nationally within the next 3 months. Normal clearance procedures would not allow IES to collect this data before schools have completed the 2020-21 school year, and the opportunity for data collection will have been lost. NCES will publish a Federal Register Notice soliciting 30 days of public comment on this collection concurrent with data collection. This memo also documents some of the ways this data collection changes established NAEP instruments to collect better data on the impact of the coronavirus pandemic on American schools and education practices.

The National Assessment of Educational Progress (NAEP), conducted by the National Center for Education Statistics (NCES), is a federally authorized survey of student achievement at grades 4, 8, and 12 in various subject areas, such as mathematics, reading, writing, science, U.S. history, civics, and technology and engineering literacy (TEL). The National Assessment of Educational Progress Authorization Act (Public Law 107-279 Title III, section 303) requires the assessment to collect data on specified student groups and characteristics, including information organized by race/ethnicity, gender, socio-economic status, disability, and limited English proficiency. It requires fair and accurate presentation of achievement data and permits the collection of background, noncognitive, or descriptive information that is related to academic achievement and aids in fair reporting of results. The intent of the law is to provide representative sample data on student achievement for the nation, the states, and subpopulations of students and to monitor progress over time.

Throughout 2020 NCES continued to plan for a full NAEP data collection in American schools in Spring 2021. Emergency Clearance Request OMB# 1850-0956 in December 2020 announced the postponement of the 2021 main NAEP student assessments. This amendment, OMB# 1850-0956 v.2, is submitted to explain the details of how we plan to collect voluntary responses from teachers and schools via online survey questionnaires to capture their experiences during the COVID-19 outbreak and its impact on the 2019-2020 and 2020-2021 school years.

A previous emergency clearance (OMB#1850-0957) in February 2021 allowed work on the NAEP 2021 School Survey to begin. The NAEP 2021 School Survey is a monthly collection of data from 3,500 schools that gathers information about opening status (ranging from fully in-person to fully remote), the hours of instruction for students, and how enrollment and attendance rates vary by a number of social stratifying factors including race/ethnicity, socio-economic status, English learner status, and disability status. The work proposed in this package, the NAEP 2021 School and Teacher Questionnaire Special Study, is a more in-depth data collection. Although it will collect data only once, it allows NCES a deeper and richer understanding of how schools and teachers are faring while operating during a pandemic. Because the instruments are very much the same as the instruments used during every administration of NAEP, the data collected will also allow the federal government to better understand trends in schools across and through the global coronavirus pandemic.

This request updates Part A, Part B, Appendix D2, Appendix J-2, Appendix J-3, Appendix J-S, Appendix I and Appendix I2 of the last approved NAEP 2021 package (OMB# 1850-0928 v.21, November 2020), removing references to the 2021 NAEP student assessment. In addition, these appendices have been revised to add information that pertains to separate teacher and school online survey questionnaire administration that is now currently planned. Note, some of the activities for the student assessment were conducted prior to the postponement of the assessment. These activities are noted in the package and the burden for them is included. The table below details the changes that were made to each of the documents as compared to those approved in November 13, 2020 (OMB# 1850-0928 v.21), as well as the revised burden and costs to the federal government.

Table of Changes

Document	Changes
Part A	Revised or removed all references of student assessments and all activities associated with it. Revised general language to differentiate a typical NAEP assessment from this school and teacher online survey questionnaire data collection. Schedule of Amendments has been removed. Revised the explanation on use of data to only provide information specific to schools and teachers responding to the online survey questionnaires. All analyses will be contingent on participant rates and minimum sample size requirements. Added additional context about the questionnaires that schools and teachers will be completing related to the COVID-19 outbreak. Revised the burden to eliminate the student assessment activities components, including the post-assessment feedback forms. Updated costs to reflect those for the school and teacher online survey questionnaire data collection Revised the data collection schedule from January-March 2021 to March-April 2021.
Part B	Revised or removed all references of student assessments and all activities associated with it. Revised general language to differentiate a typical NAEP assessment from this school and teacher online survey questionnaire data collection. Revised the sampling procedures to remove the student sampling. Note the school sampling did not change. Revised the weighting procedures to remove the student sampling and now account for school and teacher weighting. Revised recruitment of schools to identify two additional letters sent to districts and school administrators regarding the school and teacher online survey questionnaires, and similar materials sent to private schools. Revised school coordinator responsibilities to remove the student assessment components of their role.
Appendix D2	Added new communication and recruitment materials to be used with the School and Teacher Questionnaire Special Study. For a detailed list of new communication materials see p. 2 of Appendix D2. Removed the following materials that were not circulated prior to the announcement of canceling the NAEP 2021 administration. Materials removed: Appendix D2-7: NAEP 2021 Save-the-Date Letter from NAEP Coordinator to School Principal Appendix D2-9: NAEP in Your School Appendix D2-14: Thank You for Your Participation Letter Appendix D2-15: Preassessment Feedback Email Template Appendix D2-16: Assessment Day Feedback Email Template Appendix D2-7-S-PR (Spanish version): NAEP 2-21 Save-the-Date Letter from NAEP Coordinator to School Principals, Puerto Rico Appendix D2-8-S & S-PR (Spanish version): NAEP in Your School Appendix D2-9-S-PR NAEP 2021 Parent/Guardian Notification Letter, Public School-PR Appendix D2 10-S-PR NAEP 2021 Parent/Guardian Notification Letter, Private School-PR Appendix D2-11-S-PR NAEP 2021 Parent/Guardian Notification Letter, Private School-U.S. Spanish Speaking Populations Appendix D2-12-S-PR NAEP 2021 Parent/Guardian Notification Letter, Public School School-U.S. Spanish Speaking Populations
Appendices J2, J3, J-S	Inserted paper booklet cover pages for the printed copies of the school (J3) and teacher (J2) questionnaires, as well as their Spanish versions (J-S), that are available upon request. Revised some questions to better reflect the realities of schooling during the coronavirus pandemic (see pp. 6-7 of this memo for more detailed description).
Appendix I	Reduced MyNAEP description to the pre-assessment activities performed prior to student assessment postponement and to the Manage Questionnaires section.
Appendix I2	Reduced MyNAEP description to the pre-assessment activities performed prior to student assessment postponement and to the Manage Questionnaires section - Spanish Version.

Revised Burden

Total Burden Hours decreased by an estimated 154,974 hours [total burden hours=190,417 (OMB# 1850-0928 v.21) to total burden hours=35,443 (OMB# 1850-0956 v.2)] due to final design. A comparison of the burden tables is below.

New (This Amendment, OMB# 1850-0956 v.2) – Preparation for Student Assessment and Administration of Teacher and School Questionnaires

Subjects

Students

Teachers

School Questionnaire (school principal)

School Coordinator

SD/ELL (school personnel)

Pre-assessment and sample submission

Online SQ from remote students

Total Burden (in hours)

# of Students

Avg. minutes per response

Burden (in hours)

# of Teachers

Avg. minutes per response

Burden (in hours)

# of Schools

Avg. minutes per response

Burden (in hours)

# of Schools

Burden (in hours)1

# of Schools

Burden (in hours)

# of Schools

# of SD/ELL Students

Avg. minutes per response

Burden (in hours)

4th Grade

Operational (Math and Reading) single-subject assessment

N/A

N/A

N/A

14,044

35

8,192

3,511

40

2,341

3,511

5,630

N/A

N/A

N/A

N/A

N/A

N/A

16,163

Puerto Rico Math

N/A

N/A

N/A

600

35

350

150

40

100

150

241

N/A

N/A

N/A

N/A

N/A

N/A

691

4th Grade Totals

N/A

N/A

N/A

14,644

N/A

8,542

3,661

N/A

2,441

3,661

5,871

N/A

N/A

N/A

N/A

N/A

N/A

16,854

8th Grade

Operational (Reading, Math) single-subject assessment

N/A

N/A

N/A

20,376

25 for teachers who teach 1 subject; additional 10 for each additional subject2

10,188

3,396

40

2,264

3,396

5,445

N/A

N/A

N/A

N/A

N/A

N/A

17,897

Puerto Rico Math

N/A

N/A

N/A

870

25

363

145

40

97

145

232

N/A

N/A

N/A

N/A

N/A

N/A

692

8th Grade Totals

N/A

N/A

N/A

21,246

N/A

10,551

3,541

N/A

2,361

3,541

5,677

N/A

N/A

N/A

N/A

N/A

N/A

18,589

Total Requested Burden

N/A

N/A

N/A

35,890

N/A

19,093

7,202

N/A

4,802

7,202

11,548

N/A

N/A

N/A

N/A

N/A

N/A

35,443

Total number of respondents

50,294

Total number of responses

50,294

Old (V.21) – Includes Student Assessment

Subjects	Students				Teachers				School Questionnaire (school principal)					School Coordinator							SD/ELL (school personnel)
														Pre-assessment, sample submission, & assessment feedback			Online SQ from remote students										Total Burden (in hours)
	# of Students		Avg. minutes per response	Burden (in hours)	# of Teachers	Avg. minutes per response		Burden (in hours)	# of Schools		Avg. minutes per response		Burden (in hours)	# of Schools	Burden (in hours)1		# of Schools4			Burden (in hours)3	# of Schools	# of SD/ELL Students2			Avg. minutes per response	Burden (in hours)
4th Grade
Operational (Math and Reading) single-subject assessment	95,700		35	55,825	10,064	35		5,871	2,516		40		1,677	2,516	12,714		2,264			10,756	2,516	22,968			15	5,742	92,585
Puerto Rico Math	3,000		35	1,750	596	35		348	149		40		99	149	753		134			637	149	720			15	180	3,767
4th Grade Totals	98,700		N/A	57,575	10,660	N/A		6,219	2,665		N/A		1,776	2,665	13,467		2,399			11,393	2,665	23,688			N/A	5,922	96,352
8th Grade
Operational (Reading, Math) single-subject assessment	95,700		35	55,825	14,400	25		6,000	2,400		40		1,600	2,400	12,128		2,160			10,260	2,400	18,183			15	4,546	90,359
Puerto Rico Math	3,000		35	1,750	870	25		363	145		40		97	145	733		131			620	145	570			15	143	3,706
8th Grade Totals	98,700		N/A	57,575	15,270	N/A		6,363	2,545		N/A		1,697	2,545	12,861		2,291			10,880	2,545	18,753			N/A	4,689	94,065
Total Requested Burden	197,400		N/A	115,150	25,930	N/A		12,582	5,210		N/A		3,473	5,210	26,328		4,690			22,273	5,210	42,441			N/A	10,611	190,417
		Total number of respondents					238,960					Total number of responses							346,541

Revised Cost to the federal government

The total cost to the federal government has been reduced by $95,913,500. A comparison of the cost tables is below.

New (This Amendment, OMB# 1850-0956 v.2) – Teacher and School Questionnaires

The total cost to the federal government for the preparation, data collection, and reporting of the 2021 NAEP teacher and school questionnaires (contract costs and NCES salaries and expenses) is estimated to be $6,086,800. The revised 2021 data collection cost estimate is shown in the table below.

NCES salaries and expenses	$100,000
Contract costs	$5,986,800
Printing, packaging, and distribution, and scoring	$879,000
Item development	$305,000
Sampling, training, data collection, and weighting	$3,000,000
Recruitment and state support	$315,000
Design, analysis and reporting	$987,800
Securing and transferring DBA assessment data	$0
DBA system development	$500,000

Old (V.21) – Includes Student Assessment

The total cost to the federal government for the administrations of the 2021 NAEP data collections contract costs and NCES salaries and expenses) was estimated to be $102,000,300. The 2021 assessment cost estimate is shown in the table below.

NCES salaries and expenses	$1,400,300
Contract costs	$100,600,000
Printing, packaging, and distribution, and scoring	$8,260,000
Item Development	$5,025,000
Sampling, training, data collection, and weighting	$61,655,000
Recruitment and State Support	$1,240,000
Design, analysis and reporting	$11,600,000
Securing and transferring DBA assessment data	$200,000
DBA system development	$12,620,000

Changes to Established NAEP Instruments

Previous changes (OMB# 1850-0928 v.21, November 2020) to the projected NAEP 2021 instruments included the addition of new COVID-19 modules to the teacher and school questionnaires. Those modules can be seen in full in Appendix J2 Teacher Questionnaires on pp. 89-96, in Appendix J3 School Questionnaires on pp. 88-105, and in the Spanish versions of those instruments in Appendix J-S on pp. 62-71 and pp. 136-153 respectively.

Although NCES has chosen to leave the NAEP Teacher and School Questionnaires largely as they have been used for previous administrations in order to maintain continuity of trend data, some changes were necessary in order to reflect the changes in school and education practices in the middle of the coronavirus pandemic. Those are the changes detailed below. Many of these questions are included on multiple instruments, but those changes are only detailed below once. In addition, all changes made to the English-language instruments are replicated in the Spanish-language instruments in J-S, but only the English-language changes are shown below, for the sake of brevity. As always, NAEP instruments include tables that further detail the changes to each version of the instrument; please see the attached Appendices J2, J3, and J-S for complete details.

Appendix J3 School Questionnaires

During a typical week of school, what is the total number of regularly scheduled volunteers, including parents, working in the school. Include only volunteers physically in the school building.

1. 0

2. 1-5

3. 6-10

4. 11-15

5. 16-25

6. More than 25

Approximately what percentage of students in your school have parents or guardians who do each of the following activities?

Volunteer regularly to help in the classroom or another part of the school. (Include only volunteers physically in the school building.)

1. Not applicable

2. 0-10%

3. 11-25%

4. 26-50%

5. Over 50%

In this school year, which of the following types of computers or other digital devices are available in your school or distributed by the school for student use? Select all squares that apply.

1. Desktop computers

2. Laptop computers (including Chromebooks)

3. Tablets (for example, Surface Pro, iPad, Kindle Fire)

In your school, where are desktop computers available for students to work? Select all squares that apply.

1. In some classrooms

2. In all classrooms

3. In a media center

4. In a computer lab

5. In the school library

6. The school distributed desktop computers to students doing distance learning

In your school, where are laptop computers available for students to work? Select all squares that apply.

1. In some classrooms

2. In all classrooms

3. In a media center

4. In a computer lab

5. In the school library

6. On mobile carts

7. The school distributed laptop computers to students doing distance learning

In your school, where are tablets (for example, Surface Pro, iPad, Kindle Fire) available for students to work? Select all squares that apply.

1. In some classrooms

2. In all classrooms

3. In a media center

4. In a computer lab

5. In the school library

6. On mobile carts

7. The school distributed tablets to students doing distance learning

550 12^th Street, S.W., Washington, DC 20202

Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.